Where are they and – if yes – how many?

Fleet management using agrirouter and the GPS Info App

Intro – What are chopping chains?

Harvest time – the “shopping” of Agriculture

You can never be sure who reads this, so, first a brief excursus for our non-Ag-Readers: If you ever went to a shop twice – once during the day and once shortly before they close – you know this incident: During the day, the shop is not really crowded, the cashiers sometimes have a lot of time. Shortly before closing their doors, the game is totally different. Suddenly it looks like everyone would starve if they cannot get their bread and milk and chips *RIGHT NOW*. Long line ups and the cashiers have to cash up like it’s the olympics of grocery stores.

Taking this as a comparison, chopping season is the “shopping” of agriculture. A chopper cuts corn (the whole plant) and “throws” it to a wagon that is driving behind or next to him; mostly towed to a tractor. Once the wagon is fully loaded, the transport vehicle is heading to the storage place for unloading. At the unloading place, there is tractor who compresses the stack. Once the transport vehicle unloads, the “compression tractor” will have to push the corn up the stack. Choppers don’t have a bunker (most of them), so don’t mix that up:

“

No cashier => no shopping

No transport => no chopping

”

No chain – no gain

For a constant and smooth harvesting process, it takes multiple transport vehicles. The longer the distance between storage and field, the more vehicles are required. In case of long distances of more than 20 km, transport is handled using trucks, otherwise it’s mostly done with tractors and towed wagons. This chopping chain needs to run as smooth as possible to ensure that the chopper doesn’t need to wait. The reasons why this chain could work unsmooth are quite diverse:

Traffic jams: Depending on the traffic situation, the chopper driver might be all alone for a few minutes and then there are suddenly 3 tractors who all waited at this one traffic light that feels like it takes longer than the corn to turn green.

Missing field changes: Once a field is fully processed, the chopper driver might go on to the next field and wonder why no one is coming there while all the tractors navigate to the old field.

This leads to the 2 situations that first the chopper has to wait for the transport vehicles and later the transport vehicles have to wait for the chopper.

An app a day keeps the boredom away

I’ve seen both of these incidents back when I worked for a machine manufacturer. Sometimes it took like 20 minutes to say “Hi” to a chopper driver because he had to organize everything by phone or radio.

Over the last 10 years there has been good progress and chopping chains organize themselves with fleet management. That can either be a professional software or – hands-on but less comfortable – just sharing their GPS positions via a messenger such as WhatsApp.

Experimentierfeld Nordwest – Testing fleet management with agrirouter

agrirouter goes fleet management

One important thing upfront:

Please note, that in general, agrirouter is *not* a fleet management system. It’s a data exchange platform to connect farming software such as fleet management systems with communication units.

With a little trick however we accomplished the most basic requirements of a fleet management system:

– Spread incoming position of communication units

– “Organize” fleets => decide who should see whom

agrirouter – the basics

If you already know agrirouter, you know that you can do a couple of things there:

– Connect your farming software

– Connect communication units

– Create Groups in agrirouter

– Set routings between endpoints and groups

Remark: You can think about routings like doors in a wall. By default, no data can be exchanged between endpoints. But by setting a routing, you add a door into your wall, so that data can pass in one or both directions.

Chain management with agrirouter groups

To handle different chains, agrirouter groups can be used. Groups are a possibility to handle multiple endpoints in agrirouter the same way. If you set a routing for the group, all members in the group will have such routing.

Our smartphone app is a communication unit, one endpoint for each Smartphone is created in a central account. (Remark: it would also work if each user had its own account and those accounts were connected).

First, we create 2 groups for our chopping chains, called Chopper1 and Chopper2. For each of these groups, we create exactly one Routing: “Exchange gps:info with yourself”.

This leads to a situation where all apps within one group can only communicate with such applications that are also in this group.

Side note: Be aware that with more routings and with one app being in multiple groups, this rule could be overwritten.

Now, we onboard our apps. The software is provided to the testers via TestFlight (iOS) and the App Store Beta program
(Android). Each user needs a registration code to agrirouter.

After all apps are  onboarded, they can be distributed between the two chains where one chain could use user names, the other could use machine names.

The name of the endpoint can be edited in the agrirouter UI so that a common naming convention can be used.
Now, agrirouter is setup and the data exchange can start.
 
The app
Heard of the app is a full screen map view that shows all other chain participants and means to start/stop the sending of GPS positions. The background layer can be changed on the fly, a zoom to a specific endpoint or a zoom out to the whole fleet is possible.

The app runs on Android and iOS, on smartphones and tablets.
Now, the chains can be maintained by adding or removing endpoints to one of the groups.

For the techies

Now, after all the process stuff, let’s dive a bit into the technology.

A little background

It’s – of course – open source

As part of the “Experimentierfeld NordWest”-Project the source code was published in the DKE Data Repository DKE-Data/agrirouter-gps-info-app (github.com) . Feel free to take a look. 😊

Technology

The app was built using the Xamarin Framework of Microsoft to ensure compatibility with the 2 operating systems. It uses the agrirouter SDK and the GPS:Info Data Format which is a protobuf message. The GPS:Info message is quite close but not equal to agrirouter-EFDI messages for 2 reasons:

  1. Agrirouter-EFDI provides more information such as machine data while GPS:Info only provides position and Timestamps
  2. Agrirouter-EFDI belongs to a machine while GPS:Info belongs to a communication unit. This is important to understand as we would have to generate a pseudo-Machine to send agrirouter-efdi.

GPS:Info Analysing

The format GPS:Info is a binary, protobuf based format. You can find the specification in the agrirouter Repositories here: DKE-Data/agrirouter-tmt-protobuf-definitions: Protobuf definitions of additional technical message types (github.com)

Analysis as developer can be done using the IO-Tool (https://io.my-agrirouter.com).

We’ve also build a command line converter that can read GPS:Info and transfer it to CSV. Of course it’s open source as well; find it here: dev4Agriculture/ar_telemetry_converter: A tool to convert agrirouter gps:info efdi:timelog messages to other formats (github.com)

What’s next?

There are several ideas of what to do with the app:

  • Take pictures. Simple but powerful: Document your farmwork independent of the FMIS
  • Document simple machines: Select machines and document working progress for non-digital machines such as plows

Interested in developments for the app? Feel free to contact us 😊

/by

Terminal says “No” – first aid for ISOXML imports

, ,

With the – a bit adjusted – words of Billy Joel:

Send us some help, oh, IT support

Send us some help … tonight

‘cause we’re all in the mood for a seeding task

But the import of the taskset just failed.

Most people working with a TaskController have sooner or later experienced an issue with the import of ISOXML. Of course, it’s not a farmers responsibility to fix some weird raw data file, however, if you’re ready to go and the only thing that stops you from going to the fields is waiting for your farming software support, it might be worth trying to fix the issues on your own.

The most common issue: Folder Structure

Many users drop their ISOXML Files just somewhere on the USB Stick, hoping that the terminal may find it. That works for some terminals, but some are very strict when it comes to folder structures. So, if your terminal cannot find your ISOXML, first make sure it’s in the right place

Remark: With online exchange, that’s not a difference; make sure that you have a TASKDATA folder within the zip that you’re sending.

Case sensitive naming

Another common issue is that terminals are looking for a TASKDATA.XML, but the file is called TaskData.xml or TaskData.XML. If this is the case for your ISOXML, there are two tricky issues on windows for checking and fixing that.

Visible file extension

Windows offers the functionality to “hide known file extensions”. This stops you from checking .xml vs .XML and is a bad idea for other reasons ( see e.g. https://www.cyren.com/blog/articles/what-you-see-isnt-necessarily-what-you-get ). If you still have this artifact of old Microsoft ideas on your system, you should disable it ( search for “show known file extensions [my System]” in a search engine of your choice 😉 ).

Renaming the file

As a windows user, you can quite easily go nuts when trying to rename “TaskData.xml” to “TASKDATA.XML” as windows keeps renaming it to “TaskData.xml”. The reason is, that the file system on windows is not case sensitive, so a TaskData.xml is the same as a TASKDATA.xml for windows. To avoid this error, you can rename the file in 2 steps:

  1. Rename TaskData.xml to TaskData2.xml
  2. Rename TaskData2.xml to TASKDATA.XML

This way, windows will recognize TASKDATA.XML as a new file and perform the renaming.

Remark: When writing this article, I tested the TaskData.xml to TASKDATA.XML and … it worked :astonished: . So, either Microsoft just fixed that or my system is just having a good day. However, assuming that this problem exists out there, I’ll keep this part of the article.

Switching the ISO11783 Version in your FMIS

Many Farm management systems provide a possibility to change the TaskData version they export.

The most recent version of ISOXML is ISO11783 Part 10 V4. Unfortunally, even though this version was released in September of 2015, there are still a lot of terminals out there, that are compatible with Version 3 only.  Version 2 is – from my experience – quite outdated and does not really show up anymore.

Of course I don’t know the settings possibilities of every FarmManagementSystem out there. So, I can just advice to search for “ISO11783”, “TASKDATA.XML” or “ISO Version” in the Help file or in a search engine of your choice.

If the version cannot be changed within your Farm Management system, you can do it by hand; I’ll describe that in the next chapter.

Analysing the TASKDATA.XML

If folder structure and filename is not the solution for the error, an analysis of the ISOXML might find some errors. This is a bit tricky, but don’t forget: As long as you have a backup, you cannot break anything!

ISOXML Schema validation

A TASKDATA.XML can have several different Elements with Attributes of different type and range. If there is an unknown element or if an attribute is out of range, a TaskController might have issues importing the document. Luckily, we don’t have to check all elements by hand. We can just use Schema Validation.

Preparations

For a schema validation, you will need the schemas and a tool for validation

Get the schemas

First, we need the Schemas. They can be downloaded here  (https://www.isobus.net/isobus/file/supportingDocuments ); you will most likely need the files for version 3.3, 3.0 4.2 or 4.3. Attention: There’s a second page with important files 😉

Get the Tool: Notepad++

To run the schema validation, you will need a validator tool. We advice Notepad++ as it’s also a quite handy text editor (see next steps) You can get it here: https://notepad-plus-plus.org/downloads/ Within Notepad++, you will need the XML Tools. From the menu, open Plugins/Plugins Admin

Analyse the TASKDATA

Now that we have our tools set up, we can check the TASKDATA.

  1. Load the TaskData.xml to Notepad++ (Menu: File/Open)
  2. Find the version of the TASKDATA in the root element

Here, the version is 4.2

  1. Go to Plugins/XML Tools/Validate now
  2. A window will open, asking you to select the corresponding XSD file. This is one of the schema files that you recently downloaded. Depending on the version, you will need:
VersionRequired root file
3.3ISO11783_TaskFile_V3-3.xsd
4.2ISO11783_TaskFile_V4-2.xsd
4.3ISO11783_TaskFile_V4-3.xsd

Remark:  From V4.2, many information was extracted to the file ISO11783_Common_….xsd. Make sure, the file corresponding to the required version is available as well.

  1. Click “Validate now”

Now, there are 2 possible results:

  1. No error was detected. In this case, check the chapter “Switch the version”
  2. Errors were detected. In this case, check the chapter “Fix a broken ISOXML”

Switch the version by hand

If your FarmManagement System does not provide a version change, you can do it by hand, but it might be a bit tricky.

Switch the VersionMajor to 3 and Version Minor to 3. Now, before testing the file on the terminal, run the schema validation again.

In general, a TaskSet that is compatible with Version 3.3 should be compatible with Version 4.3 as well. The other way around, that’s not 100% sure. Some elements have been extended with extra information(e.g. a TIM element can now include TimeZones), some information like GuidanceLines were added in Version 4. However, if your fields are waiting, it might be better to remove some information (that your terminal could not read anyway) than being unable to use the TasksSet at all.

Removing Parts of a TaskData-Set

If you have the correct folder structure but can’t load your ISOXML, an analysis of the involved files is required.

Therefore, a basic understanding of XML is required. Don’t worry, this won’t be a full ISOXML or XML course, just the basics so that you more or less know, what you’re doing and can remove parts without destroying the file structure.

Important!

For any operation where you manipulate files, the golden rule is: Keep a backup! Noone will be able to tell you what went wrong with your FMIS or Terminal, if you don’t have the original file. Furthermore, you will only have one chance to fix the error. If you’re wrong, the original file is gone.

XML – The extended markup language

ISOXML files consist of Elements which have Attributes and SubElements. Those SubElements can have Attributes and SubElements as well.

If it was human understandable, ISOXML looked like that:

<PartField Designator=”Peter” Size=”52”>

              <LineString Name=”Border” >

                            <Point Latitude=”52.34333” Longitude=”12.333” />

              </LineString>

</Partfield>

In general, an Element starts with < and its name, so, e.g. “<Partfield” followed by a list of ‘Attribute=”value”’, finished with a >.

All SubElements are encapsulated, so LineString is a SubElement of Partfield.

The group of subelements is closed by </ElementName>.

If an Element does not have any SubElements, it ends with a “/>” instead; e.g. like the Point.

So, for any adjustment, the following – general – rules apply:

  • You can change a value of an attribute by exchanging the text within the “”. You may not remove the quotes, even though you might input a number
  • You can remove an Element by removing the whole <…> and </…> including all SubElements

But the question is:

What should you change

In general, you can change any element and attribute that was mentioned by the XML validation.

Remove Proprietary Elements

The elements that can exist in an ISOXML file are standardized and there is a fixed list with one exception: Any element starting with a P followed by a number (so, e.g. <P223_Test >) is proprietary and manufacturer dependent. If your terminal cannot load such a TaskSet, remove all proprietary Elements including their subelements. The reason is quite simple: If your terminal could read those data, it would be able to import the TaskSet.

Duplicated Elements

The schema validation complains about two “Customers with name CTR-2”? There are 2 possible solutions:

Delete the duplicate

Sometimes, an FMIS might export a customer, a field or a farm twice. So, you have a duplicate element, but it’s just 2 times Customer “Farmer Frank” of Partfield “Cornfield 2020”. In that case, you can just delete all duplicates

Assign a different ID

This one can be tricky: If you have 2 PFD1, you can rename one of them to “PFD2” (attention: only if PFD2 does not yet exist in the list!), but: This PFD2 is not linked anywhere, so you would have to search the whole file for references to “PFD1” (Attributes other than “A” that have the value “PFD1”) and check, which one should be PFD2 instead.

Advice: Remove the second PFD and check, if your terminal can import the TASKDATA. If so, you can go on with renaming the references

Don’t flood your terminal with useless data

If your Taskset is bigger than – let’s say – 100 kilobyte, it’s worth checking, if your FMIS just dumped its whole Database into the file. Many FMIS used to export all Crop Varieties, Products, and Workers, just in case they might be need in the field. Terminals however; especially when they are older (running Version 3.3 is a good indicator) are no super computers but small embedded systems with not too much memory and storage. So, if you have a file that looks somehow like that:

It might be worth checking, if all of these Products are required. A common way is to cut all the Products and put them to a different file (Tipp: Doubleclicking on the files bar in Notepad++ creates a new file).

and then search the TaskDataFile for “PDT.

File Encoding

Honestly, this issue didn’t show up that often, but to be complete, we’ll mention it.

There are different file encodings out there. A file encoding simply describes, what the bytes within the file mean. The most simple encoding is ASCII. In Ascii, each Letter/Number/Symbol uses 1 byte. This however only allows for 255 different symbols, so, in Greek, Chinese or Arabian, this would not be enough.

ISOXML TaskData Files are always encoded in UTF-8. This is mentioned in the first line of any TaskData.xml:

The current format of the open file is displayed in the downer right corner of Notepad++:

Remove Comments

This is one of those “It should not happen but it does” errors: XML files can include comments. From a pure XML validity, that’s OK. For a terminal manufacturer, this could however be the issue that was never tested and therefore fails. Comments in xml start with

<!– and end with –> . Everything inbetween can be dropped.

Conclusion

There are various ways how an ISOXML TASKDATA file can be broken. I hope, the tricks mentioned above help you to get your file running on your terminal. If they don’t, your Farmmanagement systems IT support should be the first one to contact. If they cannot help you, feel free to contact me with at least the following information:

  • The TaskData.xml
  • Which FarmManagementSystem are you using
  • Which Terminal are you using? 
  • Which version does the terminal have?
  • Which error is shown?
/by

(AG)IT IN A NUTSHELL – SOME BASIC THOUGHTS

,

OK, now I had this really “great” idea of writing a blog post once a month in 2021. So, let’s stumble into this and see where we go. 

I started by writing about message queues. Really interesting to see how data flows from one terminal to a cloud over an interface… OK, do I need to describe an interface? Well, that’s so off topic, might be an additional blog post? 🤔

OK, let’s start with an article on UIs. Many things to discuss about User Inter….. D’oh 🤦‍♂️

So, I think I’ll just start with the basics:

A simple view to the world

From an IT guys perspective, the world is just about 2 things: Objects and Interfaces. Objects have Inputs and Outputs, Interfaces describe the match between two Objects’ in- and outputs to transfer information (in general: force, material or data). Too nerdy? Indeed, let’s ag-lify* that.

(*ag-lify: Put something in the context of agriculture 😉)

Objects in ag

The best way to describe an object that I found is definitely this one:

(Source: https://esb1jockisch.lima-city.de/math/math13/hamm/in_out/vgrgra.htm )

Take information (worm protein), convert information ( …. no details here …), output information ( other form of protein).

For conversion, we can just say:

“A few inputs + some logic = A few outputs”

So, in general an object has inputs, some logic inside, which can be mathematical formulars or Conditional statements (“Output 1 = Input1 if Input 3 = 5, otherwise (else)  Output1 = Input2”) and one or multiple outputs.

Let the blue box be an object and the orange dots be inputs (with the one inside the object being some fixed value/attribute without any influence from outside), then this “Mozart’s hair” (actually my first shot to draw a brain 😀 ) is the conversion and the green dots are outputs.

Need another example of an object? Here we go: Tractor

The tractor has many In- and Outputs: The PTO outputs power to an attached machine. The ISOBUS breakaway outputs AND inputs ISOBUS information. For the power output, you need to input fuel into the tractor. Not to forget: The wheels output force to the ground and move it under the tractor (as long as you’re not just digging that thing deeper into the mud ).

Connecting objects

On a field, our tractor is a bit lonely, so let’s give him a friend, e.g. a sprayer.

Now they are standing here, our tractor starts to move and …. D’Oh, the sprayer stays where it is? Yep, because without a compatible coupling, the tractor cannot transfer the information (“move on, dude”) to the sprayer.

Let’s call this coupling an interface. It’s just the connection between two objects, where the output of one object (“Move on dude”) is the input for the other object (“Should I stay or should I go now?”).

In a real world, this “Stand by me” interface is just the ball hitch. It is one of multiple interfaces though that connect machine and implement.

We have the ball hitch, maybe a PTO, hydraulic cycles and electronic connections such as ISOBUS.

But is it just ISOBUS? THE ISOBUS? The one and only that we’ve heard of so much?

Onions, oh bloody layered onions

Well, I would say, it’s a Yeah….nope 😐

Of course, ISOBUS is standardized, so you can connect multiple ISOBUS devices to one ISOBUS without burning down your tractor. But ISOBUS – as most interfaces – has different layers (sidenote: The following is simplified; if you’re interested in a full list, see the OSI Model ):

  • Physical: This one describes e.g. that an INCAB connector is round and has 3 by 3 pins
  • Data Link layer: That one describes, how on-off-on-off is translated into bits and bytes
  • PresentationLayer: That one describes, how bits and bytes are to be interpreted
  • Application Layer: What to do with those data?

In ISOBUS, there are different Applications like TaskController (TC) and Universal Terminal (UT). Each of them needs a client (on machine side) and a server (on terminal side).

Connecting a machine that only provides TC with a Terminal only supporting UT is somehow like Opening a Website with your Email program. It just doesn’t work.

To check compatibility, the AEF Database  is a good place to go.

Zoom, zoom, zoom, zoom….

Let’s now go a bit closer to our tractor. If we take a closer look, it’s not just a Tractor. It’s indeed a combination of different objects: It consists of wheels, an engine, a steering wheel and sometimes a nice carpet in the drivers cap. If taking all these smaller objects (that might consist of objects which might consist of objects, which might…. you get that) together, we can recognize it as a package in a detailed view. But for a high level view, a tractor is OK as an object. All the interfaces of a tractor are those interfaces, that are not connected to any other object within the tractor.

Somehow like that: 4 Inputs (e.g. Fuel, Oxygen, Steering wheel movement, sensor input), but the inner objects convert the input information to different 3 outputs (driving speed, rotating pto, ISOXML machine Data).

(let me know if you need an example of what I meant here)

May I introduce: Process chains

But wait: If we have interfaces between 2 objects and we can convert data of one input to another output, we can build a chain. 😲

Yes, that’s correct 😊 *

(*: Imagine this like a conversation. Since I’m in home office for a while, I can just say “I’m pretty sure, that’s how conversations with other people work”)

 So, let’s do this: Lets add a terminal, a modem, a cloud and a laptop and here we go: Ag-IT in a nutshell:

We translate a physical environment status (e.g. a temperature) to a sensor signal to an ISOBUS information to an ISOXML or EFDI entry, package it in some modem signal that’s converted to a Cloud API where our data just hangs around for a while until some guy (or woman) with a laptop decides, it requests some information. Stopping the chain at the user interfaces (next blog post 😉 ), that’s a – very simplified – overview of digital farming already.

Onions, oh lovely layered onions

Agreed, this layer stuff in interfaces makes the description a bit more complicated, but it has a huge advantage: Sometimes, 2 incompatible interfaces are just incompatible in certain layers, so you just have to convert these layers. Need an example?

Imagine you’re in the fields with your laptop and want to google something:

  • Your laptop only supports WiFi
  • The transmission tower only supports 4G
  • Your Smartphone supports both and can translate between those

The upper layers (HTTPS, IP etc.) are the same, but by adding your smartphone to the system, you can setup a hotspot to “convert” from WiFi to 4G.

Asking better questions

First things first: You made it through the article. Congratulations and thank you for taking the time. If you read this, I owe you a beer. (99 bottles of beer on the wall, 99 bottles of beeeeer….) Just hit me up on LinkedIn, Xing or via mail to blog@dev4Agriculture.de.

Now, what’s the essence of all of this. Why did I even write it? Because interoperability is about compatible interfaces, interfaces with different layers and information of different types. With the implementation of IoT on farms, there are more zoom levels: Intra-Machine, Intra-Field, Intra-Process Chain, Intra-Farm, Intra-Farming Community (e.g. contractors).

IoT offers big chances but has also a potential for big confusion.

I recently talked to someone working for a college with focus on farming technology. And I really liked his answer to the question “What’s the most important thing to teach about ag IT?”:

“The best would be if people learn to ask better questions”

So, with a view to all I’ve written above, we might come up with those questions:

  • Which information do I want to transfer?
  • What’s the source of my information?
  • What’s the destination of (the need for) my information?
  • Where do I have a mismatch in my interfaces? Is there a special layer to respect?
  • Which object could be fit inbetween to convert between these incompatible interfaces?

Sure, it’s not the solution for all our problems (sorry about that 🙁), but I think it’s an interesting view worth sharing and I’ld really love any feedback.

I’m yet missing a good closing for my blog articles, some inspirational catch phrase.

Let’s try that one:

Thanks for reading, I really put my heard and soil into this…

OK, agreed, I’ll find a better one next time. Thanks for reading.

/by