RFC-0020 dispersion payload

[wes-public-apps]

I have put together a proposal to support dispersion payloads. (ie sprayers and spreaders) All of the relevant info can be found in the markdown files attached. Let me know if you have any questions!

[hamishwillee]

Thanks very much @wes-public-apps . I hope to look at this tomorrow.

Do you have a target flight stack/vehicle/ground station in mind? I ask because MAVLink can't merge protocols in "abstract" without a stakeholder that is willing to take it through to implementation.

Note also that the gimbal protocol may not be the right model to follow. Gimbals are complicated because they have to address so many use cases and mediate input from multiple possible sources. There also needs to potentially be a lot of them on a vehicle, and so on. Upshot it might be possible to do less.

[wes-public-apps]

@hamishwillee I would like to target all MAVLink based autopilots eventually since I hope to make this standard widely adopted but I would be happy to start with PX4 realistically. I feel like most of the implementation work would be in a dispersion device controller itself, leaving the only PX4 implementation to be sending AUTOPILOT_STATE, integrating with missions, logging, and adding sim support. Id be open to adding the functionality for manual control to QGroundControl but since this protocol is specific to agriculture I imagined industry specific ground stations would implement the manual control. I only know of proprietary ground stations for agriculture at this time. For the dispersion device itself I was thinking about hosting an open source repo that contained a generic dispersion device controller that people could extend. I think that is OOS here since it does not fit into the existing open source structure but I am open to thoughts.

In regards to the PX4 implementation, I must say it would be stretching my skillset. I am happy to give it a shot worst case but I would prefer to be connected with someone who knows PX4 inside and out that would be willing to implement this spec. I can also reach out to my own network. So, yes I am willing to follow through with the implementation.

[wes-public-apps]

@hamishwillee I went ahead and updated my proposal to incorporate your comments. There is a decent amount of refactoring but I overall think it yielded as simpler spec. Plus we always have the earlier commits to fall back on. I plan to attend the dev call tomorrow in case there are any questions.

[julianoes]

A couple of comments but in general I think this is good!

[julianoes]

These should re-use COMPONENT_INFORMATION_BASIC!

[wes-public-apps]

I see the COMPONENT_INFORMATION_BASIC message. When you say reuse, are you saying I can:

1. use XML to extend that pre-existing definition somehow?
2. or are you suggesting I update the common fields in my proposed message to match the definition for COMPONENT_INFORMATION_BASIC via copy and paste?
3. or are you saying have the device support a microservice for COMPONENT_INFORMATION_BASIC and delete the associated fields from my information message? I would still need an information message for the remaining values and the only relevant microservice I am seeing is the Component Metadata Protocol. How stable is this WIP?

I would like to keep the timestamp as unix time rather than time from boot per my previous comments on time synchronized systems.

[hamishwillee]

@wes-public-apps He means that the majority of this information is in COMPONENT_INFORMATION_BASIC and should be sent in COMPONENT_INFORMATION_BASIC. Your component should still send COMPONENT_INFORMATION_BASIC in any case, so no point doubling up.

@julianoes We should write a component guide to mavlink for this stuff ;-0

If there are extra fields that are not in COMPONENT_INFORMATION_BASIC that are in this message we need to decide if we really need them, and then decide where they might be included if they are needed.

[wes-public-apps]

Ok, that makes sense, I pulled out the content that can be found in COMPONENT_INFORMATION_BASIC and note that message must be supported by the dispersion device in the microservice.md file. That still leaves a very large chunk of information remaining in this message that is critical and specific to a dispersion component. IMO this is a low risk message in the grand scheme of things. It is only published on request and that request should be few and far between. So I am not seeing a strong need to eliminate or prune this message. I would be much more concerned if this content was in the STATUS messages or published in a similar way.

[julianoes]

Timestamps are usually in us, not ns. I don't think that's required, and I doubt anyone would have that information or use it.

[wes-public-apps]

It is a little ambiguous whether "that" is referring to nanoseconds vs "micoseconds". To clarify, I assume you mean you dont think ns level timestamp is required since people wont have that information or use it. I chose ns because that is what I have seen the most in the robotics systems I worked on. That being said, IMO it does make sense that realistically nanosecond level timestamps are very overkill. Sticking with convention makes sense here.

[hamishwillee]

The convention is to use us for timestamp units. We don't use ns for units unless the data is actually needed, because we try to keep the message size on the wire constrained.

IN some cases, such as https://mavlink.io/en/messages/common.html#CAMERA_TRIGGER we specify them as either/or time from boot, or unix time (in which case the name shouldn't include the units (it doesn't need to anyway really).

Before you say "that'll be great" I want to use unix time, where are you planning on getting an absolute time from? Drones aren't always connected to a reliable unix time source, and I presume your sprayer might not have that information either. Better to do from boot if you want it to work everywhere, but up to you on that.

[wes-public-apps]

us units, full support. unix vs boot. more complicated ...

TL;DR I expect time from GPS somewhere in the system and support of the time sync microservice to get time to the device. I have a strong preference to use unix, but I think additionally supporting a fallback from unix to time since boot is fine. It is not something I see any value in tho. In post, a synchronized timestamp is essential to provide accurate reporting on when a system state actually occurred. Trying to reconstruct a common time between the flight controller and dispersion device after the fact is not nearly as effective.

I guess this assumption could be false since my drone experience ends at agriculture but I have never seen a drone without GPS and thus expect almost all drones to have a reliable time source since that can be converted to unix. Even if GPS loss occurs, the clock stays stable while powered. With one base source (ground station that is occasionally connected to internet or gps connected drone) , time sync across components was pretty minimal to establish. With that infrastructure in place, post processing of data from multiple components got substantially better. If time sync is not available the default unix timestamp is used and in my experience is net no different than time since boot when doing calculations. Its easy to detect. I currently have the time sync microservice as a dependency for the dispersion device.

My stance at the moment is to use unix. I think additionally supporting a fallback from unix to time since boot is fine, but not something I feel strongly about. Time from boot is totally fine if the data stream is processed in isolation. You just lose so much value around latency/staleness detection on the ground station, using linear mapping to extrapolate data from timestamps on other data streams, or combining this into a time series datastore with all system data to query. I am clearly biased since my primary experience is in tooling that surrounds the drone, so I am definitely approaching this from what will be done with the data after leaving the drone. To give an example for this exact case, it is crucial that I understand the time delay from position measurement in the autopilot to dispersion device reporting the associated status change. Without this I cannot say where the spray actually turned on or off. My uncertainty value would depend on latency in the underlying system which will not be known unless profiled and loaded into the post processing system. That assumes a perfectly ideal and consistent bus. That will vary from system to system making such an approach not very feasible when considering the alternative. Even trying to reconstruct a common time in post based on key events will be subject to that same uncertainty. A latency of 100ms at 15m/s would give us data off by 1.5m without realizing it! With time sync, now I can use difference measurement to effectively eliminate my uncertainty. My only uncertainty is driven by limitation of time sync, which is a much much smaller uncertainty. The consensus from my network is that the little overhead with time sync and using unix time is well worth down stream gains. I have very direct experience seeing this work well as we transitioned a system from no time sync to time sync.

[julianoes]

As far as I'm aware, everything on the microcontroller side is in us, and not ns, so that's why we usually just use us, and that's why I'm suggesting it for consistency.

[wes-public-apps]

There is a gap here in that this protocol works with the operational model where nozzles are physically swapped out. This protocol should support atomizers which allow you to control droplet size as well. This could pretty easily be added by:

1. adding an atomizer capability flag to info
2. adding min/max droplet size hardware limits to info
3. adding dispersion type spray atomizer
4. add atomizer failure error + out of bounds droplet size request error
5. add droplet size to status message
6. update existing mavlink command or create a new command to set droplet size

DJI T50 demo

[hamishwillee]

This is a partial response/suggestion to the thread in #26 (comment)

We need to come up with an approach that works with both architectures. Note, I have filled below in with placeholders because I'm too lazy to do this properly. This is just to show you what I'm thinking.

Suggested change

	### Recommended Hardware Set-up
	This protocol is designed to work best with a standalone MAVLink dispersion device that shares a high throughput, low latency communication bus with its controller. While any MAVLink source can be used to control the device, a common configuration is to have an autopilot providing critical real time flight data and a ground station to manage locking or low frequency control and configuration.
	
	The protocol can be used with either standalone MAVLink dispersion devices, or devices that are directly connected to autopilot I/O. The two approaches are described below.
	### MAVLink Dispersion Device
	The protocol can be used to support independent MAVLink dispersion devices.
	In this architecture the device emits the the X, Y, Z messages, and must be able to respond to the ABC commands.
	The device will need to have the same system ID as the autopilot and should use the default dispersion device component ID (these need to be configurable).
	Commonly the device is connected to the autopilot via a (relatively high speed) local serial link. The autopilot must forward commands from a ground station and telemetry from the device.
	When executing missions, the autopilot will need to re-emit mission items as command protocol commands when connected to a MAVLink dispersion device. These should be addressed to the targeted device
	[NOTE. This is why commands used in missions need a sprayer id - otherwise you can't send to a particular sprayer from a mission]
	
	### Autopilot-connected Dispersion Device
	Dispersion devices that don't support a MAVLink interface may still be controllable via autopilot I/0 (via some other protocol/custom driver).
	For these devices the autopilot can act as a dispersion device with respect to ground stations and other external MAVLink systems. In this case the autopilot should emit X, Y, Y and respond to to spray commands ABC from a ground station. Instead of forwarding commands from a GCS, it will execute them on the connected sprayer.
	[NOTE:
	- How does a GCS discover that an autopilot might have a PWM attached sprayer?
	]
	In this architecture the autopilot mission executor will need to execute commands in the mission, rather than forwarding them to a MAVLink device.
	

Note, see https://mavlink.io/en/services/mavlink_id_assignment.html for info on ID assignment.

This is what the camera and gimbal do. They expose a mavlink API to allow detection of the available services, and use an id field in commands for addressing.

HOpe this makes some kind of sense. I'm rushing.

[wes-public-apps]

I think I understand your feedback. My understanding of your comment is, we should have a custom autopilot implementation that can directly control an ESC but expose a mavlink interface with the messages for this protocol for the ground station. I considered this approach originally but am not in favor of it. My proposal is that we only allow a standalone dispersion device solution. Maybe this is an implementation call and does not belong here? The protocol could probably be utilized in that scenario pretty easily by keeping almost all the capability flags as false but I am biased in the sense I would never recommend such an implementation. Worst case, if for some reason someone had to implement something without a dispersion device compute, then they could use the generic payload protocol to accomplish this.

My question is, should that be considered a show stopper? I believe a functional standalone dispersion device is enough. It will have the most basic capability and more. I have the following issues with incorporating a controller in the autopilot.

1. the dispersion device is not flight critical and thus should not be included as bloat in flight critical software (there is already some sentiment PX4, while an amazing flight controller, tries to do too much stuff, but I acknowledge this is opinion)
2. an autopilot implementation can provide only the absolutely most basic spray controller and not the majority of what this protocol is designed to support. This system would require extensive calibration for each hardware configuration if there is no flow rate or pressure sensor connected. This is very brittle and prone to mistakes. Even without mistakes, detection of failures such as nozzle clogging or system leaks is difficult. In theory we can add an autopilot implementation that accounts for flow rate sensors and/or pressure sensors connected through the autopilot in some way For atomizers we would also need a separate PWM per nozzle. Such an implementation gets a lot more complicated since we are plumbing a lot of independent connections through the autopilot just for the dispersion system. In my opinion it is much more sensible as an isolated unit. Having at a minimum pressure or flow rate sensor and atomizers is the current market expectation. DJI and XAG are delivering this today.
3. it does not scale well when there are multiple dispersion devices. Each dispersion device takes more critical flight controller compute where as a standalone dispersion device compute handles most of the compute overhead. If you want atomizers and pressure or flow rate sensors, the scaling gets even worse to identify what sensors go with that payload.

PS sorry for being a pain, I think this is good feedback and definitely gave me something to think about.

[hamishwillee]

@wes-public-apps I'll look at all your points tomorrow. It is never "a pain" to question any of this - the more correct we are now, the easier it is in future.

[hamishwillee]

All of that makes sense, and yet if you look at ArduPilot almost everything is designed to be attached to a flight controller first.
I'm not saying that it is a better approach, but I will say that they are a key stakeholder and can block this being part of the standard. Also, they have a real integration already, so showing you can replace it with something better is a great selling point.
And yes, the capabilities of the sprayer might be very much more limited.

We've just been through the exercise of retrofitting this support for camera and gimbal, and let me tell you, it was not fun.
What we should do here is design such that it can work when attached to a flight stack, but leave that implementation as optional/untested. We know enough about design to know what will work.

Either way we also need to decide whether we will every allow multiple sprayers. . For years we only had one camera, and now we have multiples, so I am wary of saying "will never happen".
If we will need to support multiples then we need an id in the commands that may be called from missions (that's anything that is actually controllable at runtime - not necessarily things that you could set before taking off). Again, the reason it is to allow addressing individual sprayers from missions.
IF we only need one sprayer (ever) then the flight stack should be able to discover that sprayer from knowing it is attached or by monitoring for a mavlink component of the right type.

[hamishwillee]

@wes-public-apps I've tried to respond to all your comments. Sorry if I missed anything.

We discussed this in the mav call last night. Some thoughts:

1. ArduPilot implements this sprayer https://ardupilot.org/copter/docs/sprayer.html - it is worth considering how you would integrate with this particular sprayer - confirm that your interface could provide the feature this does. Ditto for other popular sprayers.
2. IN the "other options" section, I think it is worth explaining why we have chosen to have a dedicated message for the amount of fuel rather than reusing FUEL_STATUS - specifically, this is because if you have an autopilot-connected sprayer there is no way to tell that this fuel is associated with the sprayer rather than the autopilot, so this might count to cumulative flight time fuel, and warnings/actions in the case of low fuel. We might consider this if the FUEL status was extended somehow to make it possible to differentiate its purpose.

The next step is to see if we can get some more review from other stakeholders, in particular to answer questions around:

1. Is the information output sufficient
2. Are there other errors we need to provide.
3. Are there any other control options that need to be specified.

From a MAVLink perspective we then need to get this into development.xml (either included as a separate file with WIP markup, or inserted directly) and agree what needs to happen in terms of implementation and testing for us to say that this is valid. Once that's done you go off and implement. After some time you come back and we either agree it works and merge move the messages into common.xml or you tell us that you have decided on some path, and they are removed.

That's very high level. Hopefully enough to get you going. Right now I am not sure we have finished our iteration.

[hamishwillee]

@auturgy Can you reach out to the ArduPilot dev community for people in the ag-spraying community we can get to confirm they are happy with this?

[wes-public-apps]

I definitely want to note that this protocol currently has a strong bias for UAV systems. I am going to take some time to revisit that bias and update the protocol to make sure I do not design out ground based systems. That is a huge area where this could provide value as well. If there is anyone with experience in ground systems that perspective would be extremely valuable.

[wes-public-apps]

The pull request has been updated to support ground systems as well as UAVs. The main reasons for doing so are that there is a massive market for ground based dispersion rigs, supported autopilots do technically have rover modes, the ground based technology gives some indication of potential aerial application technology to come, and this protocol can be a template for other common ground rig based standards such as ISOBUS.

[hamishwillee]

The pull request has been updated to support ground systems as well as UAVs.

Makes perfect sense to support this case. The sprayer shouldn't care particularly where it is mounted.

[hamishwillee]

Sorry, I'm being dim. What is a subcomponent?

[hamishwillee]

You are setting this here so we can address a target dispersion device component in a mission?

[flocked-agriculture]

I made a lot of changes in the last update to support ground rigs which means there is a lot of new stuff to parse. I added this note to the 0020-disperison-payload.md:

This protocol should at a minimum support the capability of existing systems and then project into the future a little bit to be a timely, durable, and valuable solution. So, this protocol is designed to support a system with multiple dispersion devices and multiple subcomponents on that device. For the sake of this design, we consider a dispersion device to be a system containing one chemical mix and the infrastructure to deliver that to a field. The dipsersion device must have one gateway compute node that is responsible for communicating over a MAVLink interface. In the most simple form, a disperison device could be a microprocessor, tank, esc, motor, and spreader wheel that needs rate assigned at specific GPS locations but it can take on a more complex form factor for large ground based spray systems. These might have multiple feeder tanks with n number of subcomponents allowing independent control of subcomponent droplet size and flow rate to support many powerful features such as spot spraying and turn compenstation.

I also added this to the microservice which might be helpful:

Key Terms:

Term	Description >
Dispersion Device	A system containing one chemical mixture and the infrastructure to deliver that to a field
Dispersion Component (aka Subcomponent)	A discrete unit of the dispersion device that has independent dispersion rate control. Depending on the hardware configuration, there could be no subcomponents, all the way to a subcomponent per nozzle for independent nozzle control.

In summary I found that with ground rigs, it is common to have ground rigs with up to like 150 nozzles and each nozzle can each independently be controlled. If it is not per nozzle then they do boom sections, maybe a group of 4 nozzles. Typically one tank feeds all of the system but either a valve or motor controls flow rate at the subcomponent level with its own flow rate or pressure sensor. So a dispersion device's stats is the summation of all subcomponents. I built this protocol for the worst case scenario, a ground rig with multiple dispersion devices and each device having multiple subcomponents for independent nozzle control. A subcomponent represents an isolated control unit (from a software perspective) that take a target disperison rate and it uses what it has (ie valve, motor, rate sensors, etc) to ensure it delivers on that target.

[hamishwillee]

Are these your "subcomponents"?

[flocked-agriculture]

Effectively. I tried to give a better description in this comment.

[hamishwillee]

Parameters are explicitly internal. That means that you can't define param_do_whatever on multiple flight stacks. The implication is that you'll tend to use thing for params that really only matter on the implementation - such as which output is mapped to which dohicky on your device.

If you'll have two flight stacks that want to set the same things, then you probably can't use params, or at least not directly.

A model used for cameras was to separate out the simple things that all cameras do into a core set of commands - zoom, capture image, pan, etc. Then a camera that supports more things can export a camera definition file that references parameters - the UI is generated from this file. I'm not advocating, since it's a lot more work. I guess I'm saying we don't have to invent everything now if we can work out an reasonable extension mechanism - and address 90% of the use cases in MAVLink messages.

[flocked-agriculture]

Interesting. I meant this as the dispersion device has its own internal parameter set and parameter protocol server. The parameter protocol would be used to assign values to the device and not the autopilot. My vision for this would not require any autopilot modification. This was also more for convenience than necessity. In most cases, a manufacturer can set a device's parameters, sell that device and it will just work indefinitely. Only in cases where the owner might need more flexibility with nozzle swapping and the system does not have feedback sensors, or they require nozzle repositioning, would they need to modify parameters. I figured that it would be convenient to connect QGC or another util to the component and write the params.

I may be making some assumptions about the mavlink architecture but I think this should work.

[hamishwillee]

So what you are describing could work, but again, I'd try and reserve this for stuff that is more or less static.

The concerns with parameters is that to MAVLink what gets sent is an id and a value. In order for something that uses the parameters to make any sense of it you need to provide metadata - such as this information in PX4 https://docs.px4.io/main/en/advanced_config/parameter_reference.html#battery-calibration

You need some way to deploy that to your ground stations and keep it in sync - though I guess if it never changes and is the same for all devices you could have a plain list somewhere. ArduPilot provides this kind of metadata to QGC in a file. PX4 has dynamic mechanisms - component metadata - for sharing the file (this might be a good approach for you).
And of course the camera stuff similarly allows a dynamic UI approach.

[wes-public-apps]

@hamishwillee I removed the messages.md and dispersion.xml. I was going crazy keeping them all in sync. I have been moving forward with prototyping in the background. I forked the mavlink repo and created a pull request on that fork with the desired changes to minimal and common. I would like to reference that for message definition content to reduce the number of files I need to touch and allow me to prototype off of it.