How to Represent "Raw" Sensor output.

[DocOtak]

Some of our users strongly prefer to have the "raw" instrument output rather than converted/calibrated values. Raw in this instance is a voltage timeseries from a SeaBird (wetlabs?) FLBB attached to a rosette. For the seabird systems at least, I think frequency is the other "raw" output you get from some sensors.

There exist some standard_names for what the converted values would be but the units of the raw sensor data are volts so there would be a name/unit mismatch.

I'm asking for ideas on how best to represent this... ideas:

• Use the "what it would be" name with the volts as units
• Propose a "sensor_output" (or something like this) stand alone standard name for any and all variables that are just the output from some sensor, sensor model information should be included in the appropriate ACDD attributes.
• Combining the above... propose a "sensor_output" standard name modifier to say that “this is the direct output of a sensor designed to measure standard_name”
• other ideas?

Thanks for any input anyone can provide.

[japamment]

Hi Andrew, If two closely related quantities have different units, then the usual practice is to give them different standard names. For example, we have mass_concentration_of_carbon_dioxide_in_air (kg m-3) and mole_concentration_of_carbon_dioxide_in_air (mol m-3). Your quantities have units of volts which isn’t something we have used before, so certainly we would need new names. Having something in the name such as ‘instrument_output’ or ‘sensor_output’ sounds sensible. What are the existing names for the converted values? Best wishes, Alison

…

----------------------------------------------------------------------------------------------------------- Alison Pamment Tel: +44 1235 778065 NCAS/Centre for Environmental Data Analysis Email: [email protected] STFC Rutherford Appleton Laboratory R25, 2.22 Harwell Oxford, Didcot, OX11 0QX, U.K. From: Andrew Barna <[email protected]> Sent: 27 June 2019 15:16 To: cf-convention/cf-conventions <[email protected]> Cc: Subscribed <[email protected]> Subject: [cf-convention/cf-conventions] How to Represent "Raw" Sensor output. (#168) Some of our users strongly prefer to have the "raw" instrument output rather than converted/calibrated values. Raw in this instance is a voltage timeseries from a SeaBird (wetlabs?) FLBB<https://www.seabird.com/eco-flbb-rtd-backscatter-and-chlorophyll-a-with-real-time-output/product-details?id=54627915247> attached to a rosette. For the seabird systems at least, I think frequency is the other "raw" output you get from some sensors. There exist some standard_names for what the converted values would be but the units of the raw sensor data are volts so there would be a name/unit mismatch. I'm asking for ideas on how best to represent this... ideas: * Use the "what it would be" name with the volts as units * Propose a "sensor_output" (or something like this) stand alone standard name for any and all variables that are just the output from some sensor, sensor model information should be included in the appropriate ACDD attributes. * Combining the above... propose a "sensor_output" standard name modifier to say that “this is the direct output of a sensor designed to measure standard_name” * other ideas? Thanks for any input anyone can provide. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#168?email_source=notifications&email_token=AB3ZVYNS5X3CPE54AMLDOKLP4TDQXA5CNFSM4H34OYIKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G4CZANA>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AB3ZVYL7XBTTCBHKSXQGOYDP4TDQXANCNFSM4H34OYIA>.

[DocOtak]

Hi Alison,

For this particular instrument, when you apply the manufacturers calibration coefficients, you would end up with:

• volume_backwards_scattering_coefficient_of_radiative_flux_in_sea_water (m-1)
• mass_concentration_of_chlorophyll_a_in_sea_water (kg m-3, though calsheet says ug/L)

The voltage data is still useful in that it can show "something" is going on in the water when other sensors and measurements are showing something funny or unexpected. It's also good for archiving purposes since the equations for these particular measurements aren't quite settled yet.

I'm also planning on using the ACDD instrument attribute at the variable level and including the correct names from NERC L22.

As another example, if the instrument was an SBE 4, then the "raw" sensor output has units of Hz but after manufacturers coefficients applied, you get sea_water_electrical_conductivity (S m-1), later converted to sea_water_practical_salinity for actual use by oceanographers

[JimBiardCics]

@DocOtak It is also OK to have a variable without a standard name. It is still CF compliant. You can
provide information about the measurement in a comment attribute.

@japamment A quasi-generic standard name is possible, but the standard name list requires a units that the quantity can be converted to using UDUNITS, right? I really don't like the idea of adding tranches of <standard_name>_instrument_output names to cover this use case.

This same issue comes up in Level 0/1 satellite data. There are all kinds of raw measurements that end up being converted through various algorithms to radiances, reflectances, etc. I like the idea of providing a means to indicate the next level of meaning that can be applied to a raw measurement when possible, but it doesn't seem to me that the standard_name mechanism, as it stands, is up to the task.

There's a related issue with lookup tables or polynomial coefficients for raw data. The lookup table units are those of the resultant quantity, but it is not, itself, measurements of the resultant quantity. Do you attach the standard name to the lookup table? In the case of polynomial coefficients, the units for the terms are the set <output units> / <input units>**n, where n represents the power the input value is multiplied by in each case. Do we have clean mechanisms in place for any of this?

[ngalbraith]

I agree with Jim; the best way to represent this kind of 'non-science' data is to not use a standard name. We carry along lots of engineering values from many types of instruments - especially ADCPs. Using a standard name with the wrong units isn't a good idea, it's likely to cause misuse of the data.
A long name, appropriate units, and a comment should make things clear.

[DocOtak]

@JimBiardCics Some clarification for my "third" idea was to have a modifier to avoid the whole <standard_name>_instrument_output propagation, instead it would result in:
"<standard_name> instrument_output" The difference being a space between instead of the underscore. Current list of modifiers is in Appendix C. Section 3.3 also says that canonical units can be changed by the presence of that modifier.

[JimBiardCics]

@DocOtak Ah. I missed that. I think the units modification mentioned in Section 3.3 is not intended to allow the units to become arbitrary, so that would represent a significant change to the way standard names are interpreted.

[ethanrd]

Hi @DocOtak and all - There was a conversation about this (starting here) on the cf-metadata email list in 2009. Very similar ideas and discussion, there was no strong agreement (other than that there's lots of complications), and then the discussion just ended without a real proposal. I suspect people at the time went with the idea @JimBiardCics suggests here of not using standard names for this type of engineering data.

[DocOtak]

Hi @ethanrd

Thanks for the link to the mailing list conversation. I tried to read it all, but it's difficult with the way that mailman works. As you said, the only real consensus I could detect from reading all those emails was that there is a need to represent this somehow and basically all my the things I asked about in this issue were discussed.

The reason this came up is actually because I was told that the SOCCOM folks want the volts rather than "science" units. Which caused a "I guess my engineering units are now science units" situation... While my use cases so far have been 1-1 in the sensor to physical quantity mappings, from reading that email thread I can see why nothing is currently in CF specifically addressing this.

[JonathanGregory]

Following the agreement in #345 I am closing this issue with the tag ǹo_change_agreed because it has been dormant for more than twelve months. It can be reopened or the subject discussed again as a new issue if the need arises.