Handling of missing values

[dosse]

From @sosna
According to previous SDMX User Guides, NaN should be used (The guide from 2012 (i.e. published after SDMX 2.1) stated “The first in the first has an OBS_VALUE of “NaN”. This is an XML expression that declared a value of “not a number”, thus allowing a “missing value” to be declared"). More recent versions of the Guide have seen significant rewriting and that section has disappeared in the process (I don’t know why). The SDMX-ML 2.0 spec states: “In some of the SDMX-ML documents, an Observation is required (as in the Utility format) or it is desirable to indicate that a numerical value does not exist. While this information may be captured in an Observation-level attribute such as OBS_STATUS, with a code indicating that the value for the observation is missing, there is also a way to reliably indicate this state in the data itself. For this purpose, missing observation values – when included in an SDMX-ML data file – should be indicated using “NaN”. In XML, this indicates “not a number”, but is still valid in numeric fields. This avoids having to use a number (such as “-9999999” or “0”), along with a status code of “missing” (or similar construct) to indicate missing numeric values”.

The appropriate approach for missing values in SDMX-XML should be clarified.
The solution should also distinguish between “values to be set to 'missing'” and “values not to be changed when appending”.

Related ticket in SDMX-JSON: sdmx-twg/sdmx-json#122
Related Ticket in SDMX-CSV: sdmx-twg/sdmx-csv#27