Document $ion_literal and work out edge cases.

[toddjonker]

This hasn't made it into the 1.1 spec somehow other than oblique mention in the intro's system symbol table section.

One open issue is how we deal with $ion_literal::$ion_symbol_table::{} and $ion_literal::$ion_1_0. If those work as expected for $ion_literal, it means 1.1 can encode a value that 1.0 cannot.

I think we need to pick one:

1. We require that both 1.0 and 1.1 represent the identical space of data model values. This means that conversion between the formats will always succeed. It also means that 1.1 must reject values that cannot be encoded by 1.0.
2. We say that data that cannot be encoded in 1.0 are indicative of defects of that format, and allow them to be encoded in 1.1

[popematt]

I think that there's another option here—we could say that values annotated with $ion_... were not really part of the user/application data model to begin with because the Ion 1.0 spec reserves $ion and symbols starting with $ion_ for Ion (and related) standards.

By convention, symbols starting with $ should be reserved [...].
In particular, the symbol $ion and all symbols starting with $ion_ are reserved for use by the Ion notation and by related standards.

(emphasis added, source)

I believe that we can say that the $ion_encoding symbol (whether a value, field name, or annotation) was already reserved in Ion 1.0, so it is not an issue that $ion_encoding::{} cannot be represented as a user value in Ion 1.1. If we take this approach, then we don't need to have $ion_literal at all.

I know that there is some risk that someone is already using $ion_encoding for something, but I still think it's worth considering this proposal. Practically speaking, this proposal is similar to option 2, but it doesn't require a magic annotation.

[toddjonker]

The working group had thorough discussions of this over a year ago, and decided we didn't want to cut more slices from the data that can be represented in Ion documents. There's a meaningful difference between "this symbol is reserved" and "you cannot write data of this shape".

More to the point: I think its important to be able to say, without qualification, that any valid 1.0 document can be transcribed into 1.1. $ion_literal makes that possible.

[popematt]

Here's an edge case that needs to be addressed—how is $ion_literal data presented to the user/application—with or without the $ion_literal annotation? See amazon-ion/ion-tests#138 (comment)