Reading null into std::vector<> (empty as array)

[ignus2]

I used to read/write json arrays with my own code like so:

// writing
std::vector<SomeClass> arr = ...; // array of stuff
nlohmann::json jarr;
for (const auto& a : arr) jarr.push_back(a.myToJson());
...
// reading
std::vector<SomeClass> arr;
for (size_t i = 0; i < jarr.size(); ++i) arr.push_back(MyFromJson(jarr[i]));

Now I switched to using simply to_json()/from_json() for everything including vector<>'s (indirectly), so now I simply do:

// writing
std::vector<SomeClass> arr = ...; // array of stuff
nlohmann::json jarr = arr;
...
// reading
jarr.get_to(arr);

The problem is that I have existing files written with the previous method, where if the input array was empty, it actually resulted in a null value.
Now when I try to read back those files I get an exception saying "type must be array but is null".
Is there anything I can do now? Or is there any option to specify "allow reading nulls into std::vector<> as empty"?
I really don't want to litter the code with checks each time I'm reading an array. It's a lot of nested structures.

[ignus2]

To answer my own question, I can actually fix this by using my own serialization macros, see #2588 how.
But my argument for allowing parsing null as an empty vector<> is that in C++ code, it is perfectly legal to iterate null nlohmann::json values, it is also legal to push_back() to them, which will morph the value to an array. So it is conceptually allowed/encouraged to treat null values as arrays in C++ code. But doing the same when parsing a file (or more accurately when assigning directly to a vector<>) it is suddenly not allowed and throws an exception. Therefore I propose to change the behaviour and allow assigning null values to vector<>s.

[nlohmann]

The problem was indeed your old code. Instead of

std::vector<SomeClass> arr = ...; // array of stuff
nlohmann::json jarr;
for (const auto& a : arr) jarr.push_back(a.myToJson());

you should have initialized the jarr as an array:

std::vector<SomeClass> arr = ...; // array of stuff
nlohmann::json jarr = nlohmann::json::array();
for (const auto& a : arr) jarr.push_back(a.myToJson());

[ignus2]

Sure, my problem is a conceptual one. The library allows treating nulls as if they were arrays, but inconsistently. It is like duck typing, but not always, and that leads to such surprises.

If this is allowed:

nlohmann::json jarr;
std::vector<SomeClass> arr;
for (size_t i = 0; i < jarr.size(); ++i) arr.push_back(jarr[i].get<SomeClass>());

why is this not:

nlohmann::json jarr;
std::vector<SomeClass> arr = jarr;

?

(EDIT: Also, the documentation on the main site encourages the "incorrect" usage https://github.com/nlohmann/json#stl-like-access :

// create an array using push_back
json j;
j.push_back("foo");
j.push_back(1);
j.push_back(true);

)

[nlohmann]

I understand. The former translation is magic inside the JSON type to allow code like

json j;
j["key"] = "value";

After adding an implicit conversation from null to object, I decided to do the same for array. This behavior exists since version 1.0.0, and I don't think changing this is a good idea now.

[ignus2]

I understand, it is potentially breaking for whomever might be depending on exceptions, ie. strict type checking when reading. What about a runtime or compiler option, or some global flag perhaps? Or a "policy" in cmake parlance... :)

[nlohmann]

Good idea in general, but I would need some help here. There are already a lot of preprocessor symbols, and I am afraid that this is less useful to the users as we all think...

[ignus2]

In my opinion the jnull.push_back(); and jnull["key"] = value; operations magically changing the jnull type underneath are the ones which unfortunately don't fit the rest of the concept of the library.

Because allowing null to be assigned to a vector<> (resulting in an empty vector<>) would raise the question: why not do the same for strings, resulting in an empty string? Or any type for the matter, resulting in a default constructed type?

So it seems to me that the library enforces strict typing almost everywhere, except for jnull.push_back() and jnull["key"]=..., which are outliers. They seem convenient, but I believe this convenience comes with the price of surprises which I ran into, which could lead to unforeseen bugs. And I'm not the first (and probably won't be the last) to be hit by this (see #1867, #1163, #27).

So I'd get rid of this convenience, because the other options will probably cause even more cans of worms to appear later on.
Trying to add elements to null values should be illegal. That seems the most natural to me.

In any case, I no longer care about this issue that much, since my main concern was that I'd have to write endless amounts of boilerplate to deal with this, since the built-in serialization macros (NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE) don't allow any customization, but since I now use my own macros (#2588), I can control what's happening, and avoid these pitfalls.