Mutating validators are difficult to debug

[stephenfin]

I saw strange issues while adding schemas to a project I was working on. Despite knowing I was passing a valid instance in, I was getting errors like {} is not valid under any of the given schemas. I eventually traced this down to one of the in-tree validators (here, fwiw), which has the unfortunate side-effect of mutating the input instance. When the schema is using oneOf, the check against the first sub-schema ended up deleting all keys from the instance, causing the checks against subsequent sub-schemas to fail.

This is 100% not an issue with jsonschema but it is something I think it might be able to help mitigate. I could see us either (a) always doing a deepcopy of the instance before passing it to the validator, or (b) doing a deepcopy and comparing against instance after the call, with an exception raised if they're not identical. There might also be other approaches possible. In any case, I'm happy to help resolve this, assuming we agree it's a thing we'd like to address.

[Julian]

Hi thanks for raising and apologies that my reply will be both late and short. Do feel free to prod again though.

(I sympathize, as you're not the first person to run into this kind of tricky error)

My brief response is that I think the real thing jsonschema can do for this sort of thing to help is to support or encourange users to not represent schemas + instances as mutable types at all (but e.g. to use rpds or another persistent data structure).

Copying is definitely both expensive and has performance costs for the 99% case (where no one is writing code that mutates incoming data -- which I'd say is a big code smell in general).

But I'd really like to resolve this anyhow via a long-needed rehaul of how Validators work in general, which is needed to support JSON Schema Annotations present in newer versions (and which can be used for things beyond just classic validation).

You can find some tickets labelled Dialects v2 which essentially are pieces of related work for this.

I say long-needed but given I'm no longer sponsored to work on jsonschema time is extremely limited to be honest, so I do not know when I'll have any time whatsoever to dedicate unless I can secure some funding to let me work on jsonschema again. But that'd be where I think we could consider this.

I'm going to close the PR you raised (Which is appreciated anyhow!) given the above, but I'm happy to leave this issue open.

Thoughts again welcome.