more correct emulation of \box and \copy primitives

[dginev]

Fixes #2412 .

Two key upgrades:

• deeply clone the box returned during \copy
• for \box, add a new removeValue method in State, which (idempotently) unbinds the nearest binding of a value in the undo table.

This gets the tests discussed in the issue to pass, and matches the pgf rendering in the PDF build (really great diagnostic by @xworld21 !)

I would need a second opinion by @brucemiller on whether this kind of "removeValue" method makes sense for latexml's State. Its code is inspired by the global assignment mode of assign_internal, which will unbind all values in all frames, and then assign a value on the cleaned up 1-element stack.

I am wondering if many of the current uses of AssignValue($key, undef) are meant as this RemoveValue behavior, which could motivate changing local uses of assign_internal with undef to also unbind from the nearest assigned frame -- without creating a new method.

Edit: Another direction for refining the code is to add a 'local' vs 'global' argument for removeValue, to indicate whether one or all bindings should be deleted.

But I am feeling confident I am following the \box unbind semantics accurately now, as explained in the TeXbook. What was missing was unbinding the actual assignment - we were adding a local undef assignment, while also keeping the original assignment in place, when it happened in a higher frame.

Thoughts?

[xworld21]

You could use this code to implement scope => 'samelevel' assignments (with a better name, if you can find it), which would be equivalent to removeValue when assigning \undef. Should you one day discover another assignment 'in place'.

[dginev]

That's also a possibility. My main worry is that assign_internal is already somewhat entangled, so adding more to it just makes life harder... Will wait for feedback from Bruce.

I enjoy the idea of having a dedicated removal method, for what that's worth. But since the removal isn't global, the name is a bit misleading. It's really UnbindLastValueAssignment, but this is not a Java project...

[xworld21]

That's also a possibility. My main worry is that assign_internal is already somewhat entangled, so adding more to it just makes life harder... Will wait for feedback from Bruce.

I was thinking the opposite: this code is very similar to assign_internal and would be better placed there for ease of maintainance... ?

I am also wondering if removing values has other side effects, e.g. will this make \ifcsname false? That's not really possible in TeX, is it? (This has no real consequence of course since removeValue is only used for the box registers.)

[dginev]

this code is very similar to assign_internal and would be better placed there for ease of maintainance... ?

Only if it can be DRY ("do not repeat yourself"). Otherwise just growing the body of the function will make it more impenetrable.

[xworld21]

Going back to the DRY idea, couldn't this code be folded into the global assignment? If I parse correctly, your new code and global assignment differ by only two lines (last; and $$frame{value}{$key} = 0). So a samelevel scope could be implemented by adding the line last if $samelevel in assign_internal (but it wouldn't remove the value as you do here, of course).

[dginev]

I am open to the idea.

Would likely seek a slightly better name ... To fit better with global and local, maybe initial is a good keyword? Fishing for inspiration from CSS initial

[xworld21]

I agree that samelevel is a terrible name... let's think about how one could document the new scope. The assignment in scope samelevel means something like:

• in the last active scope where $name was assigned a value (so last?)
• in the scope where $name received its current value/the current definition of $name was made (so current_def or current_value?)
• in the same scope where $name was assigned the current value (so same?)
• in the scope where $name was initially assigned the current value (so initial?)

I am not totally convinced by initial. It sounds too similar to global.

[dginev]

Well, the primary purpose we'll use this for is still "removing" or "unsetting" the value (CSS has an "unset" property related to initial which also comes to mind). So having an associated scope=>"unset" could also be intuitive when looked at (+ documented) from the right perspective, if the "initial" is too confusing. The thing that bothers me a little is that such calls really don't take a "value" argument, so unifying the interfaces will be inevitably clunky. So I can also see as reasonable to keep the code paths separate.

[xworld21]

Is this idempotent? I.e. if applied twice, will it remove the two previous bindings? That would be too much for \box.

[dginev]

Curious idea - we don't yet have a test for that. Something like:

\newbox\myboxD
\setbox\myboxD\vbox{D won't appear outer.}
\begingroup
  \setbox\myboxD\vbox{D will appear inner, once.}
  \box\myboxD
  \box\myboxD
\endgroup
\bye

[dginev]

Added a test. It appears to work as expected?

[dginev]

Scratch that, the test wasn't precise enough -- there is indeed a bug where unbinding twice will pop the outer frame, nice catch @xworld21

[dginev]

Ok, using exists and a 0 value for the undo counter becomes idempotent. The new test now has correct timing and matches the PDF.

But the code got extra scary. Maybe good, maybe not?

[brucemiller]

I really need to think about what TeX is really doing, conceptually, to choose a good name & API. I expect that whatever this is should be kept fairly internal, and not exported to Package, since except for this special case of a box, removing a value is almost certainly not what you'll want to do.
The cloning is curious for a different reason. It's only needed since adjustBoxColor_internal (unwisely?) modifies the boxes in place; and that's different than TeX, because we've diverged from TeX by embedding color into the Font, and embedded the Font into the box. (TeX handles color via a much different hack).

[brucemiller]

Ah, not quite there; As mentioned in some other issue, there's the problem of when Black actually should have meant "current color". Consider this variant of your tests:

\setbox\myboxC\vbox{{\color{black}Black, stays black, then black.}}
\copy\myboxC
{\color{red}\copy\myboxC}
\copy\myboxC

[brucemiller]

I'm suspecting that perhaps Font's $DEFCOLOR should be undef, rather than Black (and maybe many other Black, as well), but that'll involve tracking a number of surprises.