One of the optimisations that Fitzgibbon identifies for forward mode autodiff is loop fusion. My understanding is that the Julia compiler automatically does loop fusion on broadcast operations, but not elsewhere, and lazy evaluation has the same effect of removing intermediate allocations by delaying evaluations. Would it be worthwhile to support lazy evaluation/propagation of dual arrays and their jacobians, perhaps through LazyArrays.jl?
One of the optimisations that Fitzgibbon identifies for forward mode autodiff is loop fusion. My understanding is that the Julia compiler automatically does loop fusion on broadcast operations, but not elsewhere, and lazy evaluation has the same effect of removing intermediate allocations by delaying evaluations. Would it be worthwhile to support lazy evaluation/propagation of dual arrays and their jacobians, perhaps through
LazyArrays.jl?