Contextual source code AD transformations for sum types
We provide context-based source code transformations (SCTs) for the automatic differentiation of a simple first-order language with basic, product, and sum types. We consider and relate forward differentiation, reverse differentiation, and transposition SCTs. These SCTs are efficient (indeed they are straightforward structural recursions) and we give runtime and space efficiency bounds for the codes they produce. The target language of the SCTs is a first-order mixed linear/non-linear language. The tangent space of an element of a sum type (eg, real^2 + real^3) depends on which component of the type it inhabits. For that reason, the linear language has dependent types; in this we follow ideas of Lucatelli and Vakar.
|Contextual source code AD transformations for sum types (AD-transformations.pdf)||207KiB|