Automatic Type Tracking: Operator Result Type Priority and Applications in Algorithmic Differentiation (with Examples from Mathematical Finance using Stochastic Automatic Differentiation)

Abstract

In this paper we consider the object oriented implementation of numerical algorithms where arithmetic operators (add, mult, exp) operate on objects with more complex structure (compared to floating point numbers). Examples are objects representing vectors instead of scalars, random variables, implementations on CPUs or GPUs, or augmentations of such objects providing algorithmic differentiations.

By defining a common interface an algorithm can be formulated independently of the type of the object used in each operation.

For such cases, mixing different types of such objects in operators may be desirable, but it will introduce a dependency on the order of execution of the operators. This may lead to undesired or even wrong results (as we will illustrate). To resolve this dependency we introduce a priority which determines the result type of an operation depending on the type and priorities of their arguments. This ensures that the result type will always provide the feature set of the highest priority argument type (e.g., desired floating point accuracy or algorithmic differentiability).

The formal contract introduced here will also allow to reduce the memory requirements in certain applications of algorithmic differentiations. We give examples from the algorithmic differentiation of Monte-Carlo simulation and its applications to mathematical finance.

Keywords: Automatic Differentiation, Adjoint Automatic Differentiation, Monte Carlo Simulation, Object Oriented Implementation

JEL Classification: C15, G13, C63