Variational Calculus

The calculus of variation deals with variations, i.e. small changes in functions and functionals. A small-change in a function is evaluated by applying small changes on the input arguments, see Eq. (1). While the original function only depends on the input argument(s), its variation also depends on the variation of the input argument(s).

(1)\[ \begin{align}\begin{aligned}f &= f(x)\\\delta f &= \delta f(x, \delta x)\\\delta f &= \frac{\partial f(x)}{\partial x}\ \delta x\end{aligned}\end{align} \]

The partial derivative of a function \(f(x)\) w.r.t. its input argument \(x\) is obtained from Eq. (1) by setting the variation of the input argument to \(\delta x=1\). The same holds also for nested functions by the application of the chain rule, see Eq. (2).

(2)\[ \begin{align}\begin{aligned}y &= y(x)\\f &= f(y(x))\\\delta f &= \delta f(y(x), \delta y(x, \delta x))\end{aligned}\end{align} \]

The partial derivative of a nested function \(f(y(x))\) w.r.t. its base input argument \(x\) is obtained from Eq. (3) by setting the variation of the base input argument as before to \(\delta x=1\).

(3)\[ \begin{align}\begin{aligned}\delta f &= \frac{\partial f(y)}{\partial y}\ \delta y\\\delta f &= \frac{\partial f(y)}{\partial y}\ \frac{\partial y(x)}{\partial x}\ \delta x\end{aligned}\end{align} \]

By inserting given values for \(\delta x\) one obtains the so-called gradient-vector-product for vector-valued input arguments.

Example

Given a differentiable (nested) function along with its derivative. The variation of the nested function w.r.t. its base input argument is carried out by the chain rule, see Eq. (4) and Eq. (5).

(4)\[ \begin{align}\begin{aligned}y(x) &= x^2\\\delta y(x, \delta x) &= 2 x\ \delta x\end{aligned}\end{align} \]

(5)\[ \begin{align}\begin{aligned}f(y) &= \sin(y)\\\delta f(y, \delta y) &= \cos(y)\ \delta y\end{aligned}\end{align} \]

The default evaluation graph of the nested function is shown in Eq. (6).

(6)\[\begin{matrix} x & & \rightarrow & & y(x) = x^2 & & \rightarrow & & f(y) = \sin(y) \end{matrix}\]

By augmenting the evaluation graph with its dual counterpart, variations are computed side-by-side with the default graph, see Eq. (7).

(7)\[\begin{split}\begin{matrix} x & & \rightarrow & & y(x) = x^2 & & \rightarrow & & f(y) = \sin(y) \\ \delta x & & \rightarrow & & \delta y(x, \delta x) = 2x\ \delta x & & \rightarrow & & \delta f(y, \delta y) = \cos(y)\ \delta y \end{matrix}\end{split}\]