1 Background and Introduction

A fundamental relationship in nonequilibrium physics is the Jarzynski equality [1,2,3] which relates the free energy difference between two states of a system to the work required to force the system from one state to the other. The work done during a nonequilibrium process is described by the time-integral over (\dot{\lambda }_t \partial _{\lambda }H), where (H = H(\lambda _t)) is a (\lambda )-dependent Hamiltonian and (\lambda _t) is a time-dependent control parameter as depicted in Fig. 1(a). The Jarzynski equality states that starting the system from an equilibrium distribution, the expectation of the exponential of the negative work performed on the system is equal to the exponential of the negative free energy difference between the two states:

$$\begin{aligned} \mathbb {E}\big [e^{-\beta W} \big ] = e^{-\beta \Delta F}. \end{aligned}$$

(1)

Here, (W_t=\int _0^t \partial _{\lambda }H(z_s,\lambda _s)\dot{\lambda _{s}} \textrm{d}s) is the work performed, (\Delta F) is the free energy difference, and (\beta = 1/(k_\textrm{B} T)) is the inverse temperature.

Jarzynski also extended the equality to stochastic trajectories. His original result was then found to be a consequence of a more general fluctuation theorem proposed by Crooks [4]. In this work, Crooks considers the Markovian dynamics of a system that can be influenced through a time-dependent control parameter (\lambda _t) and that satisfies the microreversibility condition

$$\begin{aligned} \frac{\mathcal {P}[x_{+t} \mid \lambda _{+t}]}{\mathcal {P}[\bar{x}_{-t} \mid \bar{\lambda }_{-t}]} =\exp \big (-\beta Q[x_{+t}, \lambda _{+t}]\big ). \end{aligned}$$

(2)

Here, (\mathcal {P}[x(+t) \mid \lambda _{+t}]) is the probability density of the forward trajectory (x_{+t}), given the control parameter (\lambda _{+t}), and (\mathcal {P}[\bar{x}_{-t} \mid \bar{\lambda }_{-t}]) is the probability density of the time-reversed trajectory (\bar{x}_{-t}), given the time-reversed control parameter (\bar{\lambda }_{-t}). Starting from Eq. (2), Crooks found

$$\begin{aligned} \frac{\mathcal {P}_F(W)}{\mathcal {P}_R(-W)} = e^{\beta (W - \Delta F)}, \end{aligned}$$

(3)

where (\mathcal {P}_F(W)) and (\mathcal {P}_R(-W)) are the probabilities of observing work W in the forward and reverse processes, respectively. This result generalizes the Jarzynski equality to a probability density over work. As with the Jarzynski equality, derivation of the Crooks fluctuation theorem seems to require an ensemble of states sampled from equilibrium at the start of the process. However, from Jarzynski’s work, it was not entirely clear how the usual concepts of heat, entropy, and free energy in thermodynamics can be defined in a general stochastic system. In particular, for what kind of system can the microreversibility condition in Eq. (2) be satisfied?

By introducing the concept of stochastic energetics, Sekimoto [5] developed a framework connecting thermodynamics to overdamped Langevin dynamics with diffusion subject to the fluctuation-dissipation relation. Specifically, the heat can be computed according to the first law of thermodynamics as (Q = \Delta U - W), where (\Delta U) is the change in internal energy.

Later, Seifert [6] showed that microreversibility holds in overdamped Langevin dynamics obeying the fluctuation-dissipation relation. Most importantly, Seifert observed that Eq. (2) and Eq. (3) do not require the system to start from equilibrium. Instead, they hold as long as the initial distribution (\rho _0) is non-singular, i.e., can be written as an integrable function on the phase space, excluding the Dirac delta distribution. To achieve this, he explicitly introduced a trajectory-dependent, information-theoretic formulation of entropy production for nonequilibrium thermodynamics, which was also used implicitly in [4] and discussed in [7]. The trajectory-dependent entropy was defined as

$$\begin{aligned} S(x_{t},t) \equiv -k_\textrm{B} \ln \rho (x_{t},t), \end{aligned}$$

(4)

where (\rho (x_{t},t)) is the probability density in state coordinate x evaluated at the value (x_{t}) of the stochastic trajectory ({x_t: t \ge 0}) at time t. In Eq. (4), the probability density is implicitly defined relative to the uniform density (i.e., the Lebesgue measure) in the phase space such that the density is dimensionless and the logarithm of probability density is well-defined. Seifert then considered overdamped Langevin dynamics under the influence of a Hamiltonian (potential) (U(x,\lambda )) and an external force (f(x,\lambda )), both subject to a time-dependent control parameter (\lambda _t). In the stochastic differential equation (SDE) formulation, the associated dynamics obey

$$\begin{aligned} \textrm{d}x_{t} = \mu \big [-\partial _x U(x,\lambda _{t}) +f(x_{t}, \lambda _{t}) \big ] \textrm{d}t + \sqrt{2D}, \textrm{d}B_{t}, \end{aligned}$$

(5)

where (\mu ) is the mobility, D is the diffusion coefficient, and (B_{t}) is a Wiener process that generates Brownian motion. Using the path integral formulation of overdamped Langevin dynamics [8], Seifert validated Eq. (2) and Eq. (3) for initial distributions (\rho _0(x)) where (\ln (\rho _0(x))) is well defined over the whole phase space. Eq. (1) was then generalized to the following form:

$$\begin{aligned} \mathbb {E}\big [e^{-\beta (W_t -\Delta F_t)} \big ] = 1. \end{aligned}$$

(6)

Here, and in the rest of the paper, (\mathbb {E}[X_t]) is the expectation of (X_t) over all trajectories starting from a given initial distribution (\rho _0(x)) up to time t. The subscript t indicates the process sampled at time t. (W_t) is the cumulative work up to time t and (\Delta F_t = F_t - F_0) is the free energy difference between time t and time 0. Note that we have adapted previous notation [6] to be consistent with the context and derivations in the rest of the paper. In the original paper [6], the heat exchange between the system and the environment is understood in terms of entropy change in the environment. Then, considering the total entropy change in the system and the environment, we have (\Delta S_\textrm{tot} = (W- \Delta F)/T).

Fig. 1

Schematic of standard backward driving protocols and the associated probability densities. (a) The forward driving protocol (\lambda _t) as a function of forward time t. (b) The backward driving protocol (\tilde{\lambda }_{s}) is obtained by counting time backwards from the terminal time (t_1). (c) Probability densities of the forward ((\rho )) and the standard backward ((\tilde{\rho })) processes described in previous work [9]. With an initial distribution (\tilde{\rho }(x_{0},0) = \rho (t_1)), (\tilde{\rho }(s)) indexed by the backward time s evolves under the time-reversed driving protocol (\tilde{\lambda }_{s})

Subsequently, Sagawa and Ueda [10] introduced the concept of feedback control and measurement into nonequilibrium thermodynamics, generalizing the Jarzynski equality to account for systems where a feedback controller influences the dynamics. They further generalized Jarzynski’s equality starting from equilibrium to

$$\begin{aligned} \mathbb {E}\big [e^{-\beta (W_t - \Delta F_t) - I_t} \big ] = 1, \end{aligned}$$

(7)

where (I_t) is the (trajectorywise version of) mutual information between the actual system state x and the measurement outcome y, defined to be (I_t=I\big (x_t,y_t,t\big ) = - \ln \left[ ({\rho \big (x_t,y_t,t\big )}/{(\rho (x_t,t) \rho (y_t,t))}\right] ). This is the first result that explicitly connects the Jarzynski equality to information theory.

More recently, martingale properties of entropy production in stochastic systems have been explored by Neri [11] and Manzano et al. [9], who extended the Jarzynski equality from a fixed time t to a stopping time (\tau \le t). Through the use of path probability densities from a path integral formulation of an overdamped process, they derived the following martingale identity that is associated with expectations of entropy production evaluated over all trajectories of duration (\tau ) after the initial time at which the distribution is (\rho _0):

$$\begin{aligned} \mathbb {E}\big [ e^{-\beta (W_{\tau } - \Delta F_{\tau }) - \delta _{\tau }} \big ] = 1, \end{aligned}$$

(8)

where the stochastic distinguishability (\delta _{\tau }) is defined as

$$\begin{aligned} \delta _{\tau } = {\left{ \begin{array}{ll} \displaystyle {\ln \left[ \frac{\rho _\textrm{eq}(x_{\tau },\lambda _{\tau })}{\tilde{\rho }{(x_{\tau }, t-\tau )}} \right] } & (\text {Neri}, [11]) \ \displaystyle \ln \left[ \frac{\rho (x_{\tau },\tau )}{\tilde{\rho }{(x_{\tau }, t-\tau )}}\right] & (\text {Manzano}, [9]). \end{array}\right. } \end{aligned}$$

(9)

Here, (\rho _\textrm{eq}(x_{\tau }, \lambda _\tau )) refers to the equilibrium probability density of the system at (x_{\tau }) with parameter (\lambda _\tau ), (\rho (x_{\tau }, \tau )) is the probability density of the system at x, evaluated at (x_{\tau }) at time (\tau ), and (\tilde{\rho }) is the time-reversed probability density under a time-reversed driving protocol as shown in Fig. 1(b,c). In the setting of Neri [11], the initial condition is set to equilibrium, rendering his results equivalent to Jarzynski’s equality at random times. On the other hand, Manzano et al. works with entropy defined from nonequilibrium distributions, which has been further generalized by Yang and Ge [12] to decoupled auxiliary processes.

Nearly all previous results have been developed using a path integral formulation [8] for overdamped Langevin dynamics or quantum systems [9, 13]. While the path integral formulation is a convenient tool in many areas of physics, gaps in its mathematical rigor may preclude certain desirable directions of analysis. For example, the path integral integrates over the space of continuously differentiable functions but solutions to stochastic differential equations (SDEs) are nowhere differentiable. There have been several efforts to formulate the path integral in a mathematically rigorous way [14,15,16]; however, these approaches were primarily focused on quantum path integrals and typically assigned a different interpretation of the probability density in the path integral. A mathematically satisfying formulation of the path integral for classical stochastic systems can arise through Girsanov’s theorem, and was treated previously in [17]. Even though Girsanov’s theorem is a powerful tool in the theory of stochastic calculus, dealing with the path integral with both forward and backward paths is challenging in terms of precise interpretation of the probability density in the forward and backward paths. Moreover, derivations using path integrals rely on microreversibility, precluding treatment of far-from-equilibrium systems where microreversibility does not hold.

On the other hand, solving the corresponding Fokker-Planck equation to find the trajectorywise entropy (S(x_t,t)) at sufficient numerical precision requires significant computational resources. This is especially challenging for high-dimensional systems or systems with complex potentials. While biological systems can often be described as Maxwell’s demons that convert information into work [18, 19], the aforementioned computational demands limit application of the generalized work theorem to biological systems. Thus, experimental verification of the Jarzynski equality has been restricted to relatively simple artificial systems [20,21,22].

In this paper, we side-step path integration by providing an alternative mathematical proof of the martingale property of entropy production. While our method mirrors that described in a recent treatise on martingale methods for physicists [23], we further show that this proof reveals a generalization to the work theorem, extending it to a family of equations that hold for the same stochastic process but using different choices for the backward process. Our proof also strengthens the equality in Eq. (8) by explicitly evaluating the conditional expectation of the same exponential given the initial value (x_0) for any initial distribution (\rho _0). In particular, the initial condition can be singular, such as the Dirac delta distribution. Thus, it is not necessary to average over all trajectories sampled from the initial distribution (\rho (x_0,0)). Moreover, our new “forward-backward-decoupled” work theorem can be developed using underdamped dynamics described by position x and velocity v. Our generalized work theorem can be applied to high-dimensional out-of-equilibrium systems or systems with complex potentials with lower computational costs. Specifically, while the initial condition of the forward process can be arbitrary, we can set the backward process to be the flux associated with a nonequilibrium steady state. In this way, the only computations needed are solving for a stationary distribution and forward simulations. The need for solving a PDE in time is no longer required.

2 Analysis and Results

We formally derive and describe a number of results below.

2.1 Mathematical Approach

We provide the mathematical intuition behind our approach. Consider a stochastic process x that evolves according to a stochastic differential equation (SDE) of the form (\textrm{d}x_t = b_t\textrm{d}t + \sigma _t\cdot \textrm{d}B). If (b_t\equiv 0), the process (x_t) is purely driven by diffusion and has no drift bias in any direction. Once some regularity conditions are satisfied, the mean of the process (x_t) is the same as that of the initial condition, i.e., (\mathbb {E}[x_t] = \mathbb {E}[x_0]).

We formalize the intuition developed above by introducing the concept of an exponential martingale. Consider a predictable process (\theta _t) adapted to the filtration of a standard Wiener process (B_t). The exponential martingale ((M_t)_{t \ge 0}) associated with (\varphi (x_t)) is defined by:

$$\begin{aligned} M_t \equiv \exp \Big ( \int _0^t !\varphi (x_s)\cdot \text {d}B_s - \frac{1}{2} \int _0^t !\Vert \varphi (x_s)\Vert ^2 , \text {d}s \Big ), \end{aligned}$$

(10)

where (\int _0^t \varphi (x_s)\cdot \text {d}B_s) denotes the stochastic integral with respect to the Wiener process that drives (x_t) and s is the integrated-over dummy time variable. The term (\tfrac{1}{2} \int _0^t \Vert \varphi (x_s)\Vert ^2 , \text {d}s) is the compensating drift term, ensuring that (M_t) has zero mean drift.

To guarantee that (\big [M_t\big ]_{t \ge 0}) is indeed a true martingale, a sufficient condition is provided by the well-known Novikov Condition, which states that if

$$\begin{aligned} \mathbb {E}\bigg [\exp \Big (\frac{1}{2} \int _0^t \Vert \varphi (x_s)\Vert ^2 , \text {d}s \Big )\bigg ] < \infty \quad \forall , t > 0, \end{aligned}$$

(11)

holds then the process (M_t) defined in (10) is a true martingale. Throughout this paper, we will use Itô calculus rules for stochastic integrals. Here, the terms “predictable process” and “filtration” are mathematical definitions necessary for the Itô integral to be defined. In the usual context of stochastic thermodynamics, they can be thought of as being automatically satisfied when the processes of interest are functions of some “fundamental” processes (e.g., coordinates of the particle) at current time points. For example, the potential (H(x_t,t)) of (x_t) at time t is predictable with respect to the natural filtration of (x_t) or the underlying Brownian motion (B_t). Interested readers can refer to [24] for a comprehensive and pedagogical formulation of stochastic calculus.

2.2 Overdamped Dynamics

Consider overdamped Langevin dynamics defined by

$$\begin{aligned} \gamma \textrm{d} x_t&= -\nabla H(x_t,t) , \textrm{d} t + f(x_t,t) ,\textrm{d} t + \sqrt{\frac{2 \gamma }{\beta }}, \textrm{d} B_t. \end{aligned}$$

(12)

Here, x is the state of the system, H(x, t) is the Hamiltonian, f(x, t) is the external force, and (\gamma ) is the friction coefficient. The time dependence of H(x, t) and f(x, t) can include that of the control parameter (\lambda _t) in the original formulation.

The corresponding Fokker-Planck equation is given by

$$\begin{aligned} \partial _t \rho (x,t) = \frac{1}{\gamma } \nabla \cdot \left[ \big (\nabla H(x,t) - f(x,t)\big ) \rho (x,t) \right] + \frac{1}{\beta \gamma } \Delta \rho (x,t), \end{aligned}$$

(13)

where (\rho (x,t)) is the probability density of the system at time t, and (\nabla ) and (\Delta ) are the gradient and Laplacian operators, respectively, with respect to the state variable x.

Along a given trajectory (x_{s\le t}), the work performed on the system up to time t is given by [7]

$$\begin{aligned} \begin{aligned} W_t&= \int _0^t f(x_s,s) \circ \textrm{d} x_s + \int _0^t \partial _{t} H(x_s,s) ,\textrm{d}s,\ ,&= \int _0^t f(x_s,s) ,\textrm{d}x_s + \frac{1}{\gamma \beta }\int _0^t \nabla \cdot f(x_s,s) ,\textrm{d}s + \int _0^t \partial _{t}H(x_s,s) ,\textrm{d}s, \end{aligned} \end{aligned}$$

(14)

where (\circ ) denotes the Stratonovich integral. The entropy (S(x_t)) specific to the trajectory (x_t) is given by Eq. (4).

Our backward process. While the identity in Eq. (6) suggests that (\exp \big [-\beta (W - \Delta F)\big ]) is a martingale, it is actually not if (F = H - TS). In the following, we will define a general version of entropy, (\Sigma (x_t ,t) = -k_\textrm{B}\ln \psi (x_t,t)), and show that (\exp [-\beta (W_{t}- \Delta F_{t})]) is a martingale when (\Sigma ) replaces S. Here (\psi (x,t)) solves a backward Fokker-Planck equation, which can be interpreted as a probability distribution for a generalized backward process, as detailed below.

Consider a time-reversed driving protocol described by (\tilde{\lambda }_{s}) depicted in Fig. 1(b). In our analysis we will implicitly define forces, Hamiltonians, and densities under the reversed protocol by reversing the time direction of the Fokker-Planck equation by switching the sign of the time-derivative term and do not need to explicitly invoke (\tilde{\lambda }_{s}). Specifically, the Hamiltonian and external force are replaced by their time-reversed forms, (\widetilde{H}(x,s):=H(x,t_1-s)) and (\widetilde{f}(x,t):=f(x,t_1-s)) [9]. Here, (s = t_1 - t) represents how far back in time the current time is compared to the terminal time (t_1). A new “time-reversed” probability density (\tilde{\rho }(x,s)) with initial condition (\tilde{\rho }(x,0) = \rho (x,t_1)), depicted in Fig. 1(c), evolves according to the “backward” Fokker-Planck equation

$$\begin{aligned} \partial _t \tilde{\rho } = \frac{1}{\gamma } \nabla \cdot \left[ \big (\nabla \widetilde{H}(x,t) - \widetilde{f}(x,t)\big ) \tilde{\rho } \right] + \frac{1}{\beta \gamma } \Delta \tilde{\rho }. \end{aligned}$$

(15)

When the potential H(x, t) and the force f(x, t) are time-asymmetric, i.e., (H(x,t) \ne \widetilde{H}(x,t)) and (f(x,t) \ne \widetilde{f}(x,t)), the time-reversed process for (\tilde{\rho }(x,t)) is different from the original process for (\rho (x,t)) in the sense that (\tilde{\rho }(x,t) \ne \rho (x, t_1-t)) for (t \in (0,t_1)). Note that the Fokker-Planck equation is also known as the Kolmogorov forward equation, but here, the backward Fokker-Planck equation differs from the Kolmogorov backward equation which is simply the adjoint of the Kolmogorov forward equation [25].

We define our backward process by its probability density (\psi (x,t)) which obeys a specific type of backward Fokker-Planck equation

$$\begin{aligned} -\partial _t \psi {(x,t)} = \frac{1}{\gamma } \nabla \cdot \Big [ \big (\nabla H(x,t) - f(x,t)\big ) \psi (x,t) \Big ] + \frac{1}{\beta \gamma } \Delta \psi (x,t). \end{aligned}$$

(16)

Eq. (16) differs from the Fokker-Planck equation (13) by an extra minus sign in front of the time derivative. It is straightforward to verify that (\psi (x,t) = \tilde{\rho }(x,t_1 -t)) and that (\psi (x,t)) is associated with the time-reversed probability (\tilde{\rho }) used in [9].

In the setting of Crooks’ fluctuation theorem [4], a specific terminal time (t_1) was a priori chosen, and (\psi (x,t) :=\tilde{\rho }(x,t_1 - t)) solves Eq. (16) with initial condition (\psi (x,t_1) = \rho (x,t_1)). The specific time-reversed probability density (\tilde{\rho }(x,s)) is used in Eq. (9) to define the stochastic distinguishability (\delta ). (\psi (x,t_1 -s) =\tilde{\rho }(x, s)) maps the time-reversed probability density at the backward time s to the time-reversed probability density at the forward time (t = t_1 -s), as illustrated in Figs. 1(c).

To derive the generalized work theorem, (\psi (x,t)) need only satisfy Eq. (16) regardless of initial condition. Consequently, choosing different initial conditions results in different martingales and corresponding identities. Our generalization of the work theorem is based on the observation that the backward process (\psi (x,t)) can be defined with an arbitrary initial condition, not necessarily (\rho (x,t_1)), as shown in Fig. 2(a). Broadly speaking, our approach does not require specifying a terminal time (t_1).

Analogous to the entropy (S(x_t)) defined in Eq. (4), we define the “entropy” of the backward process along the trajectory (x_t) as

$$\begin{aligned} \Sigma (x_t,t) = -k_\textrm{B}\ln \psi (x_t,t). \end{aligned}$$

(17)

As with the trajectorywise entropy, (\Sigma (x_t,t)) implicitly requires a reference probability density (\psi (x_0,0)). Throughout this work, we use (A_t) to denote the trajectory-specific value of A(x, t) evaluated at (x=x_t), e.g., (\Sigma _t \equiv \Sigma (x_t,t)).

Fig. 2

Decoupling of the forward and backward processes. (a) In our derivations, we employ the probability density (\psi (x,t)) of a backward process indexed by forward time t. The initial condition is arbitrary so that in general (\psi (x,t_1) \ne \rho (x,t_1)). (b) Trajectories of the forward processes (x_t) sampled from the initial distribution (\rho (x_{0},0)) are shown in grey, while trajectories with a specific initial value (x_0) are shown in red. The original work theorem uses averages over the grey trajectories, while our generalized work theorem considers averages over the red trajectories. See Eqs. (24) and (27)

Martingale and generalized work theorem. We now consider the quantity defined by

$$\begin{aligned} \theta _t = - \beta W_t + \beta \big (\Delta H(x_t,t)-T\Delta \Sigma (x_t,t)\big ). \end{aligned}$$

(18)

This process represents the nondimensionalized “entropy production” along the trajectory (x_t) when the entropy S is replaced by that of the backward process (\Sigma ). Upon using Itô’s formula to expand terms, the time derivative of (\theta _t) is given by

$$\begin{aligned} \begin{aligned} \textrm{d} \theta _t&= - \beta f \cdot \textrm{d}x_t - \frac{1}{\gamma } \nabla \cdot f, \textrm{d}t - \beta \partial _{t} H, \textrm{d}t + \beta \nabla H \cdot ,\textrm{d}x_t + \frac{\beta }{2}\Delta H ,\big [\textrm{d}x_t\big ]^2 + \beta \partial _{t} H, \textrm{d}t\&\qquad \qquad \quad + \frac{\textrm{d}\psi }{\psi } - \frac{[\textrm{d} \psi ]^2}{2 \psi ^2}\&= \beta (\nabla H -f) \cdot \textrm{d}x_t + \frac{1}{\gamma } (\Delta H - \nabla \cdot f),\textrm{d}t + \frac{\textrm{d}\psi }{\psi } - \frac{[\textrm{d} \psi ]^2}{2 \psi ^2}. \end{aligned} \end{aligned}$$

(19)

Our goal is to express (\textrm{d} \theta _t) in terms of (\textrm{d} B_t) and (\textrm{d} t). To achieve this, we need to evaluate (\textrm{d}\psi ) as (\textrm{d}\psi (x_t,t)). Using the chain rule and Eq. (16), we find

$$\begin{aligned} \begin{aligned} \textrm{d} \psi&= \partial _t \psi ,\textrm{d}t + \nabla \psi \cdot \textrm{d}x_t + \frac{1}{2} \Delta \psi , [\textrm{d}x_t]^2\ ,&= -\frac{1}{\gamma } \nabla \cdot \big [(\nabla H - f) \psi \big ] \textrm{d}t -\frac{1}{\beta \gamma } \Delta \psi ,\textrm{d}t + \nabla \psi \cdot \textrm{d}x_t + \frac{1}{2} \Delta \psi [\textrm{d}x_t]^2\ ,&= - \frac{\psi }{\gamma } (\Delta H - \nabla \cdot f) , \textrm{d} t - \frac{1}{\gamma }\big [(\nabla H - f) \cdot \nabla \psi \big ] \textrm{d}t + \nabla \psi \cdot \textrm{d}x_t \end{aligned} \end{aligned}$$

(20)

Consequently, (\tfrac{\textrm{d}\psi }{\psi }) and (\tfrac{1}{2 \psi ^2}[\textrm{d} \psi ]^2) are given by

$$\begin{aligned} \begin{aligned} \frac{1}{\psi }, \textrm{d} \psi&= - \frac{1}{\gamma } (\Delta H - \nabla \cdot f) ,\textrm{d} t - \frac{1}{\gamma } \big [(\nabla H - f) \cdot \nabla \ln \psi \big ] \textrm{d}t + \nabla \ln \psi \cdot \textrm{d}x_t \ \frac{1}{2 \psi ^2} [\textrm{d} \psi ]^2&= \frac{1}{\gamma \beta }\frac{\Vert \nabla \psi \Vert ^2}{\psi ^{2}}\textrm{d}t. \end{aligned} \end{aligned}$$

(21)

Substituting Eqs. (21) into Eq. (19), we find

$$\begin{aligned} \begin{aligned} \textrm{d} \theta _t&= - \frac{1}{\gamma } (\nabla \ln \psi ) \cdot (\nabla H -f) ,\textrm{d} t - \frac{1}{\gamma \beta } \big \Vert \nabla \ln \psi \big \Vert ^2 ,\textrm{d}t + \big ( \beta \nabla H - \beta f + \nabla \ln \psi \big ) \cdot \textrm{d}x_t\ ,&=- \frac{1}{\gamma \beta } \big \Vert \beta (\nabla H -f) + \nabla \ln \psi \big \Vert ^2 ,\textrm{d}t + \sqrt{\frac{2}{\gamma \beta }},\Big [\beta (\nabla H -f) + \nabla \ln \psi \Big ] \cdot \textrm{d}B_t. \end{aligned} \end{aligned}$$

(22)

The exponential of (\theta _t) can now be expressed as

$$\begin{aligned} \begin{aligned} \textrm{d} e^{\theta _t}&= e^{\theta _t} \left[ \textrm{d} \theta _t + \frac{1}{2} (\textrm{d}\theta _t)^{2 \right] \ ,&= e}{\theta _t} \sqrt{\frac{2}{\gamma \beta }}, \Big [\beta (\nabla H -f) + \nabla \ln \psi \Big ] \cdot \textrm{d}B_t. \end{aligned} \end{aligned}$$

(23)

Thus, as long as the Novikov condition holds, the process (e^{\theta _t}) is a martingale with initial value one.

By the optional stopping theorem for martingales, we have

$$\begin{aligned} \mathbb {E} \left[ e^{-\beta (W_{\tau } - \Delta H_{\tau } + T \Delta \Sigma _{\tau })}, \Big |, x_{0}\right] = \mathbb {E} \left[ e^{\theta _{\tau }} \mid x_0\right] = e^{\theta _{0}} = 1, _{\text {a.s.},} \forall _{\text {bounded stopping time}}\tau . \end{aligned}$$

(24)

Eq. (24) is the main result of this paper; note that Eq. (24) is stronger than (\mathbb {E}\left[ e^{\theta _\tau }\right] = 1) since Eq. (24) is an average over trajectories starting from an arbitrary initial value (x_0), independent of the initial distribution (\rho _0(x)), while the latter is an average over all trajectories starting from the initial distribution (\rho _0(x)), shown by the red and grey trajectories in Fig. 2(b), respectively. Pigolotti et al. [26] first studied a special case, the nonequilibrium stationary system, of Eq. (23) and then Neri et al. proposed the stationary version of Eq. (24) in [27].

Yang and Ge [12] generalized the fluctuation relation using the path integral method by introducing another stochastic process (y_t) mutually absolutely continuous with the process of interest (x_t) and a third process (z_t) driven by the time-reversed protocol of (y_t) with an arbitrary initial condition. The logarithm of the ratio of the forward path probability under x to the backward path probability under z plays a similar role to the free energy change (\Delta F). When this functional is compensated by a proper log probability ratio between distributions of (z_0) and (z’_{t_{1}-t}) (where (t_{1}) is a chosen fixed time point) and exponentiated, a martingale is formally constructed. Here, (z’_t) is another process derived in the same way as (z_t) but may have a different initial condition.

In terms of overdamped dynamics, our result coincides with that in [12], if we restrict our generalized backward process to the same fixed finite interval ([0,t_{1}]) and restrict (z_t) and (z’_t) to be identically distributed as our generalized backward process (\psi _{t_{1}-t}). However, there are two important differences. First, while [12] enjoys more degrees of freedom in choosing the auxiliary processes (z_t), their construction still requires a fixed time interval to be chosen in advance, as this interval is intrinsic to the definition of their functional and compensation. By contrast, our construction does not rely on this fixed time interval and can be extended to the whole time axis. Second, the path integral or path probability method employed in prior work formally only requires that (x_t) and (y_t) to be mutually absolutely continuous. This is satisfied automatically if (y_t) is chosen to be (x_t) or effectively (z_t) is chosen to be the generalized backward process, as in our case. Our Itô calculus approach requires an additional regularity condition to be satisfied in order for the local martingale to be a martingale. In other words, predictions made by the path integral methods may fail for some unforeseeable singular cases.

Manzano’s result. We now show that Eq. (24) is a generalization of Eq. (8). The key observation is that in the definition of the backward process (\psi (x,t)), we have the freedom to choose the initial condition. For different choices of the initial condition, we will arrive at different forms of the martingale. Specifically, in the setting of [9], the backward process is chosen by fixing a final time (t_1), and choosing the “initial condition” of the backward process to be

$$\begin{aligned} \psi (x,t_1) = \rho (x,t_1),,, \forall x. \end{aligned}$$

(25)

Then, (\psi (x,t)) evolves backward in time according to Eq. (16).

Consider a stopping time (\tau ) such that (0 \le \tau \le t_1) almost surely. Note that (-\beta T \Delta \Sigma (\tau ) = \ln \psi (x_\tau ,\tau ) - \ln \psi (x_0,0) = \ln \tilde{\rho }(x_\tau ,t_1 - \tau ) - \ln \tilde{\rho }(x_0,t_1)), while (-\beta T \Delta S = \ln \rho (x_\tau ,\tau ) - \ln \rho (x_0,0)). Recalling the definition of (\delta _{\tau }) in Eq. (9), we find

$$\begin{aligned} \mathbb {E}\Big [e^{- \beta \big [W_{\tau } - \Delta F_{\tau }\big ]- \delta _\tau } \Big ] = \mathbb {E} \Big [ e^{\theta _{\tau }} \frac{\tilde{\rho }(x_0,t_1)}{\rho (x_0,0)}\Big ] = \int \tilde{\rho }(x_0,t_1) \mathbb {E} \left[ e^{\theta _t} \mid x_0 \right] \textrm{d} x_0 = 1, \end{aligned}$$

(26)

which is Eq. (8). Here, the second equality follows from conditioning on (x_0) and the last equality follows from Eq. (24). Additionally, by conditioning on (x_0), we have

$$\begin{aligned} \mathbb {E}\Big [e^{- \beta \big [W_{\tau } - \Delta F_{\tau }\big ]- \delta _\tau } \Big |~ x_0\Big ] = \mathbb {E} \bigg [ e^{\theta _t} \frac{\tilde{\rho }(x_0,t_1)}{\rho (x_0,0)}\bigg | ~x_0 \bigg ] = \frac{\tilde{\rho }(x_0,t_1)}{\rho (x_0,0)}, \quad \forall x_0. \end{aligned}$$

(27)

Eq. (27) is stronger than Eq. (8) since Eq. (8) only holds for all trajectories sampled according to the initial distribution (\rho _0(x)), while our result holds for trajectories starting from an arbitrary initial value (x_0), independent of the initial distribution (\rho _0(x)), as shown in Fig. 2(b).

This result is particularly helpful when one wants to consider the work theorem for a system where the initial distribution cannot be written as a density function over the state space, such as the Dirac delta distribution. In such cases, the trajectorywise entropy at the initial time (S(0) = - k_\textrm{B} \ln \rho (x_0,0)) is not well-defined. Our result provides a way to bypass this issue by decoupling the initial sample (x_0) from its initial distribution (\rho _0). Energy changes and external work can be evaluated by the conditional work theorem Eq. (27) for trajectories starting at (x_0) with an arbitrarily chosen well-behaved initial distribution (\rho _0).

Stationary Hamiltonian. There are other interesting choices for the backward process. For example, in the case of time-independent potentials and forces, we can choose (\psi (x,0) = \rho _\textrm{ss}(x)), the stationary distribution of Eq. (13). In this case, (\psi (x,t) \equiv \psi (x,0) = \rho _\textrm{ss}(x)) and

$$\begin{aligned} \mathbb {E} \Big [e^{-\beta \big (W_{\tau } - \Delta H_{\tau } + T \Delta S_\textrm{ss}(\tau ) \big )}\Big ] = 1, ,, \forall _{\text {bounded stopping time}}\tau , \end{aligned}$$

(28)

where (S_\textrm{ss}(t) :=-k_\textrm{B}\ln \rho _\textrm{ss}(x_t)) and the initial distribution (\rho _0(x)) of (x_0) can be different from (\rho _\textrm{ss}(x)).

The stationary case is of interest when f is a dissipative force. Such a system can be used to model nonequilibrium stochastic chemical reactions commonly found in biological systems, including kinetic proofreading [28, 29] and chemotaxis [30]. While evaluating Eq. (8) requires solution to the (d+1)-dimensional time-dependent PDE for (\rho (x,t)), computing Eq. (28) requires only the solution to the d-dimensional time-independent PDE for (\rho _\textrm{ss}(x)), leading to an easier computational evaluation. A similar identity was derived in [23] assuming that the initial distribution (\rho _0(x)) is the stationary (backward) distribution (\rho _\textrm{ss}(x)). Our result relaxes this assumption and shows that the identity holds for any initial distribution (\rho _0(x)).

Fluctuation-dissipation relation. In our formulation of the Langevin dynamics in Eq. ([12](https://link.springer.