Explicit solution to forward and backward stochastic differential equations with state delay and its application to optimal control

Abstract

This paper is concerned with linear forward–backward stochastic differential equations (FBSDEs) with state delay, the solvability which is much more complex than the case of no delay or input delay caused by the prediction of the backward processes of the future time. To overcome this difficulty, we innovatively establish the non-homogeneous relationship between the backward and forward processes with the help of the corresponding discrete-time system. The main contribution is to give the explicit solution to the FBSDEs with state delay in terms of partial Riccati equations for the first time. The presented results form the basis to solve the challenging problem of linear quadratic optimal control for multiplicative-noise stochastic systems with state delay.

Appendices

Appendices

The proof of Theorem 1 is divided into several steps. First, we consider the corresponding discrete-time FBSDEs to derive light on the the form of the solution which is shown in “Appendix A”. Then we obtain the continuous-time solution by taking the limit to the discrete-time solution which is presented in “Appendix B”. Finally, a strict proof of Theorem 1 is given by using the continuous-time technique which is presented in “Appendix C”.

Appendix A: Corresponding discrete-time FBSDEs

Noticing that the form of the solution is hard to assume in advance, and the structure of the solution is abstract, so the discrete-time form will be assisted to comprehend the form of the explicit solution to FBSDEs (1), the details are given below.

At the beginning, some definitions will be given. First, following the procedure in [21], we introduce the following partition $0=t_0<t_1<\cdots <t_{N+1}=T$, the length of each subinterval is defined as $\delta$, then let $t_{k+1}=(k+1)\delta$, $t_k=k\delta$, $h=\delta d$. Second, defining that $\Delta w(k)=w(k+1)-w(k)$ where $\Delta w(k)$ is a white noise with zero mean and covariance $t_{k+1}-t_k$. $\mathcal {\hat{F}}(k)$ is the natural filtration generated by $\Delta w(k)$, i.e. $\mathcal {\hat{F}}(k)=\sigma \{\Delta w(0),\cdots ,\Delta w(k)\}$.

Next defining $\hat{A}_0=I+\delta A_0$, $\hat{A}_1=\delta A_1$, $\hat{B}_0=\delta B_0$, $\hat{B}_1=\delta B_1$, $\hat{D}_0=I+\delta D_0$, $\hat{D}_1=\delta D_1$, $\hat{P}(N+1)=P(T)$, $\hat{Q}=\delta Q$. In order to simplify the symbols, we denote the variables at time $t_k$ as x(k), p(k) and q(k). On the basis, we further define that

$$\begin{aligned}&A_0(k)=\hat{A}_0+\Delta w(k)\bar{A}_0,\ A_1(k)=\hat{A}_1+\Delta w(k)\bar{A}_1,\\&B_0(k)=\hat{B}_0+\Delta w(k)\bar{B}_0,\ B_1(k)=\hat{B}_1+\Delta w(k)\bar{B}_1,\\&D_1(k+d)=\hat{D}_1+\Delta w(k+d)\bar{D}_1. \end{aligned}$$

Then under the above definitions, the discretized FBSDEs for (1) will be given.

Lemma 1

The discretized FBSDEs for (1) are given as follows:

$$\begin{aligned}&x(k+1) \nonumber \\&=A_0(k)x(k)+A_1(k)x(k-d) \nonumber \\&\quad +B_0(k)\mathrm{E}[p(k)\vert \mathcal {\hat{F}}(k-1)]\nonumber \\&\quad +\frac{1}{\delta }B_1(k)\mathrm{E}\big [\Delta w(k)p(k)\vert \mathcal {\hat{F}}(k-1)\big ], \end{aligned}$$

(24)

$$\begin{aligned}&p(k-1) \nonumber \\&=\mathrm{E}\big [A'_0(k)p(k)+A'_1(k+d)p(k+d)\vert \mathcal {\hat{F}}(k-1)\big ] \nonumber \\&\quad +\hat{Q}x_k, \end{aligned}$$

(25)

$$\begin{aligned}&p(N)=P(N+1)x(N+1). \end{aligned}$$

(26)

Proof

First, we will obtain the discrete-time form of the BSDE. Following the step introduced in [22], we can discretize the BSDE as follows:

$$\begin{aligned}&p(k)-p(k-1) \nonumber \\&\quad =-\delta \big \{A'_0p(k)+\bar{A}'_0q(k)+A'_1\mathrm{E}[p(k+d)\vert \mathcal {\hat{F}}(k-1)]\nonumber \\&\quad \quad +\bar{A}'_1\mathrm{E}[q(k+d)\vert \mathcal {\hat{F}}(k-1)]+Qx(k)\big \}\nonumber \\&\quad \quad +q(k)\Delta w(k), \end{aligned}$$

(27)

by multiplying $\Delta w_k$ on both sides, and taking the conditional expectation with respect to $\mathcal {F}(k-1)$ on both sides, we have

$$\begin{aligned} \begin{aligned}&\mathrm{E}[\Delta w(k)p(k)\vert \mathcal {\hat{F}}(k-1)]\\&=-\mathrm{E}\Big \{\delta \Delta w(k) \big \{A'_0p(k)+\bar{A}'_0q(k)\\&\quad +A'_1\mathrm{E}[p(k+d)\vert \mathcal {\hat{F}}(k-1)]\\&\quad +\bar{A}'_1\mathrm{E}[q(k+d)\vert \mathcal {\hat{F}}(k-1)]\\&\quad +Qx(k)\big \}\vert \mathcal {\hat{F}}(k-1)\Big \}+\delta q(k). \end{aligned} \end{aligned}$$

Therefore, we have $q(k)=\frac{1}{\delta }\mathrm{E}[\Delta w(k)p(k)\vert \mathcal {\hat{F}}(k-1)]$. This numerical scheme is consistent with the Euler scheme used in the literature for the solvability of FBSDE in [21]. Similarly we have $q(k+d)=\frac{1}{\delta }\mathrm{E}[\Delta w(k+d)p(k+d)\vert \mathcal {\hat{F}}(k+d-1)]$, Inserting q(k) and $q(k+d)$ in (27), and taking the conditional expectation with respect to $\mathcal {\hat{F}}(k-1)$ on both sides, we have the discretization of the BSDE given by

$$\begin{aligned} \begin{aligned}&p(k-1)\\&=\mathrm{E}\big \{[I+\delta A'_0+\Delta w(k)\bar{A}'_0]p(k)\\&\quad +[\delta A'_1+\Delta w(k+d) \bar{A}'_1]p(k+d)\vert \mathcal {\hat{F}}(k-1)\big \}\\&\quad +\delta Q x(k), \end{aligned} \end{aligned}$$

this is exactly (25). Therefore, we have the discrete-time BSDE. The procedure for getting the forward stochastic differential equation is same so the details are omitted here. This completes the proof. $\square$

The second step is to obtain the explicit solution of the above discrete-time FBSDEs (24)–(26) which lead us to comprehend the form of the solution to continuous-time FBSDEs, and it will be shown as below.

Lemma 2

FBSDEs (24)–(26) are uniquely solvable if the Riccati equations (31)–(33) and (38)–(40) admit a solution such that the matrix $\varGamma (k)$ is invertible. The solution for the discrete-time FBSDEs are given as follows where $s_k=\min \{d-1,N-k\}$:

$$\begin{aligned}&\begin{aligned} x(k+1)&=M(k)x(k)+M_k(k)x(k-d)\\&\quad +\textstyle \sum \limits _{i=1}^{s_k}M_{k+i}(k)x(k-d+i), \end{aligned} \end{aligned}$$

(28)

$$\begin{aligned}&p(k-1)=\hat{P}(k)x(k)+\textstyle \sum \limits _{i=0}^{s_k} \hat{P}_{k+i}(k)x(k-d+i), \end{aligned}$$

(29)

$$\begin{aligned}&p(N)=\hat{P}(N+1)x(N+1), \end{aligned}$$

(30)

which for $k>N-d$, $\hat{P}(k)$ and $\hat{P}_{k+i}(k)$ satisfy

$$\begin{aligned}&\hat{P}(k)=\mathrm{E}[A'_0(k)\hat{P}(k+1)M(k)]+\hat{Q}, \end{aligned}$$

(31)

$$\begin{aligned}&\hat{P}_{k}(k)=\mathrm{E}[A'_0(k)\hat{P}(k+1)M_k(k)], \end{aligned}$$

(32)

$$\begin{aligned}&\hat{P}_{k+i}(k) \nonumber \\&=\mathrm{E}[A'_0(k)\hat{P}(k+1)M_{k+i}(k)+D'_0(k)\hat{P}_{k+i}(k+1)], \end{aligned}$$

(33)

$$\begin{aligned}&M(k) \nonumber \\&=A_0(k)+\left[ \begin{array}{c@{~~}c} B_0(k)\hat{P}(k+1) &{} \dfrac{1}{\delta }B_1(k)\hat{P}(k+1) \\ \end{array} \right] \nonumber \\&\quad \times \varGamma ^{-1}(k+1) \left[ \begin{array}{c} \hat{A}_0 \\ \delta \bar{A}_0 \\ \end{array} \right] , \end{aligned}$$

(34)

$$\begin{aligned}&M_{k}(k) \nonumber \\&=A_1(k)+\left[ \begin{array}{c@{~~}c} B_0(k)\hat{P}(k+1) &{} \dfrac{1}{\delta }B_1(k)\hat{P}(k+1) \\ \end{array} \right] \nonumber \\&\times \varGamma ^{-1}(k+1) \left[ \begin{array}{c} \hat{A}_1 \\ \delta \bar{A}_1 \\ \end{array} \right] , \end{aligned}$$

(35)

$$\begin{aligned}&M_{k+i}(k) \nonumber \\&=B_0(k)\hat{P}_{k+i}(k+1) \nonumber \\&\quad +\left[ \begin{array}{c@{~~}c} B_0(k)\hat{P}(k+1) &{} \dfrac{1}{\delta }B_1(k)\hat{P}(k+1) \\ \end{array} \right] \nonumber \\&\quad \times \varGamma ^{-1}(k+1) \left[ \begin{array}{c} \hat{B}_0\hat{P}_{k+i}(k+1)\\ \delta \bar{B}_0\hat{P}_{k+i}(k+1) \\ \end{array} \right] , \end{aligned}$$

(36)

$$\begin{aligned}&\varGamma (k)= \left[ \begin{array}{c@{~~}c} I-\hat{B}_0\hat{P}(k) &{} -\dfrac{1}{\delta }\hat{B}_1\hat{P}(k) \\ -\delta \bar{B}_0\hat{P}(k) &{} I-\bar{B}_1\hat{P}(k) \\ \end{array} \right] , \end{aligned}$$

(37)

and for $0\le k\le N-d$, $\hat{P}(k)$ and $\hat{P}_{k+i}(k)$ satisfy:

$$\begin{aligned}&\hat{P}(k)\nonumber \\&=\mathrm{E}[A'_0(k)\hat{P}(k+1)M(k)+A'_0(k)\hat{P}_{k+d}(k+1) \nonumber \\&\quad +\hat{P}'_{k+d}(k+1)M(k) \nonumber \\&\quad +A'_1(k)\hat{P}(k+d+1)M_{k+d}(k+d)\nonumber \\&\quad +\textstyle \sum \limits _{i=0}^{d-1}P'_{k+d}(k+d-i)M_{k+d}(k+d-i-1)]+\hat{Q}, \end{aligned}$$

(38)

$$\begin{aligned}&\hat{P}_{k}(k) \nonumber \\&=\mathrm{E}[A'_0(k)\hat{P}(k+1)M_k(k)+\hat{P}'_{k+d}(k+1)M_k(k)], \end{aligned}$$

(39)

$$\begin{aligned}&\hat{P}_{k+i}(k) \nonumber \\&=\mathrm{E}[D'_0(k)\hat{P}(k+1)M_{k+i}(k)+D'_0(k)\hat{P}_{k+i}(k+1)\nonumber \\&\quad +\textstyle \sum \limits _{j=0}^{i}\hat{P}'_{k+d}(k+j)M_{k+i}(k+j-1)], \end{aligned}$$

(40)

$$\begin{aligned}&M(k) \nonumber \\&=A_0(k)+B_0(k)P_{k+d}(k+1) \nonumber \\&\quad +\left[ \begin{array}{c@{~~}c} B_0(k)\hat{P}(k+1) &{} \dfrac{1}{\delta }B_1(k)\hat{P}(k+1) \\ \end{array} \right] \nonumber \\&\quad \times \varGamma ^{-1}(k+1) \left[ \begin{array}{c} \hat{A}_0 \\ \delta \bar{A}_0 \\ \end{array} \right] , \end{aligned}$$

(41)

with the terminal value given by $\hat{P}(N+1)$. $\hat{P}(N+1+i)=0,\ \hat{P}_{j}(N+1)=0$ for any $i,j>0$, $\hat{P}_{s}(k)=0$ if $s>N$. In addition, when $0\le k\le N-d$, $M_k(k),M_{k+i}(k)$ and $\varGamma (k)$ in (38)–(41) have the same form as in (31)–(33) which given by (35)–(37).

Proof

We will prove by using the induction method. For $k=N$, using the terminal condition (26) and the FSDE (24), it yields that

$$\begin{aligned}&x(N+1)\nonumber \\&\quad =A_0(N)x(N)+A_1(N)x(N-d) \nonumber \\&\quad \quad +B_0(N)\mathrm{E}[p(N)\vert \mathcal {\hat{F}}(N-1)] \nonumber \\&\quad \quad +\frac{1}{\delta }B_1(N)\mathrm{E}[\Delta w(N)p(N)\vert \mathcal {\hat{F}}(N-1)]\nonumber \\&\quad =A_0(N)x(N)+A_1(N)x(N-d)\nonumber \\&\quad \quad +B_0(N)\hat{P}(N+1)\mathrm{E}[x(N+1)\vert \mathcal {\hat{F}}(N-1)]\nonumber \\&\quad \quad +\frac{1}{\delta }B_1(N)\hat{P}(N+1)\mathrm{E}[\Delta w(N)x(N+1)\vert \mathcal {\hat{F}}(N-1)]. \end{aligned}$$

(42)

Taking the conditional expectation with respect to $\mathcal {\hat{F}}(N-1)$ on both sides of (42), we have

$$\begin{aligned} \begin{aligned}&[I-\hat{B}_0\hat{P}(N+1)]\mathrm{E}[x(N+1)\vert \mathcal {\hat{F}}(N-1)]\\&=\hat{A}_0x(N)+\hat{A}_1x(N-d)\\&\quad +\frac{1}{\delta }\hat{B}_1\hat{P}(N+1)\mathrm{E}[\Delta w(N)x(N+1)\vert \mathcal {\hat{F}}(N-1)]. \end{aligned} \end{aligned}$$

(43)

Then by multiplying $\Delta w(N)$ on both sides of (42) and taking the conditional expectation with respect to $\mathcal {\hat{F}}(N-1)$, we can obtain the following relationship:

$$\begin{aligned} \begin{aligned}&[I-\bar{B}_1\hat{P}(N+1)]\mathrm{E}[\Delta w(N)x(N+1)\vert \mathcal {\hat{F}}(N-1)]\\&=\delta \bar{A}_0(N)x(N)+\delta \bar{A}_1x(N-d)\\&\quad +\delta \bar{B}_0\hat{P}(N+1)\mathrm{E}[x(N+1)\vert \mathcal {\hat{F}}(N-1)].\\ \end{aligned} \end{aligned}$$

(44)

Then combining (43) and (44), we have

$$\begin{aligned} \begin{aligned}&\left[ \begin{array}{c@{~~}c} I-\hat{B}_0\hat{P}(N+1) &{} -\dfrac{1}{\delta }\hat{B}_1\hat{P}(N+1)\\ -\delta \bar{B}_0\hat{P}(N+1) &{} I-\bar{B}_1\hat{P}(N+1) \\ \end{array} \right] \\&\times \left[ \begin{array}{c} \mathrm{E}[x(N+1)\vert \mathcal {\hat{F}}(N-1)] \\ \mathrm{E}[\Delta w(N)x(N+1)\vert \mathcal {\hat{F}}(N-1)] \\ \end{array} \right] \\&=\left[ \begin{array}{c} \hat{A}_0 \\ \delta \bar{A}_0 \\ \end{array} \right] x(N) +\left[ \begin{array}{c} \hat{A}_1 \\ \delta \bar{A}_1 \\ \end{array} \right] x(N-d). \end{aligned} \end{aligned}$$

This gives that

$$\begin{aligned} \begin{aligned}&\left[ \begin{array}{c} \mathrm{E}[x(N+1)\vert \mathcal {\hat{F}}(N-1)] \\ \mathrm{E}[\Delta w(N)x(N+1)\vert \mathcal {\hat{F}}(N-1)] \\ \end{array} \right] \\&=\varGamma ^{-1}(N+1) \left[ \begin{array}{c} \hat{A}_0 \\ \delta \bar{A}_0 \\ \end{array} \right] x(N)\\&\quad +\varGamma ^{-1}(N+1) \left[ \begin{array}{c} \hat{A}_1 \\ \delta \bar{A}_1 \\ \end{array} \right] x(N-d), \end{aligned} \end{aligned}$$

(45)

where

$$\begin{aligned} \begin{aligned} \varGamma (N+1)=\left[ \begin{array}{c@{~~}c} I-\hat{B}_0\hat{P}(N+1) &{} -\dfrac{1}{\delta }\hat{B}_1\hat{P}(N+1)\\ -\delta \bar{B}_0\hat{P}(N+1) &{} I-\bar{B}_1\hat{P}(N+1) \\ \end{array} \right] . \end{aligned} \end{aligned}$$

Introducing (45) into (42), we have

$$\begin{aligned} \begin{aligned}&x(N+1)\\&=A_0(N)x(N)+A_1(N)x(N-d)\\&\quad +\left[ \begin{array}{c@{~~}c} B_0(N)\hat{P}(N+1) &{} \dfrac{1}{\delta }B_1(N)\hat{P}(N+1)\\ \end{array} \right] \\&\quad \times \varGamma ^{-1}(N+1) \left[ \begin{array}{c} \hat{A}_0 \\ \delta \bar{A}_0 \\ \end{array} \right] x(N)\\&\quad +\left[ \begin{array}{c@{~~}c} B_0(N)\hat{P}(N+1) &{} \dfrac{1}{\delta }B_1(N)\hat{P}(N+1)\\ \end{array} \right] \\&\quad \times \varGamma ^{-1}(N+1) \left[ \begin{array}{c} \hat{A}_1 \\ \delta \bar{A}_1 \\ \end{array} \right] x(N-d),\\ \end{aligned} \end{aligned}$$

thus we have

$$\begin{aligned} x(N+1)\doteq&M(N)x(N)+M_N(N)x(N-d), \end{aligned}$$

(46)

where

$$\begin{aligned} M(N)&=A_0(N) \\&\quad +\left[ \begin{array}{c@{~~}c} B_0(N)\hat{P}(N+1) &{} \dfrac{1}{\delta }B_1(N)\hat{P}(N+1)\\ \end{array} \right] \\&\quad \times \varGamma ^{-1}(N+1) \left[ \begin{array}{c} \hat{A}_0 \\ \delta \bar{A}_0 \\ \end{array} \right] ,\\ M_N(N)&=A_1(N)\\&\quad +\left[ \begin{array}{c@{~~}c} B_0(N)\hat{P}(N+1) &{} \dfrac{1}{\delta }B_1(N)\hat{P}(N+1)\\ \end{array} \right] \\&\quad \times \varGamma ^{-1}(N+1) \left[ \begin{array}{c} \hat{A}_1 \\ \delta \bar{A}_1 \\ \end{array} \right] . \end{aligned}$$

Therefore, we proved that (28) is true for $k=N$, Next we will consider BSDE (29) is true for $k=N$. Combining the terminal condition (26) and (46) and inserting to (25), we have the following relationship:

$$\begin{aligned} \begin{aligned}&p(N-1)\\&\quad =\mathrm{E}[A'_0(N)p(N)\vert \mathcal {\hat{F}}(N-1)]+\hat{Q}x(N)\\&\quad =\mathrm{E}\Big \{A'_0(N)\hat{P}(N+1)[M(N)x(N)\\&\quad \quad +M_N(N)x(N-d)]\vert \mathcal {\hat{F}}(N-1)\Big \}+\hat{Q}x(N)\\&\quad =\Big \{\mathrm{E}[A'_0(N)\hat{P}(N+1)M(N)]+\hat{Q}\Big \}x(N)\\&\quad \quad +E\big [A'_0(N)\hat{P}(N+1)M_N(N)\big ]x(N-d)\\&\quad = \hat{P}(N)x(N)+\hat{P}_N(N)x(N-d), \end{aligned} \end{aligned}$$

where

$$\begin{aligned} \begin{aligned}&\hat{P}(N)=\mathrm{E}[A'_0(N)P(N+1)M(N)]+\hat{Q},\\&\hat{P}_N(N)=\mathrm{E}[A'_0(N)P(N+1)M_N(N)]. \end{aligned} \end{aligned}$$

Thus we have proved that (28) and (29) are true for $k=N$. Then taking any s with $N-d<s<N$, and assuming x(k) and $p(k-1)$ are as (28) and (29) for all $k\ge s+1$. Next we will show that these conditions are also hold for the situation of $k=s$. From (25) at s, (29) with $k=s+1$, and (28) with $k=s+1$, one has

$$\begin{aligned}&p(s-1)\\&\quad =\Big \{\mathrm{E}[A'_0(s)P(s+1)M(s)]+\hat{Q}\Big \}x(s)\\&\quad \quad +\mathrm{E}[A'_0(s)P(s+1)M_s(s)]x(s-d)\\&\quad \quad +\textstyle \sum \limits _{i=1}^{N-s}\Big \{\mathrm{E}[A'_0(s)\hat{P}(s+1)M_{s+i}(s)\\&\quad \quad +A'_0(s)\hat{P}_{s+i}(s+1)]\Big \}x(s-d+i)\\&\quad =\hat{P}(s)x(s)+\textstyle \sum \limits _{i=0}^{N-s}\hat{P}_{s+i}(s)x(s-d+i). \end{aligned}$$

Similarly, from (24) at s, (29) with $k=s+1$, we have

$$\begin{aligned}&x(s+1)\\&\quad =A_0(s)x(s)+A_1(s)x(s-d)\\&\quad \quad +B_0(s)\mathrm{E}\Big \{[\hat{P}(s+1)x(s+1)\\&\quad \quad +\textstyle \sum \limits _{i=0}^{N-s-1}\hat{P}_{s+i+1}(s)x(s-d+i+1)]\vert \mathcal {\hat{F}}(s-1)\Big \}\\&\quad \quad +\frac{1}{\delta }B_1(s)\mathrm{E}\Big \{\Delta w(s)[\hat{P}(s+1)x(s+1)\\&\quad \quad +\textstyle \sum \limits _{i=0}^{N-s-1}\hat{P}_{s+i+1}(s)x(s-d+i+1)]\vert \mathcal {\hat{F}}(s-1)\Big \}. \end{aligned}$$

Using the similar derivation we have the following relationship kept.

$$\begin{aligned} \begin{aligned}&\left[ \begin{array}{c@{~~}c} I-\hat{B}_0\hat{P}(s+1) &{} -\dfrac{1}{\delta }\hat{B}_1\hat{P}(s+1) \\ -\delta \bar{B}_0\hat{P}(s+1) &{} I-\bar{B}_1\hat{P}(s+1) \\ \end{array} \right] \\&\times \left[ \begin{array}{c} \text{ E }(x(s+1)\vert \mathcal {\hat{F}}(s-1)) \\ \text{ E }(\Delta w(s)x(s+1)\vert \mathcal {\hat{F}}(s-1)) \\ \end{array} \right] \\&=\left[ \begin{array}{c} \hat{A}_0 \\ \delta \bar{A}_0 \\ \end{array} \right] x(s) +\left[ \begin{array}{c} \hat{A}_1 \\ \delta \bar{A}_1 \\ \end{array} \right] x(s-d) \\&\quad + \textstyle \sum \limits _{i=1}^{N-s} \left[ \begin{array}{c} \hat{B}_0\hat{P}_{s+i}(s+1)\\ \delta \bar{B}_0\hat{P}_{s+i}(s+1) \\ \end{array} \right] x(s-d+i), \end{aligned} \end{aligned}$$

which gives that

$$\begin{aligned} x(s+1)&=M(s)x(s)+M_s(s)x(s-d)\\&\quad +\textstyle \sum \limits _{i=1}^{N-s}M_{s+i}(s)x(s-d+i), \end{aligned}$$

where the coefficients are given by (34)–(37). Hence, (31)–(37) are true for $k=s$, the proof for the case of $k>N-d$ is finished. The proof of the situation of $0\le k\le N-d$ is same so the details are omitted here, thus the proof is completed by the inductive approach. $\square$

Therefore, the discrete-time FBSDEs (24)–(26) were obtained, and the explicit solution to this discrete-time FBSDEs was also presented by Riccati equations (31)–(33) and (38)–(40), the next step is to obtain the explicit solution to (1) by taking limit to the discrete-time solution where the procedure is given in “Appendix B”.

Appendix B: Limitation of the discrete-time FBSDEs

We will illustrate how to obtain the continuous-time form. The first step is to obtain (2). Defining the limitation of $\hat{P}(k)$ and $\hat{P}_{k+i}(k)$ as P(t) and $P(t,t+\theta )$. Considering the discrete-time form (29) where $s=\min \{d-1,N-k\}$:

$$\begin{aligned} p(k-1)&=\hat{P}(k)x(k)+ \textstyle \sum \limits _{i=0}^{s}\hat{P}_{k+i}(k)x(k-d+i), \end{aligned}$$

by taking the limit on both sides we have

$$\begin{aligned} p(t)&=P(t)x(t)+\int _{t}^{\min \{t+h,T\}}P(t,\theta )x(\theta -h)\mathrm{d}\theta , \end{aligned}$$

which is (2). The proof of the corresponding Riccati equations will be given later.

Next we will verify (3), using the definition (29) and $q(k)=\frac{1}{\delta }\mathrm{E}[\Delta w(k)p(k)\vert \mathcal {\hat{F}}(k-1)]$. Then inserting (28), we have

$$\begin{aligned} q(k)&=[I-\bar{B}_1\hat{P}(k+1)]^{-1}\hat{P}(k+1)[\bar{A}_0x(k)\\&\quad +\bar{A}_1x(k-d)+\bar{B}_0p(k)], \end{aligned}$$

Then taking the limit on both sides, we can obtain the continuous-time form, which is (3).

Next, we will show the process of getting (10). Inserting $\hat{B}_0=\delta B_0,\hat{B}_1=\delta B_1$ in $\varGamma (k+1)$, through the calculation, we can obtain the reverse of $\varGamma (k+1)$ as follows:

$$\begin{aligned} \begin{aligned}&\varGamma ^{-1}(k+1)\\&= \left[ \begin{array}{c@{~~}c} I-\delta B_0\hat{P}(k+1) &{} -B_1\hat{P}(k+1) \\ -\delta \bar{B}_0\hat{P}(k+1) &{} I-\bar{B}_1\hat{P}(k+1) \\ \end{array} \right] ^{-1}\\&= \left[ \begin{array}{c@{~~}c} M_{1} &{} M_{2} \\ M_{3} &{} M_{4} \\ \end{array} \right] , \end{aligned} \end{aligned}$$

where

$$\begin{aligned} \begin{aligned} M_1&=[I-\delta B_0\hat{P}(k+1)-\delta B_1\hat{P}(k+1)\\&\quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1),\\ M_2&=M_1 B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1},\\ M_3&=\delta (I-\bar{B}_1\hat{P}(k+1))^{-1}\bar{B}_0\hat{P}(k+1)M_1,\\ M_4&=(I-\bar{B}_1\hat{P}(k+1))^{-1}+\delta [I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \times \bar{B}_0\hat{P}(k+1)M_1 B_1\hat{P}(k+1)\\&\quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}.\\ \end{aligned} \end{aligned}$$

Next, by replacing $\hat{A}_0=I+\delta A_0,\hat{A}_1=\delta A_1,\hat{B}_0=\delta B_0,\hat{B}_1=\delta B_1$, we have M(k) has following form:

$$\begin{aligned}&M(k)\\&\quad =I+\delta A+\delta B_0\hat{P}_{k+d}(k+1)\\&\quad \quad +\delta ^2B_0\hat{P}(k+1)M_1(A_0+B_0\hat{P}_{k+d}(k+1))\\&\quad \quad +\delta B_0\hat{P}(k+1)M_1+\delta B_1\hat{P}(k+1)[I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)M_1\\&\quad \quad +\delta ^2B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}(k+1)M_1[A_0+B_0\hat{P}_{k+d}(k+1)]\\&\quad \quad +\delta ^2B_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)[I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}[A_0+B_0\hat{P}_{k+d}(k+1)]\\&\quad \quad +\delta B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}[A_0\\&\quad \quad +B_0\hat{P}_{k+d}(k+1)]+\delta ^2B_1\hat{P}(k+1)[I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}[A_0+B_0\hat{P}_{k+d}(k+1)]\\&\quad \quad +\Delta w(k)\Big \{\bar{A}+\bar{B}_0\hat{P}_{k+d}(k+1)+\delta \bar{B}_0\hat{P}(k+1)\\&\quad \quad \times M_1[A_0+B_0\hat{P}_{k+d}(k+1)]+\bar{B}_0\hat{P}(k+1)M_1\\&\quad \quad +\bar{B}_1\hat{P}(k\!+\!1)^{-1}[I\!-\!\bar{B}_1\hat{P}(k\!+\!1)]\bar{B}_0\hat{P}(k\!+\!1)M_1\\&\quad \quad +\delta B_1\hat{P}(k+1)[I\!-\!\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}(k+1)M_1[A_0+B_0\hat{P}_{k+d}(k+1)]\\&\quad \quad +\delta B_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)[I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}[A_0+B_0\hat{P}_{k+d}(k+1)]\\&\quad \quad +\bar{B}_1[I\!-\!\bar{B}_1\hat{P}(k+1)]^{-1}[A_0+B_0\hat{P}_{k+d}(k+1)]\\&\quad \quad +\delta B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}[A_0+B_0\hat{P}_{k+d}(k+1)]\Big \}. \end{aligned}$$

Then through the same procedure, we have $M_k(k)$ and $M_{k+i}(k)$ given by

$$\begin{aligned}&M_{k}(k)\\&\quad =\delta A_1+\delta ^2\hat{P}(k+1)M_1A_1+\delta ^2B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)M_1A_1\\&\quad \quad +\delta ^2B_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_1\\&\quad \quad +\delta B_1\hat{P}(k+1) [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_1\\&\quad \quad +\delta ^2B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_1\\&\quad \quad +\Delta w(k)\Big \{\bar{A}_1+\delta \bar{B}_0\hat{P}(k+1)M_1A_1\\&\quad \quad +\delta \bar{B}_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}(k+1)M_1A_1\\&\quad \quad +\delta \bar{B}_0\hat{P}(k+1)M_1B_1[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_1\\&\quad \quad +\bar{B}_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_1\\&\quad \quad +\delta \bar{B}_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)\\&\quad \quad \times M_1B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_1\Big \}, \end{aligned}$$

and

$$\begin{aligned}&M_{k+i}(k)\\&\quad =\delta B_0\hat{P}_{k+i}(k+1)+\delta ^2B_0\hat{P}(k+1)M_1\\&\quad \quad \times B_0\hat{P}_{k+i}(k+1)+\delta ^2B_1[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0P(k+1)M_1B_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\delta ^2B_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)[I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\delta B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}_{k+i}(k+1)+\delta ^2B_1\hat{P}(k+1)[I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\Delta w(k)\Big \{\bar{B}_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\delta \bar{B}_0\hat{P}(k+1)M_1B_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\delta \bar{B}_1[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)M_1\\&\quad \quad \times B_0\hat{P}_{k+i}(k+1)+\delta \bar{B}_0\hat{P}(k+1)M_1B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\bar{B}_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\delta \bar{B}_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)\\&\quad \quad \times M_1B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}_{k+i}(k+1)\Big \}. \end{aligned}$$

Then inserting into the (28), and keeping the term with one $\delta$ remain, we can obtain that

$$\begin{aligned}&x(k+1)\\&\quad =x(k)+\delta \Big \{A_0+B_0\hat{P}_{k+d}(k+1)+B_0\hat{P}(k+1)\\&\quad \quad +B_1P(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)\\&\quad \quad +B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}[\bar{A}_0\\&\quad \quad +\bar{B}_0\hat{P}_{k+d}(k+1)]\Big \}x(k)+\delta \{A_1+B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}A_1\}x(k-d)\\&\quad \quad +\delta \textstyle \sum \limits _{i=1}^{d-1}\{B_0\hat{P}_{k+i}(k+1)+B_1\hat{P}(k+1)[I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}_{k+i}(k+1)\}x(k-d+i)\\&\quad \quad +\Delta w(k)\Big \{\bar{A}+\bar{B}_0\hat{P}_{k+d}(k+1)+\bar{B}_0\hat{P}(k+1)\\&\quad \quad +\bar{B}_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1} \bar{B}_0\hat{P}(k+1)\\&\quad \quad +\bar{B}_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1} [\bar{A}_0\\&\quad \quad +\bar{B}_0\hat{P}_{k+d}(k+1)]x(k)+\delta \{\bar{A}_1+\bar{B}_1\hat{P}(k+1)[I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_1\}x(k-d)\\&\quad \quad +\delta \textstyle \sum \limits _{i=1}^{d-1}\big [\bar{B}_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\bar{B}_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}_{k+i}(k+1)\big ]x(k-d+i)\Big \}, \end{aligned}$$

and letting $\delta \rightarrow 0$, we have

$$\begin{aligned}&\mathrm{d}x(t)\\&\quad =\Big \{[A_0+B_0P(t)+B_1P(t)[I-\bar{B}_1P(t)]^{-1} \\&\quad \quad \times [A_0+\bar{B}_0P(t)]x(t)\\&\quad \quad +[A_1+B_1P(t)][I-\bar{B}_1P(t)]^{-1}\bar{A}_1x(t-h)\\&\quad \quad +\int _{t}^{T}[B_0P(t,\theta )+B_1P(t)][I-\bar{B}_1P(t)]^{-1}\\&\quad \quad \times \bar{B}_0 P(t,\theta )x(\theta -h)\mathrm{d}\theta \Big \}\mathrm{d}t\\&\quad \quad +\Big \{[\bar{A}_0+\bar{B}_0P(t)+\bar{B}_1P(t)[I\\&\quad \quad -\bar{B}_1P(t)]^{-1}[A_0+\bar{B}_0P(t)]x(t) \\&\quad \quad +[\bar{A}_1+\bar{B}_1P(t)][I-\bar{B}_1P(t)]^{-1}\bar{A}_1x(t-h)\\&\quad \quad +\int _{t}^{T}[\bar{B}_0P(t,\theta )+\bar{B}_1P(t)][I-\bar{B}_1P(t)]^{-1}\\&\quad \quad \times \bar{B}_0 P(t,\theta )x(\theta -h)\mathrm{d}\theta \Big \}\mathrm{d}w(t), \end{aligned}$$

which is same as (10) as the case of $t>T-h$ .

Now we will show the process of getting the continuous-time form of Riccati equations (31)–(33) and (38)–(40), and it is same as the process of getting $\mathrm{d}x(t)$. We will take the situation of $k>N-d$ as the example.

By the definition of $\hat{A}_0,\hat{A}_1,\hat{Q}$, we can rewrite P(k) as follows:

$$\begin{aligned}&\hat{P}(k)\\&\quad =(I+\delta A'_0)\hat{P}(k+1)(I+\delta A_0)+\delta \bar{A}'_0\hat{P}(k+1)\bar{A}_0\\&\quad \quad +[(I+\delta A'_0)\hat{P}(k+1)\delta B_0]\hat{P}(k+1)M_1(I+\delta A_0)\\&\quad \quad +Q+\delta \bar{A}'_0\hat{P}(k+1)\bar{B}_0\hat{P}(k+1)M_1(I+\delta A_0)\\&\quad \quad +[(I+\delta A'_0)\hat{P}(k+1)\delta B_1+\delta \bar{A}'_0\hat{P}(k+1)\bar{B}_1]\\&\quad \quad \times \hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)M_1\\&\quad \quad \times (I+\delta A_0)+\delta ^2[(I+\delta A'_0)\hat{P}(k+1)B_0\\&\quad \quad +\bar{A}'_0\hat{P}(k+1)\bar{B}_0]\hat{P}(k+1)M_1B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_0+\delta [(I+\delta A'_0)\\&\quad \quad \times \hat{P}(k+1)B_1+\bar{A}'_0\hat{P}(k+1)\bar{B}_1]\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_0+\delta ^2[(I+\delta A'_0)\\&\quad \quad \times \hat{P}(k+1)B_0+\bar{A}'_0\hat{P}(k+1)\bar{B}_0]\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)\\&\quad \quad \times M_1B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_0, \end{aligned}$$

keeping the terms with one $\delta$ remaining, we can obtain P(k) as the following form

$$\begin{aligned}&\frac{1}{\delta }[\hat{P}(k)-\hat{P}(k+1)]\\&\quad =A'_0\hat{P}(k+1)+\hat{P}(k+1)A_0+\bar{A}'_0\hat{P}(k+1)\bar{A}_0\\&\quad \quad +[\hat{P}(k+1)B_0+\bar{A}'_0\hat{P}(k+1)\bar{B}_0]\hat{P}(k+1)M_1\\&\quad \quad +Q+[\hat{P}(k+1)B_1+\bar{A}'_0\hat{P}(k+1)\bar{B}_1]\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)\\&\quad \quad +[\hat{P}(k+1)B_1+\bar{A}'_0\hat{P}(k+1)\bar{B}_1]\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_0, \end{aligned}$$

and letting $\delta \rightarrow 0$ we have

$$\begin{aligned}&-\dot{P}(t)\\&\quad =A'_0P(t)+P(t)A_0+\bar{A}'_0P(t)\bar{A}_0+[P(t)B_0\\&\quad \quad +\bar{A}'_0P(t)\bar{B}_0]P(k+1)+[P(t)B_1\\&\quad \quad +\bar{A}'_0P(t)\bar{B}_1]P(t)[I-\bar{B}_1P(t)]^{-1}[\bar{B}_0P(t)+\bar{A}_0], \end{aligned}$$

which is (4). Next, by the similarly procedure, we can rewrite $P_k(k)$ as follows:

$$\begin{aligned}&P_k(k)\\&\quad =(I+\delta A'_0)\hat{P}(k+1)\delta A_1+\delta \bar{A}'_0\hat{P}(k+1)\bar{A}_1\\&\quad \quad +\delta ^2[(I+\delta A'_0)\hat{P}(k+1)B_0+\bar{A}'_0\hat{P}(k+1)\bar{B}_0]\\&\quad \quad \times \hat{P}(k+1)M_1A_1+\delta ^2[(I+\delta A'_0)\hat{P}(k+1)B_1\\&\quad \quad +\bar{A}'_0\hat{P}(k+1)\bar{B}_1]\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}(k+1)M_1A_1+\delta ^2[(I+\delta A'_0)\hat{P}(k+1)B_0\\&\quad \quad +\bar{A}'_0\hat{P}(k+1)\bar{B}_0]\hat{P}(k+1)M_1B_1\hat{P}(k+1) [I\\&\quad \quad -\bar{B}_1\hat{P}(k+1)]^{-1}\bar{A}_1+\delta [(I+\delta A'_0)\hat{P}(k+1)B_1\\&\quad \quad +\bar{A}'_0\hat{P}(k+1)\bar{B}_1]\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}A_1\\&\quad \quad +\delta ^2[(I+\delta A'_0)\hat{P}(k+1)B_1+\bar{A}'_0\hat{P}(k+1)\bar{B}_1]\\&\quad \quad \times \hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)\\&\quad \quad \times M_1B_1\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}A_1, \end{aligned}$$

and keeping the terms with one $\delta$ remain and then letting $\delta \rightarrow 0$ we have

$$\begin{aligned} P(t,t)&=P(t)A_1+\bar{A}'_0P(t)\bar{A}_1+[P(t)B_1\\&\quad \quad +\bar{A}'_0P(t)\bar{B}_1]P(t)[I-\bar{B}_1P(t)]^{-1}\bar{A}_1, \end{aligned}$$

which is (5). Finally, we will deal with the $P_{k+i}(k)$,

$$\begin{aligned}&P_{k+i}(k)\\&\quad =\mathrm{E}\Big \{A_0(k)\hat{P}_{k+i}(k+1)+A_0(k)\hat{P}(k+1)\\&\quad \quad \times B_0(k)\hat{P}_{k+i}(k+1)+\delta A_0(k)\hat{P}(k+1)B_0(k)\\&\quad \quad \times \hat{P}(k+1)M_1B_0\hat{P}_{k+i}(k+1)+\delta A_0(k)\\&\quad \quad \times \hat{P}(k+1)B_1(k)\hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\\&\quad \quad \times \bar{B}_0\hat{P}(k+1)M_1B_0\hat{P}_{k+i}(k+1)+\delta A_0(k)\\&\quad \quad \times \hat{P}(k+1)B_0(k)\hat{P}(k+1)M_1B_1\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}_{k+i}(k+1)\\&\quad \quad +\delta A_0(k)\hat{P}(k+1)B_1(k)\hat{P}(k+1)\\&\quad \quad \times [I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}(k+1)M_1B_1\\&\quad \quad \times \hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}_{k+i}(k+1)\Big \}, \end{aligned}$$

we have

$$\begin{aligned}&\hat{P}_{k+i}(k)-\hat{P}_{k+i}(k+1)\\&\quad =\delta A_0\hat{P}_{k+i}(k+1)+\delta [\hat{P}(k+1)B_0+\bar{A}'_0\hat{P}(k+1)\bar{B}_0]\\&\quad \quad \times \hat{P}_{k+i}(k+1)+\delta [\hat{P}(k+1)B_1+\bar{A}'_0\hat{P}(k+1)\bar{B}_1]\\&\quad \quad \times \hat{P}(k+1)[I-\bar{B}_1\hat{P}(k+1)]^{-1}\bar{B}_0\hat{P}_{k+i}(k+1), \end{aligned}$$

and letting $\delta \rightarrow 0$, we can obtain $P(t,\theta )$ satisfies the following relationship:

$$\begin{aligned}&\frac{\partial P(t,\theta )}{\partial t}\\&\quad =A'_0P(t,\theta )+[P(t)B_0+\bar{A}'_0P(t)\bar{B}_0]P(t,\theta )\\&\quad \quad +[P(t)B_1+\bar{A}'_0P(t)\bar{B}_1]P(t)[I-\bar{B}_1P(t)]^{-1}\\&\quad \quad \times \bar{B}_0P(t,\theta ). \end{aligned}$$

The situation of $0\le k\le N-d$ is same so the details are omitted here. Thus the proof is now completed. $\square$

It is also noted that the proof of the convergence analysis will be much more involved since the existence of random coefficients in (28) and (29), a rigor proof by using Itô’s formula has been given in “Appendix C”.

Appendix C: Proof of Theorem 1

We will verify that the derived solution (2) and (3) and (10) indeed satisfies the FBSDEs (1) by using the continuous-time technique. The detailed derivations are given as follows.

Considering (10) and combining with (2) and (3), we have

$$\begin{aligned}&\mathrm{d}x(t)\\&\quad =[A_0x(t)+A_1x(t-h)+B_0p(t)+B_1q(t)]\mathrm{d}t\\&\quad \quad +[\bar{A}_0x(t)+\bar{A}_1x(t-h)+\bar{B}_0p(t)+\bar{B}_1q(t)]\mathrm{d}w(t), \end{aligned}$$

which is same as FSDE in (1).

Next, when $t>T-h$, from (2) and (4)–(6), by applying Itô formula to (2), we have

$$\begin{aligned}&\mathrm{d}[P(t)x(t)+\int _{t}^{T}P(t,\theta )x(\theta \!-\!h)\mathrm{d}\theta ]\\&\quad =\mathrm{d}P(t)x(t)+P(t)\mathrm{d}x(t)+\mathrm{d}\left[ \int _{t}^{T}P(t,\theta )x(\theta -h)\mathrm{d}\theta \right] \\&\quad =-\Big \{A'_{0}P(t)+P(t)A_0+\bar{A}'_{0}P(t)\bar{A}_0+Q\\&\quad \quad +[P(t)B_0+\bar{A}'_{0}P(t)\bar{B}_0]P(t)\\&\quad \quad +[P(t)B_1+\bar{A}'_{0}P(t)\bar{B}_1]P(t)[I-\bar{B}_1P(t)]^{-1}\\&\quad \quad \times [\bar{B}_0P(t)+\bar{A}_0]\Big \}x(t)\mathrm{d}t+P(t)[A_0x(t)\\&\quad \quad +A_1x(t-h)+B_0p(t)+B_1q(t)]\mathrm{d}t\!+\!P(t)[\bar{A}_0x(t)\\&\quad \quad +\bar{A}_1x(t-h)+\bar{B}_0p(t)+\bar{B}_1q(t)]\mathrm{d}w(t)\\&\quad \quad +\int _{t}^{T}\Big \{A'_{0}P(t,\theta )+[P(t)B_0+\bar{A}'_{0}P(t)\bar{B}_0]P(t,\theta )\\&\quad \quad +[P(t)B_1+\bar{A}'_{0}P(t)\bar{B}_1]P(t)[I-\bar{B}_1P(t)]^{-1}\\&\quad \quad \times \bar{B}_0P(t,\theta )\Big \}x(\theta -h)\mathrm{d}\theta \\&\quad \quad -\Big \{[P(t)B_1+\bar{A}'_{0}P(t)\bar{B}_1]P(t)[I-\bar{B}_1P(t)]^{-1}\bar{A}_1\\&\quad \quad +P(t)A_1+\bar{A}'_{0}P(t)\bar{A}_1\Big \}x(t-h), \end{aligned}$$

which we can obtain that

$$\begin{aligned}&\mathrm{d}p(t)\\&\quad =-A'_0[P(t)x(t)+\int _{t}^{T}P(t,\theta )x(\theta -h)\mathrm{d}\theta ]\mathrm{d}t\\&\quad \quad -\bar{A}'_0[I-P(t)\bar{B}_1]^{-1}P(t)[\bar{A}_0x(t)+\bar{A}_1x(t-h)\\&\quad \quad +\bar{B}_0p(t)]\mathrm{d}t-Qx(t)\mathrm{d}t+[I-P(t)\bar{B}_1]^{-1}P(t)\\&\quad \quad \times [\bar{A}_0x(t)+\bar{A}_1x(t-h)+\bar{B}_0p(t)]\mathrm{d}w(t), \end{aligned}$$

that is,

$$\begin{aligned} \mathrm{d}p(t)&=-[A'_0p(t)+\bar{A}'_0q(t)+Qx(t)]\mathrm{d}t\\&\quad +q(t)\mathrm{d}w(t), \end{aligned}$$

which is same as BSDE in (1) as the case of $t>T-h$, the case of $0<t\le T-h$ can be proved similarly, so the details are omitted here. Therefore, we have that (2) and (3) and (10) are the solution to the FBSDEs (1). This completes the proof. $\square$

Appendix D: Proof of Corollary 1

Inserting $B_0=-BR^{-1}B'$, $B_1=-BR^{-1}\bar{B}'$, $\bar{B}_0=-\bar{B}R^{-1}B'$, and $\bar{B}_0=-\bar{B}R^{-1}\bar{B}'$ into Riccati equations (4)–(9), we can quickly obtain (14)–(20). Next considering the specific matrices in Theorem 1, we have that p(t) and q(t) have following forms:

$$\begin{aligned} p(t)&=P(t)x(t)+\int _{t}^{\min \{t+h,T\}}P(t,\theta )x(\theta -h)\mathrm{d}\theta ,\\ q(t)&=P(t)[\bar{A}_0x(t)+\bar{A}_1x(t-h)-\bar{B}R^{-1}B'p(t)\\&\quad -\bar{B}R^{-1}\bar{B}'q(t)], \end{aligned}$$

and combining the equilibrium condition in (13), i.e, $0=Ru(t)+\mathrm{E}[B'p(t)+\bar{B}'q(t)\vert \mathcal {F}(t)]$, we have that

$$\begin{aligned} q(t)=P(t)[\bar{A}_0x(t)+\bar{A}_1x(t-h)+\bar{B}u(t)]. \end{aligned}$$

Therefore, (22)–(23) have been proved. Considering p(t), q(t) and equilibrium condition in (13), it gives that

$$\begin{aligned} 0&=Ru(t)+\mathrm{E}[B'p(t)+\bar{B}'q(t)\vert \mathcal {F}(t)]\\&=Ru(t)+\mathrm{E}[B'P(t)x(t)\\&\quad +\int _{t}^{\min \{t+h,T\}}B'P(t,\theta )x(\theta -h)\mathrm{d}\theta ]\\&\quad +\bar{B}'P(t)[\bar{A}_0x(t)+\bar{A}_1x(t-h)+\bar{B}u(t)], \end{aligned}$$

that is,

$$\begin{aligned} u(t)&=-R^{-1}(t)[B'P(t)+\bar{B}'P(t)\bar{A}_0]x(t)\\&\quad -R^{-1}(t)\bar{B}'P(t)\bar{A}_1x(t-h) \\&\quad -R^{-1}(t)\int _{t}^{\min \{t+h,T\}}B'P(t,\theta )x(\theta \!-\!h)\mathrm{d}\theta \end{aligned}$$

with $R(t)=\bar{B}'P(t)\bar{B}+R$. Thus we can obtain that the optimal controller is as (21). This completes the proof. $\square$