Augmented Lagrangian Alternating Direction Inexact Newton (ALADIN)¶
Where ALC solves the primal problem via FPI, Augmented Lagrangian based Alternating Direction Inexact Newton (ALADIN) computes search directions \(\Delta{}^{i}d\) by Newton/SQP iterations on the KKT system [11, 9, 12]. While [11] treats affine coupling constraints \(c\), this section extends the principle to nonlinear pairwise couplings, yielding a novel distributed Newton/SQP method.
The SQP objective is the second-order Taylor approximation of the total Lagrangian \(L\), with KKT feasibility as linearized constraints. Unlike the active-set Newton interpretation of [11, 9, 12], the QP does not prescribe an active set for inequality constraints:
\[\begin{aligned}
\left\{\Delta{}^{i}d\right\}_{i\in M}
\leftarrow
& \argmin_{\Delta{}^{i}d\;\forall\; i\in M}
\quad
\sum_{i\in M}
\Biggl(
{}^{i}v_f
+
\nabla_{{}^{i}d}^{T}{}^{i}v_f\,\Delta{}^{i}d
+
\frac{1}{2}
\Delta{}^{i}d^{T}
\nabla^{2}_{{}^{i}d\,{}^{i}d}{}^{i}v_f
\,\Delta{}^{i}d
\\
& \phantom{\argmin_{\Delta{}^{i}d\;\forall\; i\in M}} \quad\quad\quad\;\; +
{}^{i}\kappa_g^{T}\,{}^{i}v_g
+
{}^{i}\kappa_g^{T}
\nabla_{{}^{i}d}{}^{i}v_g\,\Delta{}^{i}d
+
\frac{1}{2}
\Delta{}^{i}d^{T}
\left(
\sum_{q=1}^{{}^{i}n_{v_g}}
({}^{i}\kappa_g)_{q}
\nabla^{2}_{{}^{i}d\,{}^{i}d}
\bigl(({}^{i}v_g)_{q}\bigr)
\right)
\Delta{}^{i}d
\\
& \phantom{\argmin_{\Delta{}^{i}d\;\forall\; i\in M}} \quad\quad\quad\;\; +
{}^{i}\kappa_h^{T}\,{}^{i}v_h
+
{}^{i}\kappa_h^{T}
\nabla_{{}^{i}d}{}^{i}v_h\,\Delta{}^{i}d
+
\frac{1}{2}
\Delta{}^{i}d^{T}
\left(
\sum_{q=1}^{{}^{i}n_{v_h}}
({}^{i}\kappa_h)_{q}
\nabla^{2}_{{}^{i}d\,{}^{i}d}
\bigl(({}^{i}v_h)_{q}\bigr)
\right)
\Delta{}^{i}d
\\
& \phantom{\argmin_{\Delta{}^{i}d\;\forall\; i\in M}} \quad\quad\quad\;\; +
{}^{i}\kappa_d^{T}\,{}^{i}v_D
+
{}^{i}\kappa_d^{T}\nabla_{{}^{i}d}{}^{i}v_D\,\Delta{}^{i}d
\Biggr)
\\
& \phantom{\argmin_{\Delta{}^{i}d\;\forall\; i\in M}} \quad +
\sum_{i\in M}\sum_{j\in{}^{i}N}
\Biggl(
{}^{i}_{j}\lambda^{T}
\begin{bmatrix}
{}^{i}_{j}H\!\left({}^{i}r\right) - {}^{i}_{j}h \\
{}^{i}_{j}z - {}^{j}_{i}z
\end{bmatrix}
+
{}^{i}_{j}\lambda^{T}
\begin{bmatrix}
\nabla_{{}^{i}d}{}^{i}_{j}H\,\Delta{}^{i}d
- {}^{i}_{j}S_{h}\,\Delta{}^{j}d \\
{}^{i}_{j}S_{z}\,\Delta{}^{i}d
- {}^{j}_{i}S_{z}\,\Delta{}^{j}d
\end{bmatrix}
\\
& \phantom{\argmin_{\Delta{}^{i}d\;\forall\; i\in M}} \quad\quad\quad\quad\quad\quad\; +
\frac{1}{2}
\Delta{}^{i}d^{T}
\left(
\sum_{q=1}^{{}^{i}_{j}n_h}
({}^{i}_{j}\lambda_{h})_{q}
\nabla^{2}_{{}^{i}d\,{}^{i}d}
\bigl({}^{i}_{j}H\bigr)_{q}
\right)
\Delta{}^{i}d
\Biggr)
\\
& \phantom{\argmin_{\Delta{}^{i}d\;\forall\; i\in M}} \quad +
\sum_{i\in M}\sum_{j\in{}^{i}N}
\Biggl(
\left\|
{}^{i}_{j}s\circ
\begin{bmatrix}
{}^{i}_{j}H\!\left({}^{i}r\right) - {}^{i}_{j}h \\
{}^{i}_{j}z - {}^{j}_{i}z
\end{bmatrix}
\right\|_{2}^{2}
+
2\left({}^{i}_{j}s^{2}\circ
\begin{bmatrix}
{}^{i}_{j}H\!\left({}^{i}r\right) - {}^{i}_{j}h \\
{}^{i}_{j}z - {}^{j}_{i}z
\end{bmatrix}
\right)^{T}
\begin{bmatrix}
\nabla_{{}^{i}d}{}^{i}_{j}H\,\Delta{}^{i}d
- {}^{i}_{j}S_{h}\,\Delta{}^{j}d \\
{}^{i}_{j}S_{z}\,\Delta{}^{i}d
- {}^{j}_{i}S_{z}\,\Delta{}^{j}d
\end{bmatrix}
\\
& \phantom{\argmin_{\Delta{}^{i}d\;\forall\; i\in M}} \quad\quad\quad\quad\quad\quad\; +
\begin{bmatrix}
\nabla_{{}^{i}d}{}^{i}_{j}H\,\Delta{}^{i}d
- {}^{i}_{j}S_{h}\,\Delta{}^{j}d \\
{}^{i}_{j}S_{z}\,\Delta{}^{i}d
- {}^{j}_{i}S_{z}\,\Delta{}^{j}d
\end{bmatrix}^{T}
\operatorname{diag}\!\left({}^{i}_{j}s\right)^{2}
\begin{bmatrix}
\nabla_{{}^{i}d}{}^{i}_{j}H\,\Delta{}^{i}d
- {}^{i}_{j}S_{h}\,\Delta{}^{j}d \\
{}^{i}_{j}S_{z}\,\Delta{}^{i}d
- {}^{j}_{i}S_{z}\,\Delta{}^{j}d
\end{bmatrix}
\\
& \phantom{\argmin_{\Delta{}^{i}d\;\forall\; i\in M}} \quad\quad\quad\quad\quad\quad\; +
\sum_{q=1}^{{}^{i}_{j}n_h}
\left({}^{i}_{j}s_{h}^{2}\circ
\left({}^{i}_{j}H\!\left({}^{i}r\right) - {}^{i}_{j}h\right)
\right)_{q}
\Delta{}^{i}d^{T}
\nabla^{2}_{{}^{i}d\,{}^{i}d}
\bigl(({}^{i}_{j}H)_{q}\bigr)
\Delta{}^{i}d
\Biggr)
\\
&\text{s.t.} \quad
{}^{i}v_g
+
\nabla_{{}^{i}d}{}^{i}v_g\,
\Delta{}^{i}d
\leq
0
\quad \forall\; i\in M,
\\
&\phantom{\text{s.t.}} \quad
{}^{i}v_h
+
\nabla_{{}^{i}d}{}^{i}v_h\,
\Delta{}^{i}d
=
0
\quad \forall\; i\in M,
\\
&\phantom{\text{s.t.}} \quad
{}^{i}v_D
+
\nabla_{{}^{i}d}{}^{i}v_D\,
\Delta{}^{i}d
\leq
0
\quad \forall\; i\in M.
\end{aligned}\]
The bound constraint \({}^{i}d \in {}^{i}\mathcal{D}\) is expressed as an inequality whose Hessian vanishes. All quantities are evaluated at \({}^{i}d^{(k,\,l)}\), and \({}^{i}\kappa_g,\; {}^{i}\kappa_h,\; {}^{i}\kappa_d\) and \({}^{i}_{j}\lambda\) are Lagrange multipliers for \({}^{i}v_g,\; {}^{i}v_h,\; {}^{i}v_D\), and \({}^{i}_{j}\)\(c\), respectively.
Since the QP couples \(\Delta{}^{i}d\) and \(\Delta{}^{j}d\) pairwise, a controller entity assembles and solves it. ALADIN separates locally feasible subsystem iterates from the coupled SQP correction. Subsystems provide gradients, Hessians, multipliers, and active sets to the controller.
The controller prediction is
\[{}^{i}\hat{d}^{(k,\,l+1)}
:=
{}^{i}d^{(k,\,l)}
+
\Delta{}^{i}d
\quad \forall\; i\in M.\]
A proximalized auxiliary problem is introduced around this prediction:
\[\begin{aligned}
\left\{{}^{i}d^{(k,\,l+1)}\right\}_{i\in M}
\leftarrow
& \argmin_{{}^{i}d\;\forall\; i\in M}
\quad
\sum_{i\in M}
\left(
{}^{i}v_f\!\left({}^{i}r\right)
+
\sum_{j\in{}^{i}N}
{}^{i}_{j}\lambda^{T}
\,{}^{i}_{j}c\!\left({}^{i}d,\;{}^{j}d\right)
+
\frac{\nu}{2}
\left\|
{}^{i}d
-
{}^{i}\hat{d}^{(k,\,l+1)}
\right\|_{{}^{i}\Sigma}^{2}
\right)
\\
&\text{s.t.} \quad
{}^{i}v_g\!\left({}^{i}r\right)\leq 0
\quad |\; {}^{i}\kappa_g
\quad \forall\; i\in M,
\\
&\phantom{\text{s.t.}} \quad
{}^{i}v_h\!\left({}^{i}r\right)=0
\quad |\; {}^{i}\kappa_h
\quad \forall\; i\in M,
\\
&\phantom{\text{s.t.}} \quad
{}^{i}v_D\!\left({}^{i}d\right)\leq 0
\quad |\; {}^{i}\kappa_d
\quad \forall\; i\in M.
\end{aligned}\]
The proximal term regularizes around the controller prediction. Since this is separable, each subsystem \(i\) solves:
\[\begin{aligned}
{}^{i}d^{(k,\,l+1)}
\leftarrow
& \argmin_{{}^{i}d} \quad
{}^{i}v_f\!\left({}^{i}r\right)
+
\sum_{j\in{}^{i}N}
\Biggl(
{}^{i}_{j}\lambda^{T}
\begin{bmatrix}
{}^{i}_{j}H\!\left({}^{i}r\right) \\
{}^{i}_{j}S_{z}\,{}^{i}d
\end{bmatrix}
+
{}^{j}_{i}\lambda^{T}
\begin{bmatrix}
- {}^{j}_{i}S_{h}\,{}^{i}d \\
- {}^{i}_{j}S_{z}\,{}^{i}d
\end{bmatrix}
\Biggr)
\\
& \quad\quad\quad\quad\; +
\frac{\nu}{2}
\left\|
{}^{i}d
-
{}^{i}\hat{d}^{(k,\,l+1)}
\right\|_{{}^{i}\Sigma}^{2}
\\
& \text{s.t.} \quad
{}^{i}v_g\!\left({}^{i}r\right)\leq 0 \quad
|\; {}^{i}\kappa_g,
\\
& \phantom{\text{s.t.}} \quad
{}^{i}v_h\!\left({}^{i}r\right)=0 \quad
|\; {}^{i}\kappa_h,
\\
& \phantom{\text{s.t.}} \quad
{}^{i}v_D\!\left({}^{i}d\right)\leq 0 \quad
|\; {}^{i}\kappa_d.
\end{aligned}\]
The iterates \({}^{i}d^{(k,\,l+1)}\) satisfy local feasibility exactly, so constraint linearizations in the QP objective vanish and constant terms are omitted. The local Hessian \({}^{i}B\) collects:
\[{}^{i}B
:=
\nabla^{2}_{{}^{i}d\,{}^{i}d}{}^{i}v_f
+
\sum_{q=1}^{{}^{i}n_{v_g}}
({}^{i}\kappa_g)_{q}
\nabla^{2}_{{}^{i}d\,{}^{i}d}
\bigl(({}^{i}v_g)_{q}\bigr)
+
\sum_{q=1}^{{}^{i}n_{v_h}}
({}^{i}\kappa_h)_{q}
\nabla^{2}_{{}^{i}d\,{}^{i}d}
\bigl(({}^{i}v_h)_{q}\bigr).\]
For convergence analysis and necessary assumptions of the original ALADIN with affine coupling constraints \(c\), see [9, 12, 11]. A detailed convergence analysis of the generalized ALADIN algorithm presented here is reserved for future work.
All symbols are defined in the Nomenclature. The full procedure is stated in Algorithm 2.
Algorithm 2 Augmented Lagrangian Alternating Direction Inexact Newton (ALADIN)
Require: hyperparameters $\nu$, \({}^{i}\Sigma\), \({}^{i}_{j}s \;\; \forall\; j \in {}^{i}N,\; i \in M\)
Require: initial \(\Delta{}^{i}d\) in interface storage \(i \leftrightarrow \text{Controller} \;\; \forall\; i \in M\)
Require: initial \({}^{i}\lambda^{(0)} \;\; \forall\; i \in M\)
\(k \leftarrow 0\) ▷ initialize outerloop iterator
repeat
\(l \leftarrow 0\) ▷ initialize innerloop iterator
repeat
for every \(i \in M\) in parallel do
Copy \(\Delta{}^{i}d\) from interface storage \(i \leftrightarrow \text{Controller}\)
Copy \({}^{i}_{j}h,\; {}^{j}_{i}z\) from interface storage \(i \leftrightarrow j \;\; \forall\; j \in {}^{i}N\)
\[
{}^{i}\hat{d}^{(k,\,l+1)} \leftarrow {}^{i}d^{(k,\,l)} + \Delta{}^{i}d
\]
\[\begin{aligned}
{}^{i}d^{(k,\,l+1)} \leftarrow{} & \argmin_{{}^{i}d} \;\; {}^{i}v_f\!\left({}^{i}r\right)
+ \sum_{j \in {}^{i}N} \left(
\left({}^{i}_{j}\lambda\right)^{T}
\begin{bmatrix}
{}^{i}_{j}H\!\left({}^{i}r\right) \\
{}^{i}_{j}S_{z}\,{}^{i}d
\end{bmatrix}
+ \left({}^{j}_{i}\lambda\right)^{T}
\begin{bmatrix}
- {}^{j}_{i}S_{h}\,{}^{i}d \\
- {}^{i}_{j}S_{z}\,{}^{i}d
\end{bmatrix}
\right) \\
& + \frac{\nu}{2} \left\| {}^{i}d - {}^{i}\hat{d}^{(k,\,l+1)} \right\|_{{}^{i}\Sigma}^{2} \\
& \text{s.t.} \;\; {}^{i}v_g\!\left({}^{i}r\right) \leq 0 \quad | \; {}^{i}\kappa_g \\
& \phantom{\text{s.t.}} \;\; {}^{i}v_h\!\left({}^{i}r\right) = 0 \quad | \; {}^{i}\kappa_h \\
& \phantom{\text{s.t.}} \;\; {}^{i}v_D\!\left({}^{i}d\right) \leq 0 \quad | \; {}^{i}\kappa_d
\end{aligned}\]
Copy \({}^{i}B,\; \nabla_{{}^{i}d}{}^{i}v_f,\; {}^{i}v_g,\; \nabla_{{}^{i}d}{}^{i}v_g,\; \nabla_{{}^{i}d}{}^{i}v_h,\; {}^{i}v_D,\; \nabla_{{}^{i}d}{}^{i}v_D,\; \nabla^{2}_{{}^{i}d\,{}^{i}d}{}^{i}_{j}H,\; \nabla_{{}^{i}d}{}^{i}_{j}H,\; {}^{i}_{j}H\),
\({}^{j}_{i}h,\; {}^{i}_{j}z,\; {}^{j}_{i}S_{h},\; {}^{i}_{j}S_{z},\; {}^{i}_{j}\lambda,\; {}^{i}_{j}s \;\; \forall\; j \in {}^{i}N\) to interface storage \(i \leftrightarrow \text{Controller}\)
Copy \({}^{i}_{j}H,\; {}^{j}_{i}h,\; {}^{i}_{j}z\) to interface storage \(i \leftrightarrow j \;\; \forall\; j \in {}^{i}N\)
end for
for Controller do
Copy \({}^{i}B,\; \nabla_{{}^{i}d}{}^{i}v_f,\; {}^{i}v_g,\; \nabla_{{}^{i}d}{}^{i}v_g,\; \nabla_{{}^{i}d}{}^{i}v_h,\; {}^{i}v_D,\; \nabla_{{}^{i}d}{}^{i}v_D,\; \nabla^{2}_{{}^{i}d\,{}^{i}d}{}^{i}_{j}H,\; \nabla_{{}^{i}d}{}^{i}_{j}H,\; {}^{i}_{j}H\),
\({}^{j}_{i}h,\; {}^{i}_{j}z,\; {}^{j}_{i}S_{h},\; {}^{i}_{j}S_{z},\; {}^{i}_{j}\lambda,\; {}^{i}_{j}s \;\; \forall\; j \in {}^{i}N\) from interface storage \(i \leftrightarrow \text{Controller} \;\; \forall\; i \in M\)
\[\begin{aligned}
\left\{\Delta{}^{i}d\right\}_{i \in M} \leftarrow{} & \argmin_{\Delta{}^{i}d \;\forall\; i \in M} \;\;
\sum_{i \in M} \left(
\nabla_{{}^{i}d}^{T}{}^{i}v_f\,\Delta{}^{i}d
+ \frac{1}{2} \Delta{}^{i}d^{T}\,{}^{i}B\,\Delta{}^{i}d
\right) \\
& + \sum_{i \in M} \sum_{j \in {}^{i}N} \left(
\left({}^{i}_{j}\lambda\right)^{T}
\begin{bmatrix}
\nabla_{{}^{i}d}{}^{i}_{j}H\,\Delta{}^{i}d - {}^{i}_{j}S_{h}\,\Delta{}^{j}d \\
{}^{i}_{j}S_{z}\,\Delta{}^{i}d - {}^{j}_{i}S_{z}\,\Delta{}^{j}d
\end{bmatrix}
+ \frac{1}{2} \Delta{}^{i}d^{T} \left(
\sum_{q=1}^{{}^{i}_{j}n_h} \left({}^{i}_{j}\lambda_{h}\right)_{q} \nabla^{2}_{{}^{i}d\,{}^{i}d} \left({}^{i}_{j}H\right)_{q}
\right) \Delta{}^{i}d
\right) \\
& + \sum_{i \in M} \sum_{j \in {}^{i}N} \left(
2 \left( {}^{i}_{j}s^{2} \circ
\begin{bmatrix}
{}^{i}_{j}H\!\left({}^{i}r\right) - {}^{i}_{j}h \\
{}^{i}_{j}z - {}^{j}_{i}z
\end{bmatrix}
\right)^{T}
\begin{bmatrix}
\nabla_{{}^{i}d}{}^{i}_{j}H\,\Delta{}^{i}d - {}^{i}_{j}S_{h}\,\Delta{}^{j}d \\
{}^{i}_{j}S_{z}\,\Delta{}^{i}d - {}^{j}_{i}S_{z}\,\Delta{}^{j}d
\end{bmatrix}
\right. \\
& \quad\quad\quad\quad\quad\;\; +
\begin{bmatrix}
\nabla_{{}^{i}d}{}^{i}_{j}H\,\Delta{}^{i}d - {}^{i}_{j}S_{h}\,\Delta{}^{j}d \\
{}^{i}_{j}S_{z}\,\Delta{}^{i}d - {}^{j}_{i}S_{z}\,\Delta{}^{j}d
\end{bmatrix}^{T}
\operatorname{diag}\!\left({}^{i}_{j}s\right)^{2}
\begin{bmatrix}
\nabla_{{}^{i}d}{}^{i}_{j}H\,\Delta{}^{i}d - {}^{i}_{j}S_{h}\,\Delta{}^{j}d \\
{}^{i}_{j}S_{z}\,\Delta{}^{i}d - {}^{j}_{i}S_{z}\,\Delta{}^{j}d
\end{bmatrix} \\
& \left. \quad\quad\quad\quad\quad\;\; +
\sum_{q=1}^{{}^{i}_{j}n_h} \left( {}^{i}_{j}s_{h}^{2} \circ \left( {}^{i}_{j}H\!\left({}^{i}r\right) - {}^{i}_{j}h \right) \right)_{q}
\Delta{}^{i}d^{T} \nabla^{2}_{{}^{i}d\,{}^{i}d} \left( \left({}^{i}_{j}H\right)_{q} \right) \Delta{}^{i}d
\right) \\
& \text{s.t.} \;\; {}^{i}v_g + \nabla_{{}^{i}d}{}^{i}v_g\,\Delta{}^{i}d \leq 0 \quad \forall\; i \in M \\
& \phantom{\text{s.t.}} \;\; \nabla_{{}^{i}d}{}^{i}v_h\,\Delta{}^{i}d = 0 \quad \forall\; i \in M \\
& \phantom{\text{s.t.}} \;\; {}^{i}v_D + \nabla_{{}^{i}d}{}^{i}v_D\,\Delta{}^{i}d \leq 0 \quad \forall\; i \in M
\end{aligned}\]
Copy \(\Delta{}^{i}d\) to interface storage \(i \leftrightarrow \text{Controller} \;\; \forall\; i \in M\)
end for
\(l \leftarrow l + 1\)
until innerloop convergence criterion is met
\({}^{i}d^{(k+1)} \leftarrow {}^{i}d^{(k,\,l)} \;\; \forall\; i \in M\)
for every \(i \in M\) do
Copy \(\Delta{}^{i}d\) from interface storage \(i \leftrightarrow \text{Controller}\)
Copy \({}^{j}_{i}H\!\left({}^{j}r\right),\; {}^{i}_{j}h,\; {}^{j}_{i}z\) from interface storage \(i \leftrightarrow j \;\; \forall\; j \in {}^{i}N\)
\[
{}^{i}\hat{d}^{(k+1)} \leftarrow {}^{i}d^{(k+1)} + \Delta{}^{i}d
\]
Copy \({}^{i}_{j}H,\; {}^{j}_{i}h,\; {}^{i}_{j}z\) to interface storage \(i \leftrightarrow j \;\; \forall\; j \in {}^{i}N\)
Copy \({}^{i}_{j}\hat{H},\; {}^{j}_{i}\hat{h},\; {}^{i}_{j}\hat{z}\) to interface storage \(i \leftrightarrow j \;\; \forall\; j \in {}^{i}N\)
end for
for every \(i \in M\) do ▷ decentralized Dual Update
Copy \({}^{j}_{i}\hat{H}\!\left({}^{j}r\right),\; {}^{i}_{j}\hat{h},\; {}^{j}_{i}\hat{z}\) from interface storage \(i \leftrightarrow j \;\; \forall\; j \in {}^{i}N\)
Copy \({}^{j}_{i}H\!\left({}^{j}r\right),\; {}^{i}_{j}h,\; {}^{j}_{i}z\) from interface storage \(i \leftrightarrow j \;\; \forall\; j \in {}^{i}N\)
\[\begin{aligned}
{}^{i}_{j}\hat{c}^{(k+1)} &\leftarrow
\begin{bmatrix}
{}^{i}_{j}\hat{H}\!\left({}^{i}r\right) - {}^{i}_{j}\hat{h} \\
{}^{i}_{j}S_{z}\,{}^{i}\hat{d}^{(k+1)} - {}^{j}_{i}\hat{z}
\end{bmatrix},
\quad
{}^{j}_{i}\hat{c}^{(k+1)} \leftarrow
\begin{bmatrix}
{}^{j}_{i}\hat{H}\!\left({}^{j}r\right) - {}^{j}_{i}S_{h}\,{}^{i}\hat{d}^{(k+1)} \\
{}^{j}_{i}\hat{z} - {}^{i}_{j}S_{z}\,{}^{i}\hat{d}^{(k+1)}
\end{bmatrix}
\end{aligned}\]
\[\begin{aligned}
{}^{i}_{j}\lambda^{(k+1)} &\leftarrow \operatorname{DualUpdate}\!\left(
{}^{i}_{j}\lambda^{(k)},\;
{}^{i}_{j}s^{(k)},\;
{}^{i}_{j}\hat{c}^{(k+1)}
\right) \quad \forall\; j \in {}^{i}N \\
{}^{j}_{i}\lambda^{(k+1)} &\leftarrow \operatorname{DualUpdate}\!\left(
{}^{j}_{i}\lambda^{(k)},\;
{}^{j}_{i}s^{(k)},\;
{}^{j}_{i}\hat{c}^{(k+1)}
\right) \quad \forall\; j \in {}^{i}N
\end{aligned}\]
end for
\(k \leftarrow k + 1\)
until outerloop convergence criterion is met
return \(\left\{{}^{i}d^{(k)}\right\}_{i \in M}\)