noncomputable def
quasiNewtonUpdatedHessian
{n : ℕ}
(δ : ℝ)
(G : { G : Matrix (Fin n) (Fin n) ℝ // G.PosDef })
(g : EuclideanSpace ℝ (Fin n))
:
Matrix (Fin n) (Fin n) ℝ
The rank-one Hessian update
G + (δ / (1 - δ)) * (ggᵀ / ‖g‖_{G}^{*2}) from the relative-accuracy quasi-Newton scheme,
written using the canonical Chapter 7 dual norm owner.
Instances For
theorem
quasiNewtonUpdatedHessian_def
{n : ℕ}
(δ : ℝ)
(G : { G : Matrix (Fin n) (Fin n) ℝ // G.PosDef })
(g : EuclideanSpace ℝ (Fin n))
:
quasiNewtonUpdatedHessian δ G g = ↑G + (δ / (1 - δ) / (positiveDefMatrixNorm ↑G ⋯).dualNorm g ^ 2) • Matrix.vecMulVec g.ofLp g.ofLp
Expanding quasiNewtonUpdatedHessian δ G g gives the displayed rank-one update formula for
the next Hessian matrix.
theorem
quasiNewtonUpdatedHessian_posDef
{n : ℕ}
{δ : ℝ}
(hδ : δ ∈ Set.Ioo 0 1)
(G : { G : Matrix (Fin n) (Fin n) ℝ // G.PosDef })
(g : EuclideanSpace ℝ (Fin n))
:
(quasiNewtonUpdatedHessian δ G g).PosDef
The rank-one update preserves positive definiteness when δ ∈ (0, 1).
theorem
quasiNewtonRankOneHessianUpdate_and_inverse
{n : ℕ}
{δ : ℝ}
(hδ : δ ∈ Set.Ioo 0 1)
(Gk : { G : Matrix (Fin n) (Fin n) ℝ // G.PosDef })
(gk : EuclideanSpace ℝ (Fin n))
:
quasiNewtonUpdatedHessian δ Gk gk = ↑Gk + (δ / (1 - δ) / (positiveDefMatrixNorm ↑Gk ⋯).dualNorm gk ^ 2) • Matrix.vecMulVec gk.ofLp gk.ofLp ∧ (quasiNewtonUpdatedHessian δ Gk gk)⁻¹ = (↑Gk)⁻¹ - (δ / (positiveDefMatrixNorm ↑Gk ⋯).dualNorm gk ^ 2) • ((↑Gk)⁻¹ * Matrix.vecMulVec gk.ofLp gk.ofLp * (↑Gk)⁻¹)
Proposition 7.37: if ψ_k is a quadratic estimating function with positive-definite Hessian
G_k, then the next Hessian is obtained by the rank-one update
G_{k+1} = G_k + (δ / (1 - δ)) * g_k g_kᵀ / ‖g_k‖_{G_k}^{*2}, and its inverse satisfies the
Sherman-Morrison identity
G_{k+1}^{-1} = G_k^{-1} - δ * G_k^{-1} g_k g_kᵀ G_k^{-1} / ‖g_k‖_{G_k}^{*2}.