Documentation

FirstOrderMethodsOptimization_Beck_2017.Chap09.Text_9_4

theorem mirror_descent_update_objective_add_constant_eq_bregman_form {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] (ω : EEReal) (xk gradf : E) (t : ) :
(fun (x : E) => mirror_descent_update_objective (fun (y : E) => (ω y).toReal) xk gradf t x + (inner (gradient (fun (y : E) => (ω y).toReal) xk) xk - (ω xk).toReal)) = fun (x : E) => inner (t gradf) x + B[ω] x xk

The Chapter 9 owner mirror_descent_update_objective differs from the textbook Bregman-form mirror-step objective only by the constant ⟪∇ω(x^k), x^k⟫ - ω(x^k).

theorem isMinOn_mirror_descent_update_objective_iff_isMinOn_bregman_form {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] (ω : EEReal) (C : Set E) (xk xNext gradf : E) (t : ) :
xNext C IsMinOn (mirror_descent_update_objective (fun (y : E) => (ω y).toReal) xk gradf t) C xNext xNext C IsMinOn (fun (x : E) => inner (t gradf) x + B[ω] x xk) C xNext

The canonical Chapter 9 one-step owner and the textbook Bregman-form objective have the same minimizers on C.

theorem effectiveDomain_addIndicator_eq_of_isBregmanPotentialOn {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] {ω : EEReal} {C : Set E} {σ : } ( : IsBregmanPotentialOn ω C σ) :
effective_domain (ω + δ_ C) = C

Helper for Text 9.4: for a Bregman potential on C, the constrained potential ω + δ_ C has effective domain exactly C.

theorem toReal_addIndicator_eq_of_mem {E : Type u} {ω : EEReal} {C : Set E} {xk : E} (hxC : xk C) :
((ω + δ_ C) xk).toReal = (ω xk).toReal

Helper for Text 9.4: on feasible points, the constrained potential ω + δ_ C has the same real value as ω.

theorem constrainedPotential_proper_of_mem {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] {ω : EEReal} {C : Set E} {σ : } ( : IsBregmanPotentialOn ω C σ) {x : E} (hx : x C) :

Helper for Text 9.4: once one feasible point is known, the constrained potential ω + δ_ C is proper.

theorem mirrorDescentBregmanObjective_eq_constrainedPotential_minus_pairing {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {ω : EEReal} {C : Set E} {xk gradf : E} {t : } {x : E} (hx : x C) :
inner (t gradf) x + B[ω] x xk = ((ω + δ_ C) x).toReal - inner (gradient (fun (y : E) => (ω y).toReal) xk - t gradf) x + (inner (gradient (fun (y : E) => (ω y).toReal) xk) xk - (ω xk).toReal)

Helper for Text 9.4: on feasible points, the Bregman-form mirror objective is the constrained potential minus the dual pairing with ∇ω(x^k) - t g_f, up to an x-independent constant.

theorem constrainedPotentialGap_iff_mirrorDescentComparison {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {ω : EEReal} {C : Set E} {σ : } {xk xNext gradf : E} {t : } ( : IsBregmanPotentialOn ω C σ) {y : E} (hxNext : xNext C) (hy : y C) :
(inner (gradient (fun (z : E) => (ω z).toReal) xk - t gradf) (y - xNext)) (ω + δ_ C) y - (ω + δ_ C) xNext inner (t gradf) xNext + B[ω] xNext xk inner (t gradf) y + B[ω] y xk

Helper for Text 9.4: on feasible points, the constrained-potential support-gap inequality is exactly the mirror-descent comparison.

theorem constrainedPotential_properClosedConvex {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] {ω : EEReal} {C : Set E} {σ : } {xk : E} (hC_closed : IsClosed C) ( : IsBregmanPotentialOn ω C σ) (hxk : xk C subdifferential_domain ω) :
IsProperExtendedRealFunction (ω + δ_ C) LowerSemicontinuous (ω + δ_ C) is_convex_function (ω + δ_ C)

Helper for Text 9.4: the constrained potential ω + δ_ C is proper, lower semicontinuous, and convex under the standing Bregman-potential and closed-feasible-set assumptions.

theorem eval_mem_subdifferential_conjugate_of_mem_subdifferential {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] {f : EEReal} (hf_proper : IsProperExtendedRealFunction f) {x : E} {y : Module.Dual E} (hy : y f(x)) :
(Module.Dual.eval E) x conjugate_function f(y)

Helper for Text 9.4: a primal subgradient of f at x yields the bidual evaluation subgradient of conjugate_function f at the same dual vector.

theorem mem_subdifferential_of_eval_mem_subdifferential_conjugate {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {f : EEReal} (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hf_convex : is_convex_function f) {x : E} {y : Module.Dual E} (hy : (Module.Dual.eval E) x conjugate_function f(y)) :
y f(x)

Helper for Text 9.4: under proper/closed/convex hypotheses, conjugate-side evaluation subgradient membership transports back to primal subgradient membership.

theorem conjugateFunctionFiniteEverywhere_of_proper_closed_strongConvexOn {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {σ : } {f : EEReal} ( : 0 < σ) (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hstrong : StrongConvexOn (effective_domain f) σ fun (x : E) => (f x).toReal) (y : Module.Dual E) :
conjugate_function f y conjugate_function f y <

Helper for Text 9.4: strong convexity of f makes its conjugate finite at every dual vector, transported back from the continuous-dual theorem in Chapter 5.

theorem differentiableAt_primalConjugate_of_proper_closed_strongConvexOn {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {σ : } {f : EEReal} ( : 0 < σ) (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hstrong : StrongConvexOn (effective_domain f) σ fun (x : E) => (f x).toReal) (g : E) :
DifferentiableAt (fun (y : E) => (f y).toReal) g

Helper for Text 9.4: the real-valued primal conjugate is differentiable at every point once the constrained potential is proper, closed, and strongly convex.

theorem dualMap_riesz_eval_eq_riesz {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] (x : E) :
(ContinuousLinearMap.coeLM ∘ₛₗ (InnerProductSpace.toDual E).toLinearEquiv).dualMap ((Module.Dual.eval E) x) = ((InnerProductSpace.toDualMap E) x)

Helper for Text 9.4: pulling back the bidual evaluation functional along the Riesz map gives the primal Riesz functional of the same vector.

theorem primalConjugate_convex {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {f : EEReal} (hconv : is_convex_function (conjugate_function f)) :

Helper for Text 9.4: precomposing a convex conjugate with the Riesz map keeps the primal conjugate f∗ convex on E.

theorem eqAddIffLeftEqSubOfNeBot {a b c : EReal} (ha_ne_bot : a ) (hb_ne_bot : b ) (hc_ne_top : c ) :
c = a + b a = c - b

Helper for Text 9.4: for finite c and non- summands, the equality c = a + b can be rewritten as a = c - b.

theorem pairingEqAddConjugate_iff_memSubdifferential_ofProper {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] {f : EEReal} (hf_proper : IsProperExtendedRealFunction f) (x : E) (y : Module.Dual E) :
(y x) = f x + conjugate_function f y y f(x)

Helper for Text 9.4: for a proper extended-real-valued function, Fenchel--Young equality is equivalent to primal subgradient membership.

theorem toDualMap_mem_subdifferential_primalConjugate_iff_eval_mem_subdifferential_conjugate {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {f : EEReal} (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hf_convex : is_convex_function f) (hfinite : ∀ (z : E), f z f z < ) {x g : E} :
((InnerProductSpace.toDualMap E) x) fun (y : E) => (f y).toReal(g) (Module.Dual.eval E) x conjugate_function f(((InnerProductSpace.toDualMap E) g))

Helper for Text 9.4: after normalizing the primal conjugate through the Riesz map, a subgradient of x ↦ ((f∗ x).toReal : EReal) at g is the same as the evaluation subgradient of conjugate_function f at toDualMap g.

theorem evalMemSubdifferentialConjugate_iff_memEuclideanSubdifferentialAt_primalConjugate {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {f : EEReal} (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hf_convex : is_convex_function f) (hfinite : ∀ (y : E), f y f y < ) {x g : E} :
(Module.Dual.eval E) x conjugate_function f(((InnerProductSpace.toDualMap E) g)) x euclideanSubdifferentialAt (fun (y : E) => (f y).toReal) g

Helper for Text 9.4: pulling the conjugate subdifferential back along the Riesz map turns evaluation functionals into Euclidean subgradients of the primal conjugate.

theorem mirror_descent_step_isMinOn_iff_dual_mem_subdifferential_add_indicator {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {ω : EEReal} {C : Set E} {σ : } {xk xNext gradf : E} {t : } ( : IsBregmanPotentialOn ω C σ) (hxk : xk C subdifferential_domain ω) :
xNext C IsMinOn (mirror_descent_update_objective (fun (y : E) => (ω y).toReal) xk gradf t) C xNext (InnerProductSpace.toDualMap E) (gradient (fun (y : E) => (ω y).toReal) xk - t gradf) ∂ₛω + δ_ C (xNext)

Helper for Text 9.4: if hω : IsBregmanPotentialOn ω C σ and x^k ∈ C ∩ dom(∂ ω), then the Chapter 9 mirror-descent step, encoded in Lean as x⁺ ∈ C ∧ IsMinOn (mirror_descent_update_objective ((fun y ↦ (ω y).toReal)) x^k g_f t) C x⁺, is equivalent to the constrained subgradient condition ∇ω(x^k) - t g_f ∈ ∂ (ω + δ_C)(x⁺).

theorem mirror_descent_step_isMinOn_iff_eq_gradient_conjugate_add_indicator {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [FiniteDimensional E] {ω : EEReal} {C : Set E} {σ : } {xk xNext gradf : E} {t : } (hC_closed : IsClosed C) ( : IsBregmanPotentialOn ω C σ) (hxk : xk C subdifferential_domain ω) :
xNext C IsMinOn (mirror_descent_update_objective (fun (y : E) => (ω y).toReal) xk gradf t) C xNext xNext = gradient (fun (y : E) => ((ω + δ_ C) y).toReal) (gradient (fun (y : E) => (ω y).toReal) xk - t gradf)

Text 9.4: letting ω̃ = ω + δ_C, if C is closed, hω : IsBregmanPotentialOn ω C σ, and x^k ∈ C ∩ dom(∂ ω), then the mirror-descent update step, encoded in Lean as x⁺ ∈ C ∧ IsMinOn (mirror_descent_update_objective ((fun y ↦ (ω y).toReal)) x^k g_f t) C x⁺, is equivalent to the source-facing conjugate formula x⁺ = ∇ ω̃∗(∇ω(x^k) - t g_f), expressed in Lean via the real-valued restriction of ω̃∗.