Documentation

FirstOrderMethodsOptimization_Beck_2017.Chap06.Theorem_6_63

noncomputable def outer_moreau_penalty {E : Type u} [NormedAddCommGroup E] (f : EEReal) (μ : PosReal) (x : E) :
EEReal

Helper for Theorem 6.63: the outer penalized objective whose unique minimizer is the scaled proximal point of f at x.

Instances For
    @[simp]
    theorem outer_moreau_penalty_apply {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) (x y : E) :
    outer_moreau_penalty f μ x y = f y + (1 / (2 * (μ + 1)) * x - y ^ 2)

    Evaluating the outer penalty from Theorem 6.63 gives the expected sum of f and the (μ + 1)⁻¹ quadratic term.

    theorem iInf_add_real_eq {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (g : EEReal) (c : ) :
    (⨅ (y : E), g y) + c = ⨅ (y : E), g y + c

    Helper for Theorem 6.63: adding a finite real constant commutes with the infimum defining a Moreau envelope.

    theorem lineMap_sub_right_for_moreau {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (μ : PosReal) (x y : E) :
    (AffineMap.lineMap x y) (1 / (μ + 1)) - y = (μ / (μ + 1)) (x - y)

    Helper for Theorem 6.63: the affine point minimizing the inner quadratic subproblem lies on the segment from x to y, and its displacement from y is the expected scaled copy of x - y.

    theorem lineMap_sub_left_for_moreau {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (μ : PosReal) (x y : E) :
    (AffineMap.lineMap x y) (1 / (μ + 1)) - x = (1 / (μ + 1)) (y - x)

    Helper for Theorem 6.63: the same affine interpolation point has displacement from x equal to the textbook weight 1 / (μ + 1) times y - x.

    theorem scaled_square_factor_for_moreau (μ : PosReal) (n : ) :
    1 / (2 * μ) * (μ / (μ + 1) * n) ^ 2 + 1 / 2 * (1 / (μ + 1) * n) ^ 2 = 1 / (2 * (μ + 1)) * n ^ 2

    Helper for Theorem 6.63: after evaluating the inner quadratic minimizer, the two endpoint quadratic terms collapse to the single outer penalty ‖x - y‖² / (2 (μ + 1)).

    theorem cross_factor_for_moreau (μ : PosReal) (r : ) :
    1 / μ * (μ / (μ + 1) * r) - 1 / (μ + 1) * r = 0

    Helper for Theorem 6.63: the cross term in the completed-square expansion vanishes because the inner quadratic minimizer satisfies the first-order balance equation.

    theorem residual_factor_for_moreau (μ : PosReal) (n : ) :
    1 / (2 * μ) * n + 1 / 2 * n = (μ + 1) / (2 * μ) * n

    Helper for Theorem 6.63: the remaining quadratic term after completing the square has coefficient (μ + 1) / (2 μ).

    theorem transport_lineMap_eq_of_moreau {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (μ : PosReal) (x y : E) :
    x + (μ + 1) ((AffineMap.lineMap x y) (1 / (μ + 1)) - x) = y

    Helper for Theorem 6.63: the affine transport sends the textbook candidate lineMap x y (1 / (μ + 1)) back to y.

    theorem transport_sub_eq_smul_of_moreau {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (μ : PosReal) (x v y : E) :
    x + (μ + 1) (v - x) - y = (μ + 1) (v - (AffineMap.lineMap x y) (1 / (μ + 1)))

    Helper for Theorem 6.63: the affine transport T v = x + (μ + 1) • (v - x) converts the residual displacement from lineMap x y (1 / (μ + 1)) into the direct displacement from y.

    theorem transported_square_factor_for_moreau (μ : PosReal) (n : ) :
    1 / (2 * (μ * (μ + 1))) * ((μ + 1) * n) ^ 2 = (μ + 1) / (2 * μ) * n ^ 2

    Helper for Theorem 6.63: after the affine transport T, the residual Moreau-envelope coefficient is exactly the completed-square coefficient (μ + 1) / (2 μ).

    theorem moreau_two_stage_quadratic_eq_completed_square {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (μ : PosReal) (x v y : E) :
    1 / (2 * μ) * v - y ^ 2 + 1 / 2 * v - x ^ 2 = 1 / (2 * (μ + 1)) * x - y ^ 2 + (μ + 1) / (2 * μ) * v - (AffineMap.lineMap x y) (1 / (μ + 1)) ^ 2

    Helper for Theorem 6.63: the two-stage quadratic objective in the Moreau-envelope proof is a completed square centered at lineMap x y (1 / (μ + 1)).

    theorem mem_scaled_prox_iff_isMinOn_moreau_penalty {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) {x u : E} :
    u prox[μ f] x IsMinOn (fun (v : E) => f v + (1 / (2 * μ) * x - v ^ 2)) Set.univ u

    Helper for Theorem 6.63: minimizing the scaled proximal objective is equivalent to minimizing the outer penalty y ↦ f y + ‖x - y‖² / (2 μ).

    theorem scaled_outer_moreau_penalty_eq_scaled_proximal_objective {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) (x y : E) :
    ((μ + 1) outer_moreau_penalty f μ x) y = proximal_objective ((μ + 1) f) x y

    Helper for Theorem 6.63: scaling the outer penalty by μ + 1 turns it into the proximal objective of the scaled function ((μ + 1) • f) at x.

    theorem proximal_objective_toReal_eq_on_effective_domain {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (g : EEReal) (x z : E) (hbot : g z ) (hz : z effective_domain g) :
    (proximal_objective g x z).toReal = (g z).toReal + 1 / 2 * z - x ^ 2

    Helper for Theorem 6.63: on the effective domain of g, the proximal objective has the expected real-valued form g.toReal + (1 / 2) ‖z - x‖².

    theorem translated_quadratic_penalty_is_convex_function {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (x : E) (a : ) (ha : 0 a) :
    is_convex_function fun (z : E) => (a * z - x ^ 2)

    Helper for Theorem 6.63: every translated quadratic penalty with a nonnegative coefficient is convex as an extended-real-valued function.

    theorem proximal_objective_proper_closed_convex_of_proper_closed_convex {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (g : EEReal) (x : E) (hg_proper : IsProperExtendedRealFunction g) (hg_closed : LowerSemicontinuous g) (hg_convex : is_convex_function g) :

    Helper for Theorem 6.63: adding the quadratic penalty keeps a proper closed convex function proper, closed, and convex.

    theorem two_stage_moreau_integrand_eq_outer_kernel {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) (x v y : E) :
    f y + (1 / (2 * μ) * v - y ^ 2) + (1 / 2 * v - x ^ 2) = outer_moreau_penalty f μ x y + (1 / (2 * (μ * (μ + 1))) * x + (μ + 1) (v - x) - y ^ 2)

    Helper for Theorem 6.63: for fixed y, the two-stage Moreau integrand equals the outer penalty plus the transported residual kernel.

    theorem proximal_objective_moreau_envelope_eq_outer_moreau_envelope {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) (x v : E) :
    have ν := μ * (μ + 1), ; have T := fun (z : E) => x + (μ + 1) (z - x); proximal_objective M[μ, f] x v = M[ν, outer_moreau_penalty f μ x] (T v)

    Helper for Theorem 6.63: the proximal objective of the Moreau envelope is the Moreau envelope of the outer penalty after the affine transport v ↦ x + (μ + 1) • (v - x).

    theorem outer_moreau_penalty_eq_inv_scaled_proximal_objective {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) (x : E) :
    outer_moreau_penalty f μ x = (μ + 1)⁻¹ fun (z : E) => proximal_objective ((μ + 1) f) x z

    Helper for Theorem 6.63: the outer penalty is the positive inverse scaling of the proximal objective of ((μ + 1) • f) at x.

    theorem outer_moreau_penalty_proper_closed_convex {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) {x : E} (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hf_convex : is_convex_function f) :

    Helper for Theorem 6.63: the outer penalty remains proper, closed, and convex under the same owner hypotheses as f.

    theorem moreau_envelope_isMinOn_of_isMinOn {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] {G : EEReal} {ν : PosReal} {u : E} (hu : IsMinOn G Set.univ u) :
    IsMinOn M[ν, G] Set.univ u

    Helper for Theorem 6.63: a minimizer of the outer penalty remains a minimizer after Moreau smoothing, because the smoothing infimum is attained at the same point.

    theorem moreau_minimizer_eq_unique_outer_minimizer_of_value_match {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] {G : EEReal} (hG_proper : IsProperExtendedRealFunction G) (hG_closed : LowerSemicontinuous G) (hG_convex : is_convex_function G) {ν : PosReal} {u y : E} (hu : IsMinOn G Set.univ u) (huniq : ∀ (z : E), IsMinOn G Set.univ zz = u) (hy : IsMinOn M[ν, G] Set.univ y) (hvalue : M[ν, G] y = G u) :
    y = u

    Helper for Theorem 6.63: if a Moreau-envelope minimizer already attains the unique minimum value of the outer function, then the envelope minimizer is the unique outer minimizer.

    theorem eq_of_isMinOn_moreau_envelope_of_unique_outer_penalty_minimizer {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) {x : E} (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hf_convex : is_convex_function f) {ν : PosReal} {u y : E} (hu : IsMinOn (outer_moreau_penalty f μ x) Set.univ u) (huniq : ∀ (z : E), IsMinOn (outer_moreau_penalty f μ x) Set.univ zz = u) (hy : IsMinOn M[ν, outer_moreau_penalty f μ x] Set.univ y) :
    y = u

    Helper for Theorem 6.63: if the outer penalty has the unique minimizer u, then every minimizer of its Moreau envelope is also u.

    theorem lineMap_transport_self {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (μ : PosReal) (x v : E) :
    (AffineMap.lineMap x (x + (μ + 1) (v - x))) (1 / (μ + 1)) = v

    Helper for Theorem 6.63: applying lineMap to the transported point x + (μ + 1) • (v - x) recovers v.

    theorem prox_moreau_envelope_eq_singleton_of_scaled_prox_eq_singleton_strong {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hf_convex : is_convex_function f) {x u : E} (hprox : prox[(μ + 1) f] x = {u}) :
    prox[M[μ, f]] x = {(AffineMap.lineMap x u) (1 / (μ + 1))}

    Helper for Theorem 6.63: once the scaled proximal point of f at x is the singleton {u}, the proximal set of the Moreau envelope is the singleton at the transported point lineMap x u (1 / (μ + 1)).

    theorem prox_moreau_envelope_eq_singleton_of_proper_closed_convex {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hf_convex : is_convex_function f) (x : E) :
    ∃ (u : E), prox[(μ + 1) f] x = {u} prox[M[μ, f]] x = {(AffineMap.lineMap x u) (1 / (μ + 1))}

    Theorem 6.63: if f is a proper closed convex extended-real-valued function, then at every point x there is a unique proximal point u of the scaled function (μ + 1) f, and the proximal set of the Moreau envelope M[μ, f] at x is the singleton containing lineMap x u (1 / (μ + 1)). This is the chapter's set-valued rendering of the textbook formula prox_{M_f^μ}(x) = x + (1 / (μ + 1)) (prox_{(μ + 1) f}(x) - x), stated without promoting a chosen proximal point to primitive data.

    theorem prox_moreau_envelope_eq_singleton_of_scaled_prox_eq_singleton {E : Type u} [NormedAddCommGroup E] [InnerProductSpace E] [ProperSpace E] (f : EEReal) (μ : PosReal) (hf_proper : IsProperExtendedRealFunction f) (hf_closed : LowerSemicontinuous f) (hf_convex : is_convex_function f) {x u : E} (hprox : prox[(μ + 1) f] x = {u}) :
    prox[M[μ, f]] x = {(AffineMap.lineMap x u) (1 / (μ + 1))}

    If the proximal set of the scaled function (μ + 1) f at x is the singleton {u}, then the proximal set of the Moreau envelope at x is the singleton containing lineMap x u (1 / (μ + 1)), equivalently the weighted average x + (1 / (μ + 1)) • (u - x). Under the same non- hypothesis and μ > 0, this is the chapter's singleton-valued rendering of the textbook identity prox_{M_f^μ}(x) = (μ x + u) / (μ + 1).