Documentation

FirstOrderMethodsOptimization_Beck_2017.Chap09.Text_9_8

@[reducible, inline]
noncomputable abbrev uniform_simplex_point {n : } (hn : 0 < n) :
(WithLp.toLp 2 '' stdSimplex (Fin n))

The canonical uniform initial point x⁰ = (1 / n) e in the Euclidean simplex toLp 2 '' stdSimplex ℝ (Fin n), obtained by transporting stdSimplex.barycenter, for n > 0.

Instances For
    @[simp]
    theorem coe_uniform_simplex_point {n : } (hn : 0 < n) :
    (uniform_simplex_point hn) = WithLp.toLp 2 fun (x : Fin n) => 1 / n

    Coercing the uniform simplex point to the ambient Euclidean space gives the transported constant vector with coordinates 1 / n.

    theorem bregmanDistance_halfSquaredNorm_eq_half_norm_sub_sq {n : } (x y : EuclideanSpace (Fin n)) :
    B[fun (z : EuclideanSpace (Fin n)) => z ^ 2 / 2] x y = x - y ^ 2 / 2

    Helper for Text 9.8: for ω(x) = ‖x‖² / 2, the Bregman distance is ‖x - y‖² / 2.

    theorem norm_sq_sub_le_two_of_mem_simplex {n : } {x y : EuclideanSpace (Fin n)} (hx : x WithLp.toLp 2 '' stdSimplex (Fin n)) (hy : y WithLp.toLp 2 '' stdSimplex (Fin n)) :
    x - y ^ 2 2

    Helper for Text 9.8: two transported simplex points are at squared Euclidean distance at most 2.

    theorem mem_subdifferential_domain_halfSquaredNorm {n : } (z : EuclideanSpace (Fin n)) :
    z subdifferential_domain (Function.toEReal fun (z : EuclideanSpace (Fin n)) => z ^ 2 / 2)

    Helper for Text 9.8: the quadratic mirror map is subdifferentiable everywhere.

    instance half_squared_norm_isBregmanPotentialOn_simplex {n : } :
    IsBregmanPotentialOn (Function.toEReal fun (z : EuclideanSpace (Fin n)) => z ^ 2 / 2) (WithLp.toLp 2 '' stdSimplex (Fin n)) 1

    The Euclidean mirror map ω(x) = ‖x‖² / 2 is a Bregman potential with modulus 1 on the transported simplex Δ.

    theorem half_squared_norm_bregman_le_one_of_mem_simplex {n : } (hn : 0 < n) {x : EuclideanSpace (Fin n)} (hx : x WithLp.toLp 2 '' stdSimplex (Fin n)) :
    B[fun (z : EuclideanSpace (Fin n)) => z ^ 2 / 2] x (uniform_simplex_point hn) 1

    On the simplex, the Euclidean Bregman distance to the uniform initialization x⁰ = (1 / n)e is bounded by 1. This is the simplex specialization of the textbook constant Θ₀ = 1.

    theorem euclidean_simplex_mirror_descent_best_value_gap_le {n : } {f : EuclideanSpace (Fin n)EReal} {XStar : Set (EuclideanSpace (Fin n))} {fOpt : } (h_bound : SubgradientNormBoundOn f (WithLp.toLp 2 '' stdSimplex (Fin n))) (hn : 0 < n) {x g : EuclideanSpace (Fin n)} {t : } (h_problem : IsConstrainedConvexProblem f (WithLp.toLp 2 '' stdSimplex (Fin n)) XStar fOpt) (h_traj : is_mirror_descent_trajectory (fun (y : EuclideanSpace (Fin n)) => (f y).toReal) (fun (z : EuclideanSpace (Fin n)) => z ^ 2 / 2) (WithLp.toLp 2 '' stdSimplex (Fin n)) x g t) {xStar : EuclideanSpace (Fin n)} (hxStar : xStar XStar) (hx0 : x 0 = (uniform_simplex_point hn)) {N : } (h_stepsize : ∀ (k : Fin (N + 1)), t k = 2 / (h_bound.L_f * (N + 1))) :
    best_achieved_function_value (fun (y : EuclideanSpace (Fin n)) => (f y).toReal) x N - fOpt 2 * h_bound.L_f / (N + 1)

    Text 9.8: for mirror descent on the Euclidean simplex with mirror map ω(x) = ‖x‖₂² / 2, uniform initialization x⁰ = (1 / n)e, and constant stepsizes t_k = √2 / (L_{f,2} √(N + 1)) on the first N + 1 iterations, the running-best objective gap is bounded by √2 L_{f,2} / √(N + 1), with L_{f,2} represented here by h_bound.L_f.

    theorem projected_subgradient_method_is_mirror_descent_trajectory {n : } {f : EuclideanSpace (Fin n)EReal} {XStar : Set (EuclideanSpace (Fin n))} {fOpt : } (g : (WithLp.toLp 2 '' stdSimplex (Fin n))EuclideanSpace (Fin n)) (t : ) (hn : 0 < n) (h_problem : IsConstrainedConvexProblem f (WithLp.toLp 2 '' stdSimplex (Fin n)) XStar fOpt) (h_subgrad : ∀ (k : ), g k (projected_subgradient_method (WithLp.toLp 2 '' stdSimplex (Fin n)) g t (uniform_simplex_point hn) k) euclideanSubdifferentialAt (fun (y : EuclideanSpace (Fin n)) => (f y).toReal) (projected_subgradient_method_iterate (WithLp.toLp 2 '' stdSimplex (Fin n)) g t (uniform_simplex_point hn) k)) (h_stepsize_pos : ∀ (k : ), 0 < t k) :
    is_mirror_descent_trajectory (fun (y : EuclideanSpace (Fin n)) => (f y).toReal) (fun (z : EuclideanSpace (Fin n)) => z ^ 2 / 2) (WithLp.toLp 2 '' stdSimplex (Fin n)) (projected_subgradient_method_iterate (WithLp.toLp 2 '' stdSimplex (Fin n)) g t (uniform_simplex_point hn)) (fun (k : ) => g k (projected_subgradient_method (WithLp.toLp 2 '' stdSimplex (Fin n)) g t (uniform_simplex_point hn) k)) t

    The projected-subgradient iterates on the Euclidean simplex, started at the uniform point, form the specialized mirror-descent trajectory for ω(x) = ‖x‖₂² / 2. This is the explicit Text 9.5 bridge between the Chapter 8 recursive iterates and the Chapter 9 owner is_mirror_descent_trajectory.