Documentation

IntroductoryLecturesOnConvexOptimization_Nesterov_2004.Chap07.Lemma_7_11

def affineBarrierRegularizedPayoff {E : Type u} [AddCommGroup E] [Module E] (x0 : E) (β : ) ( : E →ᵃ[] ) (F : E) (x : E) :

The barrier-regularized affine payoff ℓ(x) - β (F(x) - F(x₀)) attached to an affine functional , a barrier term F, and a base point x₀.

Instances For
    theorem affineBarrierRegularizedPayoff_def {E : Type u} [AddCommGroup E] [Module E] (x0 : E) (β : ) ( : E →ᵃ[] ) (F : E) (x : E) :
    affineBarrierRegularizedPayoff x0 β F x = x - β * (F x - F x0)

    Expanding affineBarrierRegularizedPayoff x₀ β ℓ F x gives the affine value ℓ(x) minus the barrier penalty β (F(x) - F(x₀)).

    theorem affineBarrierRegularizedPayoff_max_le_affine_max {E : Type u} [AddCommGroup E] [Module E] {P : Set E} {F : E} { : E →ᵃ[] } {x0 xStar xBeta : E} {β : } ( : 0 < β) (hx0 : x0 constrainedArgmin P F) (hxStar_mem : xStar P) (hxStar_max : IsMaxOn (⇑) P xStar) (hxBeta_mem : xBeta P) (hxBeta_max : IsMaxOn (affineBarrierRegularizedPayoff x0 β F) P xBeta) :
    affineBarrierRegularizedPayoff x0 β F xBeta xStar

    Lemma 7.11 (1): if xBeta belongs to P and maximizes the barrier-regularized affine payoff there, xStar belongs to P and maximizes there, and x₀ minimizes F on P, then ℓ⋆(β) ≤ ℓ⋆.

    theorem affine_value_sub_base_on_segment {E : Type u} [AddCommGroup E] [Module E] { : E →ᵃ[] } {x0 xStar : E} (α : ) :
    (x0 + α (xStar - x0)) - x0 = α * ( xStar - x0)

    Helper for Lemma 7.11: along the segment from x₀ to xStar, the affine increment of matches the scalar increment α (ℓ(xStar) - ℓ(x₀)).

    theorem base_value_le_affineBarrierRegularizedPayoff_max {E : Type u} [AddCommGroup E] [Module E] {P : Set E} {F : E} { : E →ᵃ[] } {x0 xBeta : E} {β : } (hx0 : x0 constrainedArgmin P F) (hxBeta_max : IsMaxOn (affineBarrierRegularizedPayoff x0 β F) P xBeta) :
    x0 affineBarrierRegularizedPayoff x0 β F xBeta

    Helper for Lemma 7.11: the attained regularized maximum is at least the base affine value ℓ(x₀).

    theorem regularized_gap_bound_along_segment {E : Type u} [AddCommGroup E] [Module E] {P : Set E} {F : E} { : E →ᵃ[] } {x0 xStar xBeta : E} {β v : } ( : 0 < β) (hxStar_mem : xStar P) (hxBeta_max : IsMaxOn (affineBarrierRegularizedPayoff x0 β F) P xBeta) (hsegment_mem : ∀ ⦃x : E⦄, x P∀ ⦃α : ⦄, α Set.Ico 0 1x0 + α (x - x0) P) (hF_segment : ∀ ⦃x : E⦄, x P∀ ⦃α : ⦄, α Set.Ico 0 1F (x0 + α (x - x0)) F x0 - v * Real.log (1 - α)) {α : } ( : α Set.Ico 0 1) :
    α * ( xStar - x0) + β * v * Real.log (1 - α) affineBarrierRegularizedPayoff x0 β F xBeta - x0

    Helper for Lemma 7.11: every admissible segment point from x₀ to xStar yields the common regularized-gap inequality used in both quantitative estimates.

    theorem affineMax_le_affineBarrierRegularizedPayoff_max_add_logTerm {E : Type u} [AddCommGroup E] [Module E] {P : Set E} {F : E} { : E →ᵃ[] } {x0 xStar xBeta : E} {β v : } ( : 0 < β) (hv : 0 < v) (hx0 : x0 constrainedArgmin P F) (hxStar_mem : xStar P) (hxStar_max : IsMaxOn (⇑) P xStar) (hxBeta_mem : xBeta P) (hxBeta_max : IsMaxOn (affineBarrierRegularizedPayoff x0 β F) P xBeta) (hsegment_mem : ∀ ⦃x : E⦄, x P∀ ⦃α : ⦄, α Set.Ico 0 1x0 + α (x - x0) P) (hF_segment : ∀ ⦃x : E⦄, x P∀ ⦃α : ⦄, α Set.Ico 0 1F (x0 + α (x - x0)) F x0 - v * Real.log (1 - α)) :
    xStar affineBarrierRegularizedPayoff x0 β F xBeta + β * v * (1 + max (Real.log (( xStar - x0) / (β * v))) 0)

    Lemma 7.11 (2): under the same attained-maximizer setup, if every segment from x₀ to a point of P stays in P and satisfies the displayed barrier estimate, then ℓ⋆ ≤ ℓ⋆(β) + β v (1 + [log ((ℓ⋆ - ℓ₀) / (β v))]_+).

    theorem neg_log_one_sub_le_div_of_mem_Ico {α : } ( : α Set.Ico 0 1) :
    -Real.log (1 - α) α / (1 - α)

    Helper for Lemma 7.11: on the interval 0 ≤ α < 1, the singular logarithmic term is bounded by the rational term α / (1 - α).

    theorem affineMax_sub_base_le_sq_sqrt_add_sqrt_of_affineBarrierRegularizedPayoff_max {E : Type u} [AddCommGroup E] [Module E] {P : Set E} {F : E} { : E →ᵃ[] } {x0 xStar xBeta : E} {β v : } ( : 0 < β) (hv : 0 < v) (hx0 : x0 constrainedArgmin P F) (hxStar_mem : xStar P) (hxStar_max : IsMaxOn (⇑) P xStar) (hxBeta_mem : xBeta P) (hxBeta_max : IsMaxOn (affineBarrierRegularizedPayoff x0 β F) P xBeta) (hsegment_mem : ∀ ⦃x : E⦄, x P∀ ⦃α : ⦄, α Set.Ico 0 1x0 + α (x - x0) P) (hF_segment : ∀ ⦃x : E⦄, x P∀ ⦃α : ⦄, α Set.Ico 0 1F (x0 + α (x - x0)) F x0 - v * Real.log (1 - α)) :
    xStar - x0 ((affineBarrierRegularizedPayoff x0 β F xBeta - x0) + (β * v)) ^ 2

    Lemma 7.11 (3): under the same attained-maximizer and barrier-segment hypotheses, the affine gap from x₀ to the maximizer xStar is bounded by (sqrt (ℓ⋆(β) - ℓ₀) + sqrt (β v))².