Chapter 05 — Polynomial-Time Interior-Point Methods
Proofs omitted. Hover over highlighted terms to see their Lean types.
Definition 5.0.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.1: a convex minimization problem with inequality constraints on ℝⁿ consists of a real-valued objective function f₀ and constraint functions fⱼ, j = 1, …, m, all convex on the whole space, representing the problem of minimizing f₀(x) subject to the inequalities fⱼ(x) ≤ 0.
variable {n m : ℕ}
structure ConvexInequalityConstrainedMinimizationProblem (n m : ℕ)
extends LagrangianProblem (EuclideanSpace ℝ (Fin n)) m where
/-- The objective is convex on all of `ℝⁿ`. -/
objective_convex : ConvexOn ℝ Set.univ objective
/-- Each constraint function is convex on all of `ℝⁿ`. -/
constraints_convex (j : Fin m) : ConvexOn ℝ Set.univ (constraints j)Algorithm 5.0.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
open Matrix
variable {ι R : Type*} [Fintype ι] [DecidableEq ι] [CommSemiring R]
theorem choleskySolve_solves_system
(A L : Matrix ι ι R) (b : ι → R) (hfactor : A = L * Lᵀ) (hL : Invertible L) :
A *ᵥ (⅟(Lᵀ) *ᵥ (⅟L *ᵥ b)) = b
Definition 5.0.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.3: an unconstrained twice-differentiable minimization problem is a set-constrained minimization problem whose canonical zero-constraint owner is unconstrained and whose objective is differentiable with differentiable gradient on the ambient space.
universe u
open scoped Gradient
open FunctionalConstraintsMinimizationProblem
namespace SetConstrainedMinimizationProblem
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable (problem : SetConstrainedMinimizationProblem E)
#check (
¬ problem.toFunctionalConstraintsMinimizationProblem.IsConstrained ∧
Differentiable ℝ problem.objective ∧
Differentiable ℝ (∇ problem.objective)
)Algorithm 5.0.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Algorithm 5.0.4 recalls the canonical Newton method for unconstrained minimization as Newton's method for the stationarity equation ∇ f = 0.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable (f : E → ℝ) (x0 : E)
#check (NewtonSystem.Method (∇ f) x0)Theorem 5.0.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.0.5: if f : ℝⁿ → ℝ is twice continuously differentiable, x* is a critical point, ∇² f(x*) - μ I is positive semidefinite, and the Hessian is M-Lipschitz, then the canonical local Newton orbit started in the closed ball of radius 2 μ / (3 M) around x* stays in that ball, has nonsingular Hessian at every iterate, and satisfies the standard quadratic one-step error estimate.
open scoped Gradient
variable {n : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
theorem newton_method_has_local_quadratic_convergence
(f : E → ℝ) (xStar x0 : E) {μ : ℝ} {M : NNRealˣ}
(hf : f ∈ C22[M])
(hcrit : ∇ f xStar = 0)
(hμ : 0 < μ)
(hessian_lower : (∇² f xStar - μ • (1 : Mat)).PosSemidef)
(hx0 : ‖x0 - xStar‖ ≤ localQuadraticNewtonRadius μ M) :
let traj :=
localQuadraticNewtonOrbit hμ hf hcrit
(hessianMatrix_lower_isPositive hessian_lower) hx0
(∀ k, ‖traj k - xStar‖ ≤ localQuadraticNewtonRadius μ M) ∧
(∀ k, (hessian f (traj k)).det ≠ 0) ∧
∀ k,
‖traj (k + 1) - xStar‖ ≤
((M : ℝ) * ‖traj k - xStar‖ ^ (2 : ℕ)) /
(2 * (μ - (M : ℝ) * ‖traj k - xStar‖))Definition 5.0.6
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.6: the pullback induced by x = By is exactly the canonical composition of f with the chapter's owner linear map attached to B.
variable {n : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
variable (f : E → ℝ) (B : Matrix (Fin n) (Fin n) ℝ)
#check (f ∘ B.toEuclideanLin : E → ℝ)Definition 5.0.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.7: the textbook class C_M^{2,2}(ℝⁿ) is represented in this project by the canonical owner HasLipschitzContinuousHessian M f, written on theorem surfaces as f ∈ C22[M]. Its defining consequences are the inherited C² regularity and the displayed Hessian Lipschitz estimate ‖∇² f(x) - ∇² f(y)‖ ≤ M ‖x - y‖.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {M : NNReal} {f : E → ℝ}
recall HasLipschitzContinuousHessianDefinition 5.0.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.8: the textbook matrix f'''(x)[u], viewed as the standard-basis matrix of the canonical operator fderiv ℝ (hessian f) x u. This is only a Euclidean matrix bridge; the owner remains the operator fderiv ℝ (hessian f) x u.
open scoped Gradient
variable {n : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "e" => EuclideanSpace.basisFun (Fin n) ℝ
abbrev thirdDerivativeMatrix (f : E → ℝ) (x u : E) : Matrix (Fin n) (Fin n) ℝ :=
LinearMap.toMatrixOrthonormal e (fderiv ℝ (hessian f) x u)Definition 5.0.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.9 recalls the canonical Hessian local norm owner.
open scoped Gradient
open scoped HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
recall hessianLocalNormDefinition 5.0.10
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.10: the directional slice of f at x along u is the univariate function t ↦ f (x + t • u), from which the first, second, and third directional derivatives at x in the direction u are taken at t = 0.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [NormedSpace ℝ E]
-- Semantic recall: `iteratedDeriv_vcomp_two`, `iteratedDeriv_vcomp_three`, and
-- `iteratedFDeriv_two_apply` are the mathlib bridge lemmas for the `C²`/`C³` slice identities.
def directionalSlice (f : E → ℝ) (x u : E) : ℝ → ℝ :=
fun t ↦ f (x + t • u)Definition 5.0.11
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.11 recalls the canonical Hessian local norm owner for the textbook primal local norm.
recall hessianLocalNormDefinition 5.0.12
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.12: the associated univariate function is the reciprocal of the square root of the Hessian quadratic form of f in the direction h, along the line t ↦ x + t • h, with domain consisting of those t for which x + t • h ∈ dom and the quadratic form is positive.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
def associatedUnivariateFunction (dom : Set E) (f : E → ℝ) (x h : E) :
associatedUnivariateFunctionDomain dom f x h → ℝ :=
(associatedUnivariateFunctionDomain dom f x h).restrict
(directionalSlice (fun y ↦ 1 / ‖h‖[f; y]) x h)Definition 5.0.13
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.13: the Dikin ellipsoid of f at x with radius r is the closed ball in the Hessian local norm at x.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
def dikinEllipsoid (f : E → ℝ) (x : E) (r : ℝ) : Set E :=
{y | ‖y - x‖[f; x] ≤ r}Definition 5.0.14
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.14 specializes the chapter owner to the textbook inverse-parameter Dikin neighborhood W⁰[f; x](1 / M_f).
universe u
open scoped DikinEllipsoidNotation
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable (f : E → ℝ) (x : E) (Mf : NNReal)
#check W⁰[f; x](1 / (Mf : ℝ))Proposition 5.0.15
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.15: if f is self-concordant on dom with parameter M_f, x ∈ dom, y ∈ dom f, r < 1 / M_f, and ‖y - x‖_x < r (encoded here as hxy : y ∈ W⁰[f; x](r)), then the Hessians at x and y are comparable in Loewner order by the factors (1 - M_f r)^2 and (1 - M_f r)⁻². The explicit hypothesis hy : y ∈ dom supplies the domain-membership half of the textbook Dikin ellipsoid condition y ∈ W(x; r).
open scoped DikinEllipsoidNotation Gradient HessianLocalNorm Topology ContDiff
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOnWith
theorem hessian_loewner_bounds_of_mem_openDikinEllipsoid
{dom : Set E} {Mf : NNReal} {f : E → ℝ}
(hself : IsSelfConcordantOnWith dom Mf f)
{x y : E} {r : ℝ} (hx : x ∈ dom) (hy : y ∈ dom) (hr : r < 1 / (Mf : ℝ))
(hxy : y ∈ W⁰[f; x](r)) :
((1 - (Mf : ℝ) * r) ^ (2 : ℕ)) • hessian f x ≤ hessian f y ∧
hessian f y ≤ ((1 - (Mf : ℝ) * r) ^ (2 : ℕ))⁻¹ • hessian f xDefinition 5.0.16
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
open scoped WithTopConvexAnalysis
universe u
variable {X : Type u} [AddCommMonoid X] [Module ℝ X]
variable (f : X → WithTop ℝ)
#check ConvexOn ℝ (dom f) (withTopRealPart f)Proposition 5.0.17
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.17: if the global lower Taylor remainder bound from Theorem_5_1_8.taylor_lower_bound_of_hessian_loewner_lower holds on an open domain dom for a C³ function f, with positive parameter M_f, then at every point x ∈ dom the chapter owner thirdDirectionalDerivative f x u is bounded above by 2 M_f ‖u‖_x^3.
open scoped Gradient HessianLocalNorm SelfConcordantAuxiliaryFunction
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem thirdDirectionalDerivative_le_of_global_remainder_lower_bound
{dom : Set E} {Mf : NNReal} {f : E → ℝ}
(hopen : IsOpen dom)
(hcont : ContDiffOn ℝ 3 f dom)
(hMf : 0 < Mf)
(hremainder :
∀ ⦃x y : E⦄ (_ : x ∈ dom) (_ : y ∈ dom),
(1 / (Mf : ℝ) ^ (2 : ℕ)) *
ω (selfConcordantOmegaArg Mf ‖y - x‖[f; x]
(neg_one_lt_mf_mul_of_nonneg
(hessianLocalNorm_nonneg f x (y - x)))) ≤
f y - f x - inner ℝ (∇ f x) (y - x))
(x u : E) (hx : x ∈ dom) :
thirdDirectionalDerivative f x u ≤
2 * (Mf : ℝ) * ‖u‖[f; x] ^ (3 : ℕ)
Definition 5.0.18
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.18: the partial-minimization value function is the canonical Chapter 3 infimal-projection owner partialInfProjection domΦ (Real.toEReal ∘ Φ), with real values recovered on its finite-value domain by extendedRealRealPart.
universe u v w
open scoped ConvexAnalysis
variable {E₁ : Type u} {E₂ : Type v}
variable (domΦ : Set (E₁ × E₂)) (Φ : E₁ × E₂ → ℝ) (x : E₁)
recall partialInfProjectionProposition 5.0.19
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.19 (1): on any neighborhood where y realizes the fiberwise minima of Φ over Q, the canonical real surface of the partial infimal projection agrees with the minimizing branch u ↦ Φ (u, y u).
open scoped ConstrainedArgmin ConvexAnalysis Gradient
open ContinuousLinearMap
universe u v
variable {E₁ : Type u} {E₂ : Type v}
variable [TopologicalSpace E₁]
theorem partialMinimizationObjective_eventuallyEq_of_eventually_argmin
(Q : Set (E₁ × E₂)) (Φ : E₁ × E₂ → ℝ) (x : E₁) (y : E₁ → E₂)
(hy_argmin : ∀ᶠ u in nhds x,
y u ∈ argmin[(Prod.mk u) ⁻¹' Q] (Φ ∘ Prod.mk u)) :
extendedRealRealPart (partialInfProjection Q (Real.toEReal ∘ Φ)) =ᶠ[nhds x]
fun u ↦ Φ (u, y u)Definition 5.0.20
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.20: when the Hessian of f at x is positive and invertible, the dual local norm of a covector g is the Chapter 4 dual norm attached to the Hessian bilinear form.
open InnerProductSpace
open Module LinearMap
open scoped BInducedNorm Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
abbrev dualLocalNorm (f : E → ℝ) (x : E) (hPos : (hessian f x).IsPositive)
(hInv : (hessian f x).IsInvertible) (g : StrongDual ℝ E) : ℝ :=
let _ : Fact (hessianBilin f x).IsSymm := ⟨hessianBilin_isSymm_of_isPositive hPos⟩
let _ : Fact (hessianBilin f x).toQuadraticMap.PosDef :=
⟨hessianBilin_posDef_of_isPositive_of_isInvertible hPos hInv⟩
(hessianBilin f x).dualNorm Fact.out g.toLinearMapDefinition 5.0.21
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.21 (1): the auxiliary function ω(t) = t - log (1 + t) on (-1, ∞).
def selfConcordantOmega : Set.Ioi (-1 : ℝ) → ℝ :=
fun t ↦ t - Real.log (1 + t)Proposition 5.0.21
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.21: if B is symmetric, nonnegative on diagonal values, and nondegenerate, then the pairing between a covector and a vector is bounded by the B-dual norm of the covector times the B-primal norm of the vector. The positive-definite owner data are obtained canonically from these textbook hypotheses.
universe u
open LinearMap
open scoped LinearMap.BilinForm.BInducedNorm
variable {E : Type u} [AddCommGroup E] [Module ℝ E]
variable [FiniteDimensional ℝ E]
theorem abs_apply_le_dualNorm_mul_primalSeminorm (B : LinearMap.BilinForm ℝ E)
(hB_nonneg : ∀ v : E, 0 ≤ B v v) (hB_symm : B.IsSymm) (hB_nondeg : B.Nondegenerate)
(g : Module.Dual ℝ E) (h : E) :
let hPos := B.posDef_of_nonneg_of_nondegenerate hB_nonneg hB_symm hB_nondegProposition 5.0.22
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.22 (1): the ambient real-function view of the Chapter 5 owner ω is convex on (-1, ∞).
open scoped SelfConcordantAuxiliaryFunction
theorem selfConcordantOmega_convexOn :
ConvexOn ℝ (Set.Ioi (-1 : ℝ))
(Function.extend Subtype.val ω 0)Definition 5.0.23
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.23, generalized from the textbook Euclidean setting: an unconstrained self-concordant minimization problem consists of a domain and an objective to be minimized over that domain, where the objective is self-concordant on the domain and has positive definite Hessian there. The ambient problem data are owned canonically by SetConstrainedMinimizationProblem, while self-concordance is expressed through the chapter owner IsSelfConcordantOnWith.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
structure UnconstrainedSelfConcordantMinimizationProblem (E : Type u)
[NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
extends SetConstrainedMinimizationProblem E where
/-- The self-concordance constant of the objective on the feasible set. -/
selfConcordanceConstant : NNReal
/-- The problem canonically supplies the quantitative self-concordance owner for its objective
on its feasible set. -/
toIsSelfConcordantOnWith :
IsSelfConcordantOnWith feasibleSet selfConcordanceConstant objective
/-- The problem canonically supplies positive-definite Hessian data for its objective on its
feasible set. -/
toHasPositiveDefiniteHessianOn : HasPositiveDefiniteHessianOn feasibleSet objectiveDefinition 5.0.24
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.24: when the Hessian of f at x is positive and invertible, the Newton decrement is the dual local norm of the gradient at x.
open InnerProductSpace
open scoped Gradient HessianDualLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
variable [FiniteDimensional ℝ E]
abbrev newtonDecrement (f : E → ℝ) (x : E)
(hPos : (hessian f x).IsPositive) (hInv : (hessian f x).IsInvertible) : ℝ :=
‖(toDual ℝ E) (∇ f x)‖*[f; x | hPos; hInv]Theorem 5.0.25
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.0.25: if an objective admits a recession ray from x whose values are bounded above by an affine function with negative slope σ, then the objective is not bounded below on the whole space. The owner-level conclusion is the canonical range statement ¬ BddBelow (Set.range f).
universe u
open scoped LevelSetNotation
variable {E : Type u} [AddCommGroup E] [Module ℝ E]
variable {f : E → ℝ}
theorem not_bddBelow_range_of_negative_recession_ray
{x d : E} {σ : ℝ} (hσ : σ < 0)
(hray : ∀ t : ℝ, 0 ≤ t → f (x + t • d) ≤ f x + t * σ) :
¬ BddBelow (Set.range f)Algorithm 5.0.26
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Algorithm 5.0.26 lies in the self-concordant damped-Newton domain. Sampled owner-style declarations: - selfConcordantNewtonNextPoint in Definition_5_2_1, the Chapter 5 owner of one-step self-concordant Newton updates; - DampedNewton.Method.IsSelfConcordant in Definition_5_2_1, the Chapter 5 refinement of the recursive damped Newton owner; - NewtonDecrement.ofDetNeZero in Definition_5_0_24, the canonical Chapter 5 owner for the Newton decrement at a domain point with nondegenerate Hessian; - DampedNewton.step and DampedNewton.Method in Chapter 1, the general damped-Newton bridge owners. Best owner abstraction: - source-facing: the .damped specialization of selfConcordantNewtonNextPoint and DampedNewton.Method.IsSelfConcordant; - core/canonical: the Chapter 5 owners selfConcordantNewtonNextPoint, DampedNewton.Method.IsSelfConcordant, DampedNewton.Method, and NewtonDecrement.ofDetNeZero; - bridge/view: the explicit damped step-size formula inside the inherited Chapter 1 method owner. Primitive data: - an objective f : E → ℝ; - a self-concordance parameter Mf; - a self-concordant domain dom; - a point x ∈ dom with nondegenerate Hessian. Derived API: - the Chapter 5 damped-step specialization selfConcordantNewtonNextPoint f Mf SelfConcordantNewtonVariant.damped x hx hH; - the direct Chapter 1 bridge DampedNewton.step f ⟨x, hH⟩ (1 / (1 + M_f λ_f(x))); - the Chapter 1 damped-Newton method together with its Chapter 5 self-concordant refinement. Source/core/bridge triage: - source-facing: the .damped Chapter 5 specializations; - core/canonical: the Chapter 5 self-concordant update and decrement owners; - bridge/view: the conversion to the Chapter 1 damped-Newton owner layer. Algorithm 5.0.26 is therefore centered on the Chapter 5 damped specialization itself. The Chapter 1 damped-Newton owner appears directly as the canonical recursive owner layer.
open scoped Gradient NewtonDecrement
open SelfConcordantNewtonVariant
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
recall selfConcordantNewtonNextPointDefinition 5.0.27
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.0.27 recalls the source-facing Fenchel-dual owner on ℝⁿ.
universe u
open scoped ConvexAnalysis WithTopConvexAnalysis
variable {n : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
recall fenchelDualProposition 5.0.28
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.28: the image of the gradient of the perspective-square function is the parabola in ℝ × ℝ cut out by g₁ = -(g₂)² / 4, where the gradient is taken on the genuine differentiability region τ > 0.
open scoped Gradient WithTopConvexAnalysis
local notation "P" => ℝ × ℝ
local notation "Q" => (perspectiveCone ℝ : Set P)
local notation "perspectiveSquare" =>
perspectiveTransform (fun y : ℝ ↦ y ^ (2 : ℕ))
local notation "F" =>
fenchelPrimalExtension Q perspectiveSquare
theorem perspectiveSquare_gradient_image_eq_parabola :
(fun z : P ↦ ∇ (withTopRealPart F) z) '' {z : P | 0 < z.1} =
{g : P | g.1 = -(g.2 ^ (2 : ℕ)) / 4}Proposition 5.0.29
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.29: at a single dual-domain slope s, a unique interior Fenchel-support maximizer x gives the inverse-Hessian identity ∇² (extendedRealRealPart (f⋆)) s = [∇² (withTopRealPart f) x]⁻¹.
open scoped ConvexAnalysis Gradient WithTopConvexAnalysis
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {f : E → WithTop ℝ} {xStar : E → E}
-- Semantic recall check: LeanSearch did not expose a direct owner theorem for differentiability
-- of Fenchel conjugates, so this file keeps the chapter-local bridge API.
variable (f) (xStar)
theorem fenchelConjugate_hessian_eq_inverse_atSupportMaximizer
(hf_closedConvex : ClosedConvexFunction f)
(hself : IsSelfConcordantOn (interior (dom f)) (withTopRealPart f))
(hinterior_nonempty : (interior (dom f)).Nonempty)
(hxStar_mem : ∀ ⦃s : E⦄, s ∈ dom (f⋆) → xStar s ∈ interior (dom f))
(hxStar_isMaximizer :
∀ ⦃s : E⦄, s ∈ dom (f⋆) →
IsMaxOn (fun y : E ↦ inner ℝ s y - withTopRealPart f y) (dom f) (xStar s))
(hxStar_unique :
∀ ⦃s y : E⦄, s ∈ dom (f⋆) → y ∈ dom f →
IsMaxOn (fun z : E ↦ inner ℝ s z - withTopRealPart f z) (dom f) y → y = xStar s)
{s : E} (hs : s ∈ dom (f⋆)) :
hessian (extendedRealRealPart (f⋆)) s =
(hessian (withTopRealPart f) (xStar s)).inverseProposition 5.0.30
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.30: let xStar : E → E be a chosen Fenchel-support maximizer branch on dom (f⋆). Assume f is three times differentiable on interior (dom f), that xStar s ∈ interior (dom f) and xStar s maximizes y ↦ ⟪s, y⟫ - f y on dom f for every s ∈ dom (f⋆), that the source stationarity equation ∇ (withTopRealPart f) (xStar s) = s holds on dom (f⋆), and that the source inverse-Hessian identity (5.u341) is represented in Lean by hessian (extendedRealRealPart (f⋆)) s = (hessian (withTopRealPart f) (xStar s)).inverse. Because ContinuousLinearMap.inverse is totalized in Lean, we also record the genuine primal-Hessian invertibility needed to express the source inverse-Hessian hypothesis faithfully. Because Lean's differential operators are totalized, the source-facing D³ f⋆ identity is stated here in the within-domain form closest to the textbook meaning; the stronger ambient fderiv consequence on an open dual domain remains available as the companion theorem fenchelConjugate_hessianDerivative_formula_of_isOpen.
open scoped ConvexAnalysis Gradient WithTopConvexAnalysis
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
theorem fenchelConjugate_hessianDerivative_formula
{f : E → WithTop ℝ} {xStar : E → E}
(hf_contDiff : ContDiffOn ℝ 3 (withTopRealPart f) (interior (dom f)))
(hxStar_mem : ∀ ⦃s : E⦄, s ∈ dom (f⋆) → xStar s ∈ interior (dom f))
(hxStar_isMaximizer :
∀ ⦃s : E⦄, s ∈ dom (f⋆) →
IsMaxOn (fun y : E ↦ inner ℝ s y - withTopRealPart f y) (dom f) (xStar s))
(hxStar_gradient_eq :
∀ ⦃s : E⦄, s ∈ dom (f⋆) →
∇ (withTopRealPart f) (xStar s) = s)
(hxStar_hessian_invertible :
∀ ⦃s : E⦄, s ∈ dom (f⋆) →
(hessian (withTopRealPart f) (xStar s)).IsInvertible)
(hdual_hessian_eq_inverse :
∀ ⦃s : E⦄, s ∈ dom (f⋆) →
hessian (extendedRealRealPart (f⋆)) s =
(hessian (withTopRealPart f) (xStar s)).inverse)
{s : E} (hs : s ∈ dom (f⋆))
(hunique : UniqueDiffWithinAt ℝ (dom (f⋆)) s)
(hxStar_tendsto :
Filter.Tendsto xStar (nhdsWithin s (dom (f⋆)))
(nhdsWithin (xStar s) (interior (dom f))))
(h : E) :
fderivWithin ℝ (hessian (extendedRealRealPart (f⋆))) (dom (f⋆)) s h =
-((hessian (extendedRealRealPart (f⋆)) s).comp
((fderiv ℝ (hessian (withTopRealPart f)) (xStar s)
((hessian (extendedRealRealPart (f⋆)) s) h)).comp
(hessian (extendedRealRealPart (f⋆)) s)))Proposition 5.0.31
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.0.31 uses the canonical Dikin-owner inclusion theorem IsSelfConcordantOnWith.openDikinEllipsoid_inv_constant_subset for the neighborhood W⁰[f; sBar](1 / (Mf : ℝ)).
open scoped DikinEllipsoidNotation Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
recall IsSelfConcordantOnWith.openDikinEllipsoid_inv_constant_subset
-- Proof sketch: rewrite the displayed quadratic bound as membership of `s` in the canonical
-- Dikin ellipsoid `W⁰[f; s̄](1 / M_f)` via the inverse-parameter quadratic bridge from
-- `Definition_5_0_14`, then apply the owner-level inclusion theorem
-- `IsSelfConcordantOnWith.openDikinEllipsoid_inv_constant_subset`. When `Mf = 0`, the bridge
-- identifies the displayed neighborhood with the empty Dikin ellipsoid, so the inclusion remains
-- vacuous without an extra positivity binder.
5.1 — Self-concordant Functions
Corollary 5.1.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.1.1, forward direction: the Chapter 5 owner IsSelfConcordantOnWith dom Mf f implies the Hessian-direction operator inequality D³f(x)[u] ≤ 2 M_f ‖u‖_{∇² f(x)} ∇² f(x) on dom.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
namespace IsSelfConcordantOnWith
theorem thirdDerivative_operator_le
(hself : IsSelfConcordantOnWith dom Mf f) {x : E} (hx : x ∈ dom) (u : E) :
fderiv ℝ (hessian f) x u ≤
(2 * (Mf : ℝ) * ‖u‖[f; x]) • hessian f xDefinition 5.1.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.1.1: f is self-concordant with constant M_f on the open convex domain dom when f is a convex C³ function on dom and the third directional derivative is bounded by 2 M_f times the cube of the local Hessian norm; the case M_f = 1 is the standard self-concordant condition.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
open scoped HessianLocalNorm
-- Proof sketch: unfold `hessianLocalNorm`.
class IsSelfConcordantOnWith (dom : Set E) (Mf : NNReal) (f : E → ℝ) : Prop where
/-- The domain of definition is open. -/
isOpen_domain : IsOpen dom
/-- The objective is three-times continuously differentiable on `dom`. -/
contDiffOn : ContDiffOn ℝ 3 f dom
/-- The objective is convex on the domain. -/
convexOn : ConvexOn ℝ dom f
/-- The third directional derivative is controlled by the cube of the local Hessian norm. -/
third_deriv_bound {x : E} (hx : x ∈ dom) (u : E) :
|thirdDirectionalDerivative f x u| ≤
2 * (Mf : ℝ) * ‖u‖[f; x] ^ (3 : ℕ)Example 5.1.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.1.1 is the zero-quadratic specialization of quadraticAffineObjective_isSelfConcordantOnWith_zero.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable (α : ℝ) (a : E)
#check
(show IsSelfConcordantOnWith (Set.univ : Set E) 0 (fun x ↦ α + inner ℝ a x) from by
simpa using
quadraticAffineObjective_isSelfConcordantOnWith_zero
α a (0 : E →L[ℝ] E) ContinuousLinearMap.isPositive_zero)Lemma 5.1.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.1.1, owner form: the Hessian quadratic form of a continuous affine pullback f ∘ g at x in the direction u is the Hessian quadratic form of f at g x in the image direction g.contLinear u, provided f is C² at g x.
open scoped Gradient HessianLocalNorm
universe u v
variable {E : Type u} {E₁ : Type v}
variable [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable [NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
theorem hessianQuadraticForm_comp_affine
(f : E₁ → ℝ) (g : E →ᴬ[ℝ] E₁) (x u : E) (hf : ContDiffAt ℝ 2 f (g x)) :
inner ℝ u (hessian (f ∘ g) x u) =
inner ℝ (g.contLinear u) (hessian f (g x) (g.contLinear u))Theorem 5.1.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.1: if f₁ and f₂ are self-concordant on dom₁ and dom₂ with constants M₁ and M₂, then for positive weights α and β the weighted sum (α : ℝ) • f₁ + (β : ℝ) • f₂ is self-concordant on the intersection domain dom₁ ∩ dom₂ with self-concordance constant max (M₁ / √α) (M₂ / √β).
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOnWith
theorem weightedSum
{dom₁ dom₂ : Set E} {M₁ M₂ : NNReal} {α β : NNRealˣ} {f₁ f₂ : E → ℝ}
(h₁ : IsSelfConcordantOnWith dom₁ M₁ f₁)
(h₂ : IsSelfConcordantOnWith dom₂ M₂ f₂) :
IsSelfConcordantOnWith (dom₁ ∩ dom₂)
(max (M₁ / NNReal.sqrt α) (M₂ / NNReal.sqrt β))
((α : ℝ) • f₁ + (β : ℝ) • f₂)Corollary 5.1.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.1.2: if f is self-concordant on an open convex domain dom ⊆ E with self-concordance constant M_f, then the quadratic-affine perturbation φ(x) = α + ⟪a, x⟫ + (1 / 2) ⟪Ax, x⟫ + f(x) is self-concordant on dom with the same self-concordance constant M_f whenever A is positive semidefinite. This reuses the canonical Chapter 5 owner quadraticAffineObjective α a A.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOnWith
theorem add_quadraticAffineObjective
{dom : Set E} {Mf : NNReal} {f : E → ℝ}
(hf : IsSelfConcordantOnWith dom Mf f)
(α : ℝ) (a : E) (A : E →L[ℝ] E) (hA : A.IsPositive) :
IsSelfConcordantOnWith dom Mf (quadraticAffineObjective α a A + f)Example 5.1.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.1.2: if A is positive, then the quadratic-affine objective f(x) = α + ⟪a, x⟫ + (1 / 2) ⟪A x, x⟫ on all of E is self-concordant with self-concordance constant M_f = 0.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
variable [CompleteSpace E]
theorem quadraticAffineObjective_isSelfConcordantOnWith_zero
(α : ℝ) (a : E) (A : E →L[ℝ] E) (hA : A.IsPositive) :
IsSelfConcordantOnWith (Set.univ : Set E) 0 (quadraticAffineObjective α a A)Lemma 5.1.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.1.2: for a C³ function with positive-semidefinite Hessian on an open set dom, the diagonal cubic bound in the definition of M_f-self-concordance is equivalent to the full trilinear estimate on the third derivative.
open scoped Gradient
open scoped HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem selfConcordant_diagonal_bound_iff_trilinear_bound
{dom : Set E} {Mf : NNReal} {f : E → ℝ}
(hdom_open : IsOpen dom)
(hcont : ContDiffOn ℝ 3 f dom)
(hH : ∀ {x : E} (hx : x ∈ dom), (hessian f x).IsPositive) :
(∀ {x : E} (hx : x ∈ dom) (u : E),
|thirdDirectionalDerivative f x u| ≤
2 * (Mf : ℝ) * ‖u‖[f; x] ^ (3 : ℕ)) ↔
∀ {x : E} (hx : x ∈ dom) (u₁ u₂ u₃ : E),Theorem 5.1.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.2: precomposing a self-concordant function with a continuous affine map preserves self-concordance on the affine preimage with the same self-concordance constant. This is the canonical owner-level affine-pullback theorem.
universe u v
open scoped HessianLocalNorm
variable {E : Type u} {E₁ : Type v}
variable [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable [NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
namespace IsSelfConcordantOnWith
theorem comp_continuousAffineMap
{dom : Set E₁} {Mf : NNReal} {f : E₁ → ℝ}
(hf : IsSelfConcordantOnWith dom Mf f) (g : E →ᴬ[ℝ] E₁) :
IsSelfConcordantOnWith (g ⁻¹' dom) Mf (f ∘ g)Corollary 5.1.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.1.3: if f is self-concordant on dom with constant Mf, then for every positive scalar α the rescaled function (α : ℝ) • f is self-concordant on the same domain with constant Mf / √α.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOnWith
-- Proof sketch: scale the owner fields directly. Openness is unchanged, `C³` regularity is
-- preserved by `ContDiffOn.const_smul`, convexity by `ConvexOn.smul`, the third directional
-- derivative scales linearly with `α`, and the Hessian local norm scales by `√α`.
theorem pos_smul
{dom : Set E} {Mf : NNReal} {f : E → ℝ}
(h : IsSelfConcordantOnWith dom Mf f) (α : NNRealˣ) :
IsSelfConcordantOnWith dom (Mf / NNReal.sqrt α) ((α : ℝ) • f)Example 5.1.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.1.3: the univariate logarithmic barrier x ↦ -log x on (0, ∞) is standard self-concordant.
open scoped Gradient HessianLocalNorm
instance negLog_isStandardSelfConcordantOn :
IsStandardSelfConcordantOn (Set.Ioi (0 : ℝ)) (fun x : ℝ ↦ -Real.log x)Lemma 5.1.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.1.3: if f is C³ on an open set dom and the quotient |D³f(x + t h)[h,h,h]| / (2 ‖h‖[f; x + t • h]^3) is bounded by M_f throughout the natural domain of the associated univariate function, then the derivative within that domain of the canonical ambient representative directionalSlice (fun y ↦ 1 / ‖h‖[f; y]) x h, equivalently associatedUnivariateFunction dom f x h, has absolute value at most M_f at every t ∈ associatedUnivariateFunctionDomain dom f x h.
open scoped Gradient HessianLocalNorm Topology ContDiff
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {dom : Set E} {f : E → ℝ} {x h : E}
local notation "sliceDomain" => associatedUnivariateFunctionDomain dom f x h
local notation "reciprocalLocalNormSlice" =>
directionalSlice (fun y ↦ 1 / ‖h‖[f; y]) x h
theorem abs_derivWithin_associatedUnivariateFunction_le
{Mf : NNReal} {t : ℝ}
(hdom_open : IsOpen dom)
(hcont : ContDiffOn ℝ 3 f dom)
(ht : t ∈ sliceDomain)
(hbound :
∀ {s : ℝ}, s ∈ sliceDomain →
|thirdDirectionalDerivative f (x + s • h) h| /
(2 * ‖h‖[f; x + s • h] ^ (3 : ℕ)) ≤ (Mf : ℝ)) :Theorem 5.1.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.1.3: if f is self-concordant on dom with constant Mf, then for every positive scalar α the rescaled function (α : ℝ) • f is self-concordant on the same domain with constant Mf / √α.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOnWith
-- Proof sketch: scale the owner fields directly. Openness is unchanged, `C³` regularity is
-- preserved by `ContDiffOn.const_smul`, convexity by `ConvexOn.smul`, the third directional
-- derivative scales linearly with `α`, and the Hessian local norm scales by `√α`.
theorem pos_smul
{dom : Set E} {Mf : NNReal} {f : E → ℝ}
(h : IsSelfConcordantOnWith dom Mf f) (α : NNReal) (hα : 0 < α) :
IsSelfConcordantOnWith dom (Mf / NNReal.sqrt α) ((α : ℝ) • f)
Corollary 5.1.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5 1 4: if f is self-concordant on an open convex domain dom and x ∈ dom, then the natural domain of the associated univariate function t ↦ (⟪∇² f (x + t • h) h, h⟫)^{-1/2} contains the interval (-1 / (M_f ‖h‖[f; x]), 1 / (M_f ‖h‖[f; x])) whenever the reciprocal radius is well-defined, that is, under 0 < (M_f : ℝ) * ‖h‖[f; x].
open scoped HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOnWith
theorem associatedUnivariateFunctionDomain_contains_interval
{dom : Set E} {Mf : NNReal} {f : E → ℝ}
(hself : IsSelfConcordantOnWith dom Mf f) {x h : E} (hx : x ∈ dom)
(hMh : 0 < (Mf : ℝ) * ‖h‖[f; x]) :
Set.Ioo (-(1 / ((Mf : ℝ) * ‖h‖[f; x]))) (1 / ((Mf : ℝ) * ‖h‖[f; x])) ⊆
associatedUnivariateFunctionDomain dom f x h
Example 5.1.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.1.4: if A is positive, then the logarithmic barrier attached to the affine- quadratic potential φ(x) = α + ⟪a, x⟫ - (1 / 2) ⟪A x, x⟫ is standard self-concordant on its positivity domain {x | 0 < φ(x)}. The generic Chapter 5 sublevel-barrier owners are only a proof bridge behind this source-facing formulation.
open Filter
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
variable [CompleteSpace E]
theorem logAffineQuadraticBarrier_isStandardSelfConcordantOn
(α : ℝ) (a : E) (A : E →L[ℝ] E) (hA : A.IsPositive) :
IsStandardSelfConcordantOn
{x : E | 0 < α + inner ℝ a x - (1 / 2 : ℝ) * inner ℝ (A x) x}
(fun x ↦ -Real.log (α + inner ℝ a x - (1 / 2 : ℝ) * inner ℝ (A x) x))
Lemma 5.1.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.1.4 belongs to the one-variable self-concordant auxiliary-function / Fenchel-conjugacy domain. Relevant owner-style declarations sampled before refinement: * selfConcordantOmega, selfConcordantOmegaStar, selfConcordantOmegaDeriv, and selfConcordantOmegaPrimeStar in Definition_5_0_21, the chapter owners for ω, ω_*, ω', and ω'_*; * selfConcordantOmegaArg and selfConcordantOmegaStarArg in Definition_5_0_21, the canonical Chapter 5 constructors for the natural ω and ω_* subtype arguments; * fenchelDual in Definition_5_0_27, the project-level canonical owner for Fenchel conjugacy in higher dimension; * IsMaxOn in mathlib, the canonical owner-level maximizer predicate behind the scalar Fenchel support formulas. Best owner abstraction: * source-facing: the fixed chapter owners ω, ω_*, together with their explicit derivative and inverse branches ω', ω'_*, and the textbook maximization identities built from them; * core/canonical: the already-defined chapter auxiliary functions from Definition_5_0_21; * bridge/view: the explicit scalar formulas specialized to those owners. Primitive data: * ω; * ω_*. Derived API: * ω' and ω'_*; * the canonical subtype constructors selfConcordantOmegaArg and selfConcordantOmegaStarArg; * the domain-membership lemmas for ω' and ω'_*; * the seven source-facing Fenchel-conjugacy identities of Lemma 5.1.4, stated directly against the canonical owners on their mathematically forced domains: the full owner domains for the inverse identities (1), (2), (6), (7), and the constrained nonnegative maximization regimes for (3), (4), (5), without a parallel local point/maximand wrapper API; the maximizer data for parts (3) and (4) is carried by the numbered statements themselves.
open Set
open scoped SelfConcordantAuxiliaryFunction
theorem neg_one_lt_selfConcordantUnit_of_nonneg {t : ℝ} (ht : 0 ≤ t) : -1 < tTheorem 5.1.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.4 (1): on the strict sublevel set {x | f x < β}, the logarithmic barrier x ↦ -log (β - f x) is well defined because its argument is positive.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u}
theorem sublevelLogBarrier_arg_pos_of_mem_domain
(f : E → ℝ) (β : ℝ) {x : E} (hx : f x < β) :
0 < β - f xCorollary 5.1.5
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.1.5 (lower bound): if f is self-concordant on dom with positive parameter M_f, x ∈ dom, and y ∈ W⁰[f; x](1 / (Mf : ℝ)), then the average Hessian along the segment from x to y dominates the explicit lower Loewner bound built from ∇² f(x). The positivity of M_f and the membership y ∈ dom are consequences of the displayed open-Dikin hypothesis.
open scoped Gradient HessianLocalNorm DikinEllipsoidNotation
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
namespace IsSelfConcordantOnWith
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
variable [CompleteSpace E]
variable (hself : IsSelfConcordantOnWith dom Mf f) {x y : E} (hx : x ∈ dom)
variable (hxy : y ∈ W⁰[f; x](1 / (Mf : ℝ)))
theorem segmentAverageHessian_lower_bound
(hself : IsSelfConcordantOnWith dom Mf f) {x y : E} (hx : x ∈ dom)
(hxy : y ∈ W⁰[f; x](1 / (Mf : ℝ)))
:
let r := ‖y - x‖[f; x]
let G := ∫ τ in (0 : ℝ)..1, hessian f (x + τ • (y - x))
(1 - (Mf : ℝ) * r + ((Mf : ℝ) ^ (2 : ℕ) * r ^ (2 : ℕ)) / 3) • hessian f x ≤ G
Example 5.1.5
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.1.5: for p > 0, the regularized power barrier f_p(x) = (1 / 2) x^2 + 1 / (p x^p) - 1 / p is self-concordant on (0, ∞) with self-concordance constant M_f = 1 + p / 2.
open Filter
open scoped Topology Gradient HessianLocalNorm
theorem regularizedPowerBarrier_isSelfConcordantOnWith
{p : ℝ} (hp : 0 < p) :
IsSelfConcordantOnWith (Set.Ioi (0 : ℝ)) (Real.toNNReal (1 + p / 2))
(regularizedPowerBarrier p)
Lemma 5.1.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.1.5: for t ≥ 0, the self-concordant auxiliary function ω(t) lies between the rational bounds t² / (2 * (1 + (2 / 3) t)) and t² / (2 + t).
open scoped SelfConcordantAuxiliaryFunction
theorem selfConcordantOmega_bounds
{t : ℝ} (ht : 0 ≤ t) :
let tω := selfConcordantOmegaArg 1 t (by
have h : -1 < ((1 : NNReal) : ℝ) * t := neg_one_lt_mf_mul_of_nonneg ht
simpa using h)
t ^ 2 / (2 * (1 + (2 / 3 : ℝ) * t)) ≤ ω tω ∧
ω tω ≤ t ^ 2 / (2 + t)Theorem 5.1.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.5 (1): every point of the reciprocal-radius open Dikin ellipsoid around a point of domain remains in domain, provided the canonical +∞-extension is a closed convex function. This is the public real-valued Chapter 5 surface corresponding to the source-faithful effective-domain theorem above.
universe u
open scoped Gradient HessianLocalNorm DikinEllipsoidNotation WithTopConvexAnalysis
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOnWith
variable {Mf : NNRealˣ} {f : E → WithTop ℝ}
-- Semantic recall note: `lean_leansearch` did not surface a matching mathlib owner for these
-- self-concordant Dikin transport statements, so the public API stays local to this chapter as
-- methods of `IsSelfConcordantOnWith`.
variable {domain : Set E} {f : E → ℝ}
variable {Mf : NNRealˣ}
theorem openDikinEllipsoid_inv_constant_subset
(hf_closedConvex : ClosedConvexFunction (fenchelPrimalExtension domain f))
(hself : IsSelfConcordantOnWith domain (Mf : NNReal) f)
{x y : E} (hx : x ∈ domain)
(hxy : y ∈ W⁰[f; x](1 / (Mf : ℝ))) :
y ∈ domainExample 5.1.6
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.1.6: if f is strongly convex on all of E with parameter σ₂, belongs to the chapter smoothness class C22[L₃], and is C³ at x, then the directional derivative of its Hessian satisfies the operator inequality D³f(x)[u] ≤ (L₃ / (σ₂ * √σ₂)) ‖u‖_{∇² f(x)} ∇²f(x). This keeps the example at the source-facing operator layer used by Corollary 5.1.1.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {f : E → ℝ} {σ2 : ℝ} {L3 : NNReal}
namespace StrongConvexOn
variable (hf_strong : StrongConvexOn Set.univ σ2 f) (hσ2 : 0 < σ2) (hf_hessian : f ∈ C22[L3])
include hf_strong hσ2 hf_hessian
theorem thirdDerivative_operator_le_of_mem_C22
{x u : E} (h_contDiffAt : ContDiffAt ℝ 3 f x) :
fderiv ℝ (hessian f) x u ≤
(((L3 : ℝ) / (σ2 * Real.sqrt σ2)) * ‖u‖[f; x]) • hessian f xLemma 5.1.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.1.6 (1): if f : E → ℝ ∪ {+∞} is a closed proper convex function on a finite-dimensional real inner-product space, differentiable on its open effective domain, and of Legendre type, then the Fenchel dual has a closed convex effective epigraph.
open scoped ConvexAnalysis Gradient WithTopConvexAnalysis
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
[FiniteDimensional ℝ E]
-- Semantic recall: LeanSearch exposed no direct owner theorem for the full source-level
-- Legendre statement, so this file keeps the source clauses explicit on the existing Fenchel-dual
-- API.
variable {f : E → WithTop ℝ}
theorem fenchelDual_effectiveEpigraph_closed_convex_of_legendre
(hf : ClosedConvexFunction f)
(hdom : (dom f).Nonempty)
(hdom_open : IsOpen (dom f))
(hdiff : DifferentiableOn ℝ (withTopRealPart f) (dom f))
(hleg : IsLegendreType f) :
IsClosed (effectiveEpigraph (f⋆)) ∧
Convex ℝ (effectiveEpigraph (f⋆))
Theorem 5.1.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.6: if f is self-concordant on dom, the constrained epigraph of f over dom is closed, and dom contains no affine line, then the Hessian of f is positive definite on dom.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOn
variable {dom : Set E} {f : E → ℝ}
theorem hasPositiveDefiniteHessianOn_of_no_affine_line
(hself : IsSelfConcordantOn dom f)
(hclosed : IsClosed (constrainedEpigraph dom (fun y ↦ (f y : WithTop ℝ))))
(hnoAffineLine : ∀ ⦃x h : E⦄, h ≠ 0 → ¬ ∀ τ : ℝ, x + τ • h ∈ dom) :
HasPositiveDefiniteHessianOn dom f
Example 5.1.7
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.1.7: the standard self-concordant condition is the case Mf = 1.
open scoped Gradient abbrev IsStandardSelfConcordantOn (dom : Set ℝ) (f : ℝ → ℝ) : Prop := IsSelfConcordantOnWith dom 1 f
Lemma 5.1.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.1.7, source-facing form: if f is convex and C² on an open domain, the Fenchel conjugate of fenchelPrimalExtension domain f is self-concordant on its effective domain with positive constant M_f, and the Hessian of f is positive definite on domain, then the primal Hessians at x and y satisfy the standard self-concordant Loewner-order comparison whenever d := gradientDifferenceDualLocalNorm domain f x y hx = ‖∇ f x - ∇ f y‖_x^* satisfies d < 1 / M_f.
open scoped ConvexAnalysis DikinEllipsoidNotation Gradient HessianDualLocalNorm HessianLocalNorm
MatrixOrder WithTopConvexAnalysis
universe u
open InnerProductSpace
variable {X : Type u} [NormedAddCommGroup X] [InnerProductSpace ℝ X]
variable [FiniteDimensional ℝ X]
variable {domain : Set X} {Mf : NNReal} {f : X → ℝ} {x y : X}
local notation "F" => fenchelPrimalExtension domain f
-- Semantic recall check: LeanSearch did not expose a direct owner theorem for this
-- conjugate-Hessian comparison, so the source-facing statement is kept explicit here.
theorem hessian_loewner_bounds_of_fenchelDual_selfConcordant
[HasPositiveDefiniteHessianOn domain f]
(hconv : ConvexOn ℝ domain f)
(hC2 : ContDiffOn ℝ 2 f domain)
(hdomOpen : IsOpen domain)
(hdual : IsSelfConcordantOnWith (dom (F⋆)) Mf (extendedRealRealPart (F⋆)))
(hMf_pos : 0 < (Mf : ℝ))
(hx : x ∈ domain) (hy : y ∈ domain)
(hd_lt : gradientDifferenceDualLocalNorm domain f x y hx < 1 / (Mf : ℝ)) :
let d := gradientDifferenceDualLocalNorm domain f x y hx
((1 - (Mf : ℝ) * d) ^ (2 : ℕ)) • hessian f x ≤ hessian f y ∧
hessian f y ≤ ((1 - (Mf : ℝ) * d) ^ (2 : ℕ))⁻¹ • hessian f xTheorem 5.1.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.7: if f is C³ on an open set containing the segment from x to y, its third directional derivative satisfies the Chapter 5 local-norm bound with constant M_f along that segment, the Hessian is positive along the segment, and y ∈ W⁰[f; x](1 / M_f), then with r = ‖y - x‖[f; x] the Hessians at x and y satisfy the Loewner-order bounds (1 - M_f r)^2 • ∇²f(x) ≤ ∇²f(y) ≤ (1 - M_f r)⁻² • ∇²f(x).
open scoped Gradient HessianLocalNorm DikinEllipsoidNotation
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem hessian_loewner_bounds_along_segment
{dom : Set E} {Mf : NNReal} {f : E → ℝ} {x y : E}
(hdom_open : IsOpen dom)
(hcont : ContDiffOn ℝ 3 f dom)
(hsegment : segment ℝ x y ⊆ dom)
(hpsd : ∀ ⦃z : E⦄, z ∈ segment ℝ x y → (hessian f z).IsPositive)
(hthird : ∀ ⦃z : E⦄ (_hz : z ∈ segment ℝ x y) (u : E),
|thirdDirectionalDerivative f z u| ≤
2 * (Mf : ℝ) * ‖u‖[f; z] ^ (3 : ℕ))
(hy : y ∈ W⁰[f; x](1 / (Mf : ℝ))) :
let r := ‖y - x‖[f; x]
((1 - (Mf : ℝ) * r) ^ (2 : ℕ)) • hessian f x ≤ hessian f y ∧
hessian f y ≤ ((1 - (Mf : ℝ) * r) ^ (2 : ℕ))⁻¹ • hessian f xTheorem 5.1.8
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Under the Chapter 5 owner IsSelfConcordantOnWith dom Mf f, every admissible Dikin step y ∈ W⁰[f; x](1 / (Mf : ℝ)) satisfies both lower bounds from Theorem 5.1.8: the gradient pairing dominates ‖y - x‖_x² / (1 + M_f ‖y - x‖_x), and the function value dominates the affine Taylor approximation at x with the explicit self-concordant remainder ω. This is the canonical owner-level bridge from self-concordance to the source-facing lower estimates.
open scoped DikinEllipsoidNotation Gradient HessianLocalNorm SelfConcordantAuxiliaryFunction
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantOnWith
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
-- Proof sketch: derive `y ∈ dom` from the Dikin-step hypothesis via
-- `openDikinEllipsoid_inv_constant_subset`, use convexity of `dom` to place the whole segment
-- from `x` to `y` inside `dom`, and apply
-- `hessian_loewner_bounds_of_mem_openDikinEllipsoid` pointwise to the intermediate points
-- `x + τ • (y - x)` to recover the lower segment-wise Loewner hypothesis required by the
-- source-facing Theorem 5.1.8. The two displayed estimates then follow by the local theorems
-- `gradient_difference_inner_ge_hessianLocalNorm_sq_div` and
-- `taylor_lower_bound_of_hessian_loewner_lower`.
-- The repaired source-facing theorems above now match this owner-level open-domain surface.
theorem gradient_difference_inner_and_taylor_lower_bounds_of_mem_openDikinEllipsoid
(hself : IsSelfConcordantOnWith dom Mf f)
{x y : E} (hx : x ∈ dom) (hxy : y ∈ W⁰[f; x](1 / (Mf : ℝ))) :
let r := ‖y - x‖[f; x]
let tω := selfConcordantOmegaArg Mf r (by
exact neg_one_lt_mf_mul_of_nonneg (by
simpa [r] using hessianLocalNorm_nonneg f x (y - x)))
inner ℝ (∇ f y - ∇ f x) (y - x) ≥
r ^ (2 : ℕ) / (1 + (Mf : ℝ) * r) ∧
f y ≥
f x + inner ℝ (∇ f x) (y - x) +
if hMf : Mf = 0 then
r ^ (2 : ℕ) / 2
else
(1 / (Mf : ℝ) ^ (2 : ℕ)) * ω tω
Theorem 5.1.9
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.9: if f is C² on dom and every admissible local step h from a point x ∈ dom stays in dom, satisfies the local-norm transport bound ‖h‖[f; x + h] ≤ ‖h‖[f; x] / (1 - M_f ‖h‖[f; x]), and has nonnegative Hessian quadratic form at the endpoint, then every admissible Dikin step y ∈ W⁰[f; x](1 / M_f) satisfies the gradient pairing and first-order Taylor upper bounds from the textbook statement.
open scoped DikinEllipsoidNotation Gradient HessianLocalNorm SelfConcordantAuxiliaryFunction
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
variable (hf_C2 : ContDiffOn ℝ 2 f dom)
variable (hstep :
∀ ⦃x h : E⦄, x ∈ dom → ‖h‖[f; x] < 1 / (Mf : ℝ) → x + h ∈ dom)
variable (htransport :
∀ ⦃x h : E⦄, x ∈ dom → x + h ∈ dom →
‖h‖[f; x] < 1 / (Mf : ℝ) →
‖h‖[f; x + h] ≤ ‖h‖[f; x] / (1 - (Mf : ℝ) * ‖h‖[f; x]))
variable (hquad_nonneg :
∀ ⦃x h : E⦄, x ∈ dom → x + h ∈ dom →
‖h‖[f; x] < 1 / (Mf : ℝ) →
0 ≤ inner ℝ h (hessian f (x + h) h))
include hf_C2 hstep htransport hquad_nonneg
theorem localNorm_gradient_pairing_and_value_upper_bounds_of_transport_data
{x y : E} (hx : x ∈ dom) (hxy : y ∈ W⁰[f; x](1 / (Mf : ℝ))) :
let r := ‖y - x‖[f; x]
inner ℝ (∇ f y - ∇ f x) (y - x) ≤
r ^ (2 : ℕ) / (1 - (Mf : ℝ) * r) ∧
f y ≤
f x + inner ℝ (∇ f x) (y - x) +
(1 / ((Mf : ℝ) ^ (2 : ℕ))) *
(-(Mf : ℝ) * r - Real.log (1 - (Mf : ℝ) * r))
Theorem 5.1.10
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.10 (1) is the canonical Dikin-inclusion theorem IsSelfConcordantOnWith.openDikinEllipsoid_inv_constant_subset.
open scoped DikinEllipsoidNotation HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
recall IsSelfConcordantOnWith.openDikinEllipsoid_inv_constant_subsetTheorem 5.1.11
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.11: let Q ⊆ E₁ × E₂ be a nonempty convex set. If Φ is self-concordant with positive constant M_Φ on interior Q for the canonical intrinsic L² product lift of E₁ × E₂, and if for every x in the displayed source domain partialMinimizationSourceDom Q the fiber minimum is attained at the chosen unique interior minimizer branch y x with positive-definite frozen yy Hessian ∇² (Φ ∘ Prod.mk x) (y x), then the canonical real partial-minimization objective is self-concordant on that source domain with the same constant. The bridge partialMinimizationSourceDom_eq_dom_of_attainedInteriorArgmin records the optional comparison with the canonical effective-domain owner.
open scoped ConstrainedArgmin ConvexAnalysis Gradient
open ContinuousLinearMap
universe u₁ u₂
variable {E₁ : Type u₁} {E₂ : Type u₂}
variable [NormedAddCommGroup E₁] [NormedAddCommGroup E₂]
variable [InnerProductSpace ℝ E₁] [FiniteDimensional ℝ E₁]
variable [InnerProductSpace ℝ E₂] [FiniteDimensional ℝ E₂]
local notation "Z" => WithLp 2 (E₁ × E₂)
theorem partialMinimizationObjective_isSelfConcordantOnWith
{Q : Set (E₁ × E₂)} {Mf : NNRealˣ} {Φ : E₁ × E₂ → ℝ} {y : E₁ → E₂}
(hQ_convex : Convex ℝ Q)
(hQ_nonempty : Q.Nonempty)
(hself :
IsSelfConcordantOnWith ((WithLp.ofLp : Z → E₁ × E₂) ⁻¹' interior Q) (Mf : NNReal)
(Φ ∘ WithLp.ofLp))
(hy_mem_interior :
∀ ⦃x : E₁⦄, x ∈ partialMinimizationSourceDom Q → (x, y x) ∈ interior Q)
(hy_argmin :
∀ ⦃x : E₁⦄, x ∈ partialMinimizationSourceDom Q →
y x ∈ argmin[(Prod.mk x) ⁻¹' Q] (Φ ∘ Prod.mk x))
(hy_unique :
∀ ⦃x : E₁⦄, x ∈ partialMinimizationSourceDom Q → ∀ ⦃y' : E₂⦄,
y' ∈ argmin[(Prod.mk x) ⁻¹' Q] (Φ ∘ Prod.mk x) → y' = y x)
(hyy_pos :
∀ ⦃x : E₁⦄, x ∈ partialMinimizationSourceDom Q → ∀ v : E₂, v ≠ 0 →
0 < inner ℝ v ((hessian (Φ ∘ Prod.mk x) (y x)) v)) :
IsSelfConcordantOnWith (partialMinimizationSourceDom Q) (Mf : NNReal)
(extendedRealRealPart (partialInfProjection Q (Real.toEReal ∘ Φ)))Theorem 5.1.12
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.12: if f is self-concordant on dom with constant M_f, then the value at y is bounded below by the affine Taylor model at x plus the remainder term M_f⁻² ω(M_f ‖∇ f(y) - ∇ f(x)‖*_y), interpreted as (1 / 2) ‖∇ f(y) - ∇ f(x)‖*²_y when M_f = 0. In the same zero-parameter limit, the upper branch also reduces to the quadratic remainder (1 / 2) ‖∇ f(y) - ∇ f(x)‖*²_y; otherwise, if the dual local norm of the gradient difference at y is smaller than 1 / M_f, then f y is bounded above by the affine model plus M_f⁻² ω_*(M_f ‖∇ f(y) - ∇ f(x)‖*_y).
open InnerProductSpace
open scoped DikinEllipsoidNotation Gradient HessianDualLocalNorm HessianLocalNorm
SelfConcordantAuxiliaryFunction
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
variable [IsSelfConcordantOnWith dom Mf f] [HasPositiveDefiniteHessianOn dom f]
theorem selfConcordant_value_bounds_of_dualLocalNorm_gradient_sub
{x y : E} (hx : x ∈ dom) (hy : y ∈ dom) :
let δ :=
HessianDualLocalNorm.ofPosDefMem f hy
((toDual ℝ E) (∇ f y - ∇ f x))
let taylor := firstOrderTaylorModelAt f x y
let tω := selfConcordantOmegaArg Mf δ
(neg_one_lt_mf_mul_of_nonneg (gradientDifferenceDualLocalNorm_nonneg x y hy))
f y ≥
taylor +
(if Mf = 0 then
δ ^ (2 : ℕ) / 2
else
(1 / (Mf : ℝ) ^ (2 : ℕ)) * ω tω) ∧
(if Mf = 0 then
f y ≤
taylor +
δ ^ (2 : ℕ) / 2
else
∀ hδ : δ < 1 / (Mf : ℝ),
let τω := selfConcordantOmegaStarArg Mf δ (mf_mul_lt_one_of_lt_inv hδ)
f y ≤
taylor +
(1 / (Mf : ℝ) ^ (2 : ℕ)) * ω_* τω)
Theorem 5.1.13
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.13: if x ∈ dom satisfies λ_f(x) < 1 / M_f and the problem min {f(u) | u ∈ dom} attains a minimum on dom, then it admits a unique solution x_f*, and the standard ω_* suboptimality bound holds at x. The current local Chapter 5 owners still admit openQuadraticObjectiveCounterexample, so this source-facing theorem keeps the missing attainment premise explicit rather than suppressing the open-domain counterexample.
open scoped Gradient HessianLocalNorm NewtonDecrement SelfConcordantAuxiliaryFunction
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
variable [IsSelfConcordantOnWith dom Mf f] [HasPositiveDefiniteHessianOn dom f]
theorem existsUnique_isMinOn_with_suboptimality_bound_of_newtonDecrement_lt_inv
{x : E} (hx : x ∈ dom)
(hlambda : λ[f; x | hx] < 1 / (Mf : ℝ))
(hattain : ∃ xStar : dom, IsMinOn f dom (xStar : E)) :
let τω := NewtonDecrement.omegaStarArgOfPosDefMem Mf f x hx hlambda
∃! xStar : dom,
IsMinOn f dom (xStar : E) ∧
f x - f xStar ≤
(1 / (Mf : ℝ) ^ (2 : ℕ)) * ω_* τωTheorem 5.1.14
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.14: if f is self-concordant with positive parameter M_f on dom, the direction h is a recession direction for dom, the backward ray x - τ h from a chosen point x ∈ dom meets frontier dom at finite distance, and h is a nonascent direction for f throughout dom, then the local Hessian norm of h at x is bounded by M_f times the pairing of h with the negative gradient.
open scoped Gradient HessianLocalNorm
universe u
namespace IsSelfConcordantOnWith
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
theorem hessianLocalNorm_le_neg_gradient_inner_of_recession_direction
(hself : IsSelfConcordantOnWith dom Mf f) (hMf : 0 < Mf) {h : E}
(hrecession : ∀ ⦃x : E⦄, x ∈ dom → ∀ t : ℝ, 0 ≤ t → x + t • h ∈ dom)
{x : E} (hx : x ∈ dom)
(hfrontier : ∃ τ : ℝ, 0 < τ ∧ x - τ • h ∈ frontier dom)
(hnonascent : ∀ ⦃y : E⦄, y ∈ dom → inner ℝ (∇ f y) h ≤ 0) :
‖h‖[f; x] ≤ (Mf : ℝ) * inner ℝ (-∇ f x) hTheorem 5.1.15
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.15: the damped Newton step x ↦ x - (1 + M_f λ_f(x))⁻¹ (∇² f(x))⁻¹ ∇ f(x) decreases the objective by at least M_f⁻² ω(M_f λ_f(x)) for a positive self-concordance parameter M_f.
open scoped DikinEllipsoidNotation Gradient HessianLocalNorm NewtonDecrement
open scoped SelfConcordantAuxiliaryFunction
open SelfConcordantNewtonVariant
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
theorem selfConcordant_dampedNewtonStep_value_decrease
{dom : Set E} {Mf : NNReal} {f : E → ℝ}
[IsSelfConcordantOnWith dom Mf f]
{x : E} (hx : x ∈ dom) (hH : (hessian f x).det ≠ 0) (hMf : Mf ≠ 0) :
let δ := NewtonDecrement.ofDetNeZero Mf f hx hH
f (selfConcordantNewtonNextPoint f Mf .damped x hx hH) ≤
f x -
(1 / (Mf : ℝ) ^ (2 : ℕ)) * ω (NewtonDecrement.omegaArgOfDetNeZero Mf f hx hH)
Theorem 5.1.16
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.16: on a finite-dimensional real inner-product space, if a self-concordant objective on the whole space has positive-definite Hessian everywhere and is bounded below, then it attains a unique global minimum.
open scoped NewtonDecrement SelfConcordantAuxiliaryFunction
open SelfConcordantNewtonVariant
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
namespace IsSelfConcordantOnWith
variable {Mf : NNReal} {f : E → ℝ}
variable [IsSelfConcordantOnWith Set.univ Mf f] [HasPositiveDefiniteHessianOn Set.univ f]
theorem existsUnique_isMinOn_of_bddBelow
{M0 : NNReal} [IsSelfConcordantOnWith Set.univ M0 f]
[HasPositiveDefiniteHessianOn Set.univ f]
(hbelow : BddBelow (Set.range f)) :
∃! xStar : E, IsMinOn f Set.univ xStar
Theorem 5.1.17
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.1.17: let F be the +∞-extension of a real-valued function f off Q. If f is self-concordant on Q with constant M_f, if the constrained epigraph of f over Q is closed and Q contains no affine line, then the finite real part of the Fenchel dual F⋆ is self-concordant on its finite-value domain with the same constant M_f.
open scoped ConvexAnalysis Gradient HessianLocalNorm WithTopConvexAnalysis
universe u
variable {E : Type u}
variable [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
[FiniteDimensional ℝ E]
variable {Q : Set E} {Mf : NNReal} {f : E → ℝ}
local notation "F" => fenchelPrimalExtension Q f
-- Semantic recall check: LeanSearch did not expose a direct mathlib owner for this Chapter 5
-- Fenchel-duality self-concordance transfer, so this theorem stays on the local project API.
theorem fenchelPrimalExtension_dualRealPart_isSelfConcordantOnWith
(hclosed : IsClosed (constrainedEpigraph Q (fun y ↦ (f y : WithTop ℝ))))
(hnoAffineLine : ∀ ⦃x h : E⦄, h ≠ 0 → ¬ ∀ τ : ℝ, x + τ • h ∈ Q)
(hself : IsSelfConcordantOnWith Q Mf f) :
IsSelfConcordantOnWith
(dom (F⋆))
Mf
(extendedRealRealPart (F⋆))
5.2 — Minimizing Self-concordant Functions
Definition 5.2.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.1: a self-concordant Newton method of variant variant for f on dom with parameter M_f is a Chapter 1 damped Newton method whose iterates stay in dom and whose step-size schedule is the canonical self-concordant one (1 + ξ_k)⁻¹, where (A) ξ_k = 0 for the standard Newton method, (B) ξ_k = M_f λ_f(x_k) for the damped Newton method, (C) ξ_k = M_f^2 λ_f(x_k)^2 / (1 + M_f λ_f(x_k)) for the intermediate Newton method.
open scoped Gradient
open NewtonDecrement
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
structure DampedNewton.Method.IsSelfConcordant
(dom : Set E) {f : E → ℝ} (Mf : NNReal) [IsSelfConcordantOnWith dom Mf f]
(variant : SelfConcordantNewtonVariant) {x0 : E} (method : DampedNewton.Method f x0) :
Prop where
/-- Every iterate belongs to `dom`. -/
iterates_mem : ∀ k : ℕ, method k ∈ dom
/-- The Chapter 1 damping factors are exactly the canonical self-concordant step sizes from
Definition 5.2.1. -/
stepSize_eq : ∀ k : ℕ,
method.stepSize k =
selfConcordantNewtonStepSize f Mf variant (method k) (iterates_mem k)
(method.hessian_nondegenerate k)Lemma 5.2.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.2.1: along the intermediate self-concordant Newton method (5.2.1)C, the objective drop from x_k to x_{k+1} is bounded below by the explicit rational function of the Newton decrement λ_k. This is the textbook inequality (5.2.2).
open scoped Gradient HessianDualLocalNorm HessianLocalNorm NewtonDecrement
open scoped SelfConcordantAuxiliaryFunction
open SelfConcordantNewtonVariant
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
variable [IsSelfConcordantOnWith dom Mf f]
lemma intermediateNewton_value_drop_lower_bound
{x0 : E}
(method : DampedNewton.Method f x0)
(hmethod : method.IsSelfConcordant dom Mf .intermediate)
(k : ℕ) :
let δ :=
NewtonDecrement.ofDetNeZero Mf f (hmethod.iterates_mem k) (method.hessian_nondegenerate k)
f (method k) - f (method (k + 1)) ≥
δ ^ 2 / (2 * (1 + (Mf : ℝ) * δ + (Mf : ℝ) ^ 2 * δ ^ 2)) +
(Mf : ℝ) * δ ^ 3 / (2 * (1 + (Mf : ℝ) * δ) * (3 + 2 * (Mf : ℝ) * δ))Proposition 5.2.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.2.1: the region of quadratic convergence for the self-concordant Newton method (5.2.1) C is the subset 𝒟_f = {x ∈ dom f | λ_f(x) < 1 / (2 M_f)}.
open scoped Gradient NewtonDecrement
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
def intermediateNewtonQuadraticConvergenceRegion
(dom : Set E) (f : E → ℝ) (Mf : NNRealˣ)
[HasPositiveDefiniteHessianOn dom f] : Set E :=
{x | ∃ hx : x ∈ dom, λ[f; x | hx] < 1 / (2 * (Mf : ℝ))}Remark 5.2.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Remark 5.2.1: in dual coordinates s = ∇ f(y), the central path is the straight line s(t) = t ∇ f(y₀), and the approximate centering condition (5.2.13) is exactly the statement that s lies in the inverse-Hessian quadratic neighborhood of that line with radius β / M_f. This is the project's library-facing form of the textbook dual interpretation using ∇² f_* (s).
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
-- Proof sketch: unfold `satisfies_approximate_centering_condition` and then expand
-- `dualLocalNorm` via `dualLocalNorm_def`. Writing `s = ∇ f y`, the shifted covector becomes
-- `s - t ∇ f(y₀)`, so the condition is exactly the inverse-Hessian quadratic-neighborhood bound
-- around the straight line `t ↦ t ∇ f(y₀)` in dual coordinates.
theorem approximateCenteringCondition_iff_dualCentralPathNeighborhood
{dom : Set E} (f : E → ℝ) [HasPositiveDefiniteHessianOn dom f]
(y0 : dom) (t : ℝ) (y : E) (hy : y ∈ dom) (Mf : NNRealˣ) (β : ℝ) :
satisfies_approximate_centering_condition f y0 t y hy Mf β ↔
let s := ∇ f y
Real.sqrt
(inner ℝ (s - (t : ℝ) • ∇ f (y0 : E))
((hessian f y).inverse (s - (t : ℝ) • ∇ f (y0 : E)))) ≤
β / (Mf : ℝ)
Theorem 5.2.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.2.1: if xStar minimizes a self-concordant function f on dom and the Newton decrement at x is smaller than 1 / M_f, then the scaled suboptimality M_f^2 (f x - f xStar) lies between ω(M_f λ_f(x)) and ω_*(M_f λ_f(x)). This is the textbook inequality (5.2.3).
open scoped DikinEllipsoidNotation Gradient HessianLocalNorm NewtonDecrement
SelfConcordantAuxiliaryFunction
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {dom : Set E} {Mf : NNReal} {f : E → ℝ}
variable [IsSelfConcordantOnWith dom Mf f]
variable [FiniteDimensional ℝ E]
variable [HasPositiveDefiniteHessianOn dom f]
theorem selfConcordant_suboptimality_bounds_of_newtonDecrement_lt_inv
{x : E} (xStar : dom) (hx : x ∈ dom) (hmin : IsMinOn f dom (xStar : E))
(hlambda : λ[f; x | hx] < 1 / (Mf : ℝ)) :
let tω := NewtonDecrement.omegaArgOfPosDefMem Mf f x hx
let τω := NewtonDecrement.omegaStarArgOfPosDefMem Mf f x hx hlambda
ω tω ≤ (Mf : ℝ) ^ (2 : ℕ) * (f x - f xStar) ∧
(Mf : ℝ) ^ (2 : ℕ) * (f x - f xStar) ≤ ω_* τω
Definition 5.2.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.2: the two-stage strategy for self-concordant minimization chooses the damped Newton method when the current Newton decrement is at least 1 / (2 M_f), and otherwise chooses the intermediate method from Definition 5.2.1(C), viewed as the canonical Newton variant.
def selfConcordantTwoStageStrategy
(Mf : NNRealˣ) (decrement : ℝ) : SelfConcordantNewtonVariant :=
if 1 / (2 * (Mf : ℝ)) ≤ decrement then
.damped
else
.intermediate
-- Proof sketch: unfold `selfConcordantTwoStageStrategy`; the result is definitionally the
-- displayed `if` expression selecting between the two canonical variants.Lemma 5.2.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.2.2: if (t, y) satisfies the approximate centering condition (5.2.13) with β = τ² (1 + τ + τ / (1 + τ + τ²)) and τ ≤ 1 / 2, then the path-following update 𝒫_γ(t, y) also satisfies the same centering condition whenever its computed first coordinate lies in [0, 1] and |γ| ≤ τ - τ² (1 + τ + τ / (1 + τ + τ²)). The updated point is read through the canonical intermediate Newton owner for the tilted objective ψ(t₊; ·), so its domain membership is derived rather than assumed separately.
open InnerProductSpace
open scoped Gradient NewtonDecrement AuxiliaryCentralPathNewtonDecrement DikinEllipsoidNotation
HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {dom : Set E} {f : E → ℝ} {Mf : NNRealˣ}
open scoped PathFollowingUpdate
-- Proof sketch: unfold `pathFollowingUpdate`.
theorem pathFollowingUpdate_preserves_approximate_centering_condition
[IsSelfConcordantOnWith dom (Mf : NNReal) f]
[HasPositiveDefiniteHessianOn dom f]
(y0 : dom) (t : Set.Icc (0 : ℝ) 1) {y : E} (hy : y ∈ dom) {τ gamma : ℝ}
(hObjectiveNorm :
0 < HessianDualLocalNorm.ofPosDefMem f hy ((toDual ℝ E) (∇ f (y0 : E))))
(htau : τ ≤ 1 / 2)
(hcenter : satisfies_approximate_centering_condition f y0 t y hy Mf
(pathFollowingCenteringBeta τ))
(hgamma : |gamma| ≤ pathFollowingGammaRadius τ) :
satisfies_approximate_centering_condition f y0
(𝒫[f; Mf; y0 | hy; hObjectiveNorm; gamma](t, y)).1
(𝒫[f; Mf; y0 | hy; hObjectiveNorm; gamma](t, y)).2
(pathFollowingUpdate_snd_mem y0 t hy hObjectiveNorm htau hcenter hgamma) Mf
(pathFollowingCenteringBeta τ)
Proposition 5.2.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.2.2: if the first N iterates of the damped segment of the two-stage self-concordant Newton method remain in Stage 1, then N ≤ M_f^2 (f(x_0) - f(x_f^*)) / ω(1 / 2), where x_f^* minimizes f on the domain.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
open scoped NewtonDecrement
open scoped BigOperators
open scoped SelfConcordantAuxiliaryFunction
theorem selfConcordantTwoStage_stageOneLength_le
{dom : Set E} {f : E → ℝ} {Mf : NNRealˣ} [IsSelfConcordantOnWith dom (Mf : NNReal) f]
{x0 : E}
(method : DampedNewton.Method f x0)
(hmethod : method.IsSelfConcordant dom (Mf : NNReal) SelfConcordantNewtonVariant.damped)
{xStar : E} {N : ℕ}
(hmin : IsMinOn f dom xStar)
(hstage : hmethod.IsStageOneUpTo N) :
(N : ℝ) ≤
(Mf : ℝ) ^ (2 : ℕ) * (f x0 - f xStar) /
selfConcordantOmegaAtOneHalf
Theorem 5.2.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.2.2: the three Chapter 5 Newton updates satisfy the source-facing domain-membership and Newton-decrement bounds for the standard, damped, and intermediate variants.
open InnerProductSpace
open scoped DikinEllipsoidNotation Gradient HessianLocalNorm NewtonDecrement
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {dom : Set E} {Mf : NNRealˣ} {f : E → ℝ}
variable [IsSelfConcordantOnWith dom (Mf : NNReal) f] [HasPositiveDefiniteHessianOn dom f]
theorem selfConcordantNewton_mem_and_decrement_bounds
{x : E} (hx : x ∈ dom) :
let standardBound : Prop :=
(λ[f; x | hx] < 1 / (Mf : ℝ)) →
∃ hxPlus :
selfConcordantNewtonNextPoint f (Mf : NNReal) .standard x hx
(HasPositiveDefiniteHessianOn.hessian_det_ne_zero_of_mem hx) ∈ dom,
λ[f; selfConcordantNewtonNextPoint f (Mf : NNReal) .standard x hx
(HasPositiveDefiniteHessianOn.hessian_det_ne_zero_of_mem hx) | hxPlus] ≤
((Mf : ℝ) * (λ[f; x | hx]) ^ (2 : ℕ)) /
(1 - (Mf : ℝ) * λ[f; x | hx]) ^ (2 : ℕ)
let dampedBound : Prop :=
∃ hxPlus :
selfConcordantNewtonNextPoint f (Mf : NNReal) .damped x hx
(HasPositiveDefiniteHessianOn.hessian_det_ne_zero_of_mem hx) ∈ dom,
λ[f; selfConcordantNewtonNextPoint f (Mf : NNReal) .damped x hx
(HasPositiveDefiniteHessianOn.hessian_det_ne_zero_of_mem hx) | hxPlus] ≤
((Mf : ℝ) * (λ[f; x | hx]) ^ (2 : ℕ)) *
(1 + 1 / (1 + (Mf : ℝ) * λ[f; x | hx]))
let intermediateBound : Prop :=
((Mf : ℝ) * λ[f; x | hx] + (Mf : ℝ) ^ (2 : ℕ) * (λ[f; x | hx]) ^ (2 : ℕ) +
(Mf : ℝ) ^ (3 : ℕ) * (λ[f; x | hx]) ^ (3 : ℕ) ≤ 1) →
∃ hxPlus :
selfConcordantNewtonNextPoint f (Mf : NNReal) .intermediate x hx
(HasPositiveDefiniteHessianOn.hessian_det_ne_zero_of_mem hx) ∈ dom,
λ[f; selfConcordantNewtonNextPoint f (Mf : NNReal) .intermediate x hx
(HasPositiveDefiniteHessianOn.hessian_det_ne_zero_of_mem hx) | hxPlus] ≤
((Mf : ℝ) * (λ[f; x | hx]) ^ (2 : ℕ)) *
(1 + (Mf : ℝ) * λ[f; x | hx] +
((Mf : ℝ) * λ[f; x | hx]) /
(1 + (Mf : ℝ) * λ[f; x | hx] +
(Mf : ℝ) ^ (2 : ℕ) * (λ[f; x | hx]) ^ (2 : ℕ))) ∧
((Mf : ℝ) * (λ[f; x | hx]) ^ (2 : ℕ)) *
(1 + (Mf : ℝ) * λ[f; x | hx] +
((Mf : ℝ) * λ[f; x | hx]) /
(1 + (Mf : ℝ) * λ[f; x | hx] +
(Mf : ℝ) ^ (2 : ℕ) * (λ[f; x | hx]) ^ (2 : ℕ))) ≤
((Mf : ℝ) * (λ[f; x | hx]) ^ (2 : ℕ)) *
(1 + 2 * (Mf : ℝ) * λ[f; x | hx])
standardBound ∧ dampedBound ∧ intermediateBound
Definition 5.2.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
The tilted objective ψ(t; y) = f(y) - t ⟪∇ f(y₀), y⟫ whose minimizers define the auxiliary central path.
open scoped Gradient
open scoped ConstrainedArgmin
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
def auxiliaryCentralPathObjective
{dom : Set E} (f : E → ℝ) (y0 : dom) (t : ℝ) : E → ℝ :=
fun y ↦ f y - (t : ℝ) * inner ℝ (∇ f (y0 : E)) yProposition 5.2.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.2.3: assume the iterate sequence x_k satisfies the global rate f(x_k) - f(x*) ≤ (2^(5/2) c M_f / k^p) * (f(x₀) - f(x*))^(3/2), where x* is a global minimizer of f, M_f = L₃ / (2 * σ₂^(3 / 2)), then N is bounded by the corresponding constant multiple of Δ_f(x₀)^(3 / p), with the natural-number rounding written as 1 + (2^(11/2) c Δ_f(x₀)^3)^(1 / p). Here N is the least index for which x_N enters the Chapter 5 quadratic region 𝒬[f | f(x*), M_f]. Under L₃ > 0, the Chapter 5 threshold f(x_N) - f(x*) ≤ 1 / (8 M_f^2) is equivalent to the older Chapter 4 comparison-region condition 2 L₃² (f(x_N) - f(x*)) ≤ σ₂³.
universe u
open scoped SelfConcordantQuadraticRegion StronglyConvexScaledGap
variable {E : Type u} [NormedAddCommGroup E] [NormedSpace ℝ E]
variable {σ2 : ℝ} {L3 : NNReal}
theorem stronglyConvex_firstSourceQuadraticConvergenceRegionEntryIndex_le
{f : E → ℝ}
(x : ℕ → E) (xStar : E) {c p : ℝ}
(hσ2 : 0 < σ2)
(hmin : IsMinOn f Set.univ xStar)
(hc : 0 < c) (hp : 0 < p)
(hrate :
∀ k : ℕ, 0 < k →
f (x k) - f xStar ≤
(Real.rpow (2 : ℝ) (5 / 2 : ℝ) * c *
(strongConvexSelfConcordanceConstant σ2 L3 : ℝ) *
Real.rpow (f (x 0) - f xStar) (3 / 2 : ℝ)) /
Real.rpow (k : ℝ) p)
{N : ℕ}
(hN :
IsLeast
{n : ℕ | x n ∈ 𝒬[f | f xStar, strongConvexSelfConcordanceConstant σ2 L3]}
N) :
(N : ℝ) ≤
1 +
Real.rpow
(Real.rpow (2 : ℝ) (11 / 2 : ℝ) * c *
(Δ (strongConvexSelfConcordanceConstant σ2 L3 : ℝ) (f (x 0) - f xStar)) ^ (3 : ℕ))
(1 / p)Theorem 5.2.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
A linear tilt preserves the positive-definite-Hessian owner on dom.
open InnerProductSpace
open scoped Gradient NewtonDecrement AuxiliaryCentralPathNewtonDecrement
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
instance auxiliaryCentralPathObjective_hasPositiveDefiniteHessianOn
{dom : Set E} (f : E → ℝ) [HasPositiveDefiniteHessianOn dom f]
(y0 : dom) (t : ℝ) :
HasPositiveDefiniteHessianOn dom (auxiliaryCentralPathObjective f y0 t)
Definition 5.2.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.4 recalls the Chapter 5 Newton-decrement owner specialized to the tilted objective auxiliaryCentralPathObjective f y₀ t.
open InnerProductSpace
open scoped Gradient NewtonDecrement
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
recall newtonDecrementProposition 5.2.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.2.4: the index k_p from Definition 5.2.10 grows at most on the order of Δ_f(x₀)^(1 / p), expressed here as (Δ M_f (f(x₀) - f*))^(1 / p). The asymptotic target is nontrivial only in the source-relevant regime M_f > 0, which is therefore part of the theorem surface.
open Asymptotics Filter
open scoped StronglyConvexHalfGapIndex
open scoped StronglyConvexScaledGap
theorem stronglyConvexHalfGapIndex_isBigO_scaledGap_rpow
(c p : ℝ) (Mf : NNReal) (hMf : 0 < Mf) (hp : 0 < p) :
(fun initialGap ↦ (k[c, Mf; p](initialGap) : ℝ)) =O[atTop]
(fun initialGap ↦ Real.rpow (Δ (Mf : ℝ) initialGap) (1 / p))Theorem 5.2.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.2.4: for a path-following process generated by (5.2.16), every index above the bound (5.2.22) yields an iterate in the quadratic-convergence region of the intermediate Newton method. The textbook statement writes 𝒬_f; here the conclusion is formalized by the region 𝒟[f | dom, M_f] proved in the accompanying argument.
open scoped Gradient NewtonDecrement SelfConcordantAuxiliaryFunction
IntermediateNewtonQuadraticConvergenceRegion
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
theorem selfConcordantPathFollowing_mem_intermediateNewtonQuadraticConvergenceRegion_of_large_index
{dom : Set E} {Mf : NNRealˣ} {f : E → ℝ} [IsSelfConcordantOnWith dom (Mf : NNReal) f]
[HasPositiveDefiniteHessianOn dom f]
{τ : ℝ} (htau : τ ≤ 0.23) (y0 : dom) (xStar : E)
(hmin : IsMinOn f dom xStar)
(process : SelfConcordantPathFollowingProcess f Mf y0 τ)
(hstable :
∀ k : ℕ,
process.y k ∈ 𝒟[f | dom, Mf] →
process.y (k + 1) ∈ 𝒟[f | dom, Mf])
(hdecay :
∀ m : ℕ,
(∀ k : ℕ, k ≤ m →
process.y k ∉ 𝒟[f | dom, Mf]) →
(process.t m : ℝ) ≤
Real.exp
(-(pathFollowingGammaRadius τ * pathFollowingKappa τ * (m : ℝ) ^ (2 : ℕ) /
selfConcordantScaledSuboptimality f Mf (y0 : E) (f xStar))))
(hentry :
∀ m : ℕ,
(process.t m : ℝ) ≤ pathFollowingQuadraticRegionTimeThreshold f (Mf : NNReal) y0 xStar τ →
process.y m ∈ 𝒟[f | dom, Mf])
{N : ℕ}
(hN : pathFollowingQuadraticRegionEntryBound f (Mf : NNReal) y0 xStar τ ≤ (N : ℝ)) :
process.y N ∈ 𝒟[f | dom, Mf]
Definition 5.2.5
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.5 recalls the chapter owner for the approximate centering condition.
recall satisfies_approximate_centering_condition
Theorem 5.2.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.2.5 (1): let k_p = k[c, M_f; p](f(x₀) - f*), and consider the multistage strategy (5.2.28) whose k-th stage (k ≥ 1) runs the lower-level method for t_k = ⌈k_p / 2^((k - 1) / (2p))⌉ steps and stops once f(y_k) - f* ≤ 1 / (8 M_f^2). If each stage output satisfies the source rate estimate f(y_{k+1}) - f* ≤ (2^(5/2) c M_f / t_{k+1}^p) * (f(y_k) - f*)^(3/2), then the total number of stages satisfies T ≤ 4 + log₂ ((Δ M_f (f(x₀) - f*))^2), equivalently T ≤ 4 + log₂ (M_f^2 (f(x₀) - f*)), provided that p > 0 and the initial point starts outside the terminal region 𝒬[f | f*, M_f].
open scoped BigOperators SelfConcordantQuadraticRegion
open scoped StronglyConvexHalfGapIndex StronglyConvexMultiStageAccelerationNotation
open scoped StronglyConvexScaledGap
universe u
variable {E : Type u} {f : E → ℝ}
variable {innerIterate : ℕ → E → E} {c p fStar : ℝ} {Mf : NNReal} {x0 : E}
theorem selfConcordantStronglyConvexStrategy_totalStages_le
{T : ℕ}
(hT :
IsStronglyConvexMultiStageAccelerationStoppingStage
innerIterate 𝒬[f | fStar, Mf] (k[c, Mf; p](f x0 - fStar)) p x0 T)
(hx0 : x0 ∉ 𝒬[f | fStar, Mf])
(hp : 0 < p)
(hstage_rate :
∀ k : ℕ, k < T →
f (y[innerIterate | k[c, Mf; p](f x0 - fStar); p; x0] (k + 1)) - fStar ≤
((Real.rpow (2 : ℝ) (5 / 2 : ℝ) * c * (Mf : ℝ)) /
Real.rpow (t[k[c, Mf; p](f x0 - fStar); p] k : ℝ) p) *
Real.rpow (f (y[innerIterate | k[c, Mf; p](f x0 - fStar); p; x0] k) - fStar)
(3 / 2 : ℝ)) :
(T : ℝ) ≤ 4 + Real.logb 2 ((Δ (Mf : ℝ) (f x0 - fStar)) ^ (2 : ℕ))Definition 5.2.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.6 recalls the chapter owner for the path-following update map 𝒫_γ.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
recall pathFollowingUpdate
Definition 5.2.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.7: the metric attached to a dual-valued operator B : E → E⋆, written on the owner surface as a bilinear form B : LinearMap.BilinForm ℝ E, is recalled through the Chapter 4 owner LinearMap.BilinForm.primalSeminorm.
universe u
open Module LinearMap
open scoped LinearMap.BilinForm.BInducedNorm
variable {E : Type u} [AddCommGroup E] [Module ℝ E]
recall LinearMap.BilinForm.primalSeminorm
(B : LinearMap.BilinForm ℝ E) (hPos : B.toQuadraticMap.PosDef) :
Seminorm ℝ EDefinition 5.2.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.8: a real-valued objective on a real normed space is in the strongly convex C³ minimization regime with parameters σ₂ and L₃ when σ₂ > 0, the objective is σ₂-strongly convex on all of E, it belongs to the chapter smoothness class C22[L₃], and it is three-times continuously differentiable.
universe u
variable {E : Type u} [NormedAddCommGroup E] [NormedSpace ℝ E]
class IsStrongConvexC22C3Objective
(σ2 : ℝ) (L3 : NNReal) (f : E → ℝ) : Prop
extends HasLipschitzContinuousHessian L3 f where
/-- The strong-convexity parameter `σ₂` is positive. -/
sigma_pos : 0 < σ2
/-- The objective is `σ₂`-strongly convex on the whole space. -/
strongConvex : StrongConvexOn Set.univ σ2 f
/-- The objective is three-times continuously differentiable. -/
contDiffThree : ContDiff ℝ 3 fDefinition 5.2.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.9: the Chapter 5 quadratic-convergence region, written in the zero-safe multiplication form 8 M_f² (f(x) - f*) ≤ 1. When M_f = 0, this is all of E, matching the degenerate quadratic regime.
universe u
variable {E : Type u}
def selfConcordantQuadraticRegion
(f : E → ℝ) (fStar : ℝ) (Mf : NNReal) : Set E :=
{x | (8 : ℝ) * (Mf : ℝ) ^ (2 : ℕ) * (f x - fStar) ≤ 1}Definition 5.2.10
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.10: for a positive initial gap and exponent p > 0, the half-gap admissible index set is nonempty. This is the canonical existence API used to discharge the auxiliary witness behind the source threshold index k_p.
theorem stronglyConvexHalfGapAdmissibleIndices_nonempty
{c p initialGap : ℝ} {Mf : NNReal} (hp : 0 < p) (hgap : 0 < initialGap) :
(stronglyConvexHalfGapAdmissibleIndices c Mf p initialGap).NonemptyDefinition 5.2.11
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.2.11: T is the stopping stage of the multistage acceleration strategy (5.2.28) when T is the first outer-stage index whose output lies in the terminal region Q_f.
universe u
variable {E : Type u}
open scoped StronglyConvexMultiStageAccelerationNotation
-- Proof sketch: if `kp ≥ 1`, then the real-valued stage-length expression is strictly positive,
-- so its ceiling is at least `1`.
def IsStronglyConvexMultiStageAccelerationStoppingStage
(innerIterate : ℕ → E → E) (Qf : Set E) (kp : ℕ) (p : ℝ) (x0 : E) (T : ℕ) : Prop :=
IsLeast {n : ℕ | y[innerIterate | kp; p; x0] n ∈ Qf} T5.3 — Self-concordant Barriers
Definition 5.3.0.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.0.1 recalls the canonical owner for standard self-concordance.
universe u
recall IsStandardSelfConcordantOnDefinition 5.3.0.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Membership in Dom f means membership in the closure of the effective domain.
universe u
open scoped WithTopConvexAnalysis
@[simp] theorem mem_Dom_iff {X : Type u} [TopologicalSpace X]
{f : X → WithTop ℝ} {x : X} :
x ∈ Dom f ↔ x ∈ closure (dom f)Corollary 5.3.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.3.1: if f is a standard self-concordant function on dom, then the epigraph barrier, viewed on the canonical L² product owner WithLp 2 (E × ℝ) through ofZ, is also standard self-concordant on the strict epigraph domain from Theorem 5.3.5.
universe u
open Set Topology
open scoped Gradient HessianLocalNorm
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
local notation "Z" => WithLp 2 (E × ℝ)
local notation "ofZ" => (WithLp.ofLp : Z → E × ℝ)
theorem epigraphLogBarrier_isStandardSelfConcordantOn
{dom : Set E} {f : E → ℝ}
(h : IsStandardSelfConcordantOn dom f) :
IsStandardSelfConcordantOn
(ofZ ⁻¹' strictConstrainedEpigraph dom f)
(epigraphLogBarrier f ∘ ofZ)
Definition 5.3.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.1 uses the existing constrained-problem owner together with the existing Chapter 5 standard self-concordance owner.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable (dom : Set E) (c : E) (F : E → ℝ)
recall SetConstrainedMinimizationProblemLemma 5.3.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.3.1: for a standard self-concordant function F on dom and a positive barrier parameter ν, being a ν-self-concordant barrier is equivalent to concavity of the exponential transform x ↦ exp (-(F x / ν)) on dom.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem isSelfConcordantBarrierOnWith_iff_concaveOn_exp_neg_div
{dom : Set E} {ν : NNReal} {F : E → ℝ}
(hFsc : IsStandardSelfConcordantOn dom F) (hν : 0 < (ν : ℝ)) :
IsSelfConcordantBarrierOnWith dom ν F ↔
ConcaveOn ℝ dom (fun x ↦ Real.exp (-(F x / (ν : ℝ))))Proposition 5.3.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.3.1: if x*(t) minimizes the tilted objective x ↦ t ⟪c, x⟫ + f(x) for every t ≥ 0, then at each parameter t where dom is a neighborhood of x*(t) and f is differentiable at x*(t), it satisfies the central-path stationarity equation (t : ℝ) • c + ∇ f(x*(t)) = 0.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
-- Proof sketch: differentiate the linear term `x ↦ (t : ℝ) * ⟪c, x⟫`, whose gradient is
-- `t • c`, then add the canonical gradient of `f`.
theorem centralPath_stationarity_eq_zero
(dom : Set E) (c : E) (f : E → ℝ) (xStar : Set.Ici (0 : ℝ) → dom)
(hpath : IsCentralPath dom c f xStar)
(t : Set.Ici (0 : ℝ))
(hdom : dom ∈ nhds (xStar t : E))
(hf : DifferentiableAt ℝ f (xStar t : E)) :
(t : ℝ) • c + ∇ f (xStar t : E) = 0Theorem 5.3.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.1: if F is standard self-concordant on dom, then the linear perturbation x ↦ ⟪c, x⟫ + F(x) is standard self-concordant on dom. This is the self-concordance part of the textbook statement for a self-concordant barrier.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
-- Proof sketch: apply Corollary 5.1.2 to the standard self-concordant function `F` with
-- quadratic-affine perturbation data `α = 0`, `a = c`, and `A = 0`. The zero operator is
-- positive semidefinite, and `quadraticAffineObjective_zero_operator` identifies this
-- specialization with the linear term `x ↦ inner ℝ c x`.
theorem selfConcordantBarrier_add_linear_isStandardSelfConcordantOn
(dom : Set E) (F : E → ℝ) (c : E) (hF : IsStandardSelfConcordantOn dom F) :
IsStandardSelfConcordantOn dom (fun x ↦ inner ℝ c x + F x)Definition 5.3.1.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.1.1 is a source-facing recall in the chapter's central-path / interior-point domain. Primary domain: - central paths for barrier-penalized linear minimization on a feasible set in a real inner-product space. Sampled owner-style declarations: - centralPathPenaltyObjective in Definition_5_3_6_1, the chapter owner for the penalty objective x ↦ t ⟪c, x⟫ + F x; - centralPathPenaltyObjective_apply, the defining evaluation theorem for that owner; - IsCentralPath in Definition_5_3_6_1, the chapter owner predicate for central paths; - isCentralPath_iff, the companion pointwise minimizer expansion on the canonical IsMinOn owner. Best owner abstraction: - source-facing/core: centralPathPenaltyObjective and IsCentralPath; - bridge/view: centralPathPenaltyObjective_apply and isCentralPath_iff. Primitive data: - dom : Set E - c : E - f : E → ℝ - xStar : Set.Ici (0 : ℝ) → dom Derived API: - the penalty formula centralPathPenaltyObjective c f t x = (t : ℝ) * inner ℝ c x + f x; - the pointwise minimizer expansion of IsCentralPath dom c f xStar. Source/core/bridge triage: - source-facing: the textbook penalty objective and central-path predicate of Definition 5.3.1.1; - core/canonical: the existing Chapter 5 owner declarations in Definition_5_3_6_1; - bridge/view: the evaluation and iff companion theorems. The previous version duplicated those owner declarations verbatim, creating a second public copy of the same central-path surface. This file now recalls the existing owner declarations directly instead of maintaining parallel local definitions.
universe u
recall centralPathPenaltyObjectiveExample 5.3.1.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.3.1.1: the affine function x ↦ α + a x on all of ℝ is not a ν-self-concordant barrier whenever a ≠ 0. In particular, the nonconstant linear specialization x ↦ a x cannot be a self-concordant barrier because its Hessian is zero.
open scoped Gradient
theorem affineFunction_not_isSelfConcordantBarrierOnWith
(α a : ℝ) (ν : NNReal) (ha : a ≠ 0) :
¬ IsSelfConcordantBarrierOnWith (Set.univ : Set ℝ) ν (fun x : ℝ ↦ α + a * x)
Example 5.3.1.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.3.1.2: if the quadratic form u ↦ ⟪u, A u⟫ is strictly positive on nonzero directions, then the quadratic function x ↦ α + ⟪a, x⟫ + (1 / 2) ⟪A x, x⟫ on all of E is not a ν-self-concordant barrier.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
variable [CompleteSpace E]
theorem quadraticAffineObjective_not_isSelfConcordantBarrierOnWith
[Nontrivial E]
(α : ℝ) (a : E) (A : E →L[ℝ] E) (ν : NNReal)
(hApos : ∀ u : E, u ≠ 0 → 0 < inner ℝ u (A u)) :
¬ IsSelfConcordantBarrierOnWith (Set.univ : Set E) ν
(quadraticAffineObjective α a A)Example 5.3.1.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.3.1.3: the logarithmic barrier x ↦ -log x, defined on (0, ∞), is a 1-self-concordant barrier for the nonnegative ray {x : ℝ | 0 ≤ x}.
open scoped Gradient HessianLocalNorm
theorem negLog_isSelfConcordantBarrierOnWith_nonnegativeRay :
IsSelfConcordantBarrierOnWith (Set.Ioi (0 : ℝ)) 1 (fun x : ℝ ↦ -Real.log x)Example 5.3.1.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Example 5.3.1.4: if the quadratic form u ↦ ⟪u, A u⟫ is nonnegative, then the logarithmic barrier associated to the concave affine-quadratic potential φ(x) = α + ⟪a, x⟫ - (1 / 2) ⟪A x, x⟫ is a 1-self-concordant barrier on its positivity domain {x | φ(x) > 0}.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem negLog_concaveQuadratic_isSelfConcordantBarrierOnWith
(α : ℝ) (a : E) (A : E →L[ℝ] E)
(hA_posSemidef : ∀ u : E, 0 ≤ inner ℝ u (A u)) :
IsSelfConcordantBarrierOnWith
{x : E | 0 < α + inner ℝ a x - (1 / 2 : ℝ) * inner ℝ (A x) x}
1
(fun x ↦ -Real.log (α + inner ℝ a x - (1 / 2 : ℝ) * inner ℝ (A x) x))
Corollary 5.3.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.3.2: if F is a ν-self-concordant barrier on dom and h is a recession direction of dom, then at every x ∈ dom the Hessian local norm of h is bounded by the pairing of h with the negative gradient.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantBarrierOnWith
variable {dom : Set E} {ν : NNReal} {F : E → ℝ}
variable {h x : E}
theorem hessianLocalNorm_le_neg_gradient_inner_of_recession_direction
(hF : IsSelfConcordantBarrierOnWith dom ν F)
(hrecession : ∀ ⦃y : E⦄, y ∈ dom → ∀ τ : ℝ, 0 ≤ τ → y + τ • h ∈ dom)
(hx : x ∈ dom)
:
‖h‖[F; x] ≤ inner ℝ (-∇ F x) hDefinition 5.3.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.2: a standard self-concordant function F is a ν-self-concordant barrier for dom when, for every x ∈ dom, every direction u satisfies the barrier inequality 2 ⟪∇F(x), u⟫ - ⟪u, ∇²F(x)u⟫ ≤ ν. The constant ν is the barrier parameter.
open Set Topology
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
class IsSelfConcordantBarrierOnWith (dom : Set E) (ν : NNReal) (F : E → ℝ) : Prop where
/-- A self-concordant barrier is standard self-concordant on its open convex domain. -/
toIsStandardSelfConcordantOn : IsStandardSelfConcordantOn dom F
/-- The barrier parameter bounds the gradient term by the Hessian quadratic form at each point
of the domain. -/
barrier_parameter_bound {x : E} (hx : x ∈ dom) (u : E) :
2 * inner ℝ (∇ F x) u - inner ℝ u (hessian F x u) ≤ (ν : ℝ)Lemma 5.3.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.3.2: if x in the domain of a ν-self-concordant barrier F satisfies the approximate-centering condition ‖t c + ∇ F(x)‖*ₓ ≤ β for some t > 0, then the dual local norm of the objective vector is bounded by ‖c‖*ₓ ≤ (β + √ν) / t.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
theorem dualLocalNorm_objectiveVector_le_add_sqrt_barrierParameter_div
{dom : Set E} {ν : NNReal} {F : E → ℝ}
[IsSelfConcordantBarrierOnWith dom ν F]
(c : E) {t β : ℝ} (ht : 0 < t)
{x : dom}
(hH : (fderiv ℝ (∇ F) (x : E)).det ≠ 0)
(happrox :
HessianDualLocalNorm.ofDetNeZero F (x : E)
(IsSelfConcordantOnWith.hessian_isPositive_of_mem 1 x.2) hH
((InnerProductSpace.toDual ℝ E) (t • c + ∇ F (x : E))) ≤ β) :
HessianDualLocalNorm.ofDetNeZero F (x : E)
(IsSelfConcordantOnWith.hessian_isPositive_of_mem 1 x.2) hH
((InnerProductSpace.toDual ℝ E) c) ≤
(β + Real.sqrt (ν : ℝ)) / tProposition 5.3.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.3.2, pointwise owner form: if the Hessian at x is positive and invertible, then the barrier inequality from Definition 5.3.2 is equivalent to the Chapter 5 pointwise Newton-decrement bound λ[F; x | hPos; hInv] ≤ √ν.
open scoped Gradient NewtonDecrement
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
-- Proof sketch: complete the square with
-- `u - (hessian F x).inverse (∇ F x)` and then rewrite the maximizing value by
-- `newtonDecrement_def`, identifying it with the pointwise Newton decrement
-- `λ[F; x | hPos; hInv]`.
theorem barrier_parameter_bound_iff_newtonDecrement_le_sqrt
{F : E → ℝ} {ν : NNReal} {x : E} (hPos : (hessian F x).IsPositive)
(hInv : (hessian F x).IsInvertible) :
(∀ u : E,
2 * inner ℝ (∇ F x) u - inner ℝ u (hessian F x u) ≤ (ν : ℝ)) ↔
λ[F; x | hPos; hInv] ≤ Real.sqrt (ν : ℝ)Theorem 5.3.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.2: the pointwise sum of a ν₁-self-concordant barrier F₁ on dom₁ and a ν₂-self-concordant barrier F₂ on dom₂ is a (\nu₁ + \nu₂)-self-concordant barrier on the intersection domain dom₁ ∩ dom₂.
open Set Topology
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantBarrierOnWith
-- Proof sketch: the standard self-concordance component is the owner theorem
-- `IsStandardSelfConcordantOn.add`. For the barrier parameter, expand the gradient and Hessian of
-- the sum at `x ∈ dom₁ ∩ dom₂`; the displayed expression splits into the two summand barrier
-- expressions, which are bounded by `ν₁` and `ν₂`.
theorem add
{dom₁ dom₂ : Set E} {ν₁ ν₂ : NNReal} {F₁ F₂ : E → ℝ}
(h₁ : IsSelfConcordantBarrierOnWith dom₁ ν₁ F₁)
(h₂ : IsSelfConcordantBarrierOnWith dom₂ ν₂ F₂) :
IsSelfConcordantBarrierOnWith (dom₁ ∩ dom₂) (ν₁ + ν₂) (F₁ + F₂)Corollary 5.3.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.3.3: along the segment from x to y inside the domain of a ν-self-concordant barrier, the barrier value at x + α • (y - x) is bounded above by F x - ν log (1 - α) for every α ∈ [0, 1).
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantBarrierOnWith
-- Proof sketch: use the owner theorem
-- `IsSelfConcordantBarrierOnWith.concaveOn_exp_neg_div` to make
-- `x ↦ exp (-(F x / p))` concave on `dom`, with `p = ν` when `ν > 0` and `p = 1` when `ν = 0`.
-- Evaluate concavity on the convex combination `(1 - α) • x + α • y = x + α • (y - x)`, drop the
-- nonnegative `α`-endpoint term, take logarithms, and rearrange. In the degenerate case `ν = 0`,
-- the same concavity argument with `p = 1` still yields the stronger monotonicity bound
-- `F (x + α • (y - x)) ≤ F x`, because `log (1 - α) ≤ 0`.
theorem segment_upper_bound_log_one_sub
{dom : Set E} {ν : NNReal} {F : E → ℝ}
(hF : IsSelfConcordantBarrierOnWith dom ν F)
{x y : E} (hx : x ∈ dom) (hy : y ∈ dom) {α : ℝ} (hα : α ∈ Set.Ico (0 : ℝ) 1) :
F (x + α • (y - x)) ≤ F x - (ν : ℝ) * Real.log (1 - α)Definition 5.3.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.3: the analytic center of the convex set dom, generated by the barrier F, is exactly a point xStar ∈ dom whose ambient point (xStar : E) minimizes F on dom.
universe u
variable {E : Type u}
variable (dom : Set E) (F : E → ℝ) (xStar : dom)
recall IsMinOnLemma 5.3.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.3.3: along an auxiliary central path y*(t) for a bounded ν-self-concordant barrier, the dual local norm of the gradient at time t ≥ 0 is bounded by (ν + 2 √ν) times the dual local norm of the initial gradient at the analytic center x_F^*, multiplied by t.
open scoped BInducedNorm Gradient HessianDualLocalNorm
universe u
open InnerProductSpace
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
variable [FiniteDimensional ℝ E]
namespace IsSelfConcordantBarrierOnWith
theorem dualLocalNorm_gradient_auxiliaryCentralPath_le_barrierParameter_add_two_sqrt_mul_initial
{dom : Set E} {ν : NNReal} {F : E → ℝ}
(hF : IsSelfConcordantBarrierOnWith dom ν F)
[HasPositiveDefiniteHessianOn dom F]
(y0 : dom) (yStar : Set.Ici (0 : ℝ) → dom)
(xFStar : dom) (hxFStar : IsMinOn F dom (xFStar : E))
(hpath : IsCentralPath dom (-∇ F (y0 : E)) F yStar)
(t : Set.Ici (0 : ℝ)) :
HessianDualLocalNorm.ofPosDefMem F (yStar t).2 (toDual ℝ E (∇ F (yStar t : E))) ≤
(((ν : ℝ) + 2 * Real.sqrt (ν : ℝ)) *
HessianDualLocalNorm.ofPosDefMem F xFStar.2 (toDual ℝ E (∇ F (y0 : E)))) *
(t : ℝ)
Proposition 5.3.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.3.3, pointwise owner form: at a fixed point x with positive Hessian, the barrier inequality 2 ⟪∇ F(x), u⟫ - ⟪∇² F(x)u, u⟫ ≤ ν for every direction u is equivalent to the quadratic-form bound ⟪∇ F(x), u⟫² ≤ ν ‖u‖[F; x]^2 for every u, written on the canonical Chapter 5 local-norm surface.
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem barrier_parameter_bound_iff_gradient_inner_sq_le
{F : E → ℝ} {ν : NNReal} {x : E} (hPos : (hessian F x).IsPositive) :
(∀ u : E,
2 * inner ℝ (∇ F x) u - inner ℝ u (hessian F x u) ≤ (ν : ℝ)) ↔
∀ u : E,
(inner ℝ (∇ F x) u) ^ (2 : ℕ) ≤
(ν : ℝ) * ‖u‖[F; x] ^ (2 : ℕ)Theorem 5.3.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.3: if F is a ν-self-concordant barrier on dom ⊆ E₁, then its precomposition with a continuous affine map g : E →ᴬ[ℝ] E₁ is a ν-self-concordant barrier on the affine preimage g ⁻¹' dom. This is the owner-level affine-pullback theorem.
universe u v
open scoped Gradient HessianLocalNorm
variable {E : Type u} {E₁ : Type v}
variable [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable [NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
namespace IsSelfConcordantBarrierOnWith
-- Proof sketch: apply `IsSelfConcordantOnWith.comp_continuousAffineMap` to the standard
-- self-concordance field of the barrier owner. For the barrier parameter, fix `x ∈ g ⁻¹' dom`
-- and rewrite the gradient and Hessian quadratic form of `F ∘ g` in the direction `u` through
-- the image direction `g.toAffineMap.linear u`; then apply the owner bound for `F` at `g x`.
theorem comp_continuousAffineMap
{dom : Set E₁} {ν : NNReal} {F : E₁ → ℝ}
(h : IsSelfConcordantBarrierOnWith dom ν F) (g : E →ᴬ[ℝ] E₁) :
IsSelfConcordantBarrierOnWith (g ⁻¹' dom) ν (F ∘ g)Corollary 5.3.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Corollary 5.3.4: if xStar is an analytic center of a ν-self-concordant barrier on dom, then for every x ∈ dom the Hessian at x dominates the Hessian at xStar in Loewner order by the factor (ν + 2 √ν)⁻².
open InnerProductSpace
open scoped Gradient HessianDualLocalNorm HessianLocalNorm DikinEllipsoidNotation
universe u
namespace IsSelfConcordantBarrierOnWith
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem hessian_loewner_lower_bound_of_isMinOn
{dom : Set E} {ν : NNReal} {F : E → ℝ}
(hF : IsSelfConcordantBarrierOnWith dom ν F)
(xStar : dom) (hcenter : IsMinOn F dom (xStar : E))
(x : dom) :
(1 / (((ν : ℝ) + 2 * Real.sqrt (ν : ℝ)) ^ (2 : ℕ))) •
hessian F (xStar : E) ≤
hessian F x
Proposition 5.3.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.3.4: let F_* = extendedRealRealPart (f⋆). Assume the canonical dual satisfies relation (5.1.34), namely ∇F_*(s) = ∇²F_*(s) s, is standard self-concordant on dom (f⋆), and obeys ⟪s, ∇²F_*(s) s⟫ ≤ ν on dom (f⋆). Then F_* is a ν-self-concordant barrier on its effective domain.
open InnerProductSpace
open scoped ConvexAnalysis Gradient WithTopConvexAnalysis
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {f : E → WithTop ℝ}
-- Proof sketch: for fixed `s ∈ dom (f⋆)` and direction `u`, positivity of
-- `hessian (extendedRealRealPart (f⋆)) s` gives
-- `0 ≤ ⟪u - s, ∇²F_*(s) (u - s)⟫`. Expanding this quadratic form and using the symmetry of a
-- positive Hessian shows
-- `2 ⟪∇²F_*(s) s, u⟫ - ⟪u, ∇²F_*(s) u⟫ ≤ ⟪s, ∇²F_*(s) s⟫`. Relation `(5.1.34)` rewrites the left
-- side as the barrier expression, so the assumed source bound is exactly the defining barrier
-- parameter inequality.
theorem fenchelConjugate_realPart_isSelfConcordantBarrierOnWith
{ν : NNReal}
(hdual_gradient_eq_hessian_apply_self :
∀ ⦃s : E⦄, s ∈ dom (f⋆) →
∇ (extendedRealRealPart (f⋆)) s =
hessian (extendedRealRealPart (f⋆)) s s)
(hsc : IsStandardSelfConcordantOn (dom (f⋆)) (extendedRealRealPart (f⋆)))
(hbound :
∀ s ∈ dom (f⋆),
inner ℝ s (hessian (extendedRealRealPart (f⋆)) s s) ≤ (ν : ℝ)) :
IsSelfConcordantBarrierOnWith (dom (f⋆)) ν (extendedRealRealPart (f⋆))Theorem 5.3.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.4: if f is self-concordant on dom with constant M_f and if f x ≥ f^* on dom, then for every ν ≥ 1 + M_f^2 (β - f^*) the scaled logarithmic barrier x ↦ -ν log (β - f x) is a ν-self-concordant barrier on the strict level-set domain {x ∈ dom | f x < β}.
open Set Topology
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
-- Proof sketch: apply Theorem 5.1.4 to `φ x = -log (β - f x)` to obtain self-concordance of `φ`
-- on the strict sublevel domain. If `ν = 0`, then the lower bound and the parameter inequality
-- force the strict sublevel domain to be empty, so the barrier statement is vacuous. If `ν > 0`,
-- pick a point in the strict sublevel domain to deduce `β - fStar > 0`, rewrite the owner
-- constant from Theorem 5.1.4 without `Real.toNNReal`, rescale by `ν`, and use Corollary 5.1.3
-- to make the self-concordance constant standard. Finally, Proposition 5.3.3 upgrades the
-- Hessian-gradient inequality from Theorem 5.1.4 to the barrier parameter bound with parameter
-- `ν`.
theorem sublevelLogBarrier_smul_isSelfConcordantBarrierOnWith
{dom : Set E} {Mf ν : NNReal} {f : E → ℝ} {β fStar : ℝ}
(hself : IsSelfConcordantOnWith dom Mf f)
(h_lower : ∀ ⦃x : E⦄, x ∈ dom → fStar ≤ f x)
(hν : 1 + (Mf : ℝ) ^ (2 : ℕ) * (β - fStar) ≤ (ν : ℝ)) :
IsSelfConcordantBarrierOnWith
{x : E | x ∈ dom ∧ f x < β}
ν
((ν : ℝ) • sublevelLogBarrier f β)Definition 5.3.4.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.4.1: a main path-following scheme for minimizing ⟪c, x⟫ over the closure of the domain of a ν-self-concordant barrier F starts from t₀ = 0 and a point x₀ ∈ dom with ‖∇ F(x₀)‖*_(x₀) ≤ β, updates tₖ₊₁ = tₖ + γ / ‖c‖*_(xₖ) and xₖ₊₁ = xₖ - (1 + ξₖ)⁻¹ [∇² F(xₖ)]⁻¹ (tₖ₊₁ c + ∇ F(xₖ)) with ξₖ = λₖ² / (1 + λₖ) and λₖ = ‖tₖ₊₁ c + ∇ F(xₖ)‖*_(xₖ), and stops at the first index where tₖ reaches the threshold (5.3.29).
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
structure BarrierPathFollowingScheme
{dom : Set E} (c : E) (F : E → ℝ) (ν : NNReal)
[IsSelfConcordantBarrierOnWith dom ν F]
(x0 : dom) (β γ ε : ℝ) where
/-- The scalar path parameters `t₀, t₁, t₂, ...`. -/
t : ℕ → ℝ
/-- The primal iterates `x₀, x₁, x₂, ...`. -/
x : ℕ → E
/-- Every iterate stays in the barrier domain. -/
mem_domain : ∀ k : ℕ, x k ∈ dom
/-- The Hessian at each iterate is nondegenerate, so the local dual norms and inverse-Hessian
correction are defined. -/
hessian_nondegenerate : ∀ k : ℕ, (fderiv ℝ (∇ F) (x k)).det ≠ 0
/-- The initial centering parameter satisfies the textbook condition `β < 1`. -/
beta_lt_one : β < 1
/-- The requested accuracy is positive. -/
epsilon_pos : 0 < ε
/-- The scheme starts from `t₀ = 0`. -/
t_zero : t 0 = 0
/-- The initial iterate is the prescribed point `x₀`. -/
x_zero : x 0 = x0
/-- The starting point satisfies the initial centering condition
`‖∇ F(x₀)‖*_(x₀) ≤ β`. -/
initial_centering :
hessianDualNormVector F (x 0) (hessian_nondegenerate 0) (∇ F (x 0)) ≤ β
/-- The scalar update is `tₖ₊₁ = tₖ + γ / ‖c‖*_(xₖ)`. -/
t_step :
∀ k : ℕ,
t (k + 1) =
t k + γ / barrierPathFollowingObjectiveNorm F c (x k) (hessian_nondegenerate k)
/-- The iterate update is the intermediate Newton correction for the shifted gradient
`tₖ₊₁ c + ∇ F(xₖ)`. -/
x_step :
∀ k : ℕ,
x (k + 1) =
x k -
(1 /
(1 + selfConcordantNewtonShift SelfConcordantNewtonVariant.intermediate 1
(barrierPathFollowingResidualNorm F c (t (k + 1)) (x k)
(hessian_nondegenerate k)))) •
(hessian F (x k)).inverse
((t (k + 1)) • c + ∇ F (x k))
/-- The first index at which the stopping threshold `(5.3.29)` is reached. -/
stopIndex : ℕ
/-- Before the stopping index, the path parameter stays below the threshold from `(5.3.29)`. -/
continue_condition :
∀ ⦃k : ℕ⦄, k < stopIndex →
t k < barrierPathFollowingStoppingThreshold ν β ε
/-- At the stopping index, the path parameter reaches the threshold from `(5.3.29)`. -/
stop_condition :
barrierPathFollowingStoppingThreshold ν β ε ≤ t stopIndexTheorem 5.3.5
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.5: if f is a ν-self-concordant barrier on dom, then the textbook epigraph barrier, viewed on the canonical L² product owner WithLp 2 (E × ℝ) through WithLp.ofLp, is a (\nu + 1)-self-concordant barrier on the strict epigraph domain.
universe u
open Set Topology
open scoped Gradient HessianLocalNorm
variable {E : Type u}
variable [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
local notation "Z" => WithLp 2 (E × ℝ)
local notation "ofZ" => (WithLp.ofLp : Z → E × ℝ)
theorem epigraphLogBarrier_isSelfConcordantBarrierOnWith
{dom : Set E} {ν : NNReal} {f : E → ℝ}
(h : IsSelfConcordantBarrierOnWith dom ν f) :
IsSelfConcordantBarrierOnWith
(ofZ ⁻¹' strictConstrainedEpigraph dom f)
(ν + 1)
(epigraphLogBarrier f ∘ ofZ)
Definition 5.3.5.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.5.1: a stopped intermediate self-concordant Newton method on a standard self-concordant function F on dom, started at y₀ ∈ dom with stopping threshold β, consists of the underlying intermediate self-concordant Newton iteration yₖ₊₁ = yₖ - [∇²F(yₖ)]⁻¹ ∇F(yₖ) / (1 + ξₖ) with ξₖ = λₖ^2 / (1 + λₖ) and λₖ = ‖∇ F(yₖ)‖*_(yₖ), together with a stopping index where the Newton decrement first drops below the tolerance β. In the analytic-center application, the extra barrier data belongs in downstream assumptions rather than in this owner.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
structure StoppedIntermediateSelfConcordantNewtonMethod
{dom : Set E} (F : E → ℝ) [IsStandardSelfConcordantOn dom F] (y0 : dom) (β : ℝ)
extends DampedNewton.Method F (y0 : E) where
/-- The underlying Chapter 1 method is the intermediate self-concordant Newton method. -/
isSelfConcordant :
toMethod.IsSelfConcordant dom 1 SelfConcordantNewtonVariant.intermediate
/-- The stopping index at which the Newton decrement meets the tolerance `β`. -/
stopIndex : ℕ
/-- Before the stopping index, the Newton decrement remains strictly above `β`. -/
continue_condition :
∀ ⦃k : ℕ⦄, k < stopIndex → β < isSelfConcordant.decrement k
/-- At the stopping index, the Newton decrement is at most `β`. -/
stop_condition : isSelfConcordant.decrement stopIndex ≤ βDefinition 5.3.5.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.5.2 specializes the central-path owner to the choice c = -∇ F(y₀).
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
variable {dom : Set E} (F : E → ℝ) (y0 : dom)
variable (yStar : Set.Ici (0 : ℝ) → dom)
#check IsCentralPath dom (-∇ F (y0 : E)) F yStarDefinition 5.3.5.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.5.3: for a ν-self-concordant barrier F on dom, the stopped intermediate Newton method started at y₀ with tolerance β is exactly the chapter owner StoppedIntermediateSelfConcordantNewtonMethod F y0 β. The barrier hypothesis contributes only the inferable standard self-concordance needed by that owner.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {dom : Set E} {F : E → ℝ} {ν : NNReal} [IsSelfConcordantBarrierOnWith dom ν F]
variable {y0 : dom} {β : ℝ}
#check (StoppedIntermediateSelfConcordantNewtonMethod F y0 β)Theorem 5.3.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.6: if Φ is a ν-self-concordant barrier on Q ⊆ E₁ × E₂, and if for every x ∈ D, where D = dom (partialInfProjection Q (Real.toEReal ∘ Φ)), the fiber problem min_y Φ(x, y) over (Prod.mk x) ⁻¹' Q is attained uniquely at y x, then the canonical real surface of the partial infimal projection is a ν-self-concordant barrier on its natural domain.
open scoped ConstrainedArgmin ConvexAnalysis Gradient
open ContinuousLinearMap
universe u₁ u₂
variable {E₁ : Type u₁} {E₂ : Type u₂}
variable [NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [FiniteDimensional ℝ E₁]
variable [NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [FiniteDimensional ℝ E₂]
local notation "Z" => WithLp 2 (E₁ × E₂)
variable {Q : Set (E₁ × E₂)} {ν : NNReal} {Φ : E₁ × E₂ → ℝ} {y : E₁ → E₂}
local notation "QZ" => ((WithLp.ofLp : Z → E₁ × E₂) ⁻¹' Q)
local notation "psiObj" => partialInfProjection Q (Real.toEReal ∘ Φ)
local notation "D" => dom psiObj
local notation "f" => extendedRealRealPart psiObj
theorem partialMinimizationObjective_isSelfConcordantBarrierOnWith_of_argmin
(hΦ : IsSelfConcordantBarrierOnWith QZ ν (Φ ∘ WithLp.ofLp))
(hy_argmin : ∀ ⦃x : E₁⦄, x ∈ D → y x ∈ argmin[(Prod.mk x) ⁻¹' Q] (Φ ∘ Prod.mk x))
(hy_unique : ∀ ⦃x : E₁⦄, x ∈ D → ∀ ⦃y' : E₂⦄,
y' ∈ argmin[(Prod.mk x) ⁻¹' Q] (Φ ∘ Prod.mk x) → y' = y x) :
IsSelfConcordantBarrierOnWith D ν f
Definition 5.3.6.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.3.6.1: for the problem min {(c, z) : z ∈ S, ⟪d, z⟫ ≤ 0}, choose α = ⟪d, z₀⟫ + Δ with Δ > 0; if α ≤ 0, the method falls back to the two-stage process of Definition 5.3.5.3, and if α > 0, it first finds a β-approximate analytic center of S(α), then follows the d-central path to a β-approximate analytic center of the reduced set {z ∈ S(α) | ⟪d, z⟫ ≤ 0}, and finally follows the c-central path for the reduced problem.
open scoped Gradient NewtonDecrement
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
variable [FiniteDimensional ℝ E]
variable (S : Set E) (c d : E) (F : E → ℝ) (ν : NNReal)
variable [IsSelfConcordantBarrierOnWith S ν F]
variable (z0 : S) (β Δ : ℝ)
inductive FunctionalConstraintPathFollowingScheme
(S : Set E) (c d : E) (F : E → ℝ) (ν : NNReal)
[IsSelfConcordantBarrierOnWith S ν F] (z0 : S) (β Δ : ℝ)
(delta_pos : 0 < Δ) : Type u where
| fallback
{alpha_nonpos : functionalConstraintAlpha d z0 Δ ≤ 0}
(scheme :
NonpositiveFunctionalConstraintPathFollowingScheme
S c d F ν z0 β Δ delta_pos alpha_nonpos) :
FunctionalConstraintPathFollowingScheme S c d F ν z0 β Δ delta_pos
| positive
{alpha_pos : 0 < functionalConstraintAlpha d z0 Δ}
(scheme :
PositiveFunctionalConstraintPathFollowingScheme
S c d F ν z0 β Δ delta_pos alpha_pos) :
FunctionalConstraintPathFollowingScheme S c d F ν z0 β Δ delta_posProposition 5.3.6.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.3.6.1: if ξBar bounds the original objective on the original feasible set, then the original problem and its standard-form reformulation have the same canonical Chapter 1 optimal value.
universe u
variable {X : Type u} {m : ℕ}
local notation "StdPoint" => ℝ × ℝ × X
theorem functionalConstraintOptimalValue_eq_standardFormOptimalValue
{Q : Set X} {f0 : X → ℝ} {fj : Fin m → X → ℝ} {xiBar : ℝ}
(hUpper :
∀ x : Q, x ∈ (functionalConstraintProblem Q f0 fj).feasibleSet → f0 x ≤ xiBar) :
(functionalConstraintProblem Q f0 fj).toSetConstrainedMinimizationProblem.optimalValue =
(functionalConstraintStandardFormProblem Q f0 fj xiBar).optimalValueProposition 5.3.6.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.3.6.2: if F_Q is a self-concordant barrier for Q, if F₀ is a self-concordant barrier on the canonical L² pair owner over the strict epigraph of f₀, and if each Fⱼ is a self-concordant barrier on the canonical L² pair owner over the strict epigraph of fⱼ, then F_Q(x) + F₀(x, ξ) + ∑ⱼ Fⱼ(x, κ) - log (ξBar - ξ) - log (-κ) is a self-concordant barrier for the strict reformulated barrier domain ξ < ξBar, κ < 0, x ∈ Q, f₀(x) < ξ, fⱼ(x) < κ, which sits inside the closed feasible set of Proposition 5.3.6.1, with parameter v_Q + v₀ + ∑ⱼ vⱼ + 2, all viewed on the canonical L² owners through WithLp.ofLp.
universe u
variable {X : Type u} {m : ℕ}
local notation "StdPoint" => ℝ × ℝ × X
local notation "PairPoint" => X × ℝ
local notation "StatePoint" => ℝ × X
local notation "PairPointL2" => WithLp 2 PairPoint
local notation "StatePointL2" => WithLp 2 StatePoint
local notation "StdPointL2" => WithLp 2 (ℝ × StatePointL2)
local notation "ofPairPoint" => (WithLp.ofLp : PairPointL2 → PairPoint)
local notation "ofStatePoint" => (WithLp.ofLp : StatePointL2 → StatePoint)
local notation "ofStdPoint" =>
(Prod.map id ofStatePoint) ∘ (WithLp.ofLp : StdPointL2 → ℝ × StatePointL2)
variable [NormedAddCommGroup X] [InnerProductSpace ℝ X] [CompleteSpace X]
theorem functionalConstraintStandardFormBarrier_isSelfConcordantBarrierOnWith
{Q : Set X} {f0 : X → ℝ} {fj : Fin m → X → ℝ}
{FQ : X → ℝ} {F0 : X × ℝ → ℝ} {Fj : Fin m → X × ℝ → ℝ}
{xiBar : ℝ} {vQ v0 : NNReal} {vj : Fin m → NNReal}
(hFQ : IsSelfConcordantBarrierOnWith Q vQ FQ)
(hF0 : IsSelfConcordantBarrierOnWith
(ofPairPoint ⁻¹' strictConstrainedEpigraph (Set.univ : Set X) f0)
v0 (F0 ∘ ofPairPoint))
(hFj : ∀ j : Fin m,
IsSelfConcordantBarrierOnWith
(ofPairPoint ⁻¹' strictConstrainedEpigraph (Set.univ : Set X) (fj j))
(vj j) (Fj j ∘ ofPairPoint)) :
IsSelfConcordantBarrierOnWith
(ofStdPoint ⁻¹'
functionalConstraintStandardFormBarrierDomain Q f0 fj xiBar)
(vQ + v0 + (∑ j : Fin m, vj j) + 2)
(functionalConstraintStandardFormBarrier FQ F0 Fj xiBar ∘ ofStdPoint)
Theorem 5.3.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.7: for a standard self-concordant function F, being a ν-self-concordant barrier is equivalent to the logarithmic lower Taylor bound F(y) ≥ F(x) - ν log (1 - ν⁻¹ ⟪∇ F(x), y - x⟫) on the domain, together with the required positivity of the logarithm argument.
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem isSelfConcordantBarrierOnWith_iff_logarithmic_taylor_lower_bound
{dom : Set E} {ν : NNReal} {F : E → ℝ}
(hFsc : IsStandardSelfConcordantOn dom F) (hν : 0 < (ν : ℝ)) :
IsSelfConcordantBarrierOnWith dom ν F ↔
∀ {x y : E} (hx : x ∈ dom) (hy : y ∈ dom),
let t := 1 - (1 / (ν : ℝ)) * inner ℝ (∇ F x) (y - x)
0 < t ∧ F y ≥ F x - (ν : ℝ) * Real.log tTheorem 5.3.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.8: if F is a ν-self-concordant barrier on dom and the gradient pairing ⟪∇ F(x), y - x⟫ is nonnegative, then the local norm of the chord y - x at x is bounded by ν + 2 √ν.
open scoped Gradient HessianLocalNorm DikinEllipsoidNotation
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantBarrierOnWith
variable {dom : Set E} {ν : NNReal} {F : E → ℝ}
variable {x y : E}
theorem hessianLocalNorm_sub_le_barrierParameter_add_two_sqrt_of_gradient_inner_nonneg
(hF : IsSelfConcordantBarrierOnWith dom ν F)
(hx : x ∈ dom) (hy : y ∈ dom)
(hxy_nonneg : inner ℝ (∇ F x) (y - x) ≥ 0) :
‖y - x‖[F; x] ≤ (ν : ℝ) + 2 * Real.sqrt (ν : ℝ)Theorem 5.3.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.9 (1): if xStar is an analytic center of a ν-self-concordant barrier F, then the whole domain dom lies in the Dikin ellipsoid centered at xStar with radius ν + 2 √ν. Equivalently, every x ∈ dom satisfies the local-distance bound (5.3.17).
open scoped Gradient HessianLocalNorm DikinEllipsoidNotation
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantBarrierOnWith
theorem subset_dikinEllipsoid_barrierParameter_add_two_sqrt_of_isMinOn
{dom : Set E} {ν : NNReal} {F : E → ℝ}
(hF : IsSelfConcordantBarrierOnWith dom ν F)
(xStar : dom) (hcenter : IsMinOn F dom (xStar : E)) :
dom ⊆ W[F; (xStar : E)]((ν : ℝ) + 2 * Real.sqrt (ν : ℝ))Theorem 5.3.10
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.10: if xPath is an exact central-path point for the penalty objective z ↦ t ⟪c, z⟫ + F z at some t > 0, and if another point x in dom satisfies the approximate-centering condition ‖t c + ∇ F(x)‖*ₓ ≤ β with β < 1, then the objective gap from x to any optimal point xOpt ∈ closure dom is bounded by (ν + ((β + √ν) β) / (1 - β)) / t. In particular, the exact central-path gap is recovered by the companion theorem above.
open Filter Set Topology
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
theorem centralPathApproximateCenter_objectiveGap_le_barrierParameter_add_error_div
{dom : Set E} {ν : NNReal} {F : E → ℝ}
[IsSelfConcordantBarrierOnWith dom ν F]
(c : E) (t : Set.Ici (0 : ℝ)) {β : ℝ} (ht : 0 < (t : ℝ)) (hβ : β < 1)
(xOpt : closure dom)
(hopt : ∀ y : closure dom, inner ℝ c (xOpt : E) ≤ inner ℝ c (y : E))
{xPath x : dom}
(hpath : IsMinOn (centralPathPenaltyObjective c F t) dom (xPath : E))
(hxH : (fderiv ℝ (∇ F) (x : E)).det ≠ 0)
(happrox :
HessianDualLocalNorm.ofDetNeZero F (x : E)
(IsSelfConcordantOnWith.hessian_isPositive_of_mem 1 x.2) hxH
((InnerProductSpace.toDual ℝ E) ((t : ℝ) • c + ∇ F (x : E))) ≤ β) :
inner ℝ c (x : E) - inner ℝ c (xOpt : E) ≤
((ν : ℝ) + ((β + Real.sqrt (ν : ℝ)) * β) / (1 - β)) / (t : ℝ)
Theorem 5.3.11
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.11: for every positive target accuracy ε, the Chapter 5 short-step path-following method (5.3.29) for a ν-self-concordant barrier terminates at its stopping index with the explicit logarithmic ceiling bound recorded in pathFollowingTerminationOutcome; moreover, the stopping iterate x_N already satisfies ⟪c, x_N⟫ - c* ≤ ε. The theorem is stated for a genuine Chapter 5 short-step orbit with the calibrated parameter witness IsShortStepBarrierPathFollowingParameters β γ, and the reference objective norm in the logarithmic bound is taken at the prescribed analytic center of the barrier domain.
open scoped Gradient NewtonDecrement
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
theorem pathFollowing_terminationOutcome
{dom : Set E} {ν : NNReal} {F : E → ℝ}
[IsSelfConcordantBarrierOnWith dom ν F]
(c : E)
(cStar : ℝ)
(hcStar : IsBarrierPathFollowingOptimalValue c dom cStar)
(xStar : dom)
(hcenter : IsMinOn F dom (xStar : E))
(hxStarH : (fderiv ℝ (∇ F) (xStar : E)).det ≠ 0)
{x0 : dom} {β γ ε : ℝ}
(hshort : IsShortStepBarrierPathFollowingParameters β γ)
(scheme : BarrierPathFollowingScheme c F ν x0 β γ ε) :
pathFollowingTerminationOutcome
c cStar ν β γ ε
(barrierPathFollowingObjectiveNorm F c (xStar : E) hxStarH)
scheme scheme.stopIndex
Theorem 5.3.12
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.3.12: let x_F^* be an analytic center of a ν-self-concordant barrier F, and let y*(t) be the auxiliary central path based at y₀. If a stopped intermediate Newton method started at y₀ has decrement bounded along the iterates by β + ‖∇ F(y*(t_k))‖*_{y*(t_k)} for a nonnegative parameter sequence t_k that decays like exp (-γ k / (β + √ν)), then the stopping index is at most ⌈((β + √ν) / γ) log (((ν + 2 √ν) ‖∇ F(y₀)‖*_{x_F^*}) / γ)⌉₊, provided γ > 0 and β + √ν > 0.
open InnerProductSpace
open HessianDualLocalNorm
open scoped Gradient
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {dom : Set E} {F : E → ℝ} {ν : NNReal} [IsSelfConcordantBarrierOnWith dom ν F]
variable {y0 : dom} {β γ : ℝ}
-- Proof sketch: use the positivity of `γ` and `β + √ν` so that the exponential decay rate and the
-- logarithmic denominator have their textbook sign, then combine the exponential decay estimate
-- for `t_k` with the bound
-- `λ_F(y_k) ≤ β + t_k (ν + 2 √ν) R` for a scalar reference norm `R`.
-- Since `method` stops when the ordinary Newton decrement drops below `β + γ`, solving the
-- resulting scalar inequality for `k` yields the stated natural-ceiling bound.
theorem auxiliaryPathFollowing_stopIndex_le_natCeil_terminationBound
(method : StoppedIntermediateSelfConcordantNewtonMethod F y0 (β + γ))
[HasPositiveDefiniteHessianOn dom F]
(yStar : Set.Ici (0 : ℝ) → dom)
(xFStar : dom)
(hxFStar : IsMinOn F dom (xFStar : E))
(hpath : IsCentralPath dom (-∇ F (y0 : E)) F yStar)
(t : ℕ → Set.Ici (0 : ℝ))
(hγ : 0 < γ)
(hβsqrt : 0 < β + Real.sqrt (ν : ℝ))
(ht :
∀ k : ℕ,
(t k : ℝ) ≤ Real.exp (-γ * (k : ℝ) / (β + Real.sqrt (ν : ℝ))))
(hdecrement :
∀ k : ℕ,
method.decrement k ≤
β + ofPosDefMem F (yStar (t k)).2 (toDual ℝ E (∇ F (yStar (t k) : E)))) :
method.stopIndex ≤
⌈((β + Real.sqrt (ν : ℝ)) / γ) *
Real.log
((((ν : ℝ) + 2 * Real.sqrt (ν : ℝ)) *
ofPosDefMem F xFStar.2 (toDual ℝ E (∇ F (y0 : E)))) / γ)⌉₊
5.4 — Applications to Problems with Explicit Structure
Lemma 5.4.1.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.1.1: if (\alpha, \beta) is nonempty and f is a scalar ν-self-concordant barrier on it, then the source-facing ratio supremum satisfies 1 ≤ κ. The source-facing scalar barrier owner used here packages both the Chapter 5 self-concordance/barrier-parameter data and the missing boundary divergence required by the textbook notion of barrier.
open Filter
open Topology
open scoped Gradient DikinEllipsoidNotation
variable {α : WithBot ℝ} {β : ℝ}
theorem one_le_selfConcordantBarrierKappa
{ν : NNReal} {f : ℝ → ℝ}
(hI : Set.Nonempty (scalarBarrierInterval α β))
(hbarrier : IsSelfConcordantBarrierOnScalarIntervalWith α β ν f) :
1 ≤ selfConcordantBarrierKappa α β f
Theorem 5.4.1.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.1.1: if F is a ν-self-concordant barrier for Q, then the barrier parameter satisfies ν ≥ 1. In this file the genuine-barrier hypothesis is expressed by IsSelfConcordantBarrierForWith Q ν F, which packages the Chapter 5 owner on interior Q, the Chapter 1 barrier owner for the canonical restriction to Q, and the nonvacuous-boundary condition.
open Filter
open Topology
open scoped Gradient DikinEllipsoidNotation
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
theorem one_le_selfConcordantBarrierParameter_of_nonempty_interior
{Q : Set E} {ν : NNReal} {F : E → ℝ}
(hF : IsSelfConcordantBarrierForWith Q ν F) :
1 ≤ ν
Theorem 5.4.1.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.1.2: if Q ⊆ E is a convex set in a real Hilbert space, xBar ∈ interior Q, (p i) is a finite family of recession directions of Q, each backward step xBar - βᵢ • p i leaves interior Q, and xBar - ∑ i, α i • p i ∈ Q for nonnegative scalars α i and positive scalars β i, then every ν-self-concordant barrier F on interior Q satisfies ∑ i, αᵢ / βᵢ ≤ ν.
open Filter
open scoped BigOperators Gradient HessianLocalNorm
universe u v
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
namespace IsSelfConcordantBarrierOnWith
theorem barrierParameter_ge_sum_alpha_div_beta_of_recession_directions
{ι : Type v} [Fintype ι] {Q : Set E} {ν : NNReal} {F : E → ℝ}
(hF : IsSelfConcordantBarrierOnWith (interior Q) ν F)
(hQ_convex : Convex ℝ Q)
{xBar : E} (hxBar : xBar ∈ interior Q)
(p : ι → E)
(hrecession :
∀ i, ∀ ⦃x⦄, x ∈ Q → ∀ t : ℝ, 0 ≤ t → x + t • p i ∈ Q)
(β α : ι → ℝ)
(hβ_pos : ∀ i, 0 < β i)
(hβ_exit : ∀ i, xBar - β i • p i ∉ interior Q)
(hα_nonneg : ∀ i, 0 ≤ α i)
(hy : xBar - ∑ i, α i • p i ∈ Q) :
∑ i, α i / β i ≤ (ν : ℝ)
Definition 5.4.2.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.2.1, generalized from the textbook Euclidean setting: the polar set of Q with respect to the base point xBar is the chapter polar of the displacement set Q -ᵥ ({xBar} : Set E). Specializing to E = EuclideanSpace ℝ (Fin n) recovers the textbook definition.
universe u
open scoped PolarSet
variable {E : Type u} [SeminormedAddCommGroup E] [InnerProductSpace ℝ E]
abbrev polarSetAt (Q : Set E) (xBar : E) : Set E :=
(Q -ᵥ ({xBar} : Set E))ᵒTheorem 5.4.2.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.2.1, at the intrinsic owner level: if xBar is an interior point of Q, then the based polar set P(xBar) = polarSetAt Q xBar is compact and convex. The textbook convexity and no-affine-line assumptions are redundant for this conclusion.
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
variable {Q : Set E} {xBar : E}
-- Proof sketch: `polarSetAt Q xBar` is convex unconditionally. If `xBar ∈ interior Q`, the
-- interior ball around `xBar` gives boundedness, and in finite-dimensional real inner-product
-- space boundedness plus closedness gives compactness by Heine-Borel.
theorem polarSetAt_isCompact_convex
(hxBar : xBar ∈ interior Q) :
IsCompact (polarSetAt Q xBar) ∧ Convex ℝ (polarSetAt Q xBar)Definition 5.4.2.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.2.2, stated at the intrinsic owner level: for an interior point x of a set Q in a finite-dimensional real inner-product space, universalBarrierVolume Q x is the volume of the associated based polar body P(x) = {s | ∀ y ∈ Q, ⟪s, y - x⟫ ≤ 1}. Specializing to E = EuclideanSpace ℝ (Fin n) recovers the textbook Lebesgue-volume definition on ℝⁿ. In the textbook application, Q is later assumed proper and convex.
open MeasureTheory
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
def universalBarrierVolume (Q : Set E) (x : interior Q) : ℝ :=
(volume (polarSetAt Q x)).toRealTheorem 5.4.2.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.2.2, stated with the intrinsic owner and its ambient bridge: there exist absolute positive constants c₁ and c₂ such that for every proper convex set Q in a finite-dimensional real inner-product space E, the ambient bridge of the universal barrier x ↦ c₁ * log (V(x)) is a ((c₂ : NNReal) * Module.finrank ℝ E)-self-concordant barrier on interior Q under the source hypothesis that interior Q is nonempty. Specializing to E = EuclideanSpace ℝ (Fin n) recovers the textbook (c₂ * n) parameter.
open scoped Gradient
open MeasureTheory
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
theorem exists_absolute_constants_universalBarrier_isSelfConcordantBarrierOnWith :
∃ c₁ c₂ : NNRealˣ,
∀ {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [FiniteDimensional ℝ E]
{Q : Set E},
(Convex ℝ Q) →
(∀ ⦃x h : E⦄, h ≠ 0 → ¬ ∀ τ : ℝ, x + τ • h ∈ Q) →
(interior Q).Nonempty →
IsSelfConcordantBarrierOnWith (interior Q)
((c₂ : NNReal) * Module.finrank ℝ E)
(universalBarrierAmbient (c₁ : ℝ) Q)
Definition 5.4.3.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.3.1: for A ∈ ℝ^{m × n}, b ∈ ℝ^m, and c ∈ ℝ^n with m < n, the linear optimization problem with nonnegativity constraints is the minimization of the linear functional x ↦ ⟪c, x⟫ over the nonnegative orthant subject to the linear equality constraint A x = b.
open scoped EuclideanOrthant
variable {m n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Eₘ" => EuclideanSpace ℝ (Fin m)
def linearOptimizationProblemWithNonnegativityConstraints
(A : Matrix (Fin m) (Fin n) ℝ) (b : Eₘ) (c : Eₙ) :
PrimalEqualityConstrainedProblem Eₙ Eₘ :=
SetConstrainedMinimizationProblem.toPrimalEqualityConstrainedProblem
({ feasibleSet := ℝ₊^n
objective := fun x ↦ inner ℝ c x } : SetConstrainedMinimizationProblem Eₙ)
A.toEuclideanLin b
-- Proof sketch: unfold
-- `linearOptimizationProblemWithNonnegativityConstraints`; its ambient feasible set is the Chapter
-- 1 owner `nonnegativeOrthant n`, so membership is exactly coordinatewise nonnegativity.Lemma 5.4.3.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.3.1: every ν-self-concordant barrier for the nonnegative orthant ℝ₊ⁿ has barrier parameter at least n.
open Set Topology
open EuclideanSpace (single)
open scoped BigOperators EuclideanOrthant
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
theorem nonnegativeOrthant_barrierParameter_ge_dimension
{ν : NNReal} {F : Eₙ → ℝ}
(hF : IsSelfConcordantBarrierOnWith (ℝ₊₊^n : Set Eₙ) ν F) :
(n : ℝ) ≤ (ν : ℝ)
Proposition 5.4.3.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.4.3.1 is the direct Chapter 5 LP specialization of the generic stopping theorem, with LP owner linearOptimizationProblemWithNonnegativityConstraints A b c, strict feasible-set owner linearOptimizationProblemWithNonnegativityConstraintsStrictFeasibleSet A b, and barrier standardLogarithmicBarrierAmbient n.
open Set Topology Filter
open scoped EuclideanOrthant Gradient
variable {m n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Eₘ" => EuclideanSpace ℝ (Fin m)
local notation "F" => standardLogarithmicBarrierAmbient n
theorem
linearOptimizationProblemWithNonnegativityConstraints_stopIndex_le_natCeil_terminationBound_and_objectiveGap_le_epsilon
(A : Matrix (Fin m) (Fin n) ℝ) (b : Eₘ) (c : Eₙ)
[IsSelfConcordantBarrierOnWith
(linearOptimizationProblemWithNonnegativityConstraintsStrictFeasibleSet A b)
n
F]
{β γ ε : ℝ}
(xCenter : linearOptimizationProblemWithNonnegativityConstraintsStrictFeasibleSet A b)
(hcenter :
IsMinOn F
(linearOptimizationProblemWithNonnegativityConstraintsStrictFeasibleSet A b)
(xCenter : Eₙ))
(hxCenterH : (fderiv ℝ (∇ F) (xCenter : Eₙ)).det ≠ 0)
(xOpt :
(linearOptimizationProblemWithNonnegativityConstraints A b c).equalityFeasibleSet)
(hopt :
∀ y :
(linearOptimizationProblemWithNonnegativityConstraints A b c).equalityFeasibleSet,
linearOptimizationProblemWithNonnegativityConstraints A b c (xOpt : Eₙ) ≤
linearOptimizationProblemWithNonnegativityConstraints A b c (y : Eₙ))
(t : ℕ → ℝ) (x : ℕ → Eₙ)
(mem_dom :
∀ k : ℕ, x k ∈ linearOptimizationProblemWithNonnegativityConstraintsStrictFeasibleSet A b)
(hessian_nondegenerate : ∀ k : ℕ, (fderiv ℝ (∇ F) (x k)).det ≠ 0)
(stopIndex : ℕ)
(hβ_half : β < 1 / 2)
(hγ : 0 < γ)
(hcontinue :
∀ ⦃k : ℕ⦄, k < stopIndex →
t k < barrierPathFollowingStoppingThreshold n β ε)
(hstop : barrierPathFollowingStoppingThreshold n β ε ≤ t stopIndex)
(hgrowth :
∀ k : ℕ, 1 ≤ k →
(γ * (1 - 2 * β)) /
((1 - β) *
HessianDualLocalNorm.ofDetNeZero F (xCenter : Eₙ)
(IsSelfConcordantOnWith.hessian_isPositive_of_mem 1 xCenter.2)
hxCenterH
((InnerProductSpace.toDual ℝ Eₙ) c)) *
(1 + γ / (β + Real.sqrt (n : ℝ))) ^ (k - 1) ≤
t k)
(happrox_stop :
HessianDualLocalNorm.ofDetNeZero F (x stopIndex)
(IsSelfConcordantOnWith.hessian_isPositive_of_mem 1 (mem_dom stopIndex))
(hessian_nondegenerate stopIndex)
((InnerProductSpace.toDual ℝ Eₙ)
(t stopIndex • c + ∇ F (x stopIndex))) ≤
β) :
stopIndex ≤
⌈barrierPathFollowingTerminationBound n β γ ε
(HessianDualLocalNorm.ofDetNeZero F (xCenter : Eₙ)
(IsSelfConcordantOnWith.hessian_isPositive_of_mem 1 xCenter.2) hxCenterH
((InnerProductSpace.toDual ℝ Eₙ) c))⌉₊ ∧
linearOptimizationProblemWithNonnegativityConstraints A b c (x stopIndex) -
linearOptimizationProblemWithNonnegativityConstraints A b c (xOpt : Eₙ) ≤
ε
Definition 5.4.3.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.3.2: the standard logarithmic barrier for the strict positive orthant \mathbb{R}^n_{++} is the canonical logarithmic barrier on that intrinsic domain.
open scoped BigOperators EuclideanOrthant
variable (n : ℕ)
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => ℝ₊₊^n
abbrev standardLogarithmicBarrier : C(Xₙ, ℝ) :=
(logarithmicBarrier (positiveOrthantConstraints n)).comp
(ContinuousMap.inclusion
(positiveOrthant_subset_strictConstraintSet_positiveOrthantConstraints n))
-- Proof sketch: unfold `standardLogarithmicBarrier`; this is the canonical logarithmic barrier on
-- the positive-orthant constraint family, precomposed with the subtype inclusion.Lemma 5.4.3.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.3.2 (1): logarithmic homogeneity rescales the gradient by τ⁻¹.
open Filter Set Topology
open scoped Gradient Pointwise
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E]
variable [CompleteSpace E]
variable {K : ConvexCone ℝ E} {ν : ℝ} {F : E → ℝ}
theorem gradient_pos_smul_eq_inv_smul
(hFlog : IsLogarithmicallyHomogeneousOnWith K ν F)
{x : E} (hx : x ∈ interior (K : Set E)) {τ : ℝ} (hτ : 0 < τ) :
∇ F (τ • x) = (1 / τ) • ∇ F x
Proposition 5.4.3.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.4.3.2 [Iteration complexity bound for a path-following method]: if the strict QCQP epigraph domain is nonempty and the epigraph optimization problem has finite optimal value, then there exists a logarithmic-complexity constant such that for every target accuracy ε > 0 there exists a source-facing QCQP epigraph output package whose returned feasible point is ε-accurate for the epigraph optimization problem in the sense of Definition 1.3.7 and whose stopping index satisfies the displayed O(√(m + 1) log (m / ε)) bound, written with the standard always-defined positive wrapper. The raw short-step parameters and ambient barrier scheme stay in the private helper layer.
open QuadraticallyConstrainedQuadraticOptimizationProblem
open Set Topology Filter
open scoped Gradient HessianLocalNorm BigOperators
variable {n m : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
variable (problem : QuadraticallyConstrainedQuadraticOptimizationProblem n m)
local notation "𝒟" =>
problem.strictEpigraphFeasibleSet
local notation "F" =>
problem.epigraphLogarithmicBarrierAmbient
local notation "P" =>
problem.epigraphOptimizationProblem
local notation "ℱ" =>
problem.epigraphFeasibleSet
local notation "cτ" =>
((0 : Eₙ), (1 : ℝ))
theorem exists_qcqpEpigraphPathFollowingScheme
(hstrict : Set.Nonempty 𝒟)
(hoptimal : (P).optimalValue ≠ ⊥) :
∃ C : NNRealˣ,
∀ {ε : ℝ}, 0 < ε →
Nonempty (QcqpEpigraphPathFollowingScheme problem ε C)
Definition 5.4.3.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.3.3: a function F : E → ℝ is logarithmically homogeneous on a closed convex cone K with nonempty interior and logarithmic homogeneity parameter ν when F is twice continuously differentiable on interior K and F (τ • x) = F x - ν log τ for every x ∈ interior K and every τ > 0. The cone structure itself is carried by the canonical owner K : ConvexCone ℝ E, while closedness remains part of the source-facing notion; sign conditions on ν belong to later barrier layers when they are actually needed, not to logarithmic homogeneity itself.
universe u
variable {E : Type u} [NormedAddCommGroup E] [NormedSpace ℝ E]
class IsLogarithmicallyHomogeneousOnWith (K : ConvexCone ℝ E) (ν : ℝ) (F : E → ℝ) : Prop
extends Fact (IsClosed (K : Set E)) where
/-- The cone `K` has nonempty interior. -/
interior_nonempty : (interior (K : Set E)).Nonempty
/-- The function `F` is twice continuously differentiable on `interior K`. -/
contDiffOn : ContDiffOn ℝ 2 F (interior (K : Set E))
/-- The logarithmic scaling identity holds on `interior K` for every positive scalar. -/
logarithmic_scaling {x : E} (hx : x ∈ interior (K : Set E)) {τ : ℝ} (hτ : 0 < τ) :
F (τ • x) = F x - ν * Real.log τLemma 5.4.3.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.3.3: the function (x, t) ↦ -log (t^2 - ‖x‖^2) is a 2-self-concordant barrier for the second-order cone K₂ = {(x, t) ∈ E × ℝ | ‖x‖ ≤ t}, viewed on the canonical L² product owner WithLp 2 (E × ℝ) through the canonical raw-pair bridge WithLp.ofLp. Specializing E = EuclideanSpace ℝ (Fin n) recovers the textbook ℝⁿ × ℝ statement.
open Filter
open scoped Gradient HessianLocalNorm
universe u
variable {E : Type u} [NormedAddCommGroup E]
variable [InnerProductSpace ℝ E] [CompleteSpace E]
local notation "Z" => WithLp 2 (E × ℝ)
local notation "ofZ" => (WithLp.ofLp : Z → E × ℝ)
theorem secondOrderConeBarrier_isSelfConcordantBarrierOnWith :
IsSelfConcordantBarrierOnWith
(ofZ ⁻¹' interior (secondOrderCone E : Set (E × ℝ)))
2
(secondOrderConeBarrier ∘ ofZ)Definition 5.4.3.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.3.4: A quadratically constrained quadratic optimization problem on ℝⁿ consists of quadratic functions q₀, …, q_m with positive-semidefinite quadratic parts and constraint bounds β₁, …, β_m; the problem is to minimize q₀ subject to qᵢ(x) ≤ βᵢ for i = 1, …, m, and it admits the equivalent epigraph reformulation using an auxiliary scalar variable τ.
variable {n m : ℕ}
structure QuadraticallyConstrainedQuadraticOptimizationProblem (n m : ℕ) where
α : Fin (m + 1) → ℝ
a : Fin (m + 1) → EuclideanSpace ℝ (Fin n)
A : Fin (m + 1) → Matrix (Fin n) (Fin n) ℝ
A_posSemidef : ∀ i : Fin (m + 1), (A i).PosSemidef
β : Fin m → ℝLemma 5.4.3.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.3.4: every ν-self-concordant barrier for the interior of the second-order cone K₂ = {(x, t) ∈ E × ℝ | ‖x‖ ≤ t} in a nontrivial real Hilbert space E, viewed on the canonical L² product ambient space through z ↦ z.ofLp, has barrier parameter at least 2. Specializing to E = EuclideanSpace ℝ (Fin n) with 0 < n recovers the textbook ℝⁿ statement.
open Set Topology
open scoped BigOperators
open scoped SecondOrderCone
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
[Nontrivial E]
local notation "Z" => WithLp 2 (E × ℝ)
local notation "ofZ" => (WithLp.ofLp : Z → E × ℝ)
theorem secondOrderCone_barrierParameter_ge_two
{ν : NNReal} {F : Z → ℝ}
(hF : IsSelfConcordantBarrierOnWith (ofZ ⁻¹' interior K₂[E]) ν F) :
(2 : ℝ) ≤ (ν : ℝ)
Definition 5.4.3.5
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.3.5: the logarithmic barrier for the QCQP epigraph feasible region {(x, τ) | 0 < τ - q₀(x) ∧ ∀ i, 0 < βᵢ - qᵢ(x)} is (x, τ) ↦ -log (τ - q₀(x)) - ∑ i, log (βᵢ - qᵢ(x)).
open scoped BigOperators
variable {n m : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
namespace QuadraticallyConstrainedQuadraticOptimizationProblem
def epigraphLogarithmicBarrier
(problem : QuadraticallyConstrainedQuadraticOptimizationProblem n m) :
C(problem.StrictEpigraphFeasiblePoint, ℝ) :=
logarithmicBarrier problem.strictEpigraphConstraints
Alg 5.4.4.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Alg 5.4.4.1: with the symmetry and positive-definiteness assumptions carried by the Chapter 5 owners 𝕊^n and 𝕊^n₊₊, the pair (λ, Δ) is an output of the semidefinite Newton-direction algorithm when λ solves the normal equations S λ = d and Δ = X (-U + ∑_j λ_j A_j) X.
open Matrix
open RealSymmetricMatrixSpace
open scoped BigOperators RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "SymmMat" => 𝕊^n
def IsSemidefiniteNewtonDirectionOutput
(X : 𝕊^n₊₊) (U : SymmMat) (A : Fin m → SymmMat)
(multiplier : Fin m → ℝ) (Δ : SymmMat) : Prop :=
IsSemidefiniteNewtonMultiplier X U A multiplier ∧
Δ = semidefiniteNewtonDirectionFromMultiplier X U A multiplier
-- Proof sketch: unfold `IsSemidefiniteNewtonMultiplier`; after the owner-typed refinement, the
-- right-hand side is exactly the displayed matrix normal system `S λ = d`.
Definition 5.4.4.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.4.1 is a recall-only item in the real symmetric-matrix domain. Layer targeted by this refinement: - source-facing recall of the core/canonical symmetric-matrix submodule. Primary domain: - the real vector space 𝕊^n of symmetric n × n matrices. Sampled owner-style declarations: - mathlib selfAdjointMatricesSubmodule - mathlib mem_selfAdjointMatricesSubmodule - mathlib Matrix.IsSymm - mathlib Matrix.IsSymm.eq Best owner abstraction: - source-facing: the vector space 𝕊^n of real symmetric matrices; - core/canonical: selfAdjointMatricesSubmodule (1 : Matrix (Fin n) (Fin n) ℝ); - bridge/view: membership rewritten as Matrix.IsSymm, equivalently Xᵀ = X. Primitive data: - n : ℕ Derived API: - the canonical symmetric-matrix owner selfAdjointMatricesSubmodule - the source-facing notation 𝕊^n - the owner-branded symmetry bridge RealSymmetricMatrixSpace.mem_iff_isSymm - the owner-branded transpose bridge RealSymmetricMatrixSpace.mem_iff_transpose_eq This file therefore removes the duplicate local definition realSymmetricMatrixSubspace and reuses the matrix-specific canonical owner directly. The textbook surface is then recovered by thin owner-branded companion lemmas on 𝕊^n, written directly in Matrix.IsSymm / Xᵀ = X form.
open Matrix
recall selfAdjointMatricesSubmoduleLemma 5.4.4.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.4.1 (1): at a strict-cone point, the first directional derivative of the log-determinant barrier in a symmetric direction is the Frobenius pairing with -X⁻¹.
open Matrix
open RealSymmetricMatrixSpace StrictPositiveSemidefiniteCone
open scoped Matrix.Norms.Frobenius MatrixOrder RealSymmetricMatrixSpace
variable (n : ℕ)
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "SymmMat" => 𝕊^n
local notation "logDetBarrierAmbientSymm" => logDetBarrierAmbient n
theorem logDetBarrier_lineDeriv_eq_frobeniusInner
(X : 𝕊^n₊₊) (Δ : SymmMat) :
lineDeriv ℝ logDetBarrierAmbientSymm X Δ =
⟪-inv X, Δ⟫_F
Proposition 5.4.4.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.4.4.1: for real symmetric matrices X and Y, the Frobenius pairing of X with the intrinsic symmetric-carrier square sandwich Y 1 of Y² equals Trace (YXY), and that trace is the Frobenius pairing of YXY with the identity matrix.
open Matrix
open scoped RealSymmetricMatrixSpace
variable {n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "SymmMat" => 𝕊^n
open RealSymmetricMatrixSpace
-- `sandwich Y 1` is the intrinsic symmetric-matrix representative of the ambient square `Y²`.
theorem frobenius_trace_identity_for_real_symmetric_matrices
(X Y : SymmMat) :
⟪X, sandwich Y (1 : SymmMat)⟫_F =
Matrix.trace ((Y : Mat) * (X : Mat) * (Y : Mat)) ∧
Matrix.trace ((Y : Mat) * (X : Mat) * (Y : Mat)) =
⟪(1 : SymmMat), sandwich Y X⟫_F
Theorem 5.4.4.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.4.1: the cone 𝕊ⁿ₊ of positive semidefinite real n × n matrices is a closed convex set.
open Matrix
open scoped RealSymmetricMatrixSpace
variable {n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "SymmMat" => 𝕊^n
theorem positiveSemidefiniteCone_isClosed_convex (n : ℕ) :
IsClosed (𝕊^n₊) ∧ Convex ℝ (𝕊^n₊)Definition 5.4.4.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.4.2: on 𝕊^n, the Frobenius inner product is the inherited real inner product, written on the theorem surface in Frobenius notation.
open Matrix
open scoped RealSymmetricMatrixSpace
namespace RealSymmetricMatrixSpace
abbrev frobeniusInner {n : ℕ} (X Y : 𝕊^n) : ℝ :=
inner ℝ X Y
Lemma 5.4.4.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.4.2: every ν-self-concordant barrier for the cone 𝕊ⁿ₊ of positive semidefinite real n × n matrices has barrier parameter at least n.
open Matrix
open scoped BigOperators RealSymmetricMatrixSpace
variable (n : ℕ)
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
theorem positiveSemidefiniteCone_barrierParameter_ge_dimension
{ν : NNReal} {F : 𝕊^n → ℝ}
(hF : IsSelfConcordantBarrierOnWith (𝕊^n₊₊ : Set (𝕊^n)) ν F) :
(n : ℝ) ≤ (ν : ℝ)
Proposition 5.4.4.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.4.4.2: a Newton direction is an output of the Chapter 5 semidefinite Newton system.
open RealSymmetricMatrixSpace
open scoped BigOperators Gradient RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "SymmMat" => 𝕊^n
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
theorem semidefiniteNewtonDirectionSet_output
{A : Fin m → SymmMat} {X : 𝕊^n₊₊} {U Δ : SymmMat}
(hΔ : Δ ∈ semidefiniteNewtonDirectionSet A X U) :
∃ multiplier : Fin m → ℝ,
IsSemidefiniteNewtonDirectionOutput X U A multiplier Δ
Theorem 5.4.4.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.4.2: for a positive-definite real symmetric matrix X, the Chapter 5 owner logDetBarrier equals the negative sum of the logarithms of the eigenvalues of X.
open Matrix
open RealSymmetricMatrixSpace
open scoped RealSymmetricMatrixSpace
theorem logDetBarrier_eq_neg_sum_log_eigenvalues
{n : ℕ} (X : 𝕊^n₊₊) :
logDetBarrier n X =
-∑ i : Fin n, Real.log (eigenvalues X i)
Definition 5.4.4.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.4.3 lies in the real symmetric-matrix positivity domain. Layer targeted by this refinement: - source-facing: the textbook cone notation 𝕊^n₊ inside the symmetric carrier 𝕊^n; - core/canonical: Matrix.PosSemidef and Matrix.PosDef; - bridge/view: coercion from 𝕊^n to matrices and quadratic-form characterizations through Matrix.toEuclideanLin. Sampled owner-style declarations: - Chapter 5 𝕊^n from Definition_5_4_4_1, the symmetric-matrix owner; - mathlib Matrix.PosSemidef, the canonical positive-semidefinite matrix predicate; - mathlib selfAdjoint.submodule, the canonical carrier behind 𝕊^n; - mathlib Matrix.isPositive_toEuclideanLin_iff, the Euclidean-operator positivity bridge; - mathlib Matrix.PosDef.of_dotProduct_mulVec_pos and Matrix.PosDef.dotProduct_mulVec_pos, the positive-definite owner API. Primitive data: - n : ℕ Derived API: - the source-facing notation 𝕊^n₊ : Set (𝕊^n); - the owner bridge X ∈ 𝕊^n₊ ↔ (X : Mat).PosSemidef; - the intrinsic cone bridge PositiveSemidefiniteCone.nnrpow X p for nonnegative powers of PSD matrices; - the quadratic-form and positive-definite companion characterizations. This file therefore deletes the duplicate local owner positiveSemidefiniteCone, keeps the textbook cone notation on the intrinsic symmetric carrier 𝕊^n, avoids any public Euclidean-array realization wrapper, and derives the rest of the API from the canonical matrix-positivity owners.
open Matrix
open scoped MatrixOrder NNReal RealInnerProductSpace RealSymmetricMatrixSpace
variable {n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "SymmMat" => 𝕊^n
recall Matrix.PosSemidefProposition 5.4.4.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.4.4.3: if 1 ≤ m ≤ n(n + 1) / 2, then one dense execution of the Newton-step subroutine has arithmetic work bounded by a constant multiple of n^2 * (m + n) * m, and hence has arithmetic complexity O(n^2 * (m + n) * m).
theorem semidefiniteNewtonStepDenseArithmeticComplexity_bound
{n m : ℕ} (hm : 1 ≤ m) (hmn : m ≤ n * (n + 1) / 2) :
semidefiniteNewtonStepDenseArithmeticWorkBound n m ≤
8 * n ^ 2 * (m + n) * m
Theorem 5.4.4.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.4.3: the log-determinant function X ↦ -log det X is an n-self-concordant barrier on the interior of the positive-semidefinite cone 𝕊ⁿ₊.
open Matrix
open RealSymmetricMatrixSpace
open scoped RealSymmetricMatrixSpace
open scoped BigOperators Gradient HessianLocalNorm MatrixOrder
local notation "Mat" n => Matrix (Fin n) (Fin n) ℝ
local notation "SymmMat" n => 𝕊^n
-- Proof sketch: for `X ∈ int(𝕊ⁿ₊)` and a symmetric direction `Δ`, conjugate by `X^(-1 / 2)` to
-- reduce the first three directional derivatives of `X ↦ -log det X` to sums of eigenvalue
-- powers of `Q = X^(-1 / 2) Δ X^(-1 / 2)`. Then the barrier-parameter bound with `ν = n`
-- follows from Cauchy--Schwarz, and the self-concordance inequality follows from the estimate
-- `|∑ λᵢ^3| ≤ (∑ λᵢ^2)^(3 / 2)`.
theorem negativeLogDet_isSelfConcordantBarrierOnWith_positiveSemidefiniteCone
(n : ℕ) :
IsSelfConcordantBarrierOnWith
(𝕊^n₊₊ : Set (SymmMat n))
n
(logDetBarrierAmbient n)
Definition 5.4.4.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.4.4: a semidefinite optimization problem is determined by a cost matrix C ∈ 𝕊^n, a finite family of constraint matrices A₁, …, Aₘ ∈ 𝕊^n, and a right-hand side vector b. Its canonical owner bridge is the equality-constrained problem toPrimalEqualityConstrainedProblem, whose primalProblem recovers the Chapter 1 set-constrained optimization view.
open Matrix
open scoped RealSymmetricMatrixSpace
variable {m n : ℕ}
structure SemidefiniteOptimizationProblem (n m : ℕ) where
/-- The cost matrix `C ∈ 𝕊^n`. -/
costMatrix : 𝕊^n
/-- The family of constraint matrices `A₁, …, Aₘ ∈ 𝕊^n`. -/
constraintMatrices : Fin m → 𝕊^n
/-- The right-hand side vector `b ∈ ℝᵐ`. -/
rhs : EuclideanSpace ℝ (Fin m)
Proposition 5.4.4.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.4.4.4: under the rank-one specialization Aⱼ = aⱼ aⱼᵀ, there exist primitive solver/work families for the displayed log-determinant Newton system whose arithmetic complexity is packaged by the Chapter 5 owner HasRankOneLogDetNewtonSystemArithmeticComplexityBound. That owner records a uniform cubic arithmetic-work bound O((m + n)^3) and requires the computed pair (Δy, ΔX) = (solver a X S r, rankOneLogDetNewtonDirection a X S (solver a X S r)) to satisfy the displayed equations on each compatible input tuple (a, X, S, r).
open Matrix
open scoped BigOperators RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Eₘ" => EuclideanSpace ℝ (Fin m)
local notation "SymmMat" => 𝕊^n
-- Semantic recall via `lean_leansearch` produced only generic
-- `Matrix.vecMulVec` rank-one hits, so this item stays on the local
-- Chapter 5 semidefinite Newton-system owners from `Alg_5_4_4_1`.
theorem rankOneLogDetNewtonSystem_cubic_arithmeticComplexity_bound :
∃ (solver : ∀ {n m : ℕ},
(a : Fin m → EuclideanSpace ℝ (Fin n)) →
(X : 𝕊^n₊₊) →
(S : 𝕊^n) →
(r : EuclideanSpace ℝ (Fin m)) →
EuclideanSpace ℝ (Fin m))
(arithmeticWork : ∀ {n m : ℕ},
(a : Fin m → EuclideanSpace ℝ (Fin n)) →
(X : 𝕊^n₊₊) →
(S : 𝕊^n) →
(r : EuclideanSpace ℝ (Fin m)) →
ℕ),
HasRankOneLogDetNewtonSystemArithmeticComplexityBound solver arithmeticWork
Theorem 5.4.4.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.4.4 (1): if problem.strictFeasibleSet is nonempty, xOpt is optimal for the SDP owner problem on problem.feasibleSet, and ε > 0, then there exists a positive iteration constant C₀ and a source-facing semidefinite path-following scheme whose returned feasible point is ε-accurate and whose stopping index satisfies the stated O(√n log (n / ε)) bound.
open Matrix
open scoped RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "SymmMat" => 𝕊^n
variable (problem : SemidefiniteOptimizationProblem n m)
local notation "𝓕°" => problem.strictFeasibleSet
variable {ε : ℝ}
theorem exists_semidefinitePathFollowingScheme
(hstrict : 𝓕°.Nonempty)
(xOpt : problem.feasibleSet)
(hopt : IsMinOn problem problem.feasibleSet (xOpt : SymmMat))
(hε : 0 < ε) :
∃ C₀ : NNRealˣ, Nonempty (SemidefinitePathFollowingScheme problem ε C₀)
Definition 5.4.4.5
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.4.5 uses the strict positive-definite cone 𝕊ⁿ₊₊ = int(𝕊ⁿ₊) as the intrinsic domain of the log-determinant barrier.
open Matrix open scoped MatrixOrder RealSymmetricMatrixSpace variable (n : ℕ) #check (𝕊^n₊₊ : Set (𝕊^n))
Definition 5.4.4.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.4.6: for symmetric matrices A₁, …, Aₘ ∈ 𝕊ⁿ and b ∈ ℝᵐ, the associated affine subspace is the affine subset of 𝕊ⁿ cut out by the Frobenius inner-product equations ⟨Aᵢ, X⟩_F = bᵢ for all i.
open Matrix
open scoped RealSymmetricMatrixSpace
variable {m n : ℕ}
def realSymmetricMatrixAssociatedAffineSubspace
(A : Fin m → 𝕊^n) (b : EuclideanSpace ℝ (Fin m)) : AffineSubspace ℝ (𝕊^n) :=
(AffineSubspace.mk' b (⊥ : Submodule ℝ (EuclideanSpace ℝ (Fin m)))).comap
(realSymmetricMatrixConstraintMap A).toAffineMap
Definition 5.4.4.7
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.4.7: for a strict-cone base point X, the Newton directions are the minimizers of the quadratic model ⟨U, Δ⟩_F + (1 / 2) ⟨X⁻¹ Δ X⁻¹, Δ⟩_F on the tangent kernel ⟨Aᵢ, Δ⟩_F = 0. When X also lies in the affine slice realSymmetricMatrixAssociatedAffineSubspace A b, this is exactly the textbook Newton-direction set for the barrier restricted to 𝓛.
open Matrix
open RealSymmetricMatrixSpace
open scoped ConstrainedArgmin RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "SymmMat" => 𝕊^n
def semidefiniteNewtonDirectionSet
(A : Fin m → SymmMat) (X : 𝕊^n₊₊) (U : SymmMat) : Set SymmMat :=
argmin[(realSymmetricMatrixConstraintMap A).ker]
(semidefiniteNewtonDirectionObjective X U)
Definition 5.4.4.8
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.4.8: the logarithmic-determinant barrier on the strict domain {(x, t) | t I - 𝓐(x) ∈ 𝕊ⁿ₊₊}.
open Matrix
open scoped MatrixOrder RealSymmetricMatrixSpace
variable {E : Type*} [AddCommGroup E] [Module ℝ E]
variable {n : ℕ}
local notation "SymmMat" => 𝕊^n
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
def semidefiniteAffineLogDetBarrier
(𝓐 : E →ᵃ[ℝ] SymmMat) : SemidefiniteAffineBarrierPoint 𝓐 → ℝ :=
fun xt ↦ logDetBarrier n ⟨semidefiniteAffineSlack 𝓐 xt.1, xt.2⟩
Definition 5.4.5.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.5.1: the source-facing image-form ellipsoid W(H, v) is the set of points x satisfying the textbook constraint ‖H x - v‖ ≤ 1.
open Matrix
open StrictPositiveSemidefiniteCone
open scoped EllipsoidNotation RealSymmetricMatrixSpace
variable {ι : Type*} {n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
def enclosingEllipsoid
(H : 𝕊^n₊₊) (v : Eₙ) : Set Eₙ :=
{x | ‖(toMatrix H).toEuclideanLin x - v‖ ≤ 1}
Lemma 5.4.5.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.5.1: for the ellipsoid W = {x | ⟪H⁻¹ (x - v), x - v⟫ ≤ 1} cut out by a positive-definite matrix H, the affine inequality ⟪a, x⟫ ≤ b holds on all of W exactly when the quadratic bound ⟪a, H a⟫ ≤ (b - ⟪a, v⟫)^2 holds. On the public surface, W is the chapter ellipsoid owner E(H, v).
open Matrix
open scoped EllipsoidNotation RealInnerProductSpace
variable {n : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
theorem affine_le_on_affineEllipsoid_iff
(a v : E) (b : ℝ) (H : Mat)
(hβ : 0 ≤ b - ⟪a, v⟫) (hH : H.PosDef) :
(∀ x ∈ E(H, v), ⟪a, x⟫ ≤ b) ↔
⟪a, H.toEuclideanLin a⟫ ≤ (b - ⟪a, v⟫) ^ (2 : ℕ)Proposition 5.4.5.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Proposition 5.4.5.1: let F(H, τ) be a ν-self-concordant barrier for the circumscribed-ellipsoid problem, with ν ≤ m + n + 1. Then there exists a positive iteration constant C, uniform in the target accuracy ε ∈ (0, 1), such that for every such ε the path-following method returns a source-facing circumscribed-ellipsoid output package whose stopping point is ε-accurate for the owner problem and whose Newton-iteration count satisfies the displayed O(√(m + n + 1) log ((m + n) / ε)) bound, provided the strict ambient barrier domain is nonempty and the owner optimal value is finite.
open Matrix
open Set Topology Filter
open StrictPositiveSemidefiniteCone
open scoped BigOperators Gradient RealInnerProductSpace RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "SymmMat" => 𝕊^n
variable (a : Fin m → E) (b : Fin m → ℝ) (v : E)
local notation "𝒟" => circumscribedEllipsoidBarrierAmbientDomain a b v
local notation "F" => circumscribedEllipsoidBarrierAmbient a b v
local notation "cτ" => ((0 : SymmMat), (1 : ℝ))
local notation "P" => circumscribedEllipsoidOptimizationProblem a b v
theorem exists_circumscribedEllipsoidPathFollowingScheme
{ν : NNReal}
[IsSelfConcordantBarrierOnWith 𝒟 ν F]
(hmn_pos : 0 < (m + n : ℝ))
(hstrict : Set.Nonempty 𝒟)
(hoptimal : (P).optimalValue ≠ ⊥)
(hν : ν ≤ (m + n + 1 : NNReal)) :
∃ C : NNRealˣ,
∀ {ε : ℝ}, 0 < ε → ε < 1 →
Nonempty (CircumscribedEllipsoidPathFollowingScheme (m := m) (n := n) a b v ε C)
Theorem 5.4.5.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.5.1: if the strict MVEE barrier domain 𝒟 is nonempty, then there exists a positive iteration constant C such that for every small target accuracy ε ∈ (0, Fintype.card ι + n) one can choose a source-facing MVEE path-following scheme whose preprocessing point is a unit-penalty minimizer of z ↦ ⟪cτ, z⟫ + F z, whose returned feasible point is ε-accurate, and whose stopping index is bounded by O(√(Fintype.card ι + n + 1) log ((Fintype.card ι + n) / ε)). This matches the source-facing complexity statement without asserting an impossible raw BarrierPathFollowingScheme on the ambient barrier F.
open Matrix
open WithLp
open Set Topology Filter
open scoped BigOperators Gradient RealInnerProductSpace RealSymmetricMatrixSpace
variable {ι : Type*} [Fintype ι] {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "SymmMat" => 𝕊^n
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "TailSpace" => Eₙ × ℝ
local notation "MVEEAmbientSpace" => SymmMat × TailSpace
variable (a : ι → Eₙ)
local notation "F" => minimumVolumeEnclosingEllipsoidBarrierAmbient (n := n) a
local notation "P" => minimumVolumeEnclosingEllipsoidProblem (n := n) a
local notation "𝒟" => minimumVolumeEnclosingEllipsoidBarrierAmbientDomain (n := n) a
local notation "cτ" =>
((0 : SymmMat), (0 : Eₙ), (1 : ℝ))
-- Proof sketch: specialize the standard short-step path-following existence and complexity theory
-- to the MVEE objective vector `cτ` and to the owner ambient barrier `F` from
-- `Definition_5_4_5_2`.
theorem exists_minimumVolumeEnclosingEllipsoidPathFollowingScheme
[IsSelfConcordantBarrierOnWith 𝒟 (Fintype.card ι + n + 1) F]
(hstrict : Set.Nonempty 𝒟)
:
∃ C : NNRealˣ,
∀ {ε : ℝ}, 0 < ε → ε < (Fintype.card ι + n : ℝ) →
Nonempty (MinimumVolumeEnclosingEllipsoidPathFollowingScheme ε C)
Definition 5.4.5.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.5.2: the logarithmic barrier for the minimum-volume enclosing ellipsoid problem, kept on its strict domain.
open Matrix
open scoped BigOperators RealSymmetricMatrixSpace
variable {ι : Type*} {n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "SymmMat" => 𝕊^n
def minimumVolumeEnclosingEllipsoidBarrier
[Fintype ι] (a : ι → Eₙ) : MinimumVolumeEnclosingEllipsoidBarrierPoint a → ℝ :=
fun Hvτ ↦ minimumVolumeEnclosingEllipsoidBarrierAmbient a
((Hvτ.1.1 : SymmMat), Hvτ.1.2.1, Hvτ.1.2.2)
Theorem 5.4.5.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.5.2 (Path-following complexity for problem (5.4.19)): if the feasible set of optimizationProblem a b has nonempty interior, then there exists a positive iteration constant C, uniform in ε, such that for every target accuracy ε with 0 < ε there exists a source-facing maximum-volume-inscribed-ellipsoid path-following scheme whose returned feasible point is ε-accurate and whose stopping index satisfies the displayed bound O(√(2m + n + 1) log ((m + n) / ε)).
namespace MaximumVolumeInscribedEllipsoid
open Matrix
open Set Topology Filter
open scoped BigOperators RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "SymmMat" => 𝕊^n
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "TailSpace" => E × ℝ
local notation "InscribedAmbientSpace" => SymmMat × TailSpace
variable (a : Fin m → EuclideanSpace ℝ (Fin n)) (b : Fin m → ℝ)
local notation "𝒟" => logarithmicBarrierAmbientDomain a b
local notation "F" => logarithmicBarrierAmbient a b
local notation "cτ" => ((0 : SymmMat), (0 : E), (1 : ℝ))
local notation "P" => optimizationProblem a b
-- Proof sketch: rewrite each scalar as a multiple of `1` and pull those scalars through the real
-- inner product.
theorem exists_maximumVolumeInscribedEllipsoidPathFollowingScheme
(hinterior :
(interior ((P).feasibleSet : Set (𝕊^n₊₊ × E × ℝ))).Nonempty)
:
∃ C : NNRealˣ,
∀ {ε : ℝ}, 0 < ε →
Nonempty (MaximumVolumeInscribedEllipsoidPathFollowingScheme a b ε C)
Definition 5.4.5.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.5.3: the ellipsoid E(toMatrix H, v) is a maximum-volume ellipsoid in Q centered at v when it lies inside Q, and no other ellipsoid with the same center v and strict-cone shape parameter has larger volume. The positive-definite matrix view of H is the derived bridge strictPositiveSemidefiniteCone_posDef H.
open MeasureTheory Matrix
open StrictPositiveSemidefiniteCone
open scoped EllipsoidNotation RealSymmetricMatrixSpace
variable {n : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "SymmMat" => 𝕊^n
structure IsMaximumVolumeEllipsoidIn
(Q : Set E) (v : E) (H : 𝕊^n₊₊) : Prop where
/-- The candidate ellipsoid lies inside `Q`. -/
subset : E(toMatrix H, v) ⊆ Q
/-- No other ellipsoid centered at `v` and contained in `Q` has larger volume. -/
volume_maximal {shape' : 𝕊^n₊₊}
(hsubset' : E(toMatrix shape', v) ⊆ Q) :
volume (E(toMatrix shape', v)) ≤ volume (E(toMatrix H, v))
Definition 5.4.5.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.5.4: the circumscribed-ellipsoid optimization problem minimizes the auxiliary variable τ over pairs (H, τ) satisfying H ∈ 𝕊ⁿ₊₊, logDetBarrier n H ≤ τ, and the textbook quadratic constraints ⟪H aᵢ, aᵢ⟫ ≤ (bᵢ - ⟪aᵢ, v⟫)^2. The geometric containment formulation is kept only as a bridge under the explicit center-slack hypotheses.
open Matrix
open StrictPositiveSemidefiniteCone
open scoped EllipsoidNotation RealInnerProductSpace RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "E" => EuclideanSpace ℝ (Fin n)
def circumscribedEllipsoidOptimizationProblem
(a : Fin m → E) (b : Fin m → ℝ) (v : E) :
SetConstrainedMinimizationProblem (𝕊^n₊₊ × ℝ)
Definition 5.4.5.5
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.5.5: the logarithmic barrier for the circumscribed-ellipsoid reformulation, kept on its strict domain.
open Matrix
open StrictPositiveSemidefiniteCone
open scoped BigOperators RealInnerProductSpace RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "SymmMat" => 𝕊^n
def circumscribedEllipsoidBarrier
(a : Fin m → E) (b : Fin m → ℝ) (v : E) :
CircumscribedEllipsoidBarrierPoint a b v → ℝ :=
fun Hτ ↦ circumscribedEllipsoidBarrierAmbient a b v ((Hτ.1.1 : SymmMat), Hτ.1.2)
Definition 5.4.5.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.5.6 lies in the inscribed-ellipsoid / convex-reformulation domain. Sampled owner-style declarations in this domain: * SetConstrainedMinimizationProblem in Chap01/Definition_1_3_3, the project owner of a feasible set together with a real-valued objective; * constrainedEpigraph and mem_constrainedEpigraph_iff in Chap03/Definition_3_3, the chapter owner/view for epigraph reformulations over a base feasible set; * innerLePolyhedron and mem_innerLePolyhedron_iff in Chap03/Definition_3_62, the chapter owner/view for finite half-space presentations ⟪a i, x⟫ ≤ b i; * affineEllipsoid together with the notation E(H, x̄) and mem_affineEllipsoid_iff in Chap03/Definition_3_56, the chapter owner/view for quadratic-form ellipsoids; * enclosingEllipsoid in Chap05/Definition_5_4_5_1, the adjacent Chapter 5 precedent for keeping an image-form ellipsoid owner on a strict-cone shape parameter instead of collapsing it into a quadratic-form surrogate; * logDetBarrier in Chap05/Definition_5_4_4_5, the chapter owner for the -log det contribution on 𝕊^n₊₊. Best owner abstraction: * source-facing: the image-form inscribed ellipsoid W(G, v) together with the convex reformulation base feasible set on (G, v); * core/canonical: SetConstrainedMinimizationProblem, constrainedEpigraph, innerLePolyhedron, the Chapter 3 quadratic-form ellipsoid affineEllipsoid, and logDetBarrier; * bridge/view: the comparison W(G, v) = E(G², v), the triple epigraph presentation on (G, v, τ), and the textbook formulae obtained by expanding logDetBarrier back to -log det G. Primitive data: * the half-space presentation a, b; * the strict-cone shape parameter G; * the center v. Derived API: * the source-facing ellipsoid W(G, v); * the bridge theorem W(G, v) = E(G², v); * the containment theorem for W(G, v) ⊆ innerLePolyhedron a b; * the base feasible set on (G, v) cut out by the geometric containment W(G, v) ⊆ innerLePolyhedron a b; * the reformulation feasible set on triples (G, v, τ), realized canonically as the epigraph of logDetBarrier n over that base feasible set; * the owner optimization problem optimizationProblem a b; * the formula companions obtained by expanding W(G, v) ⊆ innerLePolyhedron a b into the support inequalities and logDetBarrier into -log det G. There is no upstream owner for the source-facing image-form ellipsoid parameterized by the inscribed-ellipsoid variable G : 𝕊^n₊₊, so this file owns W(G, v) directly, keeps affineEllipsoid only as the quadratic bridge W(G, v) = E(G², v), and builds the optimization reformulation from the earlier chapter owners constrainedEpigraph, innerLePolyhedron, and logDetBarrier directly.
namespace MaximumVolumeInscribedEllipsoid
open Matrix
open StrictPositiveSemidefiniteCone
open scoped EllipsoidNotation RealInnerProductSpace RealSymmetricMatrixSpace
variable {m n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "SymmMat" => 𝕊^n
def inscribedEllipsoid
(G : 𝕊^n₊₊) (v : E) : Set E :=
{x | ∃ u : E, ‖u‖ ≤ 1 ∧ x = v + (toMatrix G).toEuclideanLin u}
Definition 5.4.5.7
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.5.7: the logarithmic barrier for the maximum-volume inscribed ellipsoid reformulation is `F(G, v, τ) = -log det G - log (τ + log det G) - ∑ᵢ log ((bᵢ - ⟪aᵢ, v⟫)^2 - ‖G aᵢ‖^2)`.
namespace MaximumVolumeInscribedEllipsoid
open Matrix
open StrictPositiveSemidefiniteCone
open scoped BigOperators RealSymmetricMatrixSpace SecondOrderCone
variable {m n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "E" => EuclideanSpace ℝ (Fin n)
local notation "SymmMat" => 𝕊^n
def logarithmicBarrier
(a : Fin m → E) (b : Fin m → ℝ) :
LogarithmicBarrierPoint a b → ℝ :=
fun Gvτ ↦ logarithmicBarrierAmbient a b ((Gvτ.1.1 : SymmMat), Gvτ.1.2.1, Gvτ.1.2.2)
Definition 5.4.6.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.1: a map ξ : Q₁ → E₂ is three times continuously differentiable and concave with respect to the closed convex cone K when Q₁ is a closed convex set, K is closed, ξ is C^3 on interior Q₁, and -D²ξ(x)[h,h] belongs to K for every x ∈ interior Q₁ and every direction h.
universe u v
variable {E₁ : Type u} {E₂ : Type v}
[NormedAddCommGroup E₁] [NormedSpace ℝ E₁]
[NormedAddCommGroup E₂] [NormedSpace ℝ E₂]
class IsThreeTimesContDiffConcaveOnWith
(Q₁ : Set E₁) (K : ConvexCone ℝ E₂) (ξ : E₁ → E₂) : Prop
extends Fact (IsClosed (K : Set E₂)) where
/-- The domain set `Q₁` is closed. -/
isClosed_domain : IsClosed Q₁
/-- The domain set `Q₁` is convex. -/
convex_domain : Convex ℝ Q₁
/-- The map `ξ` is three-times continuously differentiable on `interior Q₁`. -/
contDiffOn : ContDiffOn ℝ 3 ξ (interior Q₁)
/-- The negated second within-domain directional derivative of `ξ` belongs to `K` at every
interior point of `Q₁`. -/
neg_second_directional_derivative_mem {x : E₁} (hx : x ∈ interior Q₁) (h : E₁) :
-iteratedFDerivWithin ℝ 2 ξ (interior Q₁) x (fun _ ↦ h) ∈ KTheorem 5.4.6.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.1: for a map ξ, the cone-order bound D³ξ(x)[h,h,h] \preceq_K -3β D²ξ(x)[h,h] ‖h‖[F; x] on interior Q₁ is equivalent to the same bound with D³ξ(x)[h,h,h] replaced by -D³ξ(x)[h,h,h]. The theorem is stated on the chapter's owner surface for the vector-valued directional derivatives and the barrier Hessian local norm.
open scoped HessianLocalNorm
universe u v
variable {E₁ : Type u} {E₂ : Type v}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
[NormedAddCommGroup E₂] [NormedSpace ℝ E₂]
-- Proof sketch: for a fixed interior point `x`, replace `h` by `-h`. The second directional
-- derivative and the Hessian local norm are even in `h`, while the third directional derivative
-- is odd, so the two displayed cone-membership conditions transform into one another.
theorem betaCompatibility_sign_reversal_iff
(Q₁ : Set E₁) (K : ConvexCone ℝ E₂) (F : E₁ → ℝ) (β : ℝ) (ξ : E₁ → E₂) :
(∀ ⦃x : E₁⦄ (_ : x ∈ interior Q₁) (h : E₁),
(3 * β * ‖h‖[F; x]) • (-vectorSecondDirectionalDerivative ξ x h) -
vectorThirdDirectionalDerivative ξ x h ∈
K) ↔
∀ ⦃x : E₁⦄ (_ : x ∈ interior Q₁) (h : E₁),
(3 * β * ‖h‖[F; x]) • (-vectorSecondDirectionalDerivative ξ x h) +
vectorThirdDirectionalDerivative ξ x h ∈
KDefinition 5.4.6.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.2: a map ξ : E₁ → E₂ is β-compatible with the barrier F relative to the cone K when Q₁ is convex with nonempty interior, β ≥ 1, F is a self-concordant barrier on interior Q₁, ξ is three-times continuously differentiable on interior Q₁, and for every x ∈ interior Q₁ and every direction h the cone-order inequality D^3 ξ(x)[h,h,h] \preceq_K -3 β D^2 ξ(x)[h,h] ⟨∇²F(x) h, h⟩^{1/2} holds, encoded as membership of the difference in K.
open scoped Gradient HessianLocalNorm
universe u v
variable {E₁ : Type u} {E₂ : Type v}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
[NormedAddCommGroup E₂] [NormedSpace ℝ E₂]
class IsBetaCompatibleWith
(Q₁ : Set E₁) (K : ConvexCone ℝ E₂) (F : E₁ → ℝ) (β : NNReal) (ξ : E₁ → E₂) : Prop where
/-- The domain set `Q₁` is convex. -/
convex_domain : Convex ℝ Q₁
/-- The domain set `Q₁` has nonempty interior. -/
interior_nonempty : (interior Q₁).Nonempty
/-- The compatibility parameter satisfies `β ≥ 1`. -/
one_le_parameter : 1 ≤ β
/-- The scalar function `F` is a self-concordant barrier on `interior Q₁` for some barrier
parameter. -/
selfConcordantBarrier :
∃ ν : NNReal, IsSelfConcordantBarrierOnWith (interior Q₁) ν F
/-- The map `ξ` is three-times continuously differentiable on `interior Q₁`. -/
contDiffOn : ContDiffOn ℝ 3 ξ (interior Q₁)
/-- The third derivative of `ξ` is dominated by the negated second derivative scaled by the
local Hessian norm of the barrier, in the cone order induced by `K`. -/
compatibility_bound {x : E₁} (hx : x ∈ interior Q₁) (h : E₁) :
(3 * (β : ℝ) * ‖h‖[F; x]) •
(-vectorSecondDirectionalDerivative ξ x h) -
vectorThirdDirectionalDerivative ξ x h ∈ KTheorem 5.4.6.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.2: positive linear combinations of β-compatible functions are again β-compatible with the same self-concordant barrier.
open scoped Gradient
universe u v
variable {E₁ : Type u} {E₂ : Type v}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
[NormedAddCommGroup E₂] [NormedSpace ℝ E₂]
-- Route repair: this file only needs the source-facing compatibility owner and its barrier-side
-- support classes. We restate that small owner layer here so the theorem does not depend on the
-- blocked Chapter 5 derivative import chain through `Definition_5_0_10`.
open scoped HessianLocalNorm
namespace IsBetaCompatibleWith
variable {Q₁ : Set E₁} {K : ConvexCone ℝ E₂} {F : E₁ → ℝ} {β : NNReal}
theorem pos_combination
{ξ₁ ξ₂ : E₁ → E₂} {α₁ α₂ : ℝ}
(hξ₁ : IsBetaCompatibleWith Q₁ K F β ξ₁)
(hξ₂ : IsBetaCompatibleWith Q₁ K F β ξ₂)
(hα₁ : 0 < α₁) (hα₂ : 0 < α₂) :
IsBetaCompatibleWith Q₁ K F β (α₁ • ξ₁ + α₂ • ξ₂)
Definition 5.4.6.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.3: the feasible set obtained by composing Q₂ with the cone-order domination relation induced by ξ and K.
universe u v w uR
variable {R : Type uR}
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
variable [Semiring R] [PartialOrder R]
variable [AddCommGroup E₂] [SMul R E₂]
def coneCompositionFeasibleSet
(Q₁ : Set E₁) (K : ConvexCone R E₂) (ξ : E₁ → E₂)
(Q₂ : Set (E₂ × E₃)) : Set (E₁ × E₃) :=
{ p | ∃ y : E₂, p.1 ∈ Q₁ ∧ ξ p.1 - y ∈ K ∧ (y, p.2) ∈ Q₂ }
-- Proof sketch: unfold `coneCompositionFeasibleSet`.Theorem 5.4.6.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.3: specializing coneCompositionFeasibleSet to the power-cone data Q₁ = powerConeQ1, K = ℝ₊, ξ = powerConeGeometricMean α, and Q₂ = powerConeQ2 recovers the chapter's source-facing power cone K_α = powerCone α. The textbook assumptions 0 < α < 1 are redundant for this set identity.
open scoped PowerCone
theorem coneCompositionFeasibleSet_eq_powerCone (α : ℝ) :
coneCompositionFeasibleSet
powerConeQ1
(ConvexCone.positive ℝ ℝ)
(powerConeGeometricMean α)
powerConeQ2 =
K_[α]Definition 5.4.6.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.4: if every direction (s, 0) with s ∈ K is a recession direction of the convex set Q₂, then at every interior point (y, z) the pairing of the y-gradient of a barrier Φ with s is nonpositive. The existence of a self-concordant barrier parameter for Φ on interior Q₂ is the only barrier input needed here. The L² product owner `WithLp 2 (E₂ × E₃)` is used only internally through the canonical affine pullback along WithLp.ofLp.
open Set Topology
open scoped Gradient
universe u v
variable {E₂ : Type u} {E₃ : Type v}
variable [NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
variable [NormedAddCommGroup E₃] [InnerProductSpace ℝ E₃] [CompleteSpace E₃]
local notation "Z" => WithLp 2 (E₂ × E₃)
local notation "ofZ" => (WithLp.ofLp : Z → E₂ × E₃)
-- Proof sketch: apply
-- `IsSelfConcordantBarrierOnWith.comp_continuousAffineMap` to pull the source-facing barrier
-- `Φ` on `interior Q₂` back along the canonical map `WithLp.ofLp`, then apply
-- `IsSelfConcordantBarrierOnWith.inner_gradient_nonpos_of_recession_direction` to that internal
-- `L²` product-owner barrier and the direction `WithLp.toLp 2 (s, 0)`. Use
-- mathlib's
-- `Convex.interior` and `Convex.add_smul_mem_interior` to pass from recession directions of `Q₂`
-- to recession directions of `interior Q₂`, then rewrite the resulting product-space pairing with
-- `sum_partialGradient_pairings_eq_inner_gradient_pair` specialized to the direction `(s, 0)`.
theorem barrier_yGradient_pairing_nonpos
{Q₂ : Set (E₂ × E₃)} (hQ₂_convex : Convex ℝ Q₂) (K : ConvexCone ℝ E₂)
{Φ : E₂ × E₃ → ℝ}
(hΦ : ∃ ν : NNReal, IsSelfConcordantBarrierOnWith (interior Q₂) ν Φ)
(hK_recession :
∀ ⦃s : E₂⦄, s ∈ (K : Set E₂) →
∀ ⦃p : E₂ × E₃⦄, p ∈ Q₂ → ∀ τ : ℝ, 0 ≤ τ → p + τ • (s, (0 : E₃)) ∈ Q₂)
{y : E₂} {z : E₃} (hyz : (y, z) ∈ interior Q₂) {s : E₂} (hs : s ∈ (K : Set E₂)) :
inner ℝ (∇ (fun y' : E₂ ↦ Φ (y', z)) y) s ≤ 0Theorem 5.4.6.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.4: under the auxiliary data from Definition 5.4.6.6, if ξ is differentiable at x and Φ is differentiable at (ξ x, z), then the first directional derivative of ψ(x, z) = Φ (ξ x, z) in direction (h, v) splits into the y-gradient term paired with Dξ(x)[h] and the z-gradient term paired with v.
open scoped Gradient
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [NormedSpace ℝ E₁]
[NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
[NormedAddCommGroup E₃] [InnerProductSpace ℝ E₃] [CompleteSpace E₃]
local notation "Z" => WithLp 2 (E₂ × E₃)
theorem compositionPotential_lineDeriv_eq_sum_partialGradient_pairings
{Φ : E₂ × E₃ → ℝ} {ξ : E₁ → E₂} {x h : E₁} {z v : E₃}
(hξ : DifferentiableAt ℝ ξ x) (hΦ : DifferentiableAt ℝ Φ (ξ x, z)) :
lineDeriv ℝ (compositionPotential Φ ξ) (x, z) (h, v) =
inner ℝ (∇ (fun y : E₂ ↦ Φ (y, z)) (ξ x)) (fderiv ℝ ξ x h) +
inner ℝ (∇ (fun z' : E₃ ↦ Φ (ξ x, z')) z) vDefinition 5.4.6.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.5: given a barrier F on Q₁, a barrier Φ on Q₂, a map ξ : Q₁ → E₂, and a compatibility parameter β, the composed barrier on pairs (x, z) is Ψ(x, z) = Φ(ξ(x), z) + β^3 F(x).
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
def coneCompositionBarrier
(F : E₁ → ℝ) (Φ : E₂ × E₃ → ℝ) (ξ : E₁ → E₂) (β : NNReal) : E₁ × E₃ → ℝ :=
fun p ↦ Φ (ξ p.1, p.2) + ((β : ℝ) ^ 3) * F p.1
-- Proof sketch: unfold `coneCompositionBarrier`; the definition evaluates `Φ` at the pair
-- `(ξ x, z)` and adds the scaled barrier term `β^3 F x`.Theorem 5.4.6.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.5: if ξ is C² at x and Φ is C² at (ξ(x), z), then for ψ(x, z) = Φ(ξ(x), z), Δ₂ = D² (fun x' ↦ compositionPotential Φ ξ (x', z))(x)[d, d], σ₁ = ⟪∇² Φ(ξ(x), z) l, l⟫ with l = (Dξ(x)[d], 0), and σ₂ = ⟪∇ᵧ Φ(ξ(x), z), D²ξ(x)[d, d]⟫, the decomposition (5.4.24) reads Δ₂ = σ₁ + σ₂.
open scoped Gradient
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
variable [NormedAddCommGroup E₁] [NormedSpace ℝ E₁]
variable [NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
variable [NormedAddCommGroup E₃] [InnerProductSpace ℝ E₃] [CompleteSpace E₃]
theorem compositionPotential_secondDirectionalDerivative_eq_sigmaOne_add_sigmaTwo
{Φ : E₂ × E₃ → ℝ} {ξ : E₁ → E₂} {x d : E₁} {z : E₃}
(hξ : ContDiffAt ℝ 2 ξ x)
(hΦ : ContDiffAt ℝ 2 Φ (ξ x, z)) :
secondDirectionalDerivative (fun x' : E₁ ↦ compositionPotential Φ ξ (x', z)) x d =
compositionPotentialSigmaOne Φ ξ x z d +
compositionPotentialSigmaTwo Φ ξ x z dDefinition 5.4.6.6
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.6 (2): the textbook scalar potential ψ(x, z) = Φ (ξ x, z).
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
variable (Φ : E₂ × E₃ → ℝ) (ξ : E₁ → E₂) (x : E₁) (z : E₃)
abbrev compositionPotential : E₁ × E₃ → ℝ :=
Φ ∘ Prod.map ξ idTheorem 5.4.6.6
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.6: if the negative lifted derivative direction -l' is a recession direction of Q₂, and if Φ is some self-concordant barrier on interior Q₂, then the term σ₂ is nonnegative.
open scoped Gradient
universe u v w
open Set Topology
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [NormedSpace ℝ E₁]
[NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
[NormedAddCommGroup E₃] [InnerProductSpace ℝ E₃] [CompleteSpace E₃]
local notation "Z" => WithLp 2 (E₂ × E₃)
local notation "ofZ" => (WithLp.ofLp : Z → E₂ × E₃)
-- Proof sketch: pull the barrier back along `WithLp.ofLp` to the canonical `L²` product owner,
-- rewrite `σ₂` as the ambient gradient pairing of that pullback barrier with the lifted direction
-- `WithLp.toLp 2 l'`, transfer the recession-direction hypothesis on `-l'` from the convex set
-- `Q₂` to the barrier domain `interior Q₂`, and then apply
-- `IsSelfConcordantBarrierOnWith.inner_gradient_nonpos_of_recession_direction`.
theorem compositionPotentialSigmaTwo_nonneg_of_neg_liftedDirectionDerivative_recession
{Q₂ : Set (E₂ × E₃)} {Φ : E₂ × E₃ → ℝ} {ξ : E₁ → E₂} {x d : E₁} {z : E₃}
(hQ₂_convex : Convex ℝ Q₂)
(hΦ : ∃ μ : NNReal, IsSelfConcordantBarrierOnWith (interior Q₂) μ Φ)
(hyz : (ξ x, z) ∈ interior Q₂)
(hrecession :
∀ ⦃p : E₂ × E₃⦄, p ∈ Q₂ → ∀ τ : ℝ, 0 ≤ τ →
p + τ • (-compositionSecondLiftedDirectionDerivative ξ x d) ∈ Q₂) :
0 ≤ compositionPotentialSigmaTwo Φ ξ x z dDefinition 5.4.6.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.7: for the lifted map l(y) = (ξ' y, v), the textbook directional derivative Dl(x)[d] is the canonical Fréchet derivative application below.
universe u v w
variable {𝕜 : Type*} [NontriviallyNormedField 𝕜]
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
variable [NormedAddCommGroup E₁] [NormedSpace 𝕜 E₁]
variable [NormedAddCommGroup E₂] [NormedSpace 𝕜 E₂]
variable [NormedAddCommGroup E₃] [NormedSpace 𝕜 E₃]
variable (ξ' : E₁ → E₂) (v : E₃) (x d : E₁)
#check fderiv 𝕜 (fun y : E₁ ↦ (ξ' y, v)) x dTheorem 5.4.6.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.7: the third directional derivative Δ₃ = D³ψ(x, z)[d, d, d] of ψ(x, z) = Φ(ξ(x), z) is the sum of the third derivative of Φ in the lifted direction l = (Dξ(x)[d], 0), three times the mixed Hessian pairing with l' = (D²ξ(x)[d, d], 0), and the pairing of ∇ᵧ Φ(ξ(x), z) with D³ξ(x)[d, d, d].
open scoped Gradient
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [NormedSpace ℝ E₁]
[NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
[NormedAddCommGroup E₃] [InnerProductSpace ℝ E₃] [CompleteSpace E₃]
theorem compositionPotential_thirdDirectionalDerivative_eq
{Φ : E₂ × E₃ → ℝ} {ξ : E₁ → E₂} {x d : E₁} {z : E₃}
(hξ : ContDiffAt ℝ 3 ξ x)
(hΦ : ContDiffAt ℝ 3 Φ (ξ x, z)) :
let l : E₂ × E₃ := (fderiv ℝ ξ x d, 0);
thirdDirectionalDerivative (fun x' : E₁ ↦ compositionPotential Φ ξ (x', z)) x d =
thirdDirectionalDerivative Φ (ξ x, z) l +
(3 : ℝ) * inner ℝ (hessian Φ (ξ x, z) l)
(compositionSecondLiftedDirectionDerivative ξ x d) +
inner ℝ (∇ (fun y : E₂ ↦ Φ (y, z)) (ξ x))
(vectorThirdDirectionalDerivative ξ x d)Definition 5.4.6.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.8: the local squared norm in \mathbb E_1 is the square of the chapter's canonical Hessian local norm.
open scoped HessianLocalNorm
universe u
variable {E₁ : Type u}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
abbrev sigmaThree (F : E₁ → ℝ) (x h : E₁) : ℝ :=
‖h‖[F; x] ^ (2 : ℕ)Theorem 5.4.6.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.8: if -l' is a recession direction of Q₂ and Φ is some self-concordant barrier on interior Q₂, then at every (ξ(x), z) ∈ interior Q₂ the cross Hessian term ⟪∇² Φ(ξ(x), z) (Dξ(x)[d], 0), Dl(x)[d]⟫ is bounded above by σ₁^(1/2) σ₂, where Dl(x)[d] = compositionSecondLiftedDirectionDerivative ξ x d, σ₁ = ⟪∇² Φ(ξ(x), z) (Dξ(x)[d], 0), (Dξ(x)[d], 0)⟫, and σ₂ = ⟪∇ᵧ Φ(ξ(x), z), D²ξ(x)[d, d]⟫.
open scoped Gradient HessianLocalNorm
open Set Topology
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [NormedSpace ℝ E₁]
[NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
[NormedAddCommGroup E₃] [InnerProductSpace ℝ E₃] [CompleteSpace E₃]
-- Semantic recall: `ProdL2` provides the canonical `L²` product geometry; this file matches the
-- same `WithLp 2`-transported product structure used by `compositionPotentialSigmaOne`.
theorem compositionPotential_crossTerm_le_sqrt_sigmaOne_mul_sigmaTwo
{Q₂ : Set (E₂ × E₃)} {Φ : E₂ × E₃ → ℝ} {ξ : E₁ → E₂} {x d : E₁} {z : E₃}
(hQ₂_convex : Convex ℝ Q₂)
(hΦ : ∃ μ : NNReal, IsSelfConcordantBarrierOnWith (interior Q₂) μ Φ)
(hyz : (ξ x, z) ∈ interior Q₂)
(hrecession :
∀ ⦃p : E₂ × E₃⦄, p ∈ Q₂ → ∀ τ : ℝ, 0 ≤ τ →
p + τ • (-compositionSecondLiftedDirectionDerivative ξ x d) ∈ Q₂) :
inner ℝ (hessian Φ (ξ x, z) (fderiv ℝ ξ x d, (0 : E₃)))
(compositionSecondLiftedDirectionDerivative ξ x d) ≤
Real.sqrt (compositionPotentialSigmaOne Φ ξ x z d) *
compositionPotentialSigmaTwo Φ ξ x z dDefinition 5.4.6.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.6.9: for Ψ(x, z) = Φ (ξ x, z) + β^3 F x, the textbook quantities D₁, D₂, and D₃ are the following direct canonical directional-derivative owners applied to Ψ.
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [NormedSpace ℝ E₁]
[NormedAddCommGroup E₃] [NormedSpace ℝ E₃]
variable (F : E₁ → ℝ) (Φ : E₂ × E₃ → ℝ) (ξ : E₁ → E₂) (β : NNReal)
(x : E₁) (z : E₃) (h : E₁) (v : E₃)
local notation "Ψ" => coneCompositionBarrier F Φ ξ β
#check lineDeriv ℝ Ψ (x, z) (h, v)Theorem 5.4.6.9
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.9: if ξ is β-compatible with the barrier F on Q₁, if x lies in interior Q₁, and if -∇ᵧ Φ(ξ(x), z) belongs to the dual cone innerDual (K : Set E₂), then ⟪∇ᵧ Φ(ξ(x), z), D³ξ(x)[h, h, h]⟫ ≤ 3 β σ₂ σ₃^{1/2}, where σ₂ = compositionPotentialSigmaTwo Φ ξ x z h and σ₃ = sigmaThree F x h.
open ProperCone
open scoped Gradient HessianLocalNorm
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
[NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
theorem yGradient_pairing_vectorThirdDirectionalDerivative_le_of_betaCompatibility
{Q₁ : Set E₁} {K : ConvexCone ℝ E₂} {F : E₁ → ℝ} {Φ : E₂ × E₃ → ℝ}
{β : NNReal} {ξ : E₁ → E₂} {x h : E₁} {z : E₃}
(hξ : IsBetaCompatibleWith Q₁ K F β ξ)
(hx : x ∈ interior Q₁)
(hneg_yGradient_mem_innerDual :
-∇ (fun y : E₂ ↦ Φ (y, z)) (ξ x) ∈ innerDual (K : Set E₂)) :
inner ℝ (∇ (fun y : E₂ ↦ Φ (y, z)) (ξ x))
(vectorThirdDirectionalDerivative ξ x h) ≤
3 * (β : ℝ) * compositionPotentialSigmaTwo Φ ξ x z h *
Real.sqrt (sigmaThree F x h)
Theorem 5.4.6.10
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.10: if ξ is C³ at x, if Φ is standard self-concordant on a domain containing (ξ(x), z), if the negative lifted derivative direction satisfies the specialized local-norm estimate used in Theorem 5.4.6.8, and if ξ is β-compatible with the dual-cone sign condition from Theorem 5.4.6.9, then the third directional derivative of x' ↦ Φ(ξ(x'), z) is bounded above by 2 σ₁^(3/2) + 3 σ₁^(1/2) σ₂ + 3 β σ₂ σ₃^(1/2), where σ₁ = compositionPotentialSigmaOne Φ ξ x z d, σ₂ = compositionPotentialSigmaTwo Φ ξ x z d, and σ₃ = sigmaThree F x d.
open ProperCone
open scoped Gradient HessianLocalNorm
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
[NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
[NormedAddCommGroup E₃] [NormedSpace ℝ E₃]
[InnerProductSpace ℝ (E₂ × E₃)] [CompleteSpace (E₂ × E₃)]
theorem compositionPotential_thirdDirectionalDerivative_le_selfConcordant_sigma_bound
{dom : Set (E₂ × E₃)} {Q₁ : Set E₁} {K : ConvexCone ℝ E₂}
{F : E₁ → ℝ} {Φ : E₂ × E₃ → ℝ} {ξ : E₁ → E₂} {β : NNReal} {x d : E₁} {z : E₃}
(hξ : ContDiffAt ℝ 3 ξ x)
(hΦ_self : IsStandardSelfConcordantOn dom Φ)
(hyz : (ξ x, z) ∈ dom)
(hneg_liftedDirectionDerivative_le_sigmaTwo :
‖-compositionSecondLiftedDirectionDerivative ξ x d‖[Φ; (ξ x, z)] ≤
compositionPotentialSigmaTwo Φ ξ x z d)
(hξ_compat : IsBetaCompatibleWith Q₁ K F β ξ)
(hx : x ∈ interior Q₁)
(hneg_yGradient_mem_innerDual :
-∇ (fun y : E₂ ↦ Φ (y, z)) (ξ x) ∈ innerDual (K : Set E₂)) :
thirdDirectionalDerivative (fun x' : E₁ ↦ compositionPotential Φ ξ (x', z)) x d ≤
2 * (Real.sqrt (compositionPotentialSigmaOne Φ ξ x z d)) ^ (3 : ℕ) +
3 * Real.sqrt (compositionPotentialSigmaOne Φ ξ x z d) *
compositionPotentialSigmaTwo Φ ξ x z d +
3 * (β : ℝ) * compositionPotentialSigmaTwo Φ ξ x z d *
Real.sqrt (sigmaThree F x d)
Theorem 5.4.6.11
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.11: if F is standard self-concordant at x and ξ, Φ are C² at the relevant points, then the fixed-z slice Ψ(x') = coneCompositionBarrier F Φ ξ β (x', z) satisfies D₂ = σ₁ + σ₂ + β^3 σ₃, where σ₁, σ₂, and σ₃ are the canonical subsection owners.
open scoped Gradient
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
[NormedAddCommGroup E₂]
[NormedAddCommGroup E₃] [NormedSpace ℝ E₃]
variable [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
[InnerProductSpace ℝ (E₂ × E₃)] [CompleteSpace (E₂ × E₃)]
variable (F : E₁ → ℝ) (Φ : E₂ × E₃ → ℝ) (ξ : E₁ → E₂) (β : NNReal)
(x : E₁) (z : E₃) (h : E₁)
local notation "ψ" => fun x' : E₁ ↦ compositionPotential Φ ξ (x', z)
local notation "Ψ" => fun x' : E₁ ↦ coneCompositionBarrier F Φ ξ β (x', z)
-- Proof sketch: rewrite the directional slice of `Ψ` as the sum of the directional slice of
-- `ψ` and the scaled barrier slice `β^3 • directionalSlice F x h`. Additivity of
-- `secondDirectionalDerivative` on `C²` slices gives the decomposition
-- `D₂(Ψ) = D₂(ψ) + β^3 D₂(F)`. The standard self-concordance owner for `F` makes the Hessian
-- quadratic form nonnegative, so `D₂(F)` identifies with `sigmaThree F x h`.
-- Proof sketch: combine the slice decomposition above with the canonical upstream decomposition
-- `Δ₂ = σ₁ + σ₂` from `Theorem_5_4_6_5`.
theorem
coneCompositionBarrier_slice_secondDirectionalDerivative_eq_sigmaSum_add_betaCube_sigmaThree
{dom : Set E₁}
(hF_self : IsStandardSelfConcordantOn dom F) (hx : x ∈ dom)
(hξ : ContDiffAt ℝ 2 ξ x) (hΦ : ContDiffAt ℝ 2 Φ (ξ x, z)) :
secondDirectionalDerivative Ψ x h =
compositionPotentialSigmaOne Φ ξ x z h +
compositionPotentialSigmaTwo Φ ξ x z h +
((β : ℝ) ^ 3) * sigmaThree F x h
Theorem 5.4.6.12
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.12: assume Φ is standard self-concordant at (ξ x, z), ξ is β-compatible with the barrier F on Q₁, and the local hypotheses used in Theorem 5.4.6.10 hold for the slice x' ↦ compositionPotential Φ ξ (x', z). The hidden β ≥ 1, C³, and barrier inputs needed for the slice second-derivative estimate come from the compatibility owner hξ_compat. Then the slice Ψ(x') = coneCompositionBarrier F Φ ξ β (x', z) satisfies the third-derivative inequality D₃ ≤ 2 D₂^(3/2) at x in direction h.
open ProperCone
open scoped Gradient HessianLocalNorm
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
[NormedAddCommGroup E₂] [InnerProductSpace ℝ E₂] [CompleteSpace E₂]
[NormedAddCommGroup E₃] [InnerProductSpace ℝ E₃] [CompleteSpace E₃]
variable (F : E₁ → ℝ) (Φ : E₂ × E₃ → ℝ) (ξ : E₁ → E₂) (β : NNReal)
(x : E₁) (z : E₃) (h : E₁)
local notation "ψ" => fun x' : E₁ ↦ compositionPotential Φ ξ (x', z)
local notation "Ψ" => fun x' : E₁ ↦ coneCompositionBarrier F Φ ξ β (x', z)
-- Proof sketch: combine the third-derivative sigma bound from
-- `compositionPotential_thirdDirectionalDerivative_le_selfConcordant_sigma_bound` with the
-- second-derivative lower bound coming from a local reconstruction of the `D₂` slice estimate,
-- then apply the scalar inequality `t * (3 * s - t^2) ≤ 2 * s^(3/2)` to the resulting
-- `σ₁`, `σ₂`, `σ₃` expression.
theorem coneCompositionBarrier_slice_selfConcordant_bound
{Q₁ : Set E₁} {Q₂ : Set (E₂ × E₃)} {K : ConvexCone ℝ E₂}
(hΦ : IsStandardSelfConcordantOn (interior Q₂) Φ)
(hyz : (ξ x, z) ∈ interior Q₂)
(hneg_liftedDirectionDerivative_le_sigmaTwo :
‖-compositionSecondLiftedDirectionDerivative ξ x h‖[Φ; (ξ x, z)] ≤
compositionPotentialSigmaTwo Φ ξ x z h)
(hξ_compat : IsBetaCompatibleWith Q₁ K F β ξ)
(hx : x ∈ interior Q₁)
(hneg_yGradient_mem_innerDual :
-∇ (fun y : E₂ ↦ Φ (y, z)) (ξ x) ∈ innerDual (K : Set E₂)) :
thirdDirectionalDerivative Ψ x h ≤
2 * (Real.sqrt (secondDirectionalDerivative Ψ x h)) ^ (3 : ℕ)
Theorem 5.4.6.13
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.6.13: under the textbook hypotheses, the ambient Chapter 5 owner coneCompositionBarrier F Φ ξ β is a self-concordant barrier on the full feasible interior interior (coneCompositionFeasibleSet Q K ξ Q₂).
open Set Topology
universe u v w
variable {E₁ : Type u} {E₂ : Type v} {E₃ : Type w}
[NormedAddCommGroup E₁] [InnerProductSpace ℝ E₁] [CompleteSpace E₁]
[NormedAddCommGroup E₂] [NormedSpace ℝ E₂]
[NormedAddCommGroup E₃] [NormedSpace ℝ E₃]
variable {Q : Set E₁} {Q₂ : Set (E₂ × E₃)} {K : ConvexCone ℝ E₂}
{F : E₁ → ℝ} {Φ : E₂ × E₃ → ℝ} {ξ : E₁ → E₂}
{β μ ν : NNReal}
theorem coneCompositionBarrier_isSelfConcordantBarrierOnWith_onFeasibleInterior
[instProd13Inner : InnerProductSpace ℝ (E₁ × E₃)]
[instProd13Complete : CompleteSpace (E₁ × E₃)]
[instProd23Inner : InnerProductSpace ℝ (E₂ × E₃)]
[instProd23Complete : CompleteSpace (E₂ × E₃)]
(hξ_concave : IsThreeTimesContDiffConcaveOnWith Q K ξ)
(hξ_compat : IsBetaCompatibleWith Q K F β ξ)
(hQ₂_closed : IsClosed Q₂)
(hQ₂_convex : Convex ℝ Q₂)
(hF : IsSelfConcordantBarrierOnWith (interior Q) ν F)
(hΦ : IsSelfConcordantBarrierOnWith (interior Q₂) μ Φ)
(hK_recession :
∀ ⦃s : E₂⦄ (hs : s ∈ (K : Set E₂)) ⦃p : E₂ × E₃⦄
(hp : p ∈ Q₂) (τ : ℝ) (hτ : 0 ≤ τ),
p + τ • (s, (0 : E₃)) ∈ Q₂) :
IsSelfConcordantBarrierOnWith
(interior (coneCompositionFeasibleSet Q K ξ Q₂))
(μ + β ^ 3 * ν)
(coneCompositionBarrier F Φ ξ β)Definition 5.4.7.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.1: for α ∈ (0, 1), the power cone K_α consists of the triples ((x₁, x₂), z) with (x₁, x₂) ∈ ℝ_+² and |z| ≤ x₁^α x₂^(1 - α). The companion declarations below record the associated representation data Q₁, F, ν, K, ξ, and Q₂.
open scoped BigOperators EuclideanOrthant SecondOrderCone
local notation "E₂" => EuclideanSpace ℝ (Fin 2)
open scoped PowerConeGeometricMean
-- Proof sketch: unfold `ξ[α]`.
def powerCone (α : ℝ) : Set ((ℝ × ℝ) × ℝ) :=
coneCompositionFeasibleSet
powerConeQ1
(ConvexCone.positive ℝ ℝ)
ξ[α]
powerConeQ2Lemma 5.4.7.1
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.7.1: for 0 < α < 1, every ν-self-concordant barrier for the one-sided power cone K_α^+ has barrier parameter at least 3.
open scoped PowerConePlus
open Filter
theorem power_cone_plus_barrierParameter_ge_three
{α : ℝ} (hα₀ : 0 < α) (hα₁ : α < 1)
{ν : NNReal} {F : ((ℝ × ℝ) × ℝ) → ℝ}
(hF : IsSelfConcordantBarrierOnWith (interior (K_[α]⁺)) ν F) :
(3 : ℝ) ≤ (ν : ℝ)
Theorem 5.4.7.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.1 (1): the first directional derivative of ξ(x) = (x^(1))^α (x^(2))^(1 - α) at a point with positive coordinates satisfies Dξ(x)[h] = [α (h^(1) / x^(1)) + (1 - α) (h^(2) / x^(2))] ξ(x).
open scoped PowerConeGeometricMean
variable {α : ℝ} {x h : ℝ × ℝ}
local notation "ξ" => ξ[α]
local notation "gmean" => fun y : ℝ × ℝ ↦ ξ y
variable (α x h)
variable {α x h}
-- Proof sketch: differentiate the directional slice
-- `t ↦ ξ (x + t • h)` at `t = 0`, use the positive coordinate assumptions to identify the
-- derivatives of the two `Real.rpow` factors, and factor out `ξ x`.
theorem powerConeGeometricMean_firstDirectionalDerivative
(hx₁ : 0 < x.1) (hx₂ : 0 < x.2)
:
lineDeriv ℝ ξ x h =
(α * (h.1 / x.1) + (1 - α) * (h.2 / x.2)) * ξ xDefinition 5.4.7.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.2 recalls the (y, z)-coordinate specialization of the canonical second-order-cone barrier.
#check (secondOrderConeBarrier ∘ Prod.swap : (ℝ × ℝ) → ℝ)
-- Proof sketch: evaluate `secondOrderConeBarrier` at `(z, y)`.Lemma 5.4.7.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Lemma 5.4.7.2: any self-concordant barrier for the inverse-epigraph set 𝓘_n = {(X, Y) | X ≻ 0, Y ⪰ X⁻¹} has parameter at least 2 n.
open Matrix
open scoped MatrixOrder RealSymmetricMatrixSpace
variable {n : ℕ}
local notation "Mat" => Matrix (Fin n) (Fin n) ℝ
local notation "SymmMat" => 𝕊^n
local notation "Point" => SymmMat × SymmMat
local notation "Z" => WithLp 2 Point
local notation "ofZ" => (WithLp.ofLp : Z → Point)
local notation "BlockMat" => Matrix (Fin n ⊕ Fin n) (Fin n ⊕ Fin n) ℝ
local notation "BigMat" => Matrix (Fin (n + n)) (Fin (n + n)) ℝ
local notation "BigSymmMat" => 𝕊^(n + n)
theorem matrixInverseEpigraph_barrierParameter_ge_two_mul_dimension
{ν : NNReal} {F : Z → ℝ}
(hF : IsSelfConcordantBarrierOnWith
(ofZ ⁻¹' interior matrixInverseEpigraph) ν F) :
(2 * n : ℝ) ≤ (ν : ℝ)
Theorem 5.4.7.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.2: for 0 < α < 1, the weighted geometric mean ξ(x) = (x^(1))^α (x^(2))^(1 - α) is 1-compatible with the logarithmic barrier F(x) = -log x^(1) - log x^(2) on the orthant Q₁ = ℝ_+², relative to the scalar cone ℝ_+.
open scoped Gradient HessianLocalNorm PowerConeGeometricMean
theorem powerConeGeometricMean_isOneCompatibleWith_powerConeBarrier
{α : ℝ} (hα₀ : 0 < α) (hα₁ : α < 1) :
IsBetaCompatibleWith powerConeQ1 (ConvexCone.positive ℝ ℝ)
powerConeBarrier (1 : NNReal) ξ[α]Definition 5.4.7.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.3: for a strictly positive point x ∈ ℝ_{++}² and a direction h ∈ ℝ², the scalar σ is the squared local norm induced by the positive-orthant logarithmic barrier.
open EuclideanSpace (positiveOrthant) open scoped HessianLocalNorm RelativeDirection local notation "E₂" => EuclideanSpace ℝ (Fin 2) local notation "X₂" => positiveOrthant 2 abbrev scaledDirectionSigma (x : X₂) (h : E₂) : ℝ := sigmaThree (standardLogarithmicBarrierAmbient 2) x h
Theorem 5.4.7.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.3: for 0 < α < 1, the function Ψ_P((x₁, x₂), z) = -log ((x₁)^(2 α) (x₂)^(2 (1 - α)) - z^2) - log x₁ - log x₂ is a 4-self-concordant barrier for the power cone K_α.
open Nesterov.Chap05.Theorem_5_4_7_3
open scoped PowerConeGeometricMean
open scoped PowerCone
theorem power_cone_barrier_is_four_self_concordant_barrier
{α : ℝ} (hα₀ : 0 < α) (hα₁ : α < 1) :
IsSelfConcordantBarrierOnWith
(interior K_[α])
(4 : NNReal)
(power_cone_barrier α)
Definition 5.4.7.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.4: for α ∈ (0, 1), the one-sided power cone K_α^+ consists of the triples ((x₁, x₂), z) with (x₁, x₂) ∈ ℝ_+² and z ≤ x₁^α x₂^(1 - α).
open scoped QTwoPlus
def power_cone_plus (α : ℝ) : Set ((ℝ × ℝ) × ℝ) :=
coneCompositionFeasibleSet
powerConeQ1
(ConvexCone.positive ℝ ℝ)
(powerConeGeometricMean α)
Q₂⁺Theorem 5.4.7.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.4: for 0 < α < 1, the function Ψ_P^+((x₁, x₂), z) = -log (x₁^α x₂^(1 - α) - z) - log x₁ - log x₂ is a 3-self-concordant barrier for the one-sided power cone K_α^+.
open scoped PowerConePlus
open scoped QTwoPlus
open scoped Gradient
theorem power_cone_plus_barrier_is_three_self_concordant_barrier
{α : ℝ} (hα₀ : 0 < α) (hα₁ : α < 1) :
IsSelfConcordantBarrierOnWith
(interior (K_[α]⁺))
(3 : NNReal)
(power_cone_plus_barrier α)
Definition 5.4.7.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.5 recalls the canonical logarithmic barrier specialization for Q₂⁺ = {(y, z) : z ≤ y}.
open scoped QTwoPlus
-- Proof sketch: unfold `qTwoPlus`; membership is exactly the defining inequality `z ≤ y` for the
-- half-space `Q₂⁺`.
#check (sublevelLogBarrier (fun yz : ℝ × ℝ ↦ yz.2 - yz.1) 0 : (ℝ × ℝ) → ℝ)
-- Proof sketch: apply `sublevelLogBarrier_apply` to the affine function
-- `fun yz : ℝ × ℝ ↦ yz.2 - yz.1` and rewrite `0 - (z - y)` as `y - z`.Theorem 5.4.7.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.5: for n > 0, a point (τ, z) lies in the finite-dimensional ℓ_p epigraph cone exactly when it admits a lift to the chapter’s lifted domain with exponent 1 / p.toReal.
open scoped BigOperators
open scoped EuclideanSpaceLp
open scoped PowerCone
local notation "lpEpigraphConeLiftDomain" => lpEpigraphConeLiftDomainLocal
theorem mem_lpNormEpigraphCone_iff_exists_lift
{n : ℕ} (hn : 0 < n) (p : EuclideanSpace.LpExponent)
{τ : ℝ} {z : EuclideanSpace ℝ (Fin n)} :
(τ, z) ∈ lpNormEpigraphCone n p ↔
∃ x : EuclideanSpace ℝ (Fin n), (τ, x, z) ∈ lpEpigraphConeLiftDomain n (1 / p.toReal)Definition 5.4.7.6
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.6: the epigraph cone of the finite-dimensional ℓ_p norm is the set of pairs (τ, z) with τ ≥ ‖z‖_[p], implemented as the chapter constrained epigraph of the canonical Chapter 3 owner EuclideanSpace.lpSeminorm n p, read in the source order (τ, z).
open scoped EuclideanSpaceLp
variable (n : ℕ)
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
def lpNormEpigraphCone (p : EuclideanSpace.LpExponent) : Set (ℝ × Eₙ) :=
Prod.swap ⁻¹' constrainedEpigraph Set.univ (fun z : Eₙ ↦ (‖z‖_[p] : WithTop ℝ))Theorem 5.4.7.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.6: ξ is 1-compatible with the orthant barrier F, the corresponding specialized cone-composition barrier is 3-self-concordant on entropyEpigraphCone, and the unit slice x^(1) = 1 is described equivalently by the logarithmic and exponential epigraph inequalities from the source text.
open scoped EntropyEpigraph Gradient HessianLocalNorm
open Nesterov.Chap05.Theorem_5_4_7_6.Specialization
theorem entropyEpigraphConeBarrier_and_unitSlice_descriptions :
IsBetaCompatibleWith powerConeQ1 (ConvexCone.positive ℝ ℝ) powerConeBarrier (1 : NNReal) ξ ∧
IsSelfConcordantBarrierOnWith (interior entropyEpigraphCone) (3 : NNReal)
entropyEpigraphConeBarrier ∧
{yz : ℝ × ℝ | ((1, yz.1), yz.2) ∈ entropyEpigraphCone} =
constrainedEpigraph (Set.Ioi (0 : ℝ)) (fun x : ℝ ↦ (-Real.log x : WithTop ℝ)) ∧
{yz : ℝ × ℝ | ((1, yz.1), yz.2) ∈ entropyEpigraphCone} =
{yz : ℝ × ℝ | yz.1 ≥ Real.exp (-yz.2)}
Definition 5.4.7.7
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.7: expanding the owner interior K_[α] rewrites strict lifted-domain membership back to the coordinatewise positivity conditions from the textbook definition.
open scoped BigOperators
variable (n : ℕ)
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "LpLiftPointₙ" => ℝ × Eₙ × Eₙ
open scoped PowerCone
theorem mem_lpEpigraphConeBarrierLiftDomain_iff
(α τ : ℝ) (x z : Eₙ) :
(τ, x, z) ∈ lpEpigraphConeBarrierLiftDomain n α ↔
(∀ i : Fin n, 0 < x i) ∧
0 < τ ∧
(∀ i : Fin n,
0 < Real.rpow (x i) (2 * α) * Real.rpow τ (2 * (1 - α)) - (z i) ^ (2 : ℕ)) ∧
(∑ i : Fin n, x i) = τ
Theorem 5.4.7.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.7: restricting Ψ_L(x, y, t, τ) to the nesterovHyperplane ∑ i, y^(i) = τ yields a barrier function on the closed normalized lifted log-sum-exp region closure liftedConeLogSumExpRelativeDomain, viewed in its intrinsic relative ambient space.
open Filter Set Topology
open scoped BigOperators
universe v
variable {ι : Type v} [Fintype ι]
local notation "E" => EuclideanSpace ℝ ι
local notation "LPoint" => E × E × ℝ × ℝ
local notation "H" => (liftedConeLogSumExpNormalizationHyperplane : AffineSubspace ℝ LPoint)
local notation "D" => (liftedConeLogSumExpRelativeDomain : Set H)
theorem liftedConeLogSumExpBarrier_restriction_isBarrierFunctionOn :
IsBarrierFunctionOn
(closure D)
liftedConeLogSumExpRelativeBarrierMapDefinition 5.4.7.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.8: the entropy-epigraph cone is the constrained epigraph of the relative-entropy height -ξ(x) = x^(1) [log x^(1) - log x^(2)] on the strict positive orthant.
open scoped EntropyEpigraph
def entropyEpigraphCone : Set ((ℝ × ℝ) × ℝ) :=
constrainedEpigraph
(Set.Ioi (0 : ℝ) ×ˢ Set.Ioi (0 : ℝ))
(fun x : ℝ × ℝ ↦ ((-ξ x : ℝ) : WithTop ℝ))Theorem 5.4.7.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.8: for the positive-orthant logarithmic barrier, the local norm induced by the Hessian at a strictly positive point x agrees with the Euclidean norm of the scaled direction δ_x(h), written in Lean as δ[x](h).
open scoped Gradient HessianLocalNorm RelativeDirection
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
theorem positiveOrthantLogarithmicBarrier_localNorm_eq_norm_relativeDirection
(x : Xₙ) (h : Eₙ) :
‖h‖[standardLogarithmicBarrierAmbient n; x] = ‖δ[x](h)‖Definition 5.4.7.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.9 reuses the earlier orthant owner powerConeQ1 = ℝ_+².
recall powerConeQ1Theorem 5.4.7.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.9: for a ∈ Δₙ, the directional derivative of log ξ_a at a strictly positive point x along the ambient direction h, taken within the positive orthant, is the simplex-weighted mean Finset.univ.centerMass a (δ_x(h)) = ⟪a, δ_x(h)⟫.
open scoped BigOperators RelativeDirection MonomialXi StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
theorem lineDerivWithin_log_monomialXi_eq_centerMass_relativeDirection
(a : Δ[n]) (x : Xₙ) (h : Eₙ) :
lineDerivWithin ℝ (Real.log ∘ ambientMonomialXi a) Xₙ x h =
Finset.univ.centerMass a (δ[x](h))Definition 5.4.7.10
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.10 (1): the exponential cone is the source-facing coordinate view of the entropy-epigraph cone under ((x, y), τ) ↦ ((τ, y), -x).
def exponentialCone : Set ((ℝ × ℝ) × ℝ) :=
{p | ((p.2, p.1.2), -p.1.1) ∈ entropyEpigraphCone}
Theorem 5.4.7.10
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.10: for a ∈ Δₙ, the directional derivative of the monomial ξ_a at a strictly positive point x along h is ξ_a(x) Finset.univ.centerMass a (δ_x(h)) = ξ_a(x) ⟪a, δ_x(h)⟫.
open scoped BigOperators RelativeDirection MonomialXi StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
theorem lineDerivWithin_monomialXi_eq_monomialXi_mul_centerMass_relativeDirection
(a : Δ[n]) (x : Xₙ) (h : Eₙ) :
lineDerivWithin ℝ (ambientMonomialXi a) Xₙ x h =
ξ_[a] x * Finset.univ.centerMass a (δ[x](h))Definition 5.4.7.11
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.11: the conic-hull epigraph of the finite-family log-sum-exp function is the set of triples (x, t, τ) with τ > 0 and t ≥ τ * logSumExp (τ⁻¹ • x), i.e. the scaled-input form of t ≥ τ f(x / τ). This is the constrained epigraph of the perspective transform of logSumExp, written in the source-facing coordinates (x, t, τ). Specializing to ι = Fin n recovers the textbook n-variable cone.
open scoped BigOperators
universe v
variable {ι : Type v} [Fintype ι]
local notation "E" => EuclideanSpace ℝ ι
local notation "P" => ℝ × E
def logSumExpEpigraphCone : Set (E × ℝ × ℝ) :=
(fun p : E × ℝ × ℝ ↦ (((p.2.2, p.1), p.2.1) : P × ℝ)) ⁻¹'
constrainedEpigraph
(Set.Ioi (0 : ℝ) ×ˢ (Set.univ : Set E))
(fun z : P ↦ (perspectiveTransform logSumExp z : WithTop ℝ))Theorem 5.4.7.11
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.11: the second directional derivative of ξ_a on the positive orthant equals -ξ_a(x) S₂, where S₂ = quantityS2 a x h is the weighted second centered moment of the relative direction δ_x(h).
open scoped BigOperators RelativeDirection MonomialXi StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
theorem monomialXi_secondDirectionalDerivative_eq_neg_mul_quantityS2
(a : Δ[n]) (x : Xₙ) (h : Eₙ) :
secondDirectionalDerivative (ambientMonomialXi a) x h =
-(ξ_[a] x * quantityS2 a x h)Definition 5.4.7.12
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.12: the lifted cone hat Q for the log-sum-exp epigraph consists of the quadruples (x, y, t, τ) such that each coordinate triple ((x^(i) - t, y^(i)), τ) lies in the scalar exponential cone and the slack variables satisfy the normalization equation ∑ i, y^(i) = τ.
open scoped BigOperators
universe v
variable {ι : Type v} [Fintype ι]
local notation "E" => EuclideanSpace ℝ ι
local notation "LPoint" => E × E × ℝ × ℝ
def liftedConeLogSumExp : Set LPoint
Theorem 5.4.7.12
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.12: the third directional derivative of ξ_a on the positive orthant equals ξ_a(x) (2 S₃ + 3 m S₂), where m = ⟪a, δ_x(h)⟫, S₂ = quantityS2 a x h, and S₃ = quantityS3 a x h.
open scoped BigOperators RelativeDirection MonomialXi StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
variable (a : Δ[n]) (x : Xₙ) (h : Eₙ)
local notation "m" => Finset.univ.centerMass a (δ[x](h))
theorem monomialXi_thirdDirectionalDerivative_eq_mul_two_S3_add_three_mean_S2 :
thirdDirectionalDerivative (ambientMonomialXi a) x h =
ξ_[a] x * (2 * quantityS3 a x h + 3 * m * quantityS2 a x h)Definition 5.4.7.13
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.13 recalls the Chapter 5 owner for the lifted finite-family log-sum-exp barrier.
open scoped BigOperators
universe v
variable {ι : Type v} [Fintype ι]
recall liftedConeLogSumExpBarrier {ι : Type v} [Fintype ι] :
EuclideanSpace ℝ ι × EuclideanSpace ℝ ι × ℝ × ℝ → ℝTheorem 5.4.7.13
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.13: for a ∈ Δₙ, the monomial ξ_a(x) = x^a = ∏_{i=1}^n (x^(i))^(a^(i)) is 1-compatible with the logarithmic barrier F(x) = -\sum_{i=1}^n \log x^(i) on the strict positive orthant \mathbb{R}^n_{++}.
open scoped BigOperators RelativeDirection HessianLocalNorm MonomialXi StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
theorem monomial_isOneCompatibleWith_positiveOrthantLogarithmicBarrier
(a : Δ[n]) :
IsBetaCompatibleWith
Xₙ
(ConvexCone.positive ℝ ℝ)
(standardLogarithmicBarrierAmbient n)
(1 : NNReal)
(ambientMonomialXi a)
Definition 5.4.7.14
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.14: for a strictly positive point x ∈ ℝ^n_{++} and a direction h ∈ ℝⁿ, the relative direction δ_x(h) is the vector whose i-th coordinate is h^(i) / x^(i).
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
def relativeDirection (x : Xₙ) (h : Eₙ) : Eₙ :=
WithLp.toLp 2 fun i ↦ h i / (x : Eₙ) iTheorem 5.4.7.14
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.14: the posynomial ξ(x) = \sum_{k=1}^m α_k x^{a_k} with positive coefficients and simplex exponents is 1-compatible with the logarithmic barrier F(x) = -\sum_{i=1}^n \log x^{(i)} on the positive orthant \mathbb{R}^n_{++}.
open scoped BigOperators Gradient MonomialXi StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
theorem posynomialXi_isOneCompatibleWith_positiveOrthantLogarithmicBarrier
(m : ℕ)
(α : Fin m → Set.Ioi (0 : ℝ))
(a : Fin m → Δ[n]) :
IsBetaCompatibleWith
(positiveOrthant n)
(ConvexCone.positive ℝ ℝ)
(standardLogarithmicBarrierAmbient n)
(1 : NNReal)
(∑ k : Fin m, (α k : ℝ) • ambientMonomialXi (a k))
Definition 5.4.7.15
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.15 recalls the Chapter 1 positive orthant owner.
open scoped EuclideanOrthant
recall EuclideanSpace.positiveOrthant
(n : ℕ) :
Set (EuclideanSpace ℝ (Fin n))Theorem 5.4.7.15
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.7.15 is IsSelfConcordantBarrierOnWith.concaveOn_exp_neg_div.
universe u
recall IsSelfConcordantBarrierOnWith.concaveOn_exp_neg_divDefinition 5.4.7.16
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.16 recalls the canonical standard simplex owner.
open scoped BigOperators
open scoped StandardSimplex
variable (n : ℕ)
recall stdSimplex
(𝕜 : Type*) (ι : Type*) [Semiring 𝕜] [PartialOrder 𝕜] [Fintype ι] :
Set (ι → 𝕜)Definition 5.4.7.17
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.17: for a simplex vector a ∈ Δₙ, monomialXi a is the monomial ξ_a(x) = x^a = ∏_{i=1}^n (x^(i))^(a^(i)) on the strict positive orthant \mathbb{R}^n_{++}.
open scoped BigOperators StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
def monomialXi (a : Δ[n]) : Xₙ → ℝ :=
fun x ↦ ambientMonomialXi a xDefinition 5.4.7.18
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.18: the quantity S₂ is the simplex-weighted second centered moment of the relative direction δ_x(h), centered by its simplex-weighted mean Finset.univ.centerMass a (δ[x](h)).
open scoped BigOperators RelativeDirection StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
def quantityS2 (a : Δ[n]) (x : Xₙ) (h : Eₙ) : ℝ :=
a ⬝ᵥ fun i : Fin n ↦ (δ[x](h) i - Finset.univ.centerMass a (δ[x](h))) ^ (2 : ℕ)
-- Proof sketch: unfold `quantityS2`; the canonical dot-product notation `⬝ᵥ` is definitionally the
-- finite sum of the weighted squared centered coordinates.Definition 5.4.7.19
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.19: the quantity S₃ is the simplex-weighted third centered moment of the relative direction δ_x(h), centered by its simplex-weighted mean Finset.univ.centerMass a (δ[x](h)).
open scoped BigOperators RelativeDirection StandardSimplex
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
def quantityS3 (a : Δ[n]) (x : Xₙ) (h : Eₙ) : ℝ :=
a ⬝ᵥ fun i : Fin n ↦ (δ[x](h) i - Finset.univ.centerMass a (δ[x](h))) ^ (3 : ℕ)
-- Proof sketch: unfold `quantityS3`; the canonical dot-product notation `⬝ᵥ` is definitionally the
-- finite sum of the weighted cubed centered coordinates.Definition 5.4.7.20
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.20: for functions F, ξ : ℝ^n_{++} → ℝ, the associated barrier on 𝒟 = {(x, t) : t < ξ(x)} is Ψ(x, t) = -log (ξ(x) - t) + F(x).
open EuclideanSpace (positiveOrthant)
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
local notation "Xₙ" => positiveOrthant n
def hypographBarrierPsi (F ξ : Xₙ → ℝ) : hypographBarrierDomain ξ → ℝ :=
fun xt ↦ sublevelLogBarrier (hypographGap ξ) 0 xt.1 + F xt.1.1
-- Proof sketch: unfold `hypographBarrierPsi`; the subtype argument carries a pair `(x, t)` in the
-- domain, and evaluation substitutes that pair into the defining formula.Definition 5.4.7.21
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.21 lies in the Chapter 5 posynomial / positive-orthant domain. Primary domain: - finite positive linear combinations of simplex monomials on the strict positive orthant. Relevant owner declarations sampled before refinement: - monomialXi and monomialXi_apply in Definition_5_4_7_17, the chapter owner and evaluation bridge for the simplex monomials ξ_[a]; - EuclideanSpace.positiveOrthant from Chap01/Definition_1_10_2, the ambient owner for the strict positive orthant ℝⁿ₊₊; - posynomialXi in Theorem_5_4_7_14, the existing Chapter 5 owner for the same posynomial construction; - posynomialXi_apply in Theorem_5_4_7_14, the canonical evaluation lemma on that owner. Best owner abstraction: - posynomialXi. Primitive data: - the positive coefficients α : Fin m → Set.Ioi (0 : ℝ); - the simplex exponents a : Fin m → Δ[n]. Derived API: - the owner declaration posynomialXi; - the evaluation lemma posynomialXi_apply. Source/core/bridge triage: - source-facing: the posynomial ξ(x) = ∑ₖ αₖ x^{aₖ} on ℝⁿ₊₊; - core/canonical: the Chapter 5 owner posynomialXi; - bridge/view: the monomial notation ξ_[(a k)] used in the canonical evaluation formula. The previous version rebuilt the same owner and its apply lemma locally. This file now reuses the existing Chapter 5 declaration directly instead of keeping a parallel duplicate API.
open scoped BigOperators MonomialXi StandardSimplex
open EuclideanSpace (positiveOrthant)
recall posynomialXi
(n m : ℕ)
(α : Fin m → Set.Ioi (0 : ℝ))
(a : Fin m → Δ[n]) :
positiveOrthant n → ℝ
Definition 5.4.7.22
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.7.22 recalls the source-domain restriction of the Chapter 5 ambient exponential transform.
universe u
variable {E : Type u}
variable (dom : Set E) (p : NNRealˣ) (F : E → ℝ)
#check (dom.restrict (barrierExponentialTransform p F) : dom → ℝ)Definition 5.4.8.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.1: a separable optimization problem on a real vector space E is specified by finite families of positive weights αᵢⱼ, affine functionals ℓᵢⱼ : E →ᵃ[ℝ] ℝ, convex scalar functions fᵢⱼ : ℝ → ℝ, and right-hand sides β₁, …, βₘ, producing the functions qᵢ(x) = ∑ⱼ αᵢⱼ fᵢⱼ (ℓᵢⱼ(x)) and the problem of minimizing q₀(x) subject to qᵢ(x) ≤ βᵢ for i = 1, …, m.
universe u
open scoped BigOperators
variable {E : Type u} [AddCommGroup E] [Module ℝ E] {m : ℕ}
structure SeparableOptimizationProblem (E : Type u) [AddCommGroup E] [Module ℝ E] (m : ℕ) where
/-- The number `mᵢ` of separable summands in `qᵢ`. -/
blockSize : Fin (m + 1) → ℕ
/-- The positive coefficients `αᵢⱼ` multiplying the scalar convex terms. -/
weight (i : Fin (m + 1)) (j : Fin (blockSize i)) : ℝ
/-- Each coefficient `αᵢⱼ` is positive. -/
weight_pos (i : Fin (m + 1)) (j : Fin (blockSize i)) : 0 < weight i j
/-- The affine functionals `ℓᵢⱼ : E →ᵃ[ℝ] ℝ`. -/
affineMap (i : Fin (m + 1)) (j : Fin (blockSize i)) : E →ᵃ[ℝ] ℝ
/-- The scalar convex functions `fᵢⱼ : ℝ → ℝ`. -/
scalarFunction (i : Fin (m + 1)) (j : Fin (blockSize i)) : ℝ → ℝ
/-- Each scalar function `fᵢⱼ` is convex on all of `ℝ`. -/
scalarFunction_convex (i : Fin (m + 1)) (j : Fin (blockSize i)) :
ConvexOn ℝ Set.univ (scalarFunction i j)
/-- The right-hand sides `β₁, …, βₘ` of the inequality constraints. -/
constraintBound : Fin m → ℝTheorem 5.4.8.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.1: the original separable optimization problem and its standard-form epigraph reformulation have the same canonical Chapter 1 optimal value.
universe u
open scoped BigOperators
namespace SeparableOptimizationProblem
variable {E : Type u} [AddCommGroup E] [Module ℝ E] {m : ℕ}
open StandardFormDecisionVariable
theorem separableOptimizationProblem_optimalValue_eq_standardFormOptimalValue
(problem : SeparableOptimizationProblem E m) :
(problem : LagrangianProblem E m).toSetConstrainedMinimizationProblem.optimalValue =
(standardFormOptimizationProblem problem).optimalValueDefinition 5.4.8.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.2 recalls the chapter owner field for the block counts mᵢ.
open scoped BigOperators
namespace SeparableOptimizationProblem
universe u
variable {E : Type u} [AddCommGroup E] [Module ℝ E] {m : ℕ}
variable (problem : SeparableOptimizationProblem E m)
#check SeparableOptimizationProblem.blockSizeTheorem 5.4.8.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.2: the canonical epigraph logarithmic barrier specialized to x ↦ -\log x, namely F₁(x, t) = -\log x - \log (\log x + t), is a 2-self-concordant barrier for the canonical epigraph of Definition 5.4.8.5, viewed on the canonical L² product owner Z = WithLp 2 (ℝ × ℝ) through z ↦ z.ofLp.
local notation "Z" => WithLp 2 (ℝ × ℝ)
local notation "ofZ" => (WithLp.ofLp : Z → ℝ × ℝ)
local notation "Q₁" =>
constrainedEpigraph (Set.Ioi (0 : ℝ)) (fun x : ℝ ↦ (-Real.log x : WithTop ℝ))
-- Proof sketch: the interior of the canonical closed epigraph from Definition 5.4.8.5 is
-- obtained by replacing the boundary inequality `t ≥ -\log x` with the strict inequality
-- `t > -\log x` while keeping `x > 0`.
theorem epigraphLogBarrier_negLog_is_two_selfConcordantBarrier :
IsSelfConcordantBarrierOnWith
(ofZ ⁻¹' interior Q₁)
(2 : NNReal)
(epigraphLogBarrier (fun x : ℝ ↦ -Real.log x) ∘ ofZ)
Definition 5.4.8.3
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.3 recalls the canonical owner whose projections supply the slack variables.
namespace SeparableOptimizationProblem
universe u
variable {E : Type u} [AddCommGroup E] [Module ℝ E] {m : ℕ}
recall StandardFormDecisionVariableTheorem 5.4.8.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.3: the function F₂(x, t) = -log t - log (log t - x) is a 2-self-concordant barrier for the exponential epigraph Q₂ = {(x, t) ∈ ℝ² | t ≥ e^x}, viewed on the canonical L² product owner Z = WithLp 2 (ℝ × ℝ) through z ↦ z.ofLp. This is the affine pullback of Theorem 5.4.8.2 along (x, t) ↦ (t, -x).
local notation "Z" => WithLp 2 (ℝ × ℝ)
local notation "ofZ" => (WithLp.ofLp : Z → ℝ × ℝ)
local notation "Q₁" =>
constrainedEpigraph (Set.Ioi (0 : ℝ)) (fun x : ℝ ↦ (-Real.log x : WithTop ℝ))
theorem exponentialEpigraphBarrierF2_is_two_selfConcordantBarrier :
IsSelfConcordantBarrierOnWith
(ofZ ⁻¹' interior exponentialEpigraphQ2)
(2 : NNReal)
(fun z : Z ↦ exponentialEpigraphBarrierF2 z.ofLp)
Definition 5.4.8.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.4 recalls the canonical standard-form decision-variable owner.
namespace SeparableOptimizationProblem
universe u
variable {E : Type u} [AddCommGroup E] [Module ℝ E] {m : ℕ}
recall StandardFormDecisionVariableTheorem 5.4.8.4
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.4: the function F₃(x, t) = -\log x - \log (t - x \log x) is a 3-self-concordant barrier for the epigraph Q₃ = {(x, t) ∈ \mathbb{R}^2 \mid x ≥ 0,\ t ≥ x \log x} of x \log x, with the convention 0 \log 0 = 0.
local notation "F₃" => separableLogBarrierF3
theorem separableLogBarrierF3_is_three_selfConcordantBarrier :
IsSelfConcordantBarrierOnWith (interior Q₃) (3 : NNReal) F₃
Definition 5.4.8.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.5 recalls the chapter epigraph owner specialized to x ↦ -log x on (0, ∞).
#check constrainedEpigraph (Set.Ioi (0 : ℝ)) (fun x : ℝ ↦ (-Real.log x : WithTop ℝ))Theorem 5.4.8.5
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.5: for p ≥ 1, the function F₄(x, t) = -\log t - \log (t^(2 / p) - x^2) is a 4-self-concordant barrier for the canonical epigraph of x ↦ |x|^p, viewed on the canonical L² product owner Z = WithLp 2 (ℝ × ℝ) through z ↦ z.ofLp.
open scoped Gradient HessianLocalNorm PowerCone SecondOrderCone
variable {p : ℝ}
local notation "Z" => WithLp 2 (ℝ × ℝ)
local notation "ofZ" => (WithLp.ofLp : Z → ℝ × ℝ)
local notation "Q₄[" p "]" =>
constrainedEpigraph (Set.univ : Set ℝ)
(fun y : ℝ ↦ ((|y| ^ p : ℝ) : WithTop ℝ))
local notation "F₄" => separableLogBarrierF4 p
theorem separableLogBarrierF4_is_four_selfConcordantBarrier
(hp : 1 ≤ p) :
IsSelfConcordantBarrierOnWith
(ofZ ⁻¹' interior Q₄[p])
(4 : NNReal)
(F₄ ∘ ofZ)
Definition 5.4.8.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.6 recalls the canonical epigraph logarithmic barrier specialized to x ↦ -log x.
#check (epigraphLogBarrier (fun x : ℝ ↦ -Real.log x) : ℝ × ℝ → ℝ)
Theorem 5.4.8.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.6: for 0 < p ≤ 1, the function F₅(x, t) = -\log t - \log (t^p - x) is a 3-self-concordant barrier for the hypograph-type constraint set Q₅ = {(x, t) ∈ \mathbb{R}^2 \mid t ≥ 0,\ t^p ≥ x}.
open scoped PowerConePlus
variable (p : ℝ)
local notation "Q₅" => Q₅ p
local notation "F₅" => separableLogBarrierF5 p
-- Proof sketch: use that for `p > 0` the map `t ↦ t^p` is continuous on `(0, ∞)`, so the
-- interior of the canonical closed constrained sublevel set `Q₅` is obtained by replacing the
-- boundary conditions `t ≥ 0` and `x ≤ t^p` with the strict inequalities `t > 0` and `x < t^p`.
theorem separableLogBarrierF5_is_three_selfConcordantBarrier
(hp0 : 0 < p) (hp1 : p ≤ 1) :
IsSelfConcordantBarrierOnWith (interior Q₅) (3 : NNReal) F₅
Definition 5.4.8.7
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.7 recalls the existing chapter owner for the textbook exponential epigraph Q₂.
recall exponentialEpigraphQ2 : Set (ℝ × ℝ)
Theorem 5.4.8.7
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.7: for p > 0, the function F₆(x, t) = -\log x - \log t - \log (x^α t^(1 - α) - 1) with α = p / (p + 1) is a 3-self-concordant barrier for Q₆ = {(x, t) ∈ \mathbb{R}^2 \mid x > 0,\ t ≥ x^{-p}}.
open scoped PowerConePlus
variable (p : ℝ)
local notation "Q₆" => Q₆ p
local notation "F₆" => separableLogBarrierF6 p
theorem separableLogBarrierF6_is_three_selfConcordantBarrier
(hp : 0 < p) :
IsSelfConcordantBarrierOnWith (interior Q₆) (3 : NNReal) F₆
Definition 5.4.8.8
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.8 recalls the existing chapter owner for the textbook barrier F₂.
recall exponentialEpigraphBarrierF2 : ℝ × ℝ → ℝ
Theorem 5.4.8.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.8 lies in the Chapter 5 finite exponential-sum convexity domain. Sampled owner declarations: * sumOfExponentials from Definition_5_4_8_19, the source-facing owner for the exponential sum y ↦ ∑ⱼ αⱼ exp ⟪aⱼ, y⟫; * sumOfExponentials_apply, the defining evaluation bridge for that owner; * sumOfExponentials_convex, the upstream chapter owner theorem proving whole-space convexity under the mathematically weaker nonnegativity hypothesis on the coefficients. Best owner abstraction: * source-facing: the textbook positive-coefficient convexity statement for sumOfExponentials; * core/canonical: sumOfExponentials_convex; * bridge/view: specializing the owner theorem from nonnegative coefficients to strictly positive coefficients. Primitive data: * the coefficient family α₁, …, αᵣ; * the vectors a₁, …, aᵣ. Derived API: * the whole-space convexity conclusion for sumOfExponentials. Source/core/bridge triage: * source-facing: the positive-coefficient textbook theorem; * core/canonical: sumOfExponentials_convex; * bridge/view: the implication 0 < αⱼ → 0 ≤ αⱼ. This numbered item adds no new mathematical owner beyond the canonical theorem already proved in Definition_5_4_8_19. Since the project already exposes the stronger reusable owner statement with nonnegative coefficients, this file keeps only the direct recall surface; the textbook strict-positivity specialization should be obtained from sumOfExponentials_convex at the call site when needed.
recall sumOfExponentials_convexDefinition 5.4.8.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.9: the set Q₃, namely the epigraph of x ↦ x log x on [0, ∞).
abbrev Q₃ : Set (ℝ × ℝ) :=
constrainedEpigraph (Set.Ici (0 : ℝ))
(fun x : ℝ ↦ ((x * Real.log x : ℝ) : WithTop ℝ))Theorem 5.4.8.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.8.9: the box-constrained ℓ_p approximation problem min_{α ≤ x ≤ β} \sum_{i=1}^m |\langle a_i, x \rangle - b^{(i)}|^p and its epigraph reformulation with variables τ⁽⁰⁾, τ⁽¹⁾, …, τ⁽ᵐ⁾ have the same canonical Chapter 1 optimal value.
open scoped BigOperators RealInnerProductSpace
variable {n m : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
open LpApproximationEpigraphPoint
theorem lpApproximation_optimalValue_eq_epigraphOptimalValue
(p : ℝ) (a : Fin m → E) (b : Fin m → ℝ) (α β : E) :
(lpApproximationProblem p a b α β).optimalValue =
(lpApproximationEpigraphProblem p a b α β).optimalValueDefinition 5.4.8.10
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.10: the function F₃ on ℝ² given by F₃(x, t) = - log x - log (t - x log x).
def separableLogBarrierF3 : ℝ × ℝ → ℝ := fun p ↦ entropyEpigraphConeBarrier ((p.1, 1), p.2) -- Proof sketch: `separableLogBarrierF3` is the affine slice `x₂ = 1` of -- `entropyEpigraphConeBarrier`. Evaluate the owner and simplify `log (x / 1)` and `log 1`.
Definition 5.4.8.11
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.11 recalls the chapter epigraph owner specialized to x ↦ |x| ^ p on ℝ.
#check
(fun p : ℝ ↦ constrainedEpigraph (Set.univ : Set ℝ)
(fun x : ℝ ↦ ((|x| ^ p : ℝ) : WithTop ℝ)) : ℝ → Set (ℝ × ℝ))
-- Proof sketch: specialize `mem_constrainedEpigraph_iff` to `Q = Set.univ`; the feasible-set
-- condition is automatic, leaving exactly the inequality `|x| ^ p ≤ t`.Definition 5.4.8.12
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.12: the function F₄ on ℝ² given by F₄(x, t) = - log t - log (t^(2 / p) - x^2).
def separableLogBarrierF4 (p : ℝ) : ℝ × ℝ → ℝ :=
sublevelLogBarrier (fun q : ℝ × ℝ ↦ -q.2) 0 +
sublevelLogBarrier (fun q : ℝ × ℝ ↦ q.1 ^ (2 : ℕ) - Real.rpow q.2 (2 / p)) 0
-- Proof sketch: evaluate the two canonical `sublevelLogBarrier` factors at `(x, t)`. The first
-- becomes `-log t`, and the second becomes `-log (t^(2 / p) - x^2)`.Definition 5.4.8.13
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.13: the set Q₅, namely the constrained hypograph-type region {(x, t) ∈ ℝ² | t ≥ 0, x ≤ t^p}.
abbrev Q₅ (p : ℝ) : Set (ℝ × ℝ) :=
constrainedSublevelSet (Set.univ ×ˢ Set.Ici (0 : ℝ))
(fun xt : ℝ × ℝ ↦ ((xt.1 - xt.2.rpow p : ℝ) : WithTop ℝ))
0
-- Proof sketch: unfold `Q₅` and then `mem_constrainedSublevelSet_iff`; membership in the
-- feasible strip gives `0 ≤ t`, and the scalar inequality `x - t^p ≤ 0` is equivalent to
-- `t^p ≥ x`.Definition 5.4.8.14
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.14: the function F₅ on ℝ² given by F₅(x, t) = - log t - log (t^p - x).
def separableLogBarrierF5 (p : ℝ) : ℝ × ℝ → ℝ := fun q ↦ power_cone_plus_barrier p ((q.2, 1), q.1) -- Proof sketch: `separableLogBarrierF5` is the affine slice `((t, 1), x)` of -- `power_cone_plus_barrier p`. Evaluate the owner and simplify `log 1 = 0` and -- `1 ^ (1 - p) = 1`.
Definition 5.4.8.15
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.15: the set Q₆, namely the epigraph of the function x ↦ x^{-p} on (0, ∞).
abbrev Q₆ (p : ℝ) : Set (ℝ × ℝ) :=
constrainedEpigraph (Set.Ioi (0 : ℝ))
(fun x : ℝ ↦ ((1 / Real.rpow x p : ℝ) : WithTop ℝ))
-- Proof sketch: unfold `Q₆` and then use the Chapter 3 characterization
-- `mem_constrainedEpigraph_iff`; the feasible-set condition gives `x > 0`, and the remaining
-- inequality is exactly the epigraph inequality `t ≥ x^{-p}`.Definition 5.4.8.16
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.16: the function F₆ on ℝ² given by F₆(x, t) = - log x - log t - log (x^α t^(1 - α) - 1), where α = p / (p + 1).
def separableLogBarrierF6 (p : ℝ) : ℝ × ℝ → ℝ := fun q ↦ power_cone_plus_barrier (p / (p + 1)) (q, 1) -- Proof sketch: `separableLogBarrierF6` is the affine slice `z = 1` of -- `power_cone_plus_barrier (p / (p + 1))`. Evaluating the owner and reordering the three -- logarithmic summands gives the textbook planar formula.
Definition 5.4.8.17
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.17: a geometric optimization problem on the strict positive orthant \mathbb{R}^n_{++} is specified by block sizes m₀, …, mₘ, positive coefficients αᵢⱼ, and exponent vectors σᵢⱼ ∈ ℝⁿ, yielding the posynomials qᵢ(x) = \sum_{j=1}^{mᵢ} αᵢⱼ \prod_{k=1}^n (x^(k))^(σᵢⱼ^(k)); the problem is to minimize q₀(x) subject to the constraints qᵢ(x) ≤ 1 for i = 1, …, m.
open scoped BigOperators
open scoped EuclideanOrthant
open EuclideanSpace (positiveOrthant)
structure GeometricOptimizationProblem (n m : ℕ) where
/-- The number `mᵢ` of monomial terms in the posynomial `qᵢ`. -/
blockSize : Fin (m + 1) → ℕ
/-- The strictly positive coefficients `αᵢⱼ` multiplying the monomial terms. -/
coefficient (i : Fin (m + 1)) (j : Fin (blockSize i)) : Set.Ioi (0 : ℝ)
/-- The exponent vectors `σᵢⱼ ∈ ℝⁿ` of the monomial terms. -/
exponent (i : Fin (m + 1)) (j : Fin (blockSize i)) (k : Fin n) : ℝDefinition 5.4.8.18
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.18: the logarithmic substitution sends a strictly positive vector x ∈ \mathbb{R}^n_{++} to the vector y ∈ \mathbb{R}^n with coordinates y^(k) = log x^(k).
open scoped EuclideanOrthant
variable {n : ℕ}
local notation "Eₙ" => EuclideanSpace ℝ (Fin n)
def logarithmicSubstitution (x : ℝ₊₊^n) : Eₙ :=
WithLp.toLp 2 fun k ↦ Real.log ((x : Eₙ) k)Definition 5.4.8.19
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.19: for coefficients α₁, …, αᵣ read in ℝ and vectors a₁, …, aᵣ in a real inner product space E, the exponential sum with coefficients α and vectors a is the function g(y) = \sum_{j=1}^r αⱼ \exp(\langle aⱼ, y \rangle). Specializing to E = EuclideanSpace ℝ (Fin n) recovers the textbook ℝⁿ formula.
open scoped BigOperators RealInnerProductSpace
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] {r : ℕ}
def sumOfExponentials (α : Fin r → ℝ) (a : Fin r → E) : E → ℝ :=
fun y ↦ ∑ j : Fin r, α j * Real.exp ⟪a j, y⟫Definition 5.4.8.20
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.20: for vectors a₁, …, aₘ in a real inner product space E, targets b⁽¹⁾, …, b⁽ᵐ⁾ ∈ ℝ, and exponent p, the ℓ_p approximation objective is the residual sum x ↦ \sum_{i=1}^m |\langle a_i, x \rangle - b^{(i)}|^p. Specializing to E = EuclideanSpace ℝ (Fin n) recovers the textbook ℝⁿ presentation.
open scoped BigOperators RealInnerProductSpace
universe u
variable {E : Type u} [NormedAddCommGroup E] [InnerProductSpace ℝ E] {m : ℕ}
def lpApproximationObjective (p : ℝ) (a : Fin m → E) (b : Fin m → ℝ) : E → ℝ :=
fun x ↦ ∑ i : Fin m, |⟪a i, x⟫ - b i| ^ pDefinition 5.4.8.21
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.8.21 lies in the Chapter 5 box-constrained ℓ_p approximation epigraph domain. Sampled owner declarations: - LpApproximationEpigraphPoint in Theorem_5_4_8_9, the chapter owner for the lifted epigraph decision variable (x, τ⁽⁰⁾, τ⁽¹⁾, ..., τ⁽ᵐ⁾); - LpApproximationEpigraphPoint.objectiveSlack in Theorem_5_4_8_9, the canonical projection to the scalar slack τ⁽⁰⁾; - lpApproximationEpigraphProblem in Theorem_5_4_8_9, the chapter owner for the epigraph reformulation; - mem_lpApproximationEpigraphProblem_feasibleSet_iff in Theorem_5_4_8_9, the atomic membership view of that owner feasible set. Best owner abstraction: - source-facing: the textbook epigraph reformulation of the box-constrained ℓ_p approximation problem; - core/canonical: LpApproximationEpigraphPoint and lpApproximationEpigraphProblem; - bridge/view: the evaluation and membership lemmas expanding those owners to the displayed inequalities. Primitive data: - the lifted decision-variable type LpApproximationEpigraphPoint n m. Derived API: - the objective projection LpApproximationEpigraphPoint.objectiveSlack; - the epigraph owner lpApproximationEpigraphProblem p a b α β; - the companion lemmas lpApproximationEpigraphProblem_apply and mem_lpApproximationEpigraphProblem_feasibleSet_iff. Source/core/bridge triage: - source-facing: Definition 5.4.8.21's epigraph standard-form variables and inequalities; - core/canonical: the existing chapter owner declarations from Theorem_5_4_8_9; - bridge/view: the companion expansion lemmas for objective evaluation and feasible-set membership. This item therefore deletes the duplicate local structure LpApproximationEpigraphDecisionVariable, the duplicate feasible-set definition lpApproximationEpigraphStandardForm, and the duplicate objective alias. The existing chapter owner API should be used directly.
recall LpApproximationEpigraphPointDefinition 5.4.9.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.1: an ℓ_p approximation problem with box constraints on ℝⁿ is given by an exponent p ≥ 1, vectors a₁, ..., aₘ ∈ ℝⁿ, scalars b⁽¹⁾, ..., b⁽ᵐ⁾ ∈ ℝ, and box bounds α, β ∈ ℝⁿ. The associated optimization problem minimizes ∑ i, |⟪aᵢ, x⟫ - b⁽ⁱ⁾|^p over the box α ≤ x ≤ β.
open scoped BigOperators RealInnerProductSpace
variable {n m : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
structure LpApproximationBoxProblem (n m : ℕ) where
/-- The exponent `p` in the `ℓ_p` approximation objective, constrained by `p ≥ 1`. -/
p : Set.Ici (1 : ℝ)
/-- The vectors `a₁, ..., aₘ ∈ ℝⁿ`. -/
a : Fin m → EuclideanSpace ℝ (Fin n)
/-- The scalar targets `b⁽¹⁾, ..., b⁽ᵐ⁾ ∈ ℝ`. -/
b : Fin m → ℝ
/-- The lower box bound `α ∈ ℝⁿ`. -/
α : EuclideanSpace ℝ (Fin n)
/-- The upper box bound `β ∈ ℝⁿ`. -/
β : EuclideanSpace ℝ (Fin n)Theorem 5.4.9.1
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.9.1's source-facing total arithmetic-complexity owner for the ellipsoid method. It records the textbook bound saying that the total arithmetic work induced by the primitive iteration-count, oracle-cost, and extra-cost data is O(n^3 (m + n) log (1 / ε)) with a constant independent of the target accuracy ε ∈ (0, 1).
def HasEllipsoidMethodTotalArithmeticComplexityBound
(iterationCount : ℝ → ℕ → ℕ)
(oracleCostPerIteration : ℕ → ℕ → ℕ)
(extraCostPerIteration : ℕ → ℕ) : PropDefinition 5.4.9.2
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.2: the ellipsoid method satisfies the textbook complexity model when there exist positive constants, independent of the target accuracy ε ∈ (0, 1), such that the accuracy-indexed iteration count N_it, oracle arithmetic cost per iteration, and additional per-iteration arithmetic overhead obey the bounds N_it = O(n^2 log (1 / ε)), C_oracle = O(m n), and C_iter = O(n^2) in positive dimensions.
def HasEllipsoidMethodComplexityBounds
(iterationCount : ℝ → ℕ → ℕ)
(oracleCostPerIteration : ℕ → ℕ → ℕ)
(extraCostPerIteration : ℕ → ℕ) : Prop :=
∃ C_it C_oracle C_iter : ℝ,
0 < C_it ∧
0 < C_oracle ∧
0 < C_iter ∧
(∀ {ε : ℝ} {n : ℕ}, ε ∈ Set.Ioo (0 : ℝ) 1 → 0 < n →
(iterationCount ε n : ℝ) ≤ C_it * (n : ℝ) ^ 2 * Real.log (1 / ε)) ∧
(∀ {m n : ℕ}, 0 < m → 0 < n →
(oracleCostPerIteration m n : ℝ) ≤ C_oracle * (m : ℝ) * (n : ℝ)) ∧
(∀ {n : ℕ}, 0 < n →
(extraCostPerIteration n : ℝ) ≤ C_iter * (n : ℝ) ^ 2)Theorem 5.4.9.2
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.9.2: in the dense model, if each per-index evaluation of the source-facing block owners g₁, g₂, h₁₁, h₁₂, and h₂₂ has uniformly bounded arithmetic cost, then the actual second-order oracle for problem.barrierModelBarrier has arithmetic complexity O(m n^2) for every evaluator/work model whose outputs agree with problem.barrierSecondOrderOracle and whose arithmetic work is bounded by the dense assembly template.
open LpApproximationBoxProblem
open LpApproximationEpigraphPoint
open scoped BigOperators Gradient RealInnerProductSpace LpBarrierNewtonSystem
variable {n m : ℕ}
theorem barrierSecondOrderOracleDenseArithmeticComplexity_bound
(oracle : LpBarrierSecondOrderOracleEvaluator)
(oracleWork : LpBarrierSecondOrderOracleWork)
(g1Work g2Work h11Work h12Work h22Work : ℕ → ℕ → ℕ)
(hassembly :
LpBarrierSecondOrderOracleDenseAssemblyBound oracle oracleWork
g1Work g2Work h11Work h12Work h22Work)
(hblocks :
HasLpBarrierPerIndexSecondOrderBlockArithmeticBound
g1Work g2Work h11Work h12Work h22Work) :
HasLpBarrierSecondOrderOracleDenseArithmeticComplexityBound
oracle oracleWork
Definition 5.4.9.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.3 lies in the Chapter 5 logarithmic barrier / strict-epigraph domain. Sampled owner declarations: * separableLogBarrierF4 from Definition_5_4_8_12, the existing Chapter 5 owner for the textbook scalar barrier f(y, t) = -log t - log (t^(2 / p) - y^2); * separableLogBarrierF4_apply from Definition_5_4_8_12, the coordinate evaluation bridge for that owner; * strictConstrainedEpigraph from Theorem_5_3_5, the chapter owner for strict epigraph domains; * mem_strictConstrainedEpigraph_iff from Theorem_5_3_5, the canonical membership expansion for that owner. Best owner abstraction: * source-facing: the textbook scalar sub-function f(y, t); * core/canonical: the existing owner separableLogBarrierF4 p : ℝ × ℝ → ℝ; * bridge/view: the strict-epigraph description of the points where the two logarithmic arguments are positive. Primitive data: * the scalar exponent p. Derived API: * the recalled owner separableLogBarrierF4 p; * its coordinate formula separableLogBarrierF4_apply; * the strict-epigraph bridge describing the natural finiteness domain. This refinement removes the duplicate subtype-valued wrapper and keeps Definition 5.4.9.3 as a recall of the existing Chapter 5 owner separableLogBarrierF4, with the domain side expressed through the chapter strict-epigraph owner instead of a bespoke set definition.
recall separableLogBarrierF4
Theorem 5.4.9.3
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Theorem 5.4.9.3: Cost of solving the Newton system and total complexity.
open Matrix
open scoped BigOperators Gradient RealInnerProductSpace LpBarrierNewtonSystem
open scoped SecondOrderDerivativeBlocks
variable {n m : ℕ}
open LpApproximationBoxProblem
open LpApproximationEpigraphPoint
local notation "Q₄[" p "]" =>
constrainedEpigraph (Set.univ : Set ℝ)
(fun x : ℝ ↦ ((|x| ^ p : ℝ) : WithTop ℝ))
local notation "Z₂" => WithLp 2 (ℝ × ℝ)
local notation "ofZ₂" => (WithLp.ofLp : Z₂ → ℝ × ℝ)
theorem lpBarrierShortStepTotalArithmeticComplexity_bound :
∃ solver :
∀ {n m : ℕ} (problem : LpApproximationBoxProblem n m)
(_ : problem.StrictBarrierModelPoint)
(_ : ℝ),
LpApproximationEpigraphPoint n m,
∃ arithmeticWork :
∀ {n m : ℕ} (problem : LpApproximationBoxProblem n m)
(_ : problem.StrictBarrierModelPoint)
(_ : ℝ),
ℕ,
HasLpBarrierNewtonSystemArithmeticComplexityBound solver arithmeticWork ∧
∀ ε : ℝ,
∀ iterationCount : ℕ → ℕ → ℕ,
HasLpBarrierShortStepIterationBound ε iterationCount →
∃ totalArithmeticWork : ℕ → ℕ → ℕ,
HasLpBarrierShortStepTotalArithmeticComplexityBound ε
totalArithmeticWork ∧
∃ C_step : ℝ,
HasLpBarrierNewtonSystemArithmeticBudget solver arithmeticWork C_step ∧
totalArithmeticWork =
lpBarrierShortStepTotalArithmeticWork iterationCount C_step
Definition 5.4.9.4
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.4: the barrier-model reformulation of a box-constrained ℓ_p approximation problem is the Chapter 1 Lagrangian problem on the existing lifted decision points LpApproximationEpigraphPoint n m, where objectiveSlack plays the role of the scalar variable ξ and residualSlack plays the role of the residual bounds τ. Its inequality constraints are the residual epigraph constraints, the coupling inequality ∑ i, τᵢ ≤ ξ, and the coordinatewise box constraints.
open scoped BigOperators RealInnerProductSpace
variable {n m : ℕ}
namespace LpApproximationBoxProblem
open LpApproximationEpigraphPoint
def barrierModelProblem
(problem : LpApproximationBoxProblem n m) :
LagrangianProblem (LpApproximationEpigraphPoint n m) (m + (1 + (n + n)))Definition 5.4.9.5
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.5: the logarithmic barrier attached to the box-constrained ℓ_p approximation barrier model, kept on its strict domain and obtained by reusing the Chapter 1 owner logarithmicBarrier on the split continuous inequality family whose scalar blocks are the Chapter 5 owner separableLogBarrierF4 problem.p.
open scoped BigOperators RealInnerProductSpace
variable {n m : ℕ}
local notation "E" => EuclideanSpace ℝ (Fin n)
namespace LpApproximationBoxProblem
open LpApproximationEpigraphPoint
def barrierModelBarrier
(problem : LpApproximationBoxProblem n m) :
C(problem.StrictBarrierModelPoint, ℝ) :=
logarithmicBarrier problem.barrierModelStrictConstraintsDefinition 5.4.9.6
— source only (outside the published documentation root) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.6: a short-step iteration model with count N_it satisfies the textbook O(√(m + n) log ((m + n) / ε)) bound at the fixed small accuracy ε ∈ (0, 1) if it is controlled by a positive constant independent of m and n on positive variable dimension n.
variable (ε : ℝ) (N_it : ℕ → ℕ → ℕ)
def HasLpBarrierShortStepIterationBound : Prop :=
ε ∈ Set.Ioo (0 : ℝ) 1 ∧
∃ C_it : ℝ,
0 < C_it ∧
∀ m n : ℕ, 0 < n →
(N_it m n : ℝ) ≤
C_it * Real.sqrt ((m : ℝ) + (n : ℝ)) *
Real.log (((m + n : ℕ) : ℝ) / ε)
Definition 5.4.9.7
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.7: the auxiliary derivatives of f(y,t) are the pair whose first component is the y-gradient and whose second component is the t-derivative.
open scoped Gradient
universe u
variable {E : Type u}
variable [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
abbrev auxiliaryDerivatives (f : E × ℝ → ℝ) (y : E) (t : ℝ) : E × ℝ :=
(auxiliaryGradient f y t, auxiliaryTimeDerivative f y t)Definition 5.4.9.8
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.8: assuming f is twice differentiable, secondOrderDerivativeBlock11 f y t is the yy-block h₁₁(y, t) = ∇²_{yy} f(y, t) of the second derivative of f.
open scoped Gradient
universe u
variable {E : Type u}
variable [NormedAddCommGroup E] [InnerProductSpace ℝ E] [CompleteSpace E]
abbrev secondOrderDerivativeBlock11 (f : E × ℝ → ℝ) (y : E) (t : ℝ) : E →L[ℝ] E :=
hessian (fun y' ↦ f (y', t)) yDefinition 5.4.9.9
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
Definition 5.4.9.9: the residual shorthand sᵢ = ⟪aᵢ, x⟫ - b⁽ⁱ⁾ at the current strict barrier-model point, used in the Newton-system diagonal matrices.
open scoped BigOperators RealInnerProductSpace
open scoped SecondOrderDerivativeBlocks
variable {n m : ℕ}
namespace LpApproximationBoxProblem
open LpApproximationEpigraphPoint
namespace StrictBarrierModelPoint
def newtonSystemResidual
{problem : LpApproximationBoxProblem n m}
(decision : StrictBarrierModelPoint problem) :
Fin m → ℝ :=
fun i ↦ ⟪problem.a i, decision.1.point⟫ - problem.b i
-- Proof sketch: unfold `newtonSystemResidual`; the displayed identity is its defining formula.Supporting modules
These source modules support the indexed items and are listed separately so the chapter inventory remains complete.
-
FenchelPrimalExtension.lean(Documentation) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
L5472_profile_L6ze.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
RealProdL2.lean(Documentation) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_2_2/Common.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_2_2/Support.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_4_2_2/CoreTransport.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_4_2_2/PointwiseCore.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_4_7_3/Prereqs.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_4_7_3/Specialization.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_4_7_4/Specialization.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_4_7_6/Prereqs.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
Theorem_5_4_7_6/Specialization.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.
-
tmpcheckdyvS.lean(Documentation unavailable) (Lean source)
Lean contributors (project-level attribution): Chenyi Li, Siyuan Shao, Yijie Wang, Feiming Wang, Weiran Shi, Yuhao Jiang, Zebo Liu, Wentao Long.