Documentation

IntroductoryLecturesOnConvexOptimization_Nesterov_2004.Chap03.Proposition_3_51

theorem CompleteLevelMethod.optimalValue_le_comparison_add_of_complexity_estimate {E : Type u} [NormedAddCommGroup E] {problem : SetConstrainedMinimizationProblem E} (method : CompleteLevelMethod problem) {Mf : } {xStar : E} {ε : } {N : } (h_level_estimate : method.history.optimalValue N - problem.objective xStar 2 * Mf * method.initialPoint - xStar / N) ( : 0 < ε) (hN : 4 * Mf ^ 2 * method.initialPoint - xStar ^ 2 / ε ^ 2 N) :
method.history.optimalValue N problem.objective xStar + ε

Proposition 3.51, owner form: if a complete level-method run satisfies the standard estimate f_N^* - f(x*) ≤ 2 M_f ‖x₀ - x*‖ / √N on the owner sampled-prefix value f_N^* = fstar(method.history, N), then every budget N above 4 M_f² ‖x₀ - x*‖² / ε² forces f_N^* ≤ f(x*) + ε.

theorem CompleteLevelMethod.historyOptimalValue_le_optimalValue_add_of_complexity_estimate {E : Type u} [NormedAddCommGroup E] {problem : SetConstrainedMinimizationProblem E} (method : CompleteLevelMethod problem) {Mf : } {xStar : E} {ε : } {N : } (hxStar : xStar constrainedArgmin problem.feasibleSet problem.objective) (h_level_estimate : method.history.optimalValue N - problem.objective xStar 2 * Mf * method.initialPoint - xStar / N) ( : 0 < ε) (hN : 4 * Mf ^ 2 * method.initialPoint - xStar ^ 2 / ε ^ 2 N) :
(method.history.optimalValue N) problem.optimalValue + ε

Proposition 3.51, owner form: if x* is a constrained minimizer and the complete level method satisfies the standard comparison-point complexity estimate, then the sampled-prefix owner value f_N^* is within ε of the Chapter 1 constrained optimal value.

theorem CompleteLevelMethod.exists_isApproximateMinimizer_of_complexity_estimate {E : Type u} [NormedAddCommGroup E] {problem : SetConstrainedMinimizationProblem E} (method : CompleteLevelMethod problem) {Mf : } {xStar : E} {ε : } {N : } (hxStar : xStar constrainedArgmin problem.feasibleSet problem.objective) (h_level_estimate : method.history.optimalValue N - problem.objective xStar 2 * Mf * method.initialPoint - xStar / N) ( : 0 < ε) (hN : 4 * Mf ^ 2 * method.initialPoint - xStar ^ 2 / ε ^ 2 N) :
kN, problem.IsApproximateMinimizer ε (method.iterate k)

Proposition 3.51: under the same complexity budget, one of the iterates x₀, …, x_N of the complete level-method run is an ε-approximate minimizer of the constrained problem in the canonical Chapter 1 sense.

theorem CompleteLevelMethod.exists_isApproximateSolution_of_complexity_estimate {E : Type u} [NormedAddCommGroup E] {problem : SetConstrainedMinimizationProblem E} (method : CompleteLevelMethod problem) {Mf : } {xStar : E} {ε : } {N : } (hxStar : xStar constrainedArgmin problem.feasibleSet problem.objective) (h_level_estimate : method.history.optimalValue N - problem.objective xStar 2 * Mf * method.initialPoint - xStar / N) ( : 0 < ε) (hN : 4 * Mf ^ 2 * method.initialPoint - xStar ^ 2 / ε ^ 2 N) :
kN, IsApproximateSolution problem.objective xStar ε (method.iterate k)

The owner ε-minimizer conclusion of Proposition 3.51 recovers the textbook comparison-point form once the comparison point x* is known to lie in the constrained argmin set.