Documentation

FirstOrderMethodsOptimization_Beck_2017.Chap10.Example_10_50

noncomputable def absolute_value_log_sum_exp_smoothing (μ : PosReal) :

The two-term shifted log-sum-exp smoothing x ↦ μ log (e^{x / μ} + e^{-x / μ}) - μ log 2 of the absolute value.

Instances For
    @[simp]
    theorem absolute_value_log_sum_exp_smoothing_apply (μ : PosReal) (x : ) :
    absolute_value_log_sum_exp_smoothing μ x = μ * Real.log (Real.exp (x / μ) + Real.exp (-x / μ)) - μ * Real.log 2

    Evaluating absolute_value_log_sum_exp_smoothing μ at x gives the shifted two-term log-sum-exp formula from Example 10.50.

    theorem coordinatewiseMax_pair_eq_abs (x : ) :
    coordinatewiseMax ![x, -x].ofLp = |x|

    Helper for Example 10.50: the coordinatewise maximum of the pair (x, -x) is |x|.

    theorem absolute_value_log_sum_exp_smoothing_eq_mul_log_cosh (μ : PosReal) (x : ) :
    absolute_value_log_sum_exp_smoothing μ x = μ * Real.log (Real.cosh (x / μ))

    Helper for Example 10.50: the scalar smoothing rewrites as μ log(cosh (x / μ)).

    theorem absolute_value_log_sum_exp_smoothing_hasDerivAt (μ : PosReal) (x : ) :
    HasDerivAt (absolute_value_log_sum_exp_smoothing μ) (Real.tanh (x / μ)) x

    Helper for Example 10.50: the derivative of the scalar smoothing is tanh (x / μ).

    theorem absolute_value_log_sum_exp_smoothing_deriv_eq (μ : PosReal) (x : ) :
    deriv (absolute_value_log_sum_exp_smoothing μ) x = Real.tanh (x / μ)

    Helper for Example 10.50: the derivative formula is the pointwise derivative of the smoothing.

    theorem absolute_value_log_sum_exp_smoothing_deriv_hasDerivAt (μ : PosReal) (x : ) :
    HasDerivAt (deriv (absolute_value_log_sum_exp_smoothing μ)) ((↑μ)⁻¹ / Real.cosh (x / μ) ^ 2) x

    Helper for Example 10.50: the derivative field itself has derivative μ⁻¹ / cosh(x / μ)^2.

    theorem absolute_value_log_sum_exp_smoothing_second_deriv_eq (μ : PosReal) (x : ) :
    deriv (deriv (absolute_value_log_sum_exp_smoothing μ)) x = (↑μ)⁻¹ / Real.cosh (x / μ) ^ 2

    Helper for Example 10.50: the second derivative is μ⁻¹ / cosh(x / μ)^2.

    theorem lipschitzWith_deriv_of_is_l_smooth_on_real {f : } {L : NNReal} (hs : is_l_smooth_on f Set.univ L) :
    LipschitzWith L (deriv f)

    Helper for Example 10.50: on the real line, an L-smooth function has an L-Lipschitz ordinary derivative.

    theorem is_l_smooth_on_of_lipschitzWith_deriv_real {f : } {L : NNReal} (hdiff : Differentiable f) (hlip : LipschitzWith L (deriv f)) :
    is_l_smooth_on f Set.univ L

    Helper for Example 10.50: if the derivative is L-Lipschitz on , then the original function is L-smooth in the Chapter 5 sense.

    theorem pair_linear_map_linear_map_add (x y : ) :
    ![x + y, -(x + y)] = ![x, -x] + ![y, -y]

    Helper for Example 10.50: the affine pair map x ↦ (x, -x) as a linear map into ℝ².

    theorem pair_linear_map_linear_map_smul (c x : ) :
    ![c * x, -(c * x)] = c ![x, -x]

    Helper for Example 10.50: the pair map is homogeneous.

    theorem pair_linear_map_norm_sq (x : ) :
    ![x, -x] ^ 2 = 2 * x ^ 2

    Helper for Example 10.50: the squared norm of the pair map is 2 x².

    theorem pair_linear_map_bound (x : ) :
    ![x, -x] 2 * x

    Helper for Example 10.50: the pair map is bounded by √2.

    def pair_linear_map_linear :
    →ₗ[] EuclideanSpace (Fin 2)

    Helper for Example 10.50: the linear part of the affine pair map.

    Instances For
      noncomputable def pair_linear_map :
      →L[] EuclideanSpace (Fin 2)

      Helper for Example 10.50: the continuous linear pair map x ↦ (x, -x).

      Instances For
        @[simp]
        theorem pair_linear_map_apply (x : ) :
        pair_linear_map x = ![x, -x]

        Helper for Example 10.50: evaluating the continuous pair map recovers (x, -x).

        theorem pair_linear_opNorm_sq :
        pair_linear_map‖₊ ^ 2 = 2

        Helper for Example 10.50: the pair map has operator norm squared equal to 2.

        Helper for Example 10.50: Theorem 10.46 applied to the pair map yields the looser nonnegative pair (2, log 2).

        Helper for Example 10.50: the direct one-dimensional second-derivative computation improves the loose affine-precomposition bound to the exact nonnegative pair (1, log 2).

        theorem absolute_value_log_sum_exp_beta_lower_at_scale {μ α β : PosReal} (happrox : IsSmoothApproximation abs (absolute_value_log_sum_exp_smoothing μ) α β μ) {t : } (ht : 0 t) :
        Real.log 2 - Real.log (1 + Real.exp (-2 * t)) β

        Helper for Example 10.50: every admissible error parameter dominates the exact scaled gap at the nonnegative point x = μ t.

        Example 10.50 (1): the shifted two-term log-sum-exp smoothing of |x| is a 1 / μ-smooth approximation with parameters (1, log 2), encoded by the canonical chapter owner log_cardinality_posreal at n = 2.

        theorem absolute_value_log_sum_exp_smoothing_parameter_lower_bounds (μ α β : PosReal) (happrox : IsSmoothApproximation abs (absolute_value_log_sum_exp_smoothing μ) α β μ) :
        1 α Real.log 2 β

        Example 10.50 (2): any positive parameter pair yielding this same chapter-level smooth approximation must satisfy α ≥ 1 and β ≥ log 2.

        The affine-precomposition bound from Theorem 10.46(b) yields the looser nonnegative parameter pair (2, log 2), encoded by log_cardinality_nonneg at n = 2, for the same smoothing family.