Edit on GitHub

Algorithm Reference

Original DRSOM with 2 directions $g_k$ and $d_k$

Note

We do not currently support nonsmooth/composite optimization, this is a future plan.

DRSOM.DRSOMState — Type

Struct for DRSOM to keep the iterate state, including the following attributes e.g.,

  • x the current iterate, namely $x$;
  • fx the function value, namely $f(x)$;
  • ∇f the gradient, namely $\nabla f(x)$;
  • t the running time;
  • ϵ gradient norm $\|\nabla f(x)\|$
source
DRSOM.DRSOMIteration — Type

Iteration object for DRSOM, to initialize an iterator, you must specify the attributes, including

  • $x_0$ is initial point of the iterate
  • f is the smooth function to minimize
  • ϕ is the nonsmooth function
  • g is the gradient function
  • ga is gradient function via forward or backward diff
  • hvp is hvp function via forward or backward diff
  • H is hessian function

rest of the attributes have default options:

    t::Dates.DateTime = Dates.now()
    itermax::Int64 = 20
    fog = :forward
    sog = :forward
source
Note

This TRS solver only works for low-dimensional subproblems

DRSOM.SimpleTrustRegionSubproblem — Function

A simple procedure to solve TRS via a given regularizer This is only used in DRSOM

  • the radius free mode: compute the Lagrangian multiplier according to the adaptive $\gamma$ instead. See the paper for details.

  • the strict regularization mode: strictly solve a quadratic regularization problems given a regularizer $\lambda$

  • the strict radius mode: if you choose a radius, this function is a bisection procedure following the complexity of $O(\log\log(1/ε))$ by Ye

We use this function (radius-free mode) only in DRSOM, since the eigenvalues are easy to solve.

source