The picture shows the level set representation of Beale's testfunction
f(x,y)=(1.5-x(1-y))^2+(2.25-x(1-y^2))^2+(2.625-x(1-y^3))^2.
It has a unique global minimizer at (3,0.5) and a saddle point at (0,0).
There is a further saddle point at (0.100538,-2.644514). For x=0 or y=1 it
is constant with value 14.203125. In x<0 there exists an arc y(x)>0, where f
decreases monotonically as x-> -INF. Initial values near x=0 for large |y|
or x<0, y>0 will let descent methods fail.
The methods listed here all restrict themselves to finding one solution of grad
f(x)=0.
For trust region based methods one often can show that this solution automatically
satisfies
the second order necessary conditions.
The methods implicitly all assume that f is twice
continuously differentiable. They may fail or converge very slowly if this is not
the case. Read
M.
Powell's and
M. Wright's
papers on Direct Search methods as well as
M. Powell's paper on optimization without derivatives.
f is a sum of convex
differentiable functions each of which
has considerably less variables than n (partially separable problem)(also
for bound constraints), special quasi Newton method
A combined limited memory qN/cg
method (degrades for nonconvex f, nevertheless applicable; for ill-conditioned problem
an individual preconditioner must be provided)
A limited memory quasi Newton
method.
Directories contain software, drivers and
manuals. (dcsrch.f, the step-size-algorithm,
must be added; to be found e.g. in the
following code)
a limited memory quasi Newton
method
using a new matrix representation;
directory contains software, drivers and
a user's manual.
Not to be recommended for ill-conditioned
problems. Add your own preconditioner!
Nash's truncated Newton method
based on the Lanczos method,
without explicit eigenvalue computation. takes directions of negative curvature
into account.
truncated Newton method using the
Lanczos process with direct
computation
of a truncated spectral decomposition, uses a so called three directions method
from Heinrich,
uses directions of negative curvature. for large dimensional problems.
