If you are in search of software for your problem you will find as far as possible public domain or free-for-research software. In some cases source code may not be available, some authors only supply executables for special systems. In any case, observe the expressed or implied LICENSE conditions ! In most cases these accompany the code. Programs are in f77 unless indicated otherwise (f90, C/C++, Pascal, Matlab, Java). Links usually download files directly or put you in directory if software is not a single file.
If you really need the best possible solution to your problem and have no information about it, e.g. a currently working solution which needs improvement only, then you are faced with a problem in
Global optimization is one of centerpoints of current research. Most available codes for
Least squares (other norms -> Approximation)
identify local optimal solutions only. (Of course for linear , convex quadratic and convex semidefinite problems a local solution is also a global one). The necessary optimality conditions can also be regarded as a special problem of zerofinding. The codes in the following area however adress much more general problems and hence are also weaker in their solution power. (In most cases, you must supply a good initial guess) .
The necessary optimality conditions for a constrained problem can also be formulated as a so called
Again this class of problems is much more general. This is also an area of much active research. Often a user has not only one objective to optimize but must compromise between different ones. Then, methods of
must be applied. Discrete optimization problems require special treatment, as a rule in a problem specific way. Because of their combinatorial nature computing effort might be extreme if one aims at exact solutions. But often good suboptimal solutions can be found by approximation methods.
Often one wants to replace some complicated function expression by one which is cheap to evaluate or one wants to describe a huge set of discrete data by a simple analytical expression. Then one is faced with a problem of
Any approximation problem is an optimization problem, but there exist specialized and more efficient algorithms.
