Tools : Automatic Differentiation, Modeling Systems, Demos and Analysis Tools

We list here the above mentioned tools only. Others may be useful and/or even necessary, like preprocessors for systems of linear inequalities and equations, e.g. for eliminating fixed variables, redundant equalities and inequalities and similar tasks. There is no separate public domain software tool for these tasks. Some of the modeling systems mentioned below, however, include such options.


General note on Automatic Differentiation

While it is frequently necessary or desirable to replace gradients by numerical differentiation, a tool that became available more recently is that of automatic differentiation. NLP programs which have both options include DONLP2 and FSQP (see under programs for general nonlinear programming). A comparison of the various differentiation techniques is given in Griewank's paper.


The following are links to some general automatic differentiation software: These codes take a subprogram for computing a function as input and automatically generate a subprogram for computing the gradient and/or the Hessian as output.

ADiJaC Source-to-source Java; websubmission
Tapenade Source-to-source f77 (C, and most of f95); websubmission or local installation as JAR archive
TAMC Tangent and linear Adjoint Model Compiler; websubmission, f77 and most f90 extensions, forward and reverse modes
netlib/toms/746 method of K. Schittkowski et al, f77
ADIFOR by C. Bischof et al; f77
ADIC by C. Bischof et al; C
ADOL-C A. Griewank's packages for programs in C/C++, forward and reverse mode
ADMAT automatic differentiation in Matlab, forward and reverse mode
ADiMAT web-based automatic differentiation in Matlab, forward mode download
INTLAB automatic differentiation in Matlab, forward modes
CasADi symbolic framework for algorithmic differentiation and numerical optimization (C++, Python)
Revolve checkpointing for reverse mode
more AD tools

Modeling Languages

Modeling languages allow you to formulate a linear or nonlinear programming problem in mathematical terms in a way near your original problem and automatically generate an evaluation code appropriate for a corresponding solver. Different solvers need different interfaces, hence not all codes come with interfaces for such modeling languages. The following example shows a typical problem in mathematical terms, coded in C , coded in the modeling language AMPL, and finally coded in SIF ALKYLATION . SIF means standard input format and has been developed by Conn, Gould and Toint in connection with their LANCELOT project. It is an extension of the old linear programming MPS format. If one has coded a problem in SIF, then the SIF evaluation tools allow computation of exact first and second order derivatives. SIF is fully documented in the book A.R. Conn, N.I.M. Gould, Ph. L. Toint: "LANCELOT A Fortran Package for Large-Scale Nonlinear Optimization", Springer publisher, 1992, ISBN 3-540-55470-X.Here comes a list of some modeling languages/systems:

AMPL The AMPL modeling language is a commercial product, but demo and student version, also for solvers, are available here.
GAMS commercial modeling system with many solvers and related services
ADMB software package for the development of nonlinear statistical models, uses AUTODIF (C++, binaries, Windows, Linux, MacOS)

AIMMS commercial modeling platform, interactive graphical display, educational package
CasADi symbolic framework for algorithmic differentiation and numerical optimization (C++, Python)
Pyomo Flexible modeling of optimization problems in Python (formerly COIN-OR Coopr)
JuMP Modeling Language for linear, integer, and quadratic programming, part of Julia, benchmarks here
StructJuMP Parallel algebraic modeling framework for block structured optimization models in Julia
OptimJ extension of Java with support for optimization models, interfaces to CPLEX, MOSEK, lp_solve, LP&MPS input
CMPL modeling language and interface to several LP/MILP solvers
plam A free MILP modeling environment using lp_solve as MILP solver and SWI-Prolog
PuLP Python-based LP modeler, calls GLPK, Coin-CLP/CBC, CPLEX, GUROBI, XPRESS
SDPSL Semidefinite Programming Specification Language, SAGE interface (C++)
FLOPC++ C++ class library for algebraic modelling of linear optimization problems (LP/MIP); uses COIN OSI interface.
ZIMPL GPL-licensed LP/MILP modelling language, produces lp/MPS formats (C)
ASCEND An extensive GPL-licensed modeling system (source, binaries, docs).
IPAT-S Extensive LGPL-licensed scenario scripting language, GUI, Win executables, source; includes lp_solve.

Demos and Analysis Tools

Solver-o-matic Decision tree to choose solver for nonsmooth problems from list of opensource codes
PIPAL Penalty-Sequential Quadratic Programming with Infeasibility Detection for general NLP (Matlab, AMPL input)
Interactive Modules Java-based demos of basic optimization methods; also least squares and nonlinear equations
perPlex Tool to verify the feasibility, optimality, and integrality of a linear programming basis (C, GPL)
MProbe Analysis tool for Nonlinear Programs (AMPL, GAMS, MPS, Win student version)
MINOS(IIS) Software Tool for Analyzing Infeasible Linear Programs (Win student version)