Decision Tree for Optimization Software
 

Navigation Menu

Benchmarks

Here we provide information on testruns comparing different solution methods on standardized sets of testproblems, running on the same or on different computer systems. Benchmarking is a difficult area for nonlinear problems, since different codes use different criteria for termination. Although much effort has been invested in making results comparable, in a critical situation you should try the candidates of your choice on your specific application. Many benchmark results can be found in the literature,

For pure selections of benchmark problems see under Testcases. Below we list our own benchmarks as well as a selection of those done by others:

BENCH

Hans Mittelmann's benchmarks

Performance World

Initiative by GAMS World

COAP

COAP's Software Forum

General papers related to benchmarking:

Performance Profiles

suggestions for performance metric and associated Matlab&Python scripts

Experimental Analysis of Algorithms

recommendations on this closely related topic

Benchmarks performed by others:

GAMSlinks benchmarks

Varioua solvers run through GAMS, on GAMS libraries, results displayed using GAMS tools

Benchmarking Derivative-Free Optimization Algorithms

Comparison of NMSMAX, APPSPACK, NEWUOA on smooth, noisy, and piecewise smooth functions

Performance Assessment of Multi-Objective Optimization Algorithms

from CEC-2007 competition; also Matlab/C codes

LaGO vs BARON

Comparison for QQP, MIQQP, MINLP using GAMS Performance Tools

Discrete gradient method: a derivative free method for nonsmooth optimization

Comparison with DNLP and CONDOR

Nonlinear Regression

Comparison of codes by Borchers&Mondragon

MPECs as NLPs

Extensive comparison of SQP and IPM solvers

MILP04

Detailed comparison of eight noncommercial MILP solvers

CONDOR

a new Parallel, Constrained extension of Powell's UOBYQA algorithm. Experimental results and comparison with the DFO algorithm

COCONUT Benchmark

various codes on global optimization problems

MCNF Benchmark

Simplex and cost-scaling codes for large minimum cost flow problems

Simplex benchmark

A Comparison of Simplex Method Algorithms (with Matlab scripts)

KNITRO

KNITRO, SNOPT, LOQO, and filterSQP with the CUTE(AMPL) problems.

LOQO

KNITRO, SNOPT, and LOQO, with the CUTE(AMPL) problems.

MP 106 paper and tables

IPOPT, KNITRO, and LOQO on the CUTEr testset

Bound-constraints

Comparison of PL2 and L-BFGS-B.

LANCELOT

LANCELOT and MINOS with the CUTE problems.

SNOPT

SNOPT and MINOS with the CUTE problems.

HOPDM

Benchmarks of LP/QP problems for HOPDM-2.30.

MCP

A Comparison of Algorithms for Large Scale Mixed Complementarity Problems.

TSP

TSPLIB Benchmarks for Concorde

Global

Comparison of public-domain software for black box global optimization

Global-stochastic

Comparison of stochastic global optimization methods

The SAT-Ex Site

Comparison of solvers for satisfiability problems

Vehicle Routing Software Survey

Comparison of commercial solvers

Results for the COPS project at Argonne (Large Scale Nonlinearly Constrained Optimization Problems) can be found in a paper at

COPS

by Bondarenko, Bortz, Dolan and More