Benchmarks for Optimization Software

By Hans Mittelmann (mittelmann at asu.edu)


END OF A BENCHMARKING ERA
 For many years our benchmarking effort had included the solvers CPLEX, Gurobi, and XPRESS. Through an action by Gurobi at the 2018 INFORMS Annual Meeting this has come to an end. IBM and FICO demanded that results for their solvers be removed. See here for more details. The resulting void was filled by other developers.
In August 2024 Gurobi decided to withdraw from the benchmarks as well and their results have been removed. On 12/24/2024 MindOpt followed suit.

Note that on top of the benchmarks a link to logfiles is given!

Note also that we do not use performance profiles. See this paper and that one

We use instead the shifted geometric mean


COMBINATORIAL OPTIMIZATION

LKHcats and Hexaly on ATSPs (10-16-2026)

Concorde, Hexaly and LKH on TSPLIB (7-28-2026)

CVRP problems from size 100 to 1 million (8-4-2026)

Concorde-TSP with different LP solvers (10-1-2026)


LINEAR PROGRAMMING

The Simplex and Barrier benchmarks are replaced by benchmarks that show how well solvers find a primal-dual feasible point (as barrier methods in general do) or an optimal basic solution (as simplex methods in general do). Codes do not have to use a textbook version of either method.

LPfeas Benchmark (find a PD feasible point; also for GPUs) (9-16-2026)

LPopt Benchmark (find optimal basic solution) (10-7-2026)

Large Network-LP Benchmark (commercial vs free) (9-9-2026)


MIXED INTEGER LINEAR PROGRAMMING

MIPFEAS Benchmark - MIPLIB2017 (9-16-2026)

MILP Benchmark - MIPLIB2017 (10-10-2026)

MILP cases that are slightly pathological (10-7-2026)

Infeasibility Detection for MILP Problems (7-26-2026)


SEMIDEFINITE/SQL PROGRAMMING

SQL problems from the 7th DIMACS Challenge (8-8-2002)

Several SDP codes on sparse and other SDP problems; also on GPUs (10-10-2026)

Infeasible SDP Benchmark (9-21-2025)

Large SOCP Benchmark (9-9-2026)

MISOCP Benchmark (9-10-2026)


NONLINEAR PROGRAMMING

AMPL-NLP Benchmark (9-9-2026)


MIXED INTEGER QPs

Non-commercial convex QP Benchmark (10-6-2026)

Non-Convex QUBO-QPLIB Benchmark (9-21-2026)


MIXED INTEGER QCPs

Binary Non-Convex QPLIB Benchmark (5-9-2025)

Discrete Non-Convex QPLIB Benchmark (non-binary) (5-12-2026)

Continuous Non-Convex QPLIB Benchmark (5-17-2026)

Convex Continuous QPLIB Benchmark (also on GPUs) (8-10-2026)

Convex Discrete QPLIB Benchmark (9-7-2026)


MIXED INTEGER NONLINEAR PROGRAMMING

MINLP Benchmark (2-26-2026)


PROBLEMS WITH EQUILIBRIUM CONSTRAINTS

MPEC Benchmark (7-19-2026)