Tutorials and Books

Tutorials and online books

Cornell University Computational Optimization Open Textbook very comprehensive source for optimization and applications
A First Course in Linear Optimization an intro to LP with software
Global Optimization Algorithms – Theory and Application – book on heuristic methods
A Field Guide to Genetic Programming book on general genetic programming
LP-book an introduction to linear programming and the simplex method, with online exercises
Deterministic Modeling Linear Optimization with Applications
Linear Optimization: Theory, methods, and extensions Introduction to linear programming, sensitivity analysis, simplex and interior point methods
NEOS-Optimization-Guide A comprehensive guide with intro, algorithms, resources
Convex Optimization book by Stephen Boyd and Lieven Vandenberghe. more material on class webpages.
Tutorial on Geometric Programming by Stephen Boyd et al. (PDF)
Various lectures by Hans Bruun Nielsen optimization, least squares etc
Topology Optimization for structural problems, MEMS
A course in combinatorial optimization lecture notes by A. Schrijver
Knapsack Problems free download of book on topic
MIP Theory and Practice survey article by Bixby et al
Latest Advances in MIP Solvers talk by Bixby et al
Integer Programming Software Systems survey comparing CPLEX, XPRESS-MP, LINDO
Six Talks on MINLP by Sven Leyffer
MIP/PDECON problems by Sven Leyffer
Survey on convex MINLP handout and presentation
Survey on nonconvex MINLP by Burer and Letchford
MINLP survey paper by P. Belotti et al
OR-Notes by J. E. Beasley, deterministic and stochastic topics
OR Class Notes by M. Trick, integer programming, dynamic programming etc
Tutorials on Stochastic Programming from the COSP website
Introduction to Stochastic Programming from the COSP site
Robust control, LMI Various courses by Didier Henrion

A list of books

Optimization is a very lively area, hence standard textbooks become outdated very fast. Therefore only a very restricted and certainly subjective list of books is presented here, mainly extracted from the FAQs initiated by Gregory and presently maintained by R. Fourer.

Books on or containing a considerable amount of LP theory or practice:

Bertsimas, D. and Tsitsiklis, J.: Introduction to Linear Optimization. Athena Scientific, 1997.
Graduate-level text on linear programming, network flows, and discrete optimization.

Maros, I., Computational Techniques of the Simplex Method
Recent comprehensive monograph

Dantzig, G. B.: Linear Programming and Extensions, Princeton University Press, 1963
The most widely cited early textbook in the field.

Dantzig, George B. and Thapa, Mukund N.: Linear Programming 1: Introduction, Springer Verlag, 1997

Juenger, M. et al.: Mixed Integer Programming Computation, Springer, 2009
50 Years of Integer Programming 1958-2008

Luenberger, D. G. and Ye, Yinyu: Introduction to Linear and Nonlinear Programming, Springer, 2008
Updated version of an old classic. Well suited for beginners

Nash, S. and Sofer, A.: Linear and Nonlinear Programming, McGraw-Hill, 1996

Roos, C., Terlaky T. and Vial, J. Ph.: Theory and Algorithms for Linear Optimization: An Interior Point Approach. John Wiley, Chichester, 1997

Schrijver, A.: Theory of Linear and Integer Programming, John Wiley, 1999
Advanced, very well written

Vanderbei, R. J.: Linear Programming: Foundations and Extensions. Springer, 1996
Balanced coverage of simplex and interior-point methods. Source code available on-line for all algorithms presented.

Williams, H.P., Model Building in Mathematical Programming, John Wiley 1999, 4th edition
Little on algorithms, but excellent for learning what makes a good model

Wright, St. J.: Primal-Dual Interior-Point Methods. SIAM Publications, 1997
Covers theoretical, practical and computational aspects of the most important and useful class of interior-point algorithms

Ye, Yinyu: Interior Point Algorithms: Theory and Analysis. John Wiley, 1997

Kellerer, H. et al: Knapsack Problems. Springer-Verlag, 2003

Now a table of books mainly devoted to nonlinear programming

Conn, A.R., Scheinberg, K., and Vicente, L.N.: Introduction to Derivative-Free Optimizati. SIAM:2009
First contemporary comprehensive treatment of optimization without derivatives

Avriel, M. and Golany, B.: Mathematical Programming for Industrial Engineers. Marcel Dekker:1996
Contains introductory chapters to several areas of mathematical optimization. well suited for beginners

Bonnans, J.F., Gilbert, J.C., Lemarechal, C., Sagastizabal, C.A.: Numerical Optimization (2nd edition). Springer: 2006
Both theory and details on implementations; nonsmooth optimization, interior-point methods etc.

Bertsekas, Dimitri P.: Nonlinear Programming, second edition. Athena Scientific, 1999

Bjoerck, Ake : Numerical methods for least squares problems. Philadelphia, SIAM 1996
Very well written book with lots of nonstandard information.

Dennis, E. and Schnabel, B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice Hall, 1983. (reprinted by SIAM)
a classic in its field

Dempe, Stephan: Foundations of Bilevel Programming, Springer Verlag, 2002
Bilevel programming, Theory and algorithms.

Du, D.-Z. and Pardalos, P.: Minimax and Applications. Springer, 1995

Fiacco, A. and McCormick, G. P.: Sequential Unconstrained Minimization Techniques. Reprinted by SIAM
A classic from 1968, given new life by the interior point LP methods

Fletcher, R.: Practical Methods of Optimization. John Wiley, 2000.
"The" reference at the date of its printing.

Gill, Ph.E., Murray, W. and Wright, M.: Practical methods of optimization. New York:Acad. Press 1982
a bit dated with respect to methods, but with many hints for practitioners

M. Locatelli and F. Schoen: Global Optimization
Theory, Algorithms, and Applications

Th. Weise: Global Optimization Algorithms - Theory and Application

Hansen, E, and G.W. Walster: Global Optimization Using Interval Analysis. Dekker, 2003.
(second edition)

Horst R., Pardalos P., and Thoai, V.: Introduction to global optimization. Springer, 1995.

Horst R. and Pardalos P.: Handbook of Global Optimization. Springer, 1994.

Kelley, C.T.: Iterative methods of optimization. Philadelphia: SIAM 1999

Kelley, C.T.: Iterative methods for Linear and Nonlinear Equations. Philadelphia: SIAM 1995

Miettinen,K.: Nonlinear Multiobjective Optimization, Springer. 1999

More, J.J. and Wright, St.: Optimization Software Guide. SIAM, 1993.
Contains overview and comments existing software, mainly commercial.

Nemhauser, G.L.: Optimization. (Handbook in Operations Research and Management Science Vol I). North Holland, 1989.
Contains excellent introductions to severals areas of optimization.

Nocedal, J. and Wright, St.: Numerical Optimization, 2nd ed.. Springer Verlag, 2006.
Very well written modern introduction into continuous optimization.

Pinter, J.D.: Global Optimization in Action. Springer, 1996.
Book received 2000 INFORMS Computing Society Prize

Pinter, J.D.: Computational Global Optimization in Nonlinear Systems. Lionheart Publ., 2001.
short e-book, demo software included

Pinter, J.D.: Global Optimization with Maple, Maplesoft.
interactive e-book

Spellucci, P.: Numerische Verfahren der nichtlinearen Optimierung. Birkhäuser, Basel 1993 (in German)

Sergeyev, Y. D., Strongin, R. G. and Lera, D.: Introduction to Global Optimization Exploiting Space-Filling Curves, Springer 2013

Alexander Forrester, András Sobester, Andy Kean: Engineering Design via Surrogate Modelling: A Practical Guide, Wiley 2008

Tawarmalani, M. and Sahinidis, N. V.: Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming. Springer, 2002.
Based on BARON software.

Books on computational/automatic differentiation

Corliss, G. et al (eds.): Automatic Differentiation of Algorithms: From Simulation to Optimization
Springer 2002; Survey chapter, extensive applications chapters, and bibliography

Bücker, M. et al (eds.): Automatic Differentiation - Applications, Theory and Implementations
Springer 2006; covers the state of the art in automatic differentiation theory and practice

Griewank, A. and Walther, A. : Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Second Edition
SIAM 2008

Non-classical techniques and constraint programming

Special Issue on the Next 10 Years of Constraint Programming, downloadable (2007),
Intro and five articles

Roberto Battiti, Mauro Brunato and Franco Mascia: Reactive Search and Intelligent Optimization, downloadable (2007),
integrates machine learning techniques

Dorigo, M., and Stützle, Th.: Ant Colony Optimization,
MIT Press, 2004: comprehensive coverage of this metaheuristic including software

Corne D., Dorigo, M., and Glover, F.: New Ideas in Optimization:
Chapters on various methods: Simulated Annealing, Genetic Programming, Tabu Search, Differential Evolution etc

Michalewicz, Z., Fogel, D.B.: How to Solve It: Modern Heuristics,
Springer Verlag 2000

Siarry, P. and Michalewicz, Z.: Advances in Metaheuristics for Hard Optimization,
Springer Verlag 2007