Tutorials and Books

Tutorials and online books

Cornell University Computational Optimization Open Textbook very comprehensive source for optimization and applications
A First Course in Linear Optimization an intro to LP with software
Global Optimization Algorithms – Theory and Application – book on heuristic methods
A Field Guide to Genetic Programming book on general genetic programming
LP-book an introduction to linear programming and the simplex method, with online exercises
Deterministic Modeling Linear Optimization with Applications
Linear Optimization: Theory, methods, and extensions Introduction to linear programming, sensitivity analysis, simplex and interior point methods
NEOS-Optimization-Guide A comprehensive guide with intro, algorithms, resources
Convex Optimization book by Stephen Boyd and Lieven Vandenberghe. more material on class webpages.
Tutorial on Geometric Programming by Stephen Boyd et al. (PDF)
Various lectures by Hans Bruun Nielsen optimization, least squares etc
Topology Optimization for structural problems, MEMS
A course in combinatorial optimization lecture notes by A. Schrijver
Knapsack Problems free download of book on topic
MIP Theory and Practice survey article by Bixby et al
Latest Advances in MIP Solvers talk by Bixby et al
Integer Programming Software Systems survey comparing CPLEX, XPRESS-MP, LINDO
Six Talks on MINLP by Sven Leyffer
MIP/PDECON problems by Sven Leyffer
Survey on convex MINLP handout and presentation
Survey on nonconvex MINLP by Burer and Letchford
MINLP survey paper by P. Belotti et al
OR-Notes by J. E. Beasley, deterministic and stochastic topics
OR Class Notes by M. Trick, integer programming, dynamic programming etc
Tutorials on Stochastic Programming from the COSP website
Introduction to Stochastic Programming from the COSP site
Robust control, LMI Various courses by Didier Henrion

A list of books

Optimization is a very lively area, hence standard textbooks become outdated very fast. Therefore only a very restricted and certainly subjective list of books is presented here, mainly extracted from the FAQs initiated by Gregory and presently maintained by R. Fourer.


Books on or containing a considerable amount of LP theory or practice:


Bertsimas, D. and Tsitsiklis, J.: Introduction to Linear Optimization. Athena Scientific, 1997.
Graduate-level text on linear programming, network flows, and discrete optimization.


Maros, I., Computational Techniques of the Simplex Method
Recent comprehensive monograph


Dantzig, G. B.: Linear Programming and Extensions, Princeton University Press, 1963
The most widely cited early textbook in the field.


Dantzig, George B. and Thapa, Mukund N.: Linear Programming 1: Introduction, Springer Verlag, 1997


Juenger, M. et al.: Mixed Integer Programming Computation, Springer, 2009
50 Years of Integer Programming 1958-2008


Luenberger, D. G. and Ye, Yinyu: Introduction to Linear and Nonlinear Programming, Springer, 2008
Updated version of an old classic. Well suited for beginners


Nash, S. and Sofer, A.: Linear and Nonlinear Programming, McGraw-Hill, 1996


Roos, C., Terlaky T. and Vial, J. Ph.: Theory and Algorithms for Linear Optimization: An Interior Point Approach. John Wiley, Chichester, 1997


Schrijver, A.: Theory of Linear and Integer Programming, John Wiley, 1999
Advanced, very well written


Vanderbei, R. J.: Linear Programming: Foundations and Extensions. Springer, 1996
Balanced coverage of simplex and interior-point methods. Source code available on-line for all algorithms presented.


Williams, H.P., Model Building in Mathematical Programming, John Wiley 1999, 4th edition
Little on algorithms, but excellent for learning what makes a good model


Wright, St. J.: Primal-Dual Interior-Point Methods. SIAM Publications, 1997
Covers theoretical, practical and computational aspects of the most important and useful class of interior-point algorithms


Ye, Yinyu: Interior Point Algorithms: Theory and Analysis. John Wiley, 1997


Kellerer, H. et al: Knapsack Problems. Springer-Verlag, 2003



Now a table of books mainly devoted to nonlinear programming

Conn, A.R., Scheinberg, K., and Vicente, L.N.: Introduction to Derivative-Free Optimizati. SIAM:2009
First contemporary comprehensive treatment of optimization without derivatives


Avriel, M. and Golany, B.: Mathematical Programming for Industrial Engineers. Marcel Dekker:1996
Contains introductory chapters to several areas of mathematical optimization. well suited for beginners


Bonnans, J.F., Gilbert, J.C., Lemarechal, C., Sagastizabal, C.A.: Numerical Optimization (2nd edition). Springer: 2006
Both theory and details on implementations; nonsmooth optimization, interior-point methods etc.


Bertsekas, Dimitri P.: Nonlinear Programming, second edition. Athena Scientific, 1999


Bjoerck, Ake : Numerical methods for least squares problems. Philadelphia, SIAM 1996
Very well written book with lots of nonstandard information.


Dennis, E. and Schnabel, B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice Hall, 1983. (reprinted by SIAM)
a classic in its field


Dempe, Stephan: Foundations of Bilevel Programming, Springer Verlag, 2002
Bilevel programming, Theory and algorithms.


Du, D.-Z. and Pardalos, P.: Minimax and Applications. Springer, 1995


Fiacco, A. and McCormick, G. P.: Sequential Unconstrained Minimization Techniques. Reprinted by SIAM
A classic from 1968, given new life by the interior point LP methods


Fletcher, R.: Practical Methods of Optimization. John Wiley, 2000.
"The" reference at the date of its printing.


Gill, Ph.E., Murray, W. and Wright, M.: Practical methods of optimization. New York:Acad. Press 1982
a bit dated with respect to methods, but with many hints for practitioners


M. Locatelli and F. Schoen: Global Optimization
Theory, Algorithms, and Applications


Th. Weise: Global Optimization Algorithms - Theory and Application
downloadable


Hansen, E, and G.W. Walster: Global Optimization Using Interval Analysis. Dekker, 2003.
(second edition)


Horst R., Pardalos P., and Thoai, V.: Introduction to global optimization. Springer, 1995.


Horst R. and Pardalos P.: Handbook of Global Optimization. Springer, 1994.


Kelley, C.T.: Iterative methods of optimization. Philadelphia: SIAM 1999


Kelley, C.T.: Iterative methods for Linear and Nonlinear Equations. Philadelphia: SIAM 1995


Miettinen,K.: Nonlinear Multiobjective Optimization, Springer. 1999


More, J.J. and Wright, St.: Optimization Software Guide. SIAM, 1993.
Contains overview and comments existing software, mainly commercial.


Nemhauser, G.L.: Optimization. (Handbook in Operations Research and Management Science Vol I). North Holland, 1989.
Contains excellent introductions to severals areas of optimization.


Nocedal, J. and Wright, St.: Numerical Optimization, 2nd ed.. Springer Verlag, 2006.
Very well written modern introduction into continuous optimization.


Pinter, J.D.: Global Optimization in Action. Springer, 1996.
Book received 2000 INFORMS Computing Society Prize


Pinter, J.D.: Computational Global Optimization in Nonlinear Systems. Lionheart Publ., 2001.
short e-book, demo software included


Pinter, J.D.: Global Optimization with Maple, Maplesoft.
interactive e-book


Spellucci, P.: Numerische Verfahren der nichtlinearen Optimierung. Birkhäuser, Basel 1993 (in German)


Sergeyev, Y. D., Strongin, R. G. and Lera, D.: Introduction to Global Optimization Exploiting Space-Filling Curves, Springer 2013


Alexander Forrester, András Sobester, Andy Kean: Engineering Design via Surrogate Modelling: A Practical Guide, Wiley 2008


Tawarmalani, M. and Sahinidis, N. V.: Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming. Springer, 2002.
Based on BARON software.


Books on computational/automatic differentiation

Corliss, G. et al (eds.): Automatic Differentiation of Algorithms: From Simulation to Optimization
Springer 2002; Survey chapter, extensive applications chapters, and bibliography


Bücker, M. et al (eds.): Automatic Differentiation - Applications, Theory and Implementations
Springer 2006; covers the state of the art in automatic differentiation theory and practice


Griewank, A. and Walther, A. : Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Second Edition
SIAM 2008


Non-classical techniques and constraint programming

Special Issue on the Next 10 Years of Constraint Programming, downloadable (2007),
Intro and five articles


Roberto Battiti, Mauro Brunato and Franco Mascia: Reactive Search and Intelligent Optimization, downloadable (2007),
integrates machine learning techniques


Dorigo, M., and Stützle, Th.: Ant Colony Optimization,
MIT Press, 2004: comprehensive coverage of this metaheuristic including software


Corne D., Dorigo, M., and Glover, F.: New Ideas in Optimization:
Chapters on various methods: Simulated Annealing, Genetic Programming, Tabu Search, Differential Evolution etc


Michalewicz, Z., Fogel, D.B.: How to Solve It: Modern Heuristics,
Springer Verlag 2000


Siarry, P. and Michalewicz, Z.: Advances in Metaheuristics for Hard Optimization,
Springer Verlag 2007