Module 7: Minimization and Curve Fitting

Curve fitting is one of many ways to make sense of data. Curve fitting translates into a continuous minimization problem where we minimize the distance between a family of curves and discrete sample points. There are different ways in which we can define the distance, so that treating minimization problems is still important for the future data analyst.

7.1. Minimization in One Dimension

The field of mathematical optimization has grown to be vast, since the work by such giants as Newton, Fermat, Legendre, Gauss, Euler, and Cauchy. Optimization falls into different categories. For instance, Linear Programming finds minima of a linear goal function subject to constraints that are linear inequalities. Integer Programming is a subfield of Linear Programming, where solutions are limited to the integers. Discrete programming optimizes over a countable, usually finite set, as for example over all permutations of several objects. Clearly, it is impossible to give here a comprehensive treatment of mathematical optimization even in rough draft. We limit ourselves to describing SciPy’s optimization module. This module assumes that the objective function is at least twice differentiable. If the objective function is quadratic, then easier methods exist.