Tutorial

Here are present examples of the use of openopt

Generic framework

The module is loaded with the following instruction:

from scikits.optimization import *

Simple use of the framework

Suppose you have a simple two arguments functions with its gradient :

>>> class Function(object):
  def __call__(self, x):
    return (x[0] - 2) ** 2 + (2 * x[1] + 4) ** 2

  def gradient(self, x):
    return numpy.array((2 * (x[0] - 2), 4 * (2 * x[1] + 4)))

You then must import the optimizers (suppose you are in the solvers folder) :

>>> import numpy
>>> from scikits.optimization import *

Then you can optimize your function :

>>> fun = Function()
>>> mystep = step.GradientStep()
>>> mylinesearch = line_search.GoldenSectionSearch(min_alpha_step = 0.0001)
>>> mycriterion = criterion.criterion(ftol = 0.0001, iterations_max = 100)
>>> myoptimizer = optimizer.StandardOptimizer(function = fun, step = mystep, line_search = mylinesearch, criterion = mycriterion, x0 = numpy.zeros((2)))
array([ 2., -2.])

>>> myoptimizer.state
{'function': <__main__.Function object at 0xc67e0c>, 'new_parameters': array([ 2., -2.]), 'gradient': array([ 0.,  0.]), 'old_value': 0.0, 'iteration': 35, 'step': array([-0., -0.]), 'step_size': 4.0856349008447342e-05, 'old_parameters': array([ 2., -2.]), 'new_value': 0.0}

You can use an inexact line search with the Wolfe Powell strong rules exactly in the same way :

>>> fun = Function()
>>> mystep = step.FRConjugateGradientStep()
>>> mylinesearch = line_search.StrongWolfePowellRule()
>>> mycriterion = criterion.criterion(ftol = 0.0001, iterations_max = 100)
>>> myoptimizer = optimizer.StandardOptimizer(function = fun, step = mystep, line_search = mylinesearch, criterion = mycriterion, x0 = numpy.zeros((2)))
>>> myoptimizer.optimize()
array([ 2., -2.])
>>> myoptimizer.state
{'function': <__main__.Function object at 0x30d26c>, 'new_parameters': array([ 2., -2.]), 'gradient': array([ 0.,  0.]), 'old_value': 0.0, 'iteration': 4, 'step': array([ 0., -0.]), 'step_size': 1.0, 'old_parameters': array([ 2., -2.]), 'step_modifier': 0.0, 'new_value': 0.0}

Use of helper functions

Fitting Data

Once the fit function returns the correct result, it can be used to create a Quadratic instance (for example) that can be optimized through the framework

>>> class F1(object):
  def __call__(self, x, params):
    return params[0] + params[1] * x

  def gradient(self, x, params):
    return numpy.array((numpy.ones(len(x)), x)).T

  def hessian(self, x, params):
    return numpy.zeros((len(x), 2, 2))

>>> x = numpy.random.random_sample((10))
>>> y = 3*x+4
>>> function = helpers.Quadratic(x, y, F1())

>>> x0 = numpy.zeros(2)

>>> opt = optimizer.StandardOptimizer(function = function,
                        x0 = x0,
                        step = step.GradientStep(),
                        line_search = line_search.FibonacciSectionSearch(min_alpha_step=0.000001),
                        criterion = criterion.criterion(ftol = 0.00001, iterations_max = 1000))

>>> print opt.optimize()
[ 4.  3.]
>>> print opt.state
{'function': <optimizers.helpers.quadratic.Quadratic object at 0xe7e50c>, 'direction': array([  5.32907052e-15,   1.22827925e-15]), 'new_parameters': array([ 4.,  3.]), 'gradient': array([ -5.32907052e-15,  -1.22827925e-15]), 'old_value': 2.3665827156630354e-30, 'iteration': 10, 'alpha_step': 4.5907169276693443e-07, 'old_parameters': array([ 4.,  3.]), 'new_value': 2.3665827156630354e-30}

Finite Differences

Finite differences helps computing numerical gradient and/or hessian. If the gradient is provided, then the hessian will take advantage of it.

>>> class Function(helpers.ForwardFiniteElementDerivatives):
  def __call__(self, x):
    return (x[0] - 2) ** 2 + (2 * x[1] + 4) ** 2

>>> startPoint = numpy.zeros(2, numpy.float)
>>> optimi = optimizer.StandardOptimizer(function = Function(),
                           step = step.FRConjugateGradientStep(),
                           criterion = criterion.criterion(0.0000001, 100),
                           x0 = startPoint,
                           line_search = line_search.StrongWolfePowellRule())
>>> print optimi.optimize()
[ 2.         -2.00000004]
>>> print optimi.state
{'function': <__main__.Function object at 0xe7e8ac>, 'direction': array([ -1.13496692e-07,  -1.37756364e-08]), 'new_parameters': array([ 2.        , -2.00000004]), 'gradient': array([  9.72597028e-08,   4.72195288e-08]), 'old_value': 7.7802561238186933e-15, 'iteration': 4, 'alpha_step': 0, 'old_parameters': array([ 2.        , -2.00000004]), 'step_modifier': 0.11824091630690386, 'new_value': 7.7802561238186933e-15}

>>> startPoint = numpy.zeros(2, numpy.float)
>>> optimi = optimizer.StandardOptimizer(function = Function(),
                           step = step.NewtonStep(),
                           criterion = criterion.criterion(0.0000001, 100),
                           x0 = startPoint,
                           line_search = line_search.SimpleLineSearch())
>>> print optimi.optimize()
[ 1.99999995 -2.00000005]
>>> print optimi.state
{'function': <__main__.Function object at 0xe7e98c>, 'direction': array([ -1.11022304e-16,   1.11022302e-16]), 'new_parameters': array([ 1.99999995, -2.00000005]), 'gradient': array([  2.22044608e-16,  -8.88178414e-16]), 'old_value': 1.2500000047903369e-14, 'iteration': 5, 'alpha_step': 1.0, 'old_parameters': array([ 1.99999995, -2.00000005]), 'new_value': 1.250000007010783e-14, 'hessian': array([[  2.00000000e+00,  -4.73316543e-16],
       [ -4.73316543e-16,   7.99999998e+00]])}