Constrained least-squares fitting with Python
Scipy contains a good least-squares fitting routine, leastsq(), which implements a modified Levenberg-Marquardt algorithm. I just learned that it also has a constrained least-squared routine called fmin_slsqp(). I am using simple upper and lower bound constraints, but it’s also possible to specify more complex functional constraints. What I did not realize, at first, is that fmin_slsqprequires a different type of objective function thanleastsq. leastsqrequires you to write a function that returns a vector of residuals, andleastsqautomatically squares and sums the residuals.fmin_slsqp is actually more flexible, in that it can use any objective function that returns a single scalar value. To implement least-squares curve fitting, your objective function will need to find the residual at each data point, square the values, and sum them up. Hopefully this tip will save you some time. Check out the scipy optimization tutorial for more examples. Here is the original paper by Dieter Kraft which introduces the algorithm used by fmin_slsqp.
Comments
Comment #1 by H
H. - Jun 3, 2011
Thank you! this little post of you saved me a long day of fiddling. cheers H.
Comment #2 by I
I - Apr 3, 2012
Thanks for your post! You saved my time, too.
Comment #3 by X
X - Sep 1, 2012
Thank you!
Comment #4 by sen
sen - Aug 3, 2013
I am also facing a similar problem. In my case the matrix equation looks like [Y]nX1=[X]nXm[P]mX1, where Y and P are vectors and X is a matrix. Further, there is a equality constraint on P which is Sum(P(i))=0.0. How do I proceed to solve that? Which function of python is suitable for this? I saw few of discussion on scipy.optimize.fmin_slsqp() function. However, I need your help. I am new in SCIPY. Please help me out in this regard.
Comment #5 by Craig
Craig - Aug 0, 2013
You are on the right path. You can either use scipy.optimize.fmin_slsqp directly, or use scipy.optimize.minimize with method=‘SLSQP’. See the docs at: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html You can set equality constraints using the constraints keyword argument, which also allows you to provide a Python function that defines the constraint. In your case, this function would just sum the elements of P.
Comment #6 by Edwin Francisco Juárez Rosales
Edwin Francisco Juárez Rosales - Sep 4, 2015
Thank you! Short and sweet post. Just what I was looking for!
Comment #7 by A caffeinated adventure into optimization algorithms and numerical solver libraries in python | Aquazorcarson's Blog
A caffeinated adventure into optimization algorithms and numerical solver libraries in python | Aquazorcarson's Blog - Jul 1, 2017
[…] function in scipy: I found out about this mainly through this blog post. Upon looking at the implementation on github, I got a bit dismayed since a heavy portion of the […]