Implement a function minimization algorithm
14 comments

Nils Haferkemper commented
I found a really nice approach wich is easily generelizable: http://www.imagingshop.com/linearandnonlinearleastsquareswithmathnet/
I also applied this for several other functions and it works like charm! 
Anonymous commented
I have a quadratic minimization problem (with linear constraints) that could be solved by LevenbergMarquardt algorithm combined with the activeset method.
The objective function returns a scalar value. The hessian and the gradient are function of the current parameter values.
Which library, in C#, solve it?

Lionel du Peloux commented
Hello,
There exists a good C# wraper for the IPOPT solver (C) : https://github.com/cureos/csipopt
This wraper is also included in a nice math C# library (FuncLib) you may already know : https://funclib.codeplex.com/. FuncLib have also automatic diff and BFGS support.
Maybe this could be a good point to start ? Ipopt is very powerful and is fully opensource.
Cheers,
Lionel 
AdminChristoph Rüegg (Admin, Math.NET) commented
Thanks, that will be very useful for our implementation!
Quick note on ALGLIB: We cannot use any code from ALGLIB version >= 2.1.0 for legal/licensing reasons, however we do have explicit permission from Sergey Bochkanov (the author of ALGLIB) for version 2.0.1.

Libor commented
One excellent C# implementation of several optimization algorithms (including LM and LBFGS) is provided in ALGLIB package:

Libor commented
Here is my implementation of LevenbergMarquardt using Math.NET and C#  feel free to use it as you like:
http://libca.cz/pub/libor/nonlinearleastsquaressolver.zip
I have tested it on exponential rotation estimation and bundle adjustment (joint image alignment in panoramic image stitching).

Anonymous commented
It would be wonderful to have a couple of optimization algorithms in the library.
I wote for BFGS (turns very efficient in many practical cases) and Conjugate gradient (low memory requirements, more robust and easier to use on illconditioned problems) for nonlinear unconstrained minimization first.
Next steps could be SQP for constraint problems, and in the other direction stochastic methods (e.g. a genetic algorithm, particle swarm or similar).
Something already exist in dnAnalytics, could that be used?
http://www.koders.com/csharp/fidA522A433065B5D359C943562507882606B1AD25D.aspx?s=search#L15

AdminChristoph Rüegg (Admin, Math.NET) commented
That looks good indeed, thanks for sharing!

David Cuccia commented
Feel free to use my C# LevMarq port here: http://csmpfit.codeplex.com/

Doug commented
It would be very cool to have this in Numerics. We use an independent port of the Fortran but it's far from pretty.
I vote for LBFGSB. We've used it extensively on high (20+) dimensional problems with slow fn evaluations (10100ms) and it works as advertised for multivariate bounded minimization. DIY gradient. An automated gradient is numerically risky due to scaling and rounding, but could be a nice option.

Tom Robinson commented
Optimisation methods would be very useful.

Greg commented
What do you guys think about a Levenberg–Marquardt algorithm? It wouldn't be to hard to port over.

marcochiarini commented
Hi guys! in my opinion efforts should be addressed to the general formulation:
Min f(x)
xs.t. g(x) <= K
This wold make Math.NET really powerful.
Regards,
Marco

David Cuccia commented
Hear hear!