 |
|
The Levenberg-Marquardt algorithm is a well-suited method for finding the minimum of a sum of squares of generally nonlinear functions. This trust-region method combines effectively two different algorithms: the Gauss-Newton algorithm and the steepest descend method. In fact Levenberg-Marquardt is located between these two extremes of operation, adaptively changing its behaviour according to the problem needs.
Levenberg-Marquardt is widely used as curve fitting algorithm. Another interesting use is its implementation in back propagation training of neural networks.
In this algorithm, a penalty function is added in order to handle constraints.
The Levenberg-Marquardt is a "continuous" method and therefore all the variables should be continuous (Base=0). This algorithm ignores the bases set in the Work Flow and considers all the bases equal to 0.
The entries of the DOE table are used as a sequence of initial points for different local optimization problems.
Note:
The Levenberg-Marquardt algorithm uses the Design Target node for defining the addenda of the sum of squares to be minimized.
The user must specify:
Maximum Number of Iterations: the algorithm will always stop when the maximum number of designs have been evaluated. This is done independently from any other convergence limits.
Approximate derivatives: two options are provided, Central and Forward differences. Central Difference are more accurate but more expensive, a gradient computation requires in fact number_of_variables computations in case of Forward differences and 2*number_of_variables computations in case of Central differences.
Final Termination Accuracy: the algorithm will stop when it cannot find solution with improvements higher that the convergence value. The default is set to 1.0E-5.
Finite Difference Relative Perturbation: the value is the fraction of the variable value that is used to perturb the starting point in order to compute the derivatives. The default is set to 1.0E-7. For local refinements low values should be used.
Finite Difference Minimum Perturbation Policy: it is possible to choose between two different policies: Constant or Range Percentage.
Constant Minimum Perturbation: this field is active only when the Finite Difference Minimum Perturbation Policyis Constant. The value is the minimal value that is used to perturb the starting point. The perturbation step is then given by the maximum value between the Finite Difference Relative Perturbation and this Constant Minimum Perturbation.
Range Percentage Minimum Perturbation: this field is active only when the Finite Difference Minimum Perturbation Policyis Range Percentage. This percentage gives the minimal value that is used to perturb the starting point. For each variable this value is equal to:
Minimum_Perturbation = (Upper_bound - Lower_bound) * (Range_Percentage_Minimum_Perturbation)The perturbation step is then given by the maximum value between the Finite Difference Relative Perturbation and the Minimum_Perturbation.
Constraint penalty: the value is the constant that multiplies the distance from the feasible boundary.
The value can be problem dependent and is a function of the objective range as the penalization is added to the objective function.
The default value is 1000.
It is suggested to give a value to the penalty at least 2 orders of magnitude higher than the expected objective function value.
Note:
When finite-differences are required all the perturbed configurations can be computed in parallel, the maximum number of concurrent design evaluation, from an algorithmic point of view, is equal to number_of_variables in case of Forward differences and 2*number_of_variables computation in case of Central differences.
The number of concurrent designs evaluation can be set in the Run Project Dialog.
