Conjugate Directions method
This Conjugate Directions method is sometimes called the direct search method or Powell's method, after it originator. The basic idea behind this method is to use a series of one-dimensional searches to locate the optimum which is a function of variables X1, X2, .... Given a starting point, it seems reasonable to search for an optimum value of X1 holding X2, X3, etc. constant. Then after finding the optimum value of X1, the process is repeated for the other dimensions. The one-dimensional searches do not necessarily require numerical derivatives giving this algorithm an advantage when used with functions that are not easily differentiable. However, the procedure of searching for an optimum holding all independent variables except one constant is not necessarily efficient, as seen in the figure on the left.
It would be much more efficient to search along a set of directions that is conjugate to the objective function, as represented by Z1, Z2 in the figure on the right. The algorithm used in EES is due to Powell and described in Numerical Recipes, by Press et al, Cambridge University Press, 1989. The method attempts to orient the search direction in the conjugate directions as additional information from each iteration is collected.
Return to Min/Max.