Taylor Series and the Newton-Raphson Method

Taylor Series

For a function \( f(x) \) where \( a \) is a solution. The difference between the initial guess \( x \) and the solution is denoted by \( \delta x \). \( f(x + \delta x) \) is then the zeroth order Taylor series approximation.

Zeroth Order \( f(x + \delta x) \approx f(x)\)
First Order \( f(x + \delta x) \approx f(x) + \frac{df}{dx} \delta x\)
Second Order \( f(x + \delta x) \approx f(x) + \frac{df}{dx} \delta x + \frac{d^2 f}{dx^2} \frac{\delta x^2}{2} \)
General \( f(x + \delta x) \approx f(x) + \frac{df}{dx} \delta x + \frac{d^2 f}{dx^2} \frac{\delta x^2}{2} + \frac{d^3 f}{dx^3} \frac{\delta x^3}{3!} + \dotsb + \frac{d^n f}{dx^n} \frac{\delta x^n}{n!}\)

Newton-Raphson Method

For a system \( f(x) = 0 \), assume \( a \) is a solution. \( x_0 \) is the initial guess and \( a = x_0 + \epsilon \) where \( \epsilon \) is the difference between the guess and the true solution.

$$f(a) = 0$$
$$f(x_0 + \epsilon) \approx Taylor \ expansion$$
$$f(x_0 + \epsilon) \approx f(x_0) + \epsilon f'(x_0)$$
$$\epsilon \approx -\frac{f(x_0)}{f'(x_0)}$$

The true solution is therefore

$$a = x_0 - \frac{f(x_0)}{f'(x_0)}$$

Expressed as a recursive formula:

$$x_{r+1} = x_r - \frac{f(x_r)}{f'(x_r)}$$

Secant Method

Derived from Newton-Raphson but requires no differentiation but instead requires two initial values.

$$( f'(x_r) \approx \frac{f(x_{r-1})-f(x_r)}{x_{r-1} - x_r}$$

Recursive formula for the Secant method:

$$x_{r+1} \approx x_r - \frac{f(x_{r-1})-f(x_r)}{x_{r-1} - x_r}f(x_r)$$