An example
I can show you the general solution in all its elegance (and it is indeed elegant). But first I'd like to walk you through a simple example, involving only four data points. The data is shown in the first two columns of Table 1. In the remaining columns, I'm showing the values of f(x_i) and e_i.

The value of M is the sum of the squares of the terms in the last column.

(6)

Expanding and collecting terms gives:

(7)

This is the function we seek to minimize. Does it even have a minimum? A little reflection will show that yes, it does. If I make either a or b huge positive or huge negative numbers, M itself will be huge, Since it's a sum of squares, it's always positive. So somewhere between huge, zero, and huge again, there must always be a minimum. This is, of course, hardly a rigorous proof, but it works for me.

To find the minimum takes a little calculus, but only a little. If I take the partial derivatives of M with respect to a and b, I get:

(8)

These partials represent the slopes of M in the a and b directions. When M is at its minimum value, both slopes must be zero. So we end up with two equations to solve simultaneously:

(9)