Consider the points:

(0,1)

(1,1)

(2,1)

(3,1)

(4,1)

(5,41)

(6,1)

(7,1)

(8,1)

(9,1)

And the linear approximation y = 5.

There is nothing wrong with differences or absolute differences, rather than squared differences. It needs to fit your application, whatever it is. On the other hand, you are studying "least squares". It's a valuable tool. Learn it well. Also, it has convenient mathematical properties that make it simpler to formulate and implement.