Take a look at Taylor Series Expansion by mtu.edu for a derivation.
Basically, you look at a function and say, "How could I expand this in a power series?" The Taylor Series is one way to expand a function as a power series. Expanding a function in this way can lead to some useful approximations. For instance, lets look at sin[x] about zero:
So one can make approximations to whatever order they wish. For instance, to "zeroeth" order, sin(x) about zero = 0. To first order, sin(x) about zero = x. To third order, sin(x) about zero . Etc.
In addition, it can be helpful in finding limits of functions:
Notice that in the limit that x goes to zero, the Taylor series to any arbitrary order is exactly equal to Sin(x):
In physics we make approximations using Taylor series all the time because instruments can only measure to a certain degree of accuracy anyway.