# Thread: Smoothing algorithm for times series and slope measurement

1. ## Smoothing algorithm for times series and slope measurement

Hi,

I'm doing an experiment that gathers one data point approximately each second. I have plotted all the datapoints on a graph. The graph has many tops and bottoms. Now due to the fact that there are so many data points and that I can visualle tell that there are several overal trends: one period my data is going up, the next period it is level again for a period of time and the next period it can be down again. And in each of these periods the data can go up, down or level again, although the overal direction is up, down or level. I have searched for different smoothing methods that can follow all these small movements, but at the same time (using a second smoothing algorithm/parameter) can follow the overal movement. I was thinking about using splines or lowess. Are splines or lowess able to do this?

Second problem I'm having is, after having applied a smoothing algorithm, I need to be able to measure the slope of the smoothing line. I need to know what the slope/curvature of that smoothing line is at any giving moment in time. All the pointers you can give me are really welcome. Thanks.

Kind regards,

Nick

2. If I had a set of data and it followed a trend I would try to apply a function in your case possibly an exponential function.

Say if I had number of rats in a field at a given time per year

t = time in years
x = number of rats

I know what the initial number of rats= 20

so f(0) = 20

say if the function was the form of

$\displaystyle f(t) = A*e^t$ where A is an constant

we apply x = 20 and t = 0 we get

so $\displaystyle 20 = A*e^0$

therefore A = 20

$\displaystyle f(t) = 20e^t$

by trail and error you could change the t variable to 2t or t/3 etc until it give an APPROXIMATE of your data you have

Your second problem you could apply differentiation so you would get the tangent a given time or if that isn't what your after you could just put a year Eg 1.5 into f(x) and you would get a return value of number of rats :]

3. This is kinda a huge question. There are dozens of methods for making a function to the data, with loads of different parameters and characteristics.

Linear Basis Function Models are perhaps the easiest to get into without any pattern recognition experience and they have a closed form maximum likelihood solution (and an online version if needed), but it's still too elaborate for me to explain her without copying a book. If you need a predictive distribution over this you can also read up on the baysian variant of this... In your case with lots of ups and downs I'd suggest Gaussian basis functions. These also easily regularize to multiple outputs if needed.

A problem you might be facing is that the slopes are heavily dependent on the parameters you use, as the stiffness of the line depends on these.