Thread: Help Identifying Obvious Curves Computationally

For a final year Engineering project I am analysing brain wave spiking signals (basically a series of discrete data points over 3s). The aim is to predict where in the series certain actions are taking place during the testing.

From the original spike series the data was grouped into equally sized bins and then the spiking rate per bin calculated. To analyse the test data the groups between two points in time are taken and the probability density of each spiking rate (50ms^-1 -> 400 ms^-1) was calculated and fit to a normal distribution. Certain time periods showed a much more obvious distribution so it was expected that these would be identifiable in a "blind" test where the model sweeps through the entire series plotting points with bins of 20ms and ranges of 400ms. As expected certain traces were much more obvious than others and the next task is, in the computational model, to separate these from less significant traces.

Here is the test graph, showing the probability density distributions between the various actions that take place over one series:

Here is the graph produced from the sweeping algorithm on the same data:

From this graph it can be seen that the same obvious trace rises above the general noise. However I can't use any data from the analysis (first graph) in filtering the second graph (because I need this to work for any of the 24 tests I have data for) so I need a way of filtering this in code. Can anyone suggest steps or pseudo-code that will allow me to chuck away the majority of traces, leaving the light blue, two green, turquoise and brown traces?

I've deliberately left out how I'm storing data or the language I'm using as this is a question of the principle rather than the method. Thanks in advance.

Originally Posted by M1ke

For a final year Engineering project I am analysing brain wave spiking signals (basically a series of discrete data points over 3s). The aim is to predict where in the series certain actions are taking place during the testing.

From the original spike series the data was grouped into equally sized bins and then the spiking rate per bin calculated. To analyse the test data the groups between two points in time are taken and the probability density of each spiking rate (50ms^-1 -> 400 ms^-1) was calculated and fit to a normal distribution. Certain time periods showed a much more obvious distribution so it was expected that these would be identifiable in a "blind" test where the model sweeps through the entire series plotting points with bins of 20ms and ranges of 400ms. As expected certain traces were much more obvious than others and the next task is, in the computational model, to separate these from less significant traces.

Here is the test graph, showing the probability density distributions between the various actions that take place over one series:

Here is the graph produced from the sweeping algorithm on the same data:

From this graph it can be seen that the same obvious trace rises above the general noise. However I can't use any data from the analysis (first graph) in filtering the second graph (because I need this to work for any of the 24 tests I have data for) so I need a way of filtering this in code. Can anyone suggest steps or pseudo-code that will allow me to chuck away the majority of traces, leaving the light blue, two green, turquoise and brown traces?

I've deliberately left out how I'm storing data or the language I'm using as this is a question of the principle rather than the method. Thanks in advance.

Look at a test statistic something like the ratio of maximum to median (or mean?) for the curve, and set a detection threshold of something like 2.

CB

Unfortunately the curves are made from only 3-6 data points, and usually the median is tiny as it comes from the lower points. In the end I found that taking the standard deviation along each vertical point (i.e. the range of y values for a certain x value), then taking only graphs that had a point above that deviation filtered it partially. After that I worked out a threshold ratio of the deviation of each curve at that x value from the mean and seem to have a fairly good filter. Unfortunately I've no idea the name of this method having found it by trial and error; so I need to know what techniques I'm really using before writing it up.

Thanks for the help.