Thread: Data Mining

PLEASE HELP
THIS QUESTION MAY LOOK LONG, BUT IT’S REALLY EASY!!!
I have a list of a few thousand numbers. 9 columns in each set of numbers. You can see attached example. There are 6 columns separated by an empty one and then 3 more columns of numbers. The first 6 numbers produced a result that I wrote down in the last 3 columns. In other words, because of the first 6, the last 3 happened. For instance, if I was recording speed limits in a certain area, over a period of time, and I decided to use these criteria:
1. % of miles above speed limit
2. % above legal alcohol intake
3. at what time did the individual drive
and I did research for the past year, and I wrote them down in 3 columns . For instance:
A B C
23% 2% 3:24pm
Now that was the input, and the next 3 columns may be the results. For instance,
1. if there was an accident, how many people died
2. if there was an accident, how many people were injured
3. if there was an accident, what was the cost of the accident
so now it will be 6 columns and would look like this
A B C D E F
23% 2% 3:24pm 1 0 $34,000
and I would have many hundreds of examples taken over a long period of time. All I have is actual data. Now, I would like to know the type of mathematics or math formula that would solve, by going through all the data, and then figuring out what a future case might produce. For instance, if I typed in
A B C
19% 0.1% 12:01am
then the formula should be able to produce the last 3 results based on the analyses it has done over the other hundreds lines of data

1. What would be the name of the mathematics or the name of the mathematical formula that I would use to go through all the data and based on ALL the inputs and results find a general formula so that if I plug in a set of numbers it can predict the three results?
2. Would the type of math formula be any different if there was only 1 result as opposed to 3 results?
3. Would you happen to know any software that would do this for me?

You must have a theory which will relate numbers together. You cannot expect a computer program to figure out how to interpret data for you. It will not guess which data affect which outcome. You can use excel to help develop theories once you have developed equations to predict outcomes you can also use excel to predict outcomes.

What would be the name of the theories that would relate the numbers together?

This sounds like data mining. I would like to try classification tree, or decision tree. As I know many statistic softwares all can do this kind trick, such as SAS and R/splus. I have SAS in my computer, I get it from my school for free. And it's an academic version, it's only has 10% original modules. We have to pay if we want to use other fitures which include tree building, like I said above. These all I learned from my stats papers.

Originally Posted by eric123

What would be the name of the theories that would relate the numbers together?

Try neural networks. There are shareware or evaluation versions available on the web. Train a network with, say, 70% of your data (randomly selected) and evaluate it with the remaining 30%. For an alternative approach, consult the book by Hastie, Tibshirani, Friedman on statistical learning.

Originally Posted by MathGuru

You must have a theory which will relate numbers together. You cannot expect a computer program to figure out how to interpret data for you. It will not guess which data affect which outcome.

I disagree with this assertion completely. This is done all the time in many fields. Given sufficient observations, very useful predictive models can be built mechanically, including the selection of independent variables.

Depending on what precisely you mean by "interpret", then, no, perhaps not, but predictive modeling by automatic means is not merely possible: It is done to solve important problems every day (database marketing, credit scoring, etc.).


-Will Dwinnell
Data Mining in MATLAB