# Thread: What Is The Best Way To Measure The Accuracy Of A Prediction?

1. ## What Is The Best Way To Measure The Accuracy Of A Prediction?

Hi,

A couple of my friends and I are having a contest to see who can best predict this seasons NFL stats. We are predicting how many yards, touchdowns, etc. each player will have each week.

What would be the best way to measure how accurate each of our predictions were after the outcome is known?

2. That would quite obviously be the way that makes you come out on top?

Your only real concern is that inaccuracies cancel out each other. For example, if Runner 1 goes over the prediction by 10 and Runner 2 goes under the prediction by 10, adding them together might seem as though the prediction was perfect, +10 -10 = 0, right? For this cause, it often seems prudent to find a way to avoid this. Most often a square is used, but an absolute value will do the same thing.

(+10)^2 + (-10)^2 = 200

or

|+10|+|-10| = 20

3. Originally Posted by TKHunny
That would quite obviously be the way that makes you come out on top?
Haha, touche!

I guess I'm not even sure how to approach the problem.

Say I predict that a player will have 100 yards and the player ends up with 76 yards. My friend predicts the same player will have 98 yards. My friend's prediction would be more accurate, but what formula would I use to measure the degree of accuracy?

For example:

I predict another player will get 60 yards, and he has 65 yards. My friend predicts that same player will have 160 yards. My prediction was more accurate this time, and to a greater degree, but how do I calculate that?

4. If you've only one item to compare, you just look at it. More than one, this may be the most common measure...

$\displaystyle \frac{(observed - expected)^{2}}{observed}$

It might be called "squared relative error".

You calculate this for each prediction and add them all up.

5. Originally Posted by Amuka
Hi, Thanks for reading my question.

A couple of my friends and I are having a contest to see who can best predict this seasons NFL stats. We are predicting how many yards, touchdowns, etc. each player will have each week.

What would be the best way to measure how accurate each of our predictions were after the outcome is known?
TKHunny's method is very good.

My thoughts:
$\displaystyle ValueFactor_i \cdot \left( 1 - \dfrac{ \sqrt{(actual_i - prediction_i)^2}}{(actual_i + prediction_i)}\right)$

The Square Root is ONLY used to get the ABSOLUTE value of the difference between the guess & prediction. Whether you are over or under, the error amount is the same -- unless you allocate some minus value to a Gamer who fails to get it at least what actually occurred, or deduct something for exceeding the actual.

Example:
ValueFactor
KickingYards = 0.7
PassingYards = 1.05

RunningYards = 1.4

Predictions_Gamer1
Kicking = 1001
Passing = 801

Running = 601

Predictions_Gamer2
Kicking = 251
Passing = 201

Running = 151

ActualResults
Kicking 500
Passing 400
Running 300

Raw Ranking (without value factor)
Gamer1: (0.6623+0.6661+0.6659)=1.9943
Gamer2: (0.6684+0.6689+0.6696)=2.0069

w/ValueFactor
Gamer1: (0.4636+0.6994+0.9307)=2.0653
Gamer2: (0.4678+0.7023+0.9374)=2.1075

.

6. Awesome! I love the ValueFactor idea to place a higher value on different predictions.

My goal is really to come up with the most accurate possible measure of who was the most accurate with their prediction.

Would it make some sense to calculate a couple of different measures of predictive accuracy and then combine them into one rating, or is that just adding complexity where complexity is not required?

Obviously I am not aware of how to calculate any other measures of predictive accuracy...

Thanks a lot for the responses!

7. Originally Posted by Amuka
My goal is really to come up with the most accurate possible measure of who was the most accurate with their prediction.
It is very unlikely that you can achieve this goal as stated. "Most accurate" just cannot be defined in any generally acceptable sense. Find a good measure and use it. With some experience, you can decide if you wish to improve it or if it leads to undesirable results.

8. Originally Posted by aidan
My thoughts:
$\displaystyle ValueFactor_i \cdot \left( 1 - \dfrac{ \sqrt{(actual_i - prediction_i)^2}}{(actual_i + prediction_i)}\right)$
I've decided to use this method as a second method to rank the prediction's accuracy, and I like that it seems to put it in a decimal form. It's almost like % accuracy. I'm only using the "raw" version of it.

What is this method called?

Thanks everyone for helping me in this thread.