Results 1 to 8 of 8

Math Help - What Is The Best Way To Measure The Accuracy Of A Prediction?

  1. #1
    Newbie
    Joined
    Sep 2009
    Posts
    4

    What Is The Best Way To Measure The Accuracy Of A Prediction?

    Hi,

    Thanks for reading my question.

    A couple of my friends and I are having a contest to see who can best predict this seasons NFL stats. We are predicting how many yards, touchdowns, etc. each player will have each week.

    What would be the best way to measure how accurate each of our predictions were after the outcome is known?
    Follow Math Help Forum on Facebook and Google+

  2. #2
    MHF Contributor
    Joined
    Aug 2007
    From
    USA
    Posts
    3,110
    Thanks
    2
    That would quite obviously be the way that makes you come out on top?

    Your only real concern is that inaccuracies cancel out each other. For example, if Runner 1 goes over the prediction by 10 and Runner 2 goes under the prediction by 10, adding them together might seem as though the prediction was perfect, +10 -10 = 0, right? For this cause, it often seems prudent to find a way to avoid this. Most often a square is used, but an absolute value will do the same thing.

    (+10)^2 + (-10)^2 = 200

    or

    |+10|+|-10| = 20
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Sep 2009
    Posts
    4
    Quote Originally Posted by TKHunny View Post
    That would quite obviously be the way that makes you come out on top?
    Haha, touche!

    I guess I'm not even sure how to approach the problem.

    Say I predict that a player will have 100 yards and the player ends up with 76 yards. My friend predicts the same player will have 98 yards. My friend's prediction would be more accurate, but what formula would I use to measure the degree of accuracy?

    For example:

    I predict another player will get 60 yards, and he has 65 yards. My friend predicts that same player will have 160 yards. My prediction was more accurate this time, and to a greater degree, but how do I calculate that?
    Follow Math Help Forum on Facebook and Google+

  4. #4
    MHF Contributor
    Joined
    Aug 2007
    From
    USA
    Posts
    3,110
    Thanks
    2
    If you've only one item to compare, you just look at it. More than one, this may be the most common measure...

     <br />
\frac{(observed - expected)^{2}}{observed}<br />

    It might be called "squared relative error".

    You calculate this for each prediction and add them all up.
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Super Member
    Joined
    Jan 2009
    Posts
    591
    Quote Originally Posted by Amuka View Post
    Hi, Thanks for reading my question.

    A couple of my friends and I are having a contest to see who can best predict this seasons NFL stats. We are predicting how many yards, touchdowns, etc. each player will have each week.

    What would be the best way to measure how accurate each of our predictions were after the outcome is known?
    TKHunny's method is very good.

    My thoughts:
     ValueFactor_i \cdot \left( 1 - \dfrac{ \sqrt{(actual_i - prediction_i)^2}}{(actual_i + prediction_i)}\right)

    The Square Root is ONLY used to get the ABSOLUTE value of the difference between the guess & prediction. Whether you are over or under, the error amount is the same -- unless you allocate some minus value to a Gamer who fails to get it at least what actually occurred, or deduct something for exceeding the actual.

    Example:
    ValueFactor
    KickingYards = 0.7
    PassingYards = 1.05

    RunningYards = 1.4

    Predictions_Gamer1
    Kicking = 1001
    Passing = 801

    Running = 601

    Predictions_Gamer2
    Kicking = 251
    Passing = 201

    Running = 151

    ActualResults
    Kicking 500
    Passing 400
    Running 300

    Raw Ranking (without value factor)
    Gamer1: (0.6623+0.6661+0.6659)=1.9943
    Gamer2: (0.6684+0.6689+0.6696)=2.0069

    w/ValueFactor
    Gamer1: (0.4636+0.6994+0.9307)=2.0653
    Gamer2: (0.4678+0.7023+0.9374)=2.1075

    .
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Newbie
    Joined
    Sep 2009
    Posts
    4
    Awesome! I love the ValueFactor idea to place a higher value on different predictions.

    My goal is really to come up with the most accurate possible measure of who was the most accurate with their prediction.

    Would it make some sense to calculate a couple of different measures of predictive accuracy and then combine them into one rating, or is that just adding complexity where complexity is not required?

    Obviously I am not aware of how to calculate any other measures of predictive accuracy...

    Thanks a lot for the responses!
    Follow Math Help Forum on Facebook and Google+

  7. #7
    MHF Contributor
    Joined
    Aug 2007
    From
    USA
    Posts
    3,110
    Thanks
    2
    Quote Originally Posted by Amuka View Post
    My goal is really to come up with the most accurate possible measure of who was the most accurate with their prediction.
    It is very unlikely that you can achieve this goal as stated. "Most accurate" just cannot be defined in any generally acceptable sense. Find a good measure and use it. With some experience, you can decide if you wish to improve it or if it leads to undesirable results.
    Follow Math Help Forum on Facebook and Google+

  8. #8
    Newbie
    Joined
    Sep 2009
    Posts
    4
    Quote Originally Posted by aidan View Post
    My thoughts:
     ValueFactor_i \cdot \left( 1 - \dfrac{ \sqrt{(actual_i - prediction_i)^2}}{(actual_i + prediction_i)}\right)
    I've decided to use this method as a second method to rank the prediction's accuracy, and I like that it seems to put it in a decimal form. It's almost like % accuracy. I'm only using the "raw" version of it.

    What is this method called?

    Thanks everyone for helping me in this thread.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Best way to measure the accuracy of result of function?
    Posted in the Advanced Statistics Forum
    Replies: 2
    Last Post: June 23rd 2011, 09:01 PM
  2. Replies: 3
    Last Post: November 18th 2010, 02:04 AM
  3. Statistical test for Model Prediction Accuracy
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: May 21st 2010, 10:43 AM
  4. Weather Prediction
    Posted in the Geometry Forum
    Replies: 0
    Last Post: March 23rd 2010, 02:15 PM
  5. Replies: 3
    Last Post: September 27th 2009, 11:13 AM

Search Tags


/mathhelpforum @mathhelpforum