Results 1 to 4 of 4

Math Help - Measuring model uncertainty

  1. #1
    Newbie
    Joined
    Apr 2009
    Posts
    7

    Question Measuring model uncertainty

    I have a model with a number of parameters. I can calculate the posterior distribution of the parameter vector. This, in part, means that I can calculate the expected value and the variance of each parameter.

    Question. How do I calculate model uncertainty as a single quantity? While I have some ad hoc ideas, I'd like to take a principled approach.

    One idea is to look at the variance of the likelihood: Var( L(\theta) ) ? I'm not sure how to approach this, other than numerically.

    Another idea is to compute something like this:
    \sum_i \left | \frac {\partial L} {\partial \theta_i} \right | \sigma_i

    Any and all help appreciated.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Newbie
    Joined
    Apr 2009
    From
    US
    Posts
    16
    hi,
    I think you are thinking too complicated, you know already the variances of each parameter. If parameters are independent then system uncertainty would be the sum of the variances of each parameter. If they are correlated you need to take into the account the correlation factor too. Hope this will help. Merry Christmas.
    Sincerely
    Hametceq
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Apr 2009
    Posts
    7
    I thought of the sum of the variances at first. But I do not understand why that should be the correct approach. Why not the sum of the standard deviations, for example?

    Also, I do think that the likelihood should figure somehow into the calculation, since different parameters might have different importance in the model. An important parameter having a low variance and an unimportant one having a high variance should be better than the important parameter having a high variance and an unimportant one having a low variance.

    As an extreme example, consider a parameter that has no influence on the model at all. Then its variance (or standard deviation) should not be figured into the sum. That's why I thought of weighing the standard deviations by the derivative of the likelihood with respect to the parameters. In the case of the parameter that has no influence on the model, the derivative would be zero.
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Newbie
    Joined
    Apr 2009
    From
    US
    Posts
    16
    I see what is the question now, OK do the following,
    Strategy 1:
    STEP1: find the covariance matrix for the all parameters.
    STEP2: find the generalized variance (determinant)

    Strategy 2 (detailed analysis, I will do this for my analysis)
    STEP1: find the covariance matrix for the all parameters.
    STEP2: find eigenvalues and eigenvectors for that matrix (you can not set by yourself which one is less or more important so multivariate analysis should be applied).
    STEP4: eliminate less important eigenvalues and correspondingly you will reduce dimension.
    STEP5: find the generalized variance from the covariance matrix from the important parameters.


    Merry Christmas
    Sincerely
    Hametceq

    p.s.
    Q:Why not the sum of the standard deviations, for example?
    Ans: Not for you case, but generally because Variances are additive for independent r.v.s, but SDs are not.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Measuring R^2 through the origin
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: November 27th 2010, 03:06 PM
  2. Replies: 3
    Last Post: May 28th 2010, 03:18 PM
  3. Measuring the reals
    Posted in the Differential Geometry Forum
    Replies: 4
    Last Post: December 18th 2009, 07:02 AM
  4. Replies: 3
    Last Post: April 15th 2009, 02:18 AM
  5. Finding Absolute Uncertainty From Relative Uncertainty
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: January 16th 2009, 10:16 AM

Search Tags


/mathhelpforum @mathhelpforum