Results 1 to 14 of 14

Math Help - Multivariate Normal

  1. #1
    Junior Member
    Joined
    Feb 2009
    Posts
    42

    Multivariate Normal

    I have a couple questions (maybe all related) and I think they are because my matrix math is weak / rusty, weak and rusty

    1) Why is Z = (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) N_p (0,I)?

    2) To say Z^2 does not mean (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) * (Covariance Matrix)^(-1/2) * (X_vector - mu_vector). Instead the order changes to:

    (X_vector - mu_vector)' * (Covariance Matrix)^(-1/2)*(Covariance Matrix)^(-1/2)* (X_vector - mu_vector).

    I understand that for the calculations to work, we need to have the dimensions work, but why is the order the way it is (and how do you know?)


    3) Finally trying to show sqrt(n) * (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) is N_p (0,I).


    I bet all these show the same deficiency I have but if anyone can help explain I would be grateful. Thanks!

    Brian
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Moo
    Moo is offline
    A Cute Angle Moo's Avatar
    Joined
    Mar 2008
    From
    P(I'm here)=1/3, P(I'm there)=t+1/3
    Posts
    5,618
    Thanks
    6
    Hello,

    I'm quite skeptical about your notations... What is "(covariance matrix)^(-1/2)" ? Is it the determinant of the covariance matrix ?
    In this case, I don't see where
    Z = (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) N_p (0,I)
    comes from

    2) To say Z^2 does not mean (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) * (Covariance Matrix)^(-1/2) * (X_vector - mu_vector). Instead the order changes to:

    (X_vector - mu_vector)' * (Covariance Matrix)^(-1/2)*(Covariance Matrix)^(-1/2)* (X_vector - mu_vector).
    But Z^2 doesn't follow a normal distribution ??
    Also, what is the square of a vector ?

    3) Finally trying to show sqrt(n) * (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) is N_p (0,I).
    It's rather "goes to N_p (0,I)" as n goes to infinity ?
    But still, I don't agree with that formula...

    It looks like the multidimensional central limit theorem. And unlike the unidimensional theorem, we cannot divide by the "standard deviation" (that you would have interpreted as (Covariance matrix)^(-1/2)) to get a "standard" normal distribution...

    Where did you take these formulae from ?
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Junior Member
    Joined
    Feb 2009
    Posts
    42
    Hello.

    First: (covariance matrix)^(-1/2) is the inverse of the square root matrix of the covariance matrix. I dont know how to type in this forum with symbols. When I type N_p(O,I) this is a multivariate normal distribution in p variables with mean = the zero vector and variance equal to the Identity matrix.

    Second: The book shows shows Z^2 =
    (X_vector - mu_vector)' * (Covariance Matrix)^(-1/2)*(Covariance Matrix)^(-1/2)* (X_vector - mu_vector). And then notes it is distributed chi-square.

    where Z is (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) N_p (0,I)

    My question is how do you know the order to set up the multiplication to get Z^2?

    Third : Sorry, I mistyped! Should be:

    3) Finally trying to show sqrt(n) * (Covariance Matrix)^(-1/2) * (SampleMean_vector - mu_vector) is N_p (0,I).


    Applied Multivariate Statistical Analysis by Johnson and Wichern

    Are you able to help?
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Moo
    Moo is offline
    A Cute Angle Moo's Avatar
    Joined
    Mar 2008
    From
    P(I'm here)=1/3, P(I'm there)=t+1/3
    Posts
    5,618
    Thanks
    6
    Quote Originally Posted by B_Miner View Post
    Hello.

    First: (covariance matrix)^(-1/2) is the inverse of the square root matrix of the covariance matrix. I dont know how to type in this forum with symbols. When I type N_p(O,I) this is a multivariate normal distribution in p variables with mean = the zero vector and variance equal to the Identity matrix.
    Okay, I'm sorry. I just didn't know the concept of square root matrix.

    For the first question.
    Let K denote the covariance matrix of the Gaussian vector X.
    X \sim \mathcal{N}_p(\mu,K) \Rightarrow X-\mu \sim\mathcal{N}_p(0,K) (0 is the zero column vector of dimension p)

    Now, the key step is explained in the PDF I've attached.

    Y=(K^{1/2})^{-1} (X-\mu) has a 0 mean vector and a covariance matrix equal to :
    (K^{1/2})^{-1}K\left((K^{1/2})^{-1}\right)^T

    Now, we know that a covariance matrix is symmetric. I've read in the wikipedia that this implies that a square root matrix of K will then be symmetric. So K^{1/2} is symmetric.
    And so is its inverse (that should be some property from algebra lol).

    Hence (K^{1/2})^{-1}=\left((K^{1/2})^{-1}\right)^T

    So the covariance matrix of Y is (K^{1/2})^{-1}K(K^{1/2})^{-1}

    But we have K^{1/2}K^{1/2}=K. Left multiply by (K^{1/2})^{-1} :
    K^{1/2}=(K^{1/2})^{-1}K

    So finally, the covariance matrix of Y is K^{1/2}(K^{1/2})^{-1}=I\quad\quad\quad \blacksquare




    Third : Sorry, I mistyped! Should be:

    3) Finally trying to show sqrt(n) * (Covariance Matrix)^(-1/2) * (SampleMean_vector - mu_vector) is N_p (0,I).


    Applied Multivariate Statistical Analysis by Johnson and Wichern

    Are you able to help?
    The sample mean vector is (let's call it Z) Z=\frac 1n \sum_{i=1}^n X_i.

    Z-\mu=\frac 1n\sum_{i=1}^n (X_i-\mu)=\frac 1n\sum_{i=1}^n Z_i, where the Z_i are independent and follow a normal distribution \mathcal{N}_p(0,K)

    Now, I must say I'm a bit lost. I need to give it a thought later on...
    Second: The book shows shows Z^2 =
    (X_vector - mu_vector)' * (Covariance Matrix)^(-1/2)*(Covariance Matrix)^(-1/2)* (X_vector - mu_vector). And then notes it is distributed chi-square.

    where Z is (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) N_p (0,I)

    My question is how do you know the order to set up the multiplication to get Z^2?
    Same problem here. Need to think on that later on...

    Maybe you'll be able to answer your questions if, as you said, these questions are completely related.
    It is hard for me to use square root matrices since I've known this for 30minutes only So I feel quite uneasy
    Attached Files Attached Files
    Follow Math Help Forum on Facebook and Google+

  5. #5
    MHF Contributor matheagle's Avatar
    Joined
    Feb 2009
    Posts
    2,763
    Thanks
    5
    I figured you meant the sample mean there.
    This can be found online under Affine transformation in Multivariate normal distribution - Wikipedia, the free encyclopedia
    Last edited by matheagle; June 2nd 2009 at 12:23 AM. Reason: moo
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Junior Member
    Joined
    Feb 2009
    Posts
    42
    Hi MathEagle.

    I knew of that general result. Could you help me to see how to use it to see why, for example:

    Z = (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) ~ N_p (0,I)
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Moo
    Moo is offline
    A Cute Angle Moo's Avatar
    Joined
    Mar 2008
    From
    P(I'm here)=1/3, P(I'm there)=t+1/3
    Posts
    5,618
    Thanks
    6
    Quote Originally Posted by B_Miner View Post
    Hi MathEagle.

    I knew of that general result. Could you help me to see how to use it to see why, for example:

    Z = (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) ~ N_p (0,I)
    I showed you why !!!
    Follow Math Help Forum on Facebook and Google+

  8. #8
    MHF Contributor matheagle's Avatar
    Joined
    Feb 2009
    Posts
    2,763
    Thanks
    5
    I thought Moo helped you.
    Meanwhile she's giving me an earful via the messages.
    You should thank her and she should stop barking, or mooing?
    My ear is going to fall off.
    Follow Math Help Forum on Facebook and Google+

  9. #9
    MHF Contributor matheagle's Avatar
    Joined
    Feb 2009
    Posts
    2,763
    Thanks
    5
    Quote Originally Posted by B_Miner View Post
    Hi MathEagle.

    I knew of that general result. Could you help me to see how to use it to see why, for example:

    Z = (Covariance Matrix)^(-1/2) * (X_vector - mu_vector) ~ N_p (0,I)

    (1) Moo showed that this is normal, which you should have known

    (2) Next E((X_vector - mu_vector))=0 (vector 0).
    So a multiple of it is still zero

    E(Covariance Matrix)^(-1/2) * (X_vector - mu_vector) =0

    (3) V((Covariance Matrix)^(-1/2) * (X_vector - mu_vector))=I
    via the algebra at wikipedia.
    You can kill the mean and just show
    V((Covariance Matrix)^(-1/2) * (X_vector))=I

    (2) and (3) follow even without normality.
    Last edited by matheagle; June 1st 2009 at 08:43 AM.
    Follow Math Help Forum on Facebook and Google+

  10. #10
    Junior Member
    Joined
    Feb 2009
    Posts
    42
    Sorry Moo. I did not understand your solution, it was beyond me. The wiki piece on affine transformation looksed more to my understanding level.
    Follow Math Help Forum on Facebook and Google+

  11. #11
    Moo
    Moo is offline
    A Cute Angle Moo's Avatar
    Joined
    Mar 2008
    From
    P(I'm here)=1/3, P(I'm there)=t+1/3
    Posts
    5,618
    Thanks
    6
    Quote Originally Posted by B_Miner View Post
    Sorry Moo. I did not understand your solution, it was beyond me. The wiki piece on affine transformation looksed more to my understanding level.
    Then next time, say it !!!!!

    It would have been far better to say it rather than completely ignore it !!!!!!!


    For your information, I exactly used the formula in the Wikipedia.
    Then, I just used operations over matrices.
    Point out what you didn't understand. IT IS NOT POSSIBLE TO HELP OTHERWISE
    Follow Math Help Forum on Facebook and Google+

  12. #12
    Junior Member
    Joined
    Feb 2009
    Posts
    42
    For E(Covariance Matrix)^(-1/2) * (X_vector - mu_vector) =0

    are you assuming Covariance Matrix)^(-1/2) and (X_vector - mu_vector) are independent so that you can do
    E(Covariance Matrix)^(-1/2) ) * E((X_vector - mu_vector))
    =E(Covariance Matrix)^(-1/2) ) * 0
    ?
    Follow Math Help Forum on Facebook and Google+

  13. #13
    Junior Member
    Joined
    Feb 2009
    Posts
    42
    Moo-

    Wow. No need for all that. I appreciate your help. I was trying to read the posts while at work and as I was finishing yours I read MathEagles. I did not have time to yet respond to yours. Thanks for your help though.
    Follow Math Help Forum on Facebook and Google+

  14. #14
    Flow Master
    mr fantastic's Avatar
    Joined
    Dec 2007
    From
    Zeitgeist
    Posts
    16,948
    Thanks
    5
    And on that note boys, girls and bovines .... Thread closed (otherwise someone's going to lose an eye).
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Multivariate normal distribution
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: December 20th 2010, 11:28 AM
  2. Multivariate normal Distrubution
    Posted in the Advanced Statistics Forum
    Replies: 3
    Last Post: August 31st 2010, 01:00 PM
  3. Multivariate Normal Distribution
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: November 22nd 2009, 03:43 PM
  4. Multivariate Normal Distribution
    Posted in the Advanced Statistics Forum
    Replies: 6
    Last Post: October 16th 2009, 10:01 AM
  5. Proving - multivariate normal distribution
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: November 15th 2008, 01:01 AM

Search Tags


/mathhelpforum @mathhelpforum