Results 1 to 3 of 3

Math Help - Principal components analysis

  1. #1
    Newbie
    Joined
    Jul 2010
    Posts
    2

    Principal components analysis

    For the covariance matrix S it is known that all its p \times p elements are greater than zero 0. Prove that:
    a) Coefficients of the first principal component are all of the same sign,
    b) Coefficients of each other principal component cannot be all of the same sign.

    Any help greatly appreciated.

    Thanks
    Follow Math Help Forum on Facebook and Google+

  2. #2
    MHF Contributor
    Joined
    Sep 2012
    From
    Australia
    Posts
    3,607
    Thanks
    591

    Re: Principal components analysis

    Hey kungalo.

    You have your covariance matrix which is all positive, and for the first principal component you are going to solve an eigen-decomposition problem on your covariance matrix (i.e. find the eigen-vectors and eigen-values and retain the one with the highest eigen-value since this corresponds to the variance of each component). You don't have orthogonality conditions on the first component which means you are only interested in getting the one with the highest variance.

    So what this boils down to is proving that all elements in the eigen-vector corresponding to the highest eigen-value are all positive (since the covariance matrix is positive definite, it will always have positive eigenvalues).

    As for the next one, this lies on the argument dealing with orthogonality since all later components will always be orthogonal to every other principal compnent.

    The easiest thing to do is that if the first component is <a,b,c,d,....> then if something is normal you know that X = PC1 Y = PCN then <X,Y> = 0 and the only way for PCN to have this relation is for at least one of the components of PCN to be negative (we also assume PCN is not the zero vector and has positive magnitude).
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Jul 2010
    Posts
    2

    Re: Principal components analysis

    I actually worked this one out. It relies on expressing the variance of the principal component as a sum of the elements of the covariance matrix and the elements of the coefficient vector. From there it's pretty easy to see that all the coefficients need to have the same sign in order to maximize the variance of the first principal component, and that there need to be positive and negative coefficients in the second principal component in order for it to be uncorrelated with the first.
    Last edited by kungalo; October 6th 2012 at 05:02 PM.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Components
    Posted in the Calculus Forum
    Replies: 5
    Last Post: July 13th 2011, 03:42 AM
  2. Product of non principal ideals is principal.
    Posted in the Advanced Algebra Forum
    Replies: 5
    Last Post: June 12th 2011, 01:17 PM
  3. A question in principal Component Analysis
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: March 16th 2011, 12:46 AM
  4. Principal component and factor analysis
    Posted in the Advanced Algebra Forum
    Replies: 0
    Last Post: December 8th 2009, 12:47 PM
  5. Components
    Posted in the Calculus Forum
    Replies: 6
    Last Post: May 10th 2008, 02:57 PM

Search Tags


/mathhelpforum @mathhelpforum