Linear combination of normal random variables

**kingwinner** · May 20th 2009, 10:53 PM

"Fact: Any linear combination of independent normal random variables has a normal distribution."

Is the condition "independent" here absolutely necessary? If we remove the word "independent", would the linear combination still be normally distributed? Why or why not?

Thank you!

**matheagle** · May 20th 2009, 10:56 PM

yes you need indep...

Let X be any normal rv.
Then Y=X-X is not a normal.
P(Y=0)=1 is not normal.

**kingwinner** · May 21st 2009, 12:05 AM

Say if we have X~Normal(0,1), Y~Normal(0,1)

E(X+Y) = E(X)+E(Y) = 0
V(X+Y) = V(X)+V(Y)+2Cov(X,Y) = 2+2Cov(X,Y)

Can we now say that X+Y~Normal(0, 2+2Cov(X,Y) ) ?

Thanks!

**Isomorphism** · May 21st 2009, 01:12 AM

Originally Posted by kingwinner

Say if we have X~Normal(0,1), Y~Normal(0,1)

E(X+Y) = E(X)+E(Y) = 0
V(X+Y) = V(X)+V(Y)+2Cov(X,Y) = 2+2Cov(X,Y)

Can we now say that X+Y~Normal(0, 2+2Cov(X,Y) ) ?

Thanks!

No!

(This counterexample is wrong, see Laurents post below) Counterexample:Lets say your r.v pair X,Y has a joint char function $\phi(s_1,s_2) = \mathbb{E}(e^{s_1X+s_2Y}) = \text{exp}\left({\frac{s_1 ^2 + s_2 ^2}2 + s_1s_2}\right)$ . Clearly they are marginally gaussian as you require and the sum distribution is not gaussian.

However if your question is if "independence" is necessary,then its not. Counterexample: What if X = 3Z+Y and where Z and Y are independent normal?

**kingwinner** · May 21st 2009, 02:12 AM

Unfortunately, I haven't learnt joint characteristic function in my class yet (I've learnt moment generating functions, however), so I don't have the required background to understand your example. I am sorry about that.

What if X = 3Z+Y and where Z and Y are independent normal?

Since they are indepednent, X must be normally distributed as well. Now my question is, if the random variables are NOT indepednent (see my example below), would a linear combination of those random variables also be normally distributed?

In short, my question is:
Suppose that X~Normal(0,1), Y~Normal(0,1), where X and Y are NOT independent
Can we say that X+Y~Normal(0, 2+2Cov(X,Y) ) for sure?

Thanks!

**Isomorphism** · May 21st 2009, 02:50 AM

Originally Posted by kingwinner

Since they are indepednent, X must be normally distributed as well.

You didnt get my point. I wanted to say X+Y is normal distributed even though X and Y are not independent.

Now my question is, if the random variables are NOT indepednent (see my example below), would a linear combination of those random variables also be normally distributed?

In short, my question is:
Suppose that X~Normal(0,1), Y~Normal(0,1), where X and Y are NOT independent
Can we say that X+Y~Normal(0, 2+2Cov(X,Y) ) for sure?

Thanks!

My previous post is trying to tell you that there is no general answer. Some times the sum could be normal, sometimes not.

**Laurent** · May 21st 2009, 04:14 AM

Originally Posted by Isomorphism

No!

Counterexample:Lets say your r.v pair X,Y has a joint char function $\phi(s_1,s_2) = \mathbb{E}(e^{s_1X+s_2Y}) = \text{exp}\left({\frac{s_1 ^2 + s_2 ^2}2 + s_1s_2}\right)$ . Clearly they are marginally gaussian as you require and the sum distribution is not gaussian.

To be precise, what you're dealing with is rather a moment generating function (kind of Laplace transform) than a characteristic function (kind of Fourier transform).
By the way, it is not easy to prove that a function is a moment generating function for some probability distribution (it involves Bochner's theorem, which is uneasy to check in general), so that you should prove first that your function is indeed a m.g.f.. Actually it is a m.g.function, because this is that of a Gaussian vector...

Marginals are indeed standard Gaussian r.v., but the m.g.f. of the sum is $\mathbb{E}[e^{s(X+Y)}]=\exp(2s^2)$ (take $s_1=s_2=s$ ), which is the m.g.f. of a centered Gaussian with variance 4... Hence this is no counterexample.

Matheagle gave a working counterexample. Since Dirac measures can be seen as "limit cases" of Gaussian distribution when the variance goes to 0, I tend to prefer the following one: Let $X,\varepsilon$ be independent r.v. where $X$ is a standard Gaussian r.v., and $\varepsilon$ has a distribution given by $P(\varepsilon=+1)=P(\varepsilon=-1)=1/2$ . Then let $Y=\varepsilon X$ , so that $Y$ is a standard Gaussian, while $X+Y=(1+\varepsilon)X$ is 0 with probability 1/2, hence it is not Gaussian, and it is not degenerate.

**Isomorphism** · May 21st 2009, 05:21 AM

Originally Posted by Laurent

To be precise, what you're dealing with is rather a moment generating function (kind of Laplace transform) than a characteristic function (kind of Fourier transform).

By the way, it is not easy to prove that a function is a moment generating function for some probability distribution (it involves Bochner's theorem, which is uneasy to check in general), so that you should prove first that your function is indeed a m.g.f.. Actually it is a m.g.function, because this is that of a Gaussian vector...

Marginals are indeed standard Gaussian r.v., but the m.g.f. of the sum is $\mathbb{E}[e^{s(X+Y)}]=\exp(2s^2)$ (take $s_1=s_2=s$ ), which is the m.g.f. of a centered Gaussian with variance 4... Hence this is no counterexample.

Whoops!

Perhaps I should have gone with my standard counter $\phi(s_1,s_2) = \mathbb{E}(e^{s_1X+s_2Y}) = (1 + s_1s_2)\text{exp}\left({\frac{s_1 ^2 + s_2 ^2}2}\right)<br />$

In my basic random process classes these examples were considered good enough. So does this new example confirm to the rigorous approach?

Your counterexample is nice

**Laurent** · May 21st 2009, 06:20 AM

Originally Posted by Isomorphism

Whoops!

Perhaps I should have gone with my standard counter $\phi(s_1,s_2) = \mathbb{E}(e^{s_1X+s_2Y}) = (1 + s_1s_2)\text{exp}\left({\frac{s_1 ^2 + s_2 ^2}2}\right)<br />$

Where do you get this example from? If my computations are correct, this still doesn't work, because what you gave seems to be the m.g.f. of the "density" $f(x,y)=\frac{1}{2\pi}(1+xy)\exp\left(-\frac{x^2+y^2}{2}\right)$ on $\mathbb{R}^2$ , which is negative sometimes... This illustrates what I said about Bochner's theorem in my last post: not any function, even if it equals 1 at 0 and is "smooth", is a m.g.f.. Being a m.g.f. is a strong condition that can't be checked at first sight.

**matheagle** · May 21st 2009, 06:42 AM

Again, if Y=-X, then W=X+Y is not normal. P(W=0)=1.
The negative of a N(0,1) is a N(0,1), so both X and Y are st normals, but their sum is not a normal, not even continuous.

**Moo** · May 21st 2009, 12:10 PM

However if your question is if "independence" is necessary,then its not. Counterexample: What if X = 3Z+Y and where Z and Y are independent normal?

You didnt get my point. I wanted to say X+Y is normal distributed even though X and Y are not independent.

Then X=3Z-Y would be a better example

However, he talked about independence for "any linear combination" of normal distribution. Not for X+Y in particular.
With Gaussian vectors, we were told that $X_1,\dots,X_n$ form a Gaussian vector (meaning that any linear combination of its components follows a normal distribution) if (and only if ?) they're independent.

**matheagle** · May 21st 2009, 01:51 PM

He's going to keep asking this question until the COWS (MOOO) come home.
He knows the answer by now.

**kingwinner** · May 23rd 2009, 12:44 PM

How about this?
"ANY linear combination of uncorrelated normal random variables has a normal distribution."

Is this a correct statement? Why or why not?

Thanks!

**Laurent** · May 23rd 2009, 01:57 PM

Originally Posted by kingwinner

How about this?
"ANY linear combination of uncorrelated normal random variables has a normal distribution."

Is this a correct statement? Why or why not?

Thanks!

Did you check on our counterexamples first? In mine (post #7), the r.v. $X$ and $Y$ are uncorrelated.

**matheagle** · May 23rd 2009, 02:30 PM

I guess the cows aren't home yet.

Thread: Linear combination of normal random variables

LinkBack

Thread Tools

Display

Linear combination of normal random variables

Similar Math Help Forum Discussions

Expected Value of a Combination of Random Variables

Linear combination of poisson random variables

Variance of a combination of normal random variables

linear combination of Poisson random variables

linear combination of chi-squared variables in R

Search Tags