# Thread: Help Understanding MGFs and Transformations

1. ## Help Understanding MGFs and Transformations

I need some help in understanding moment generating functions and transformations of continuous random variables. I've found various webpages that provide pieces of the puzzle, but I can't find anything that explains the big picture in terms that are easy to understand for someone who isn't very mathematically inclined. If anyone can provide links to material that can help explain what MGFs are and how to do transformations I would be very happy.

And although I do need to gain an overall understanding of these concepts, there are two specific problems I am wrestling with right now. The first is finding the variance of the geometric distribution by taking the natural log of the moment generating function. According to The Geometric Distribution, the variance for a geometric series should be q/p^2. I have calculated the MGF to be ((p*e^t)/(1-q*e^t)), however, when I calculate the derivative of ln[((p*e^t)/(1-q*e^t))] I get 1/(1-q*e^t), so even when t=0 the value I get for variance is 1/(1-q), when it should be q/p^2. I would appreciate any insight into what I am doing wrong here.

The second problem I keep butting my head up against is how to calculate the density function U of two different density functions X and Y. I dont' even have an idea of how to approach this problem, so I would love any guidance here.

2. Originally Posted by doopokko
I need some help in understanding moment generating functions and transformations of continuous random variables. I've found various webpages that provide pieces of the puzzle, but I can't find anything that explains the big picture in terms that are easy to understand for someone who isn't very mathematically inclined. If anyone can provide links to material that can help explain what MGFs are and how to do transformations I would be very happy.

And although I do need to gain an overall understanding of these concepts, there are two specific problems I am wrestling with right now. The first is finding the variance of the geometric distribution by taking the natural log of the moment generating function. According to The Geometric Distribution, the variance for a geometric series should be q/p^2. I have calculated the MGF to be ((p*e^t)/(1-q*e^t)), however, when I calculate the derivative of ln[((p*e^t)/(1-q*e^t))] I get 1/(1-q*e^t), so even when t=0 the value I get for variance is 1/(1-q), when it should be q/p^2. I would appreciate any insight into what I am doing wrong here.

Mr F says: You're probably making mistakes in either or both of the following:

1. The calculation of the derivatives of the MGF. Note that $Var(X) = E(X^2) - [E(X)]^2$ so you need to calculate the first and second derivatives of the MGF and evaluate each at t = 0. Taking the logarithm of the MGF is merely a way of simplifying the calculation of the derivatives of the MGF.

2. The simplification of $Var(X) = E(X^2) - [E(X)]^2$.

Without seeing the details of your work it's impossible to know where your mistakes are.

The second problem I keep butting my head up against is how to calculate the density function U of two different density functions X and Y. I dont' even have an idea of how to approach this problem, so I would love any guidance here.

Mr F says: Post the question you can't do.
..

3. Originally Posted by doopokko
I need some help in understanding moment generating functions and transformations of continuous random variables. I've found various webpages that provide pieces of the puzzle, but I can't find anything that explains the big picture in terms that are easy to understand for someone who isn't very mathematically inclined. If anyone can provide links to material that can help explain what MGFs are and how to do transformations I would be very happy.

And although I do need to gain an overall understanding of these concepts, there are two specific problems I am wrestling with right now. The first is finding the variance of the geometric distribution by taking the natural log of the moment generating function. According to The Geometric Distribution, the variance for a geometric series should be q/p^2. I have calculated the MGF to be ((p*e^t)/(1-q*e^t)), however, when I calculate the derivative of ln[((p*e^t)/(1-q*e^t))] I get 1/(1-q*e^t), so even when t=0 the value I get for variance is 1/(1-q), when it should be q/p^2. I would appreciate any insight into what I am doing wrong here.

The second problem I keep butting my head up against is how to calculate the density function U of two different density functions X and Y. I dont' even have an idea of how to approach this problem, so I would love any guidance here.
$m(t) = \frac{p e^t}{1 - q e^t} \Rightarrow \ln m(t) = \ln (p e^t) - \ln (1 - q e^t) = \ln (p) + t - \ln (1 - q e^t)$.

Therefore $\frac{d \ln m(t)}{dt} = \frac{
m'(t)}{m(t)} = 1 + \frac{q e^t}{1 - q e^t} \Rightarrow m'(t) = m(t) \left( 1 + \frac{q e^t}{1 - q e^t}\right)$
.

Therefore $m'(0) = m(0) \left( 1 + \frac{q}{1 - q}\right) = \frac{p}{1 - q} \left( \frac{1}{1 - q} \right) = \frac{1}{p}$, as expected.

Now you need to calculate $E(X^2) = m''(0)$ and substitute the results into the formula for the variance (see post #2).

Note that $m'(t) = m(t) \left( 1 + \frac{q e^t}{1 - q e^t}\right) = \frac{p e^t}{\left( 1 - q e^t\right)^2}$.

For someone "who isn't very mathematically inclined" you have chosen a difficult subject to study ....

4. First Problem:

"Differentiate the moment-generating function $m(t) = (pe^t)/(1-qe^t)$ to find $E(Y)$ and $E(Y^2)$. Then find $V(Y)$ directly using derivatives of $ln m(t)$."

I calculated the derivative of the MGF given above as $(pe^t)/(1-qe^t)^2$. Then by setting $t=0$ I found that $E(Y)=1/p$, which is correct according to The Geometric Distribution.

I don't really understand how to use the derivatives of the log of $m(t)$ to find $V(Y)$. I tried just taking the derivative of $ln(m(t))$, like the wording of the problem seemed to suggest. This is $1/(1-qe^t)$, and if you let $t=0$ that evaluates to $1/(1-q)$ which doesn't match the expected variance of $q/p^2$ as given by The Geometric Distribution.

What is the technique to calculate $V(Y)$ directly from the derivatives of $ln m(t)$?

5. Second Problem:

"In a process of sintering (heating) two types of copper powder, the density function for Y1, the volume proportion of solid copper in a sample, was given by
$f1(y1) = 6(y1)(1-(y1))$ when $0<=(y1)<=1$, and 0 elsewhere.

The density function for Y2, the proportion of type A crystals among the solid copper, was given as
$f2(y2) = 3(y2)^2$ when $0<=(y2)<=1$, and 0 elsewhere.

The variable $U=(Y1)(Y2)$ gives the proprtion of the sample volume due to type A crystals. If Y1 and Y2 are independent, find the probability density function for U."

I don't really even know how to approach this problem. I would appreciate any advice on how to tackle it.

6. Originally Posted by mr fantastic
$m(t) = \frac{p e^t}{1 - q e^t} \Rightarrow \ln m(t) = \ln (p e^t) - \ln (1 - q e^t) = \ln (p) + t - \ln (1 - q e^t)$.

Therefore $\frac{d \ln m(t)}{dt} = \frac{
m'(t)}{m(t)} = 1 + \frac{q e^t}{1 - q e^t} \Rightarrow m'(t) = m(t) \left( 1 + \frac{q e^t}{1 - q e^t}\right)$
.

Therefore $m'(0) = m(0) \left( 1 + \frac{q}{1 - q}\right) = \frac{p}{1 - q} \left( \frac{1}{1 - q} \right) = \frac{1}{p}$, as expected.

Now you need to calculate $E(X^2) = m''(0)$ and substitute the results into the formula for the variance (see post #2).

For someone "who isn't very mathematically inclined" you have chosen a difficult subject to study ....

Thanks. Sorry, I made post#4 above in response to #2, before I saw #3.

In regards to pstat being a difficult subject to study, I agree whole-heartedly. I haven't chosen to study it, however. I am a computer science student, and this course is required for my major. I'm very greatful that there are helpful people like you out there to walk me through some of the harder parts.

7. Originally Posted by doopokko
Second Problem:

"In a process of sintering (heating) two types of copper powder, the density function for Y1, the volume proportion of solid copper in a sample, was given by
$f1(y1) = 6(y1)(1-(y1))$ when $0<=(y1)<=1$, and 0 elsewhere.

The density function for Y2, the proportion of type A crystals among the solid copper, was given as
$f2(y2) = 3(y2)^2$ when $0<=(y2)<=1$, and 0 elsewhere.

The variable $U=(Y1)(Y2)$ gives the proprtion of the sample volume due to type A crystals. If Y1 and Y2 are independent, find the probability density function for U."

I don't really even know how to approach this problem. I would appreciate any advice on how to tackle it.
Since Y1 and Y2 are independent their joint density function is given by $f(y_1, y_2) = [6y_1 (1 - y_1)] [3 y_2^2] = 18 y_1 (1 - y_1) y_2^2$ for $0 \leq y_1 \leq 1, ~ 0 \leq y_2 \leq 1$ and zero elsewhere.

Calculate the cumulative density function of U:

$F(u) = \Pr(U < u) = \Pr(Y_1 Y_2 < u) = \Pr\left(Y_1 < \frac{u}{Y_2} \right)$ for $0 \leq u \leq 1$

$= \int_{y_2 = 0}^{y_2 = u} \int_{y_1 =0}^{y_1 = 1} 18 y_1 (1 - y_1) y_2^2 \, dy_1 \, dy_2 + \int_{y_2 = u}^{y_2 = 1} \int_{y_1 =0}^{y_1 = u/y_2} 18 y_1 (1 - y_1) y_2^2 \, dy_1 \, dy_2$

(you'll need to draw a diagram to see where the integral terminals come from)

$= \left[y_2^3\right]_0^u \left[3 y_1^2 - 2 y_1^3\right]_0^1 + \int_{y_2 = u}^{y_2 = 1} 3 y_2^2 \left[3y_1^2 - 2y_1^3\right]_0^{u/y_2} \, dy_2$

$= u^3 + \int_{y_2 = u}^{y_2 = 1} 3 y_2^2 \left( \frac{3 u^2}{y_2^2} - \frac{2u^3}{y_2^3} \right) \, dy_2$

$= u^3 + \int_{y_2 = u}^{y_2 = 1} 9u^2 - \frac{6u^3}{y_2} \, dy_2$

$= u^3 + 9u^2 - 9u^3 -6u^3 \left[ \ln y_2 \right]_u^1$

$= 9u^2 - 8u^3 -6u^3 \ln u$.

To summarise: $F(u) = 9u^2 - 8u^3 -6u^3 \ln u$ for $0 \leq u \leq 1$.

Then the pdf of U is given by $f(u) = \frac{dF}{du} = 18u - 24u^2 - (18u^2 \ln u + 6u^2)$ $= 18u -30u^2 - 18u^2 \ln u$ for $0 \leq u \leq 1$ and zero elsewhere.

8. Originally Posted by doopokko
First Problem:

"Differentiate the moment-generating function $m(t) = (pe^t)/(1-qe^t)$ to find $E(Y)$ and $E(Y^2)$. Then find $V(Y)$ directly using derivatives of $ln m(t)$."

I calculated the derivative of the MGF given above as $(pe^t)/(1-qe^t)^2$. Then by setting $t=0$ I found that $E(Y)=1/p$, which is correct according to The Geometric Distribution.

I don't really understand how to use the derivatives of the log of $m(t)$ to find $V(Y)$. I tried just taking the derivative of $ln(m(t))$, like the wording of the problem seemed to suggest. This is $1/(1-qe^t)$, and if you let $t=0$ that evaluates to $1/(1-q)$ which doesn't match the expected variance of $q/p^2$ as given by The Geometric Distribution.

What is the technique to calculate $V(Y)$ directly from the derivatives of $ln m(t)$?
The first derivative of $\ln m(t)$ evaluated at t = 0 gives the mean.
The second derivative of $\ln m(t)$ evaluated at t = 0 gives the variance.