CRLB of Amplitude Estimate in Gamma Noise

Hello everyone,

This is regarding a research problem I am working on. Trying to see if I am doing any mistake in the calculation of the CRLB.

Consider the case of a signal **s** in additive gamma noise **w **with N independent elements w_{n}

$\displaystyle \utilde{\mathbf{x}} = A\mathbf{s} + \utilde{\mathbf{w}} $

I am trying to find the CRLB for $\displaystyle \hat{A}$. I am skipping a few trivial steps...

The log-likelihood function for **x **is - $\displaystyle L(\mathbf{x};A) = \displaystyle\sum_{n=0}^{N-1}\ln\left[\frac{1}{\Gamma(k_n)\theta_n^{k_n}}\right] (k_n-1) \ln(x_n-As_n)-\frac{(x_n-As_n)}{\theta_n}\right] $

So now, $\displaystyle E\left[ \frac{\partial^2 L(\mathbf{x};A)}{\partial A^2}\right] &= E\left[ \displaystyle\sum_{n=0}^{N-1} -(k_n-1) \left(\dfrac{s_n}{x_n-As_n}\right)^2 \right] = \displaystyle\sum_{n=0}^{N-1} -(k_n-1)s_n^2\, E\left[\dfrac{1}{ \left(x_n-As_n\right)^2} \right] $

Let $\displaystyle x_n - As_n = y_n$. So,

$\displaystyle E\left[\dfrac{1}{ y_n^2} \right] &= \int_0^\infty \dfrac{1}{y_n^2} \frac{1}{\Gamma(k_n)\theta_n^{k_n}} y_n^{k_n-1} e^{-\dfrac{y_n}{\theta_n}} dy\notag $

$\displaystyle = \frac{1}{\Gamma(k_n)\theta_n^{k_n}} \int_0^\infty y_n^{k_n-3} e^{-\dfrac{y_n}{\theta_n}}dy_n \quad \mbox{, Substituting $y_n = \theta z$}\notag$

$\displaystyle = \frac{1}{\Gamma(k_n)\theta_n^{k_n}} \int_0^\infty (\theta z)^{k_n-3} e^{-z} \theta dz $

$\displaystyle = \frac{\theta_n^{k_n-2}}{\Gamma(k_n)\theta_n^{k_n}} \int_0^\infty z^{k_n-3} e^{-z} dz$

$\displaystyle = \frac{\Gamma(k_n-2)}{\Gamma(k_n)\theta_n^{2}}$

$\displaystyle = \frac{1}{(k_n-1)(k_n-2)\theta_n^2}$

The Fisher Information is,

$\displaystyle E\left[ \frac{\partial^2 L(\mathbf{x};A)}{\partial A^2}\right] = \displaystyle\sum_{n=0}^{N-1} -\frac{s_n^2} {(k_n-2)\theta_n^2}$

Hence CRLB is,

$\displaystyle var(\hat{A}) \ge \dfrac{1}{-E\left[ \frac{\partial^2 l_A(\mathbf{x})}{\partial A^2}\right]} = \dfrac{1}{\displaystyle\sum_{n=0}^{N-1} \frac{s_n^2} {(k_n-2)\theta_n^2}} $

Does this look right? I am a little unsure about the substitution $\displaystyle y_n = x_n - As_n $.

TIA for the help.

Re: CRLB of Amplitude Estimate in Gamma Noise

Hey ronbeck.

I'm wondering if you could outline the calculation for the 2nd derivative of the log-likelihood: I'm sure it's correct but I'm wondering where a lot of the simplifications come from explicitly.

If xn - Asn ~ Gamma (which it looks to be from definition), then you should have no problem finding E[1/w^2].

Re: CRLB of Amplitude Estimate in Gamma Noise

Hi Chiro,

I missed the '+' sign for $\displaystyle $L(\mathbf{x};A)$$. Anyway, here is the calculation:

The Gamma distribution is defined as: (Assuming independent noise elements $\displaystyle w_n$ in $\displaystyle \mathbf{w}$)

$\displaystyle f_{\utilde{\mathbf{x}}}(\mathbf{x}) &= \displaystyle\prod_{n=0}^{N-1}\frac{1}{\Gamma(k_n)\theta_n^{k_n}} (x_n-As_n)^{k_n-1}e^{-\frac{(x_n-As_n)}{\theta_n}} \quad \mbox{for $x_n \ge As_n$}$

So the log-likelihood function is the 'log' of the above:

$\displaystyle $L(\mathbf{x};A)$ = \displaystyle\sum_{n=0}^{N-1}\ln\left\[\frac{1}{\Gamma(k_n)\theta_n^{k_n}}\right] + (k_n-1)\ln(x_n-As_n) -\frac{(x_n-As_n)}{\theta_n}}$

$\displaystyle \implies \frac{\partial L(\mathbf{x};A)}{\partial A} = \displaystyle\sum_{n=0}^{N-1} 0 + (k_n-1) \left[\dfrac{\partial\ln(x_n-As_n) }{\partial A}\right] - \dfrac{1}{\theta_n}\dfrac{\partial (x_n-As_n)} {\partial A} $

$\displaystyle \implies \frac{\partial L(\mathbf{x};A)}{\partial A} = \displaystyle\sum_{n=0}^{N-1} \left[\dfrac{-(k_n-1)s_n}{x_n-As_n} + \dfrac{s_n}{\theta_n}\right] $

$\displaystyle \implies \frac{\partial^2 L(\mathbf{x};A)}{\partial A^2} = \displaystyle\sum_{n=0}^{N-1} -(k_n-1) \dfrac{\partial}{\partial A}\left(\dfrac{s_n}{x_n-As_n}\right) + 0 $

$\displaystyle \implies \frac{\partial^2 L(\mathbf{x};A)}{\partial A^2} = \displaystyle\sum_{n=0}^{N-1} -(k_n-1) \left(\dfrac{s_n}{x_n-As_n}\right)^2 $

I just wanted to see if I am doing the CRLB calculations right.

Thanks

Re: CRLB of Amplitude Estimate in Gamma Noise

Yeah the CRLB calculations look right to me, assuming that particular distribution (Gamma distribution) for the residual noise elements.

One way to check these kinds of things is to use simulation if you ever need some kind confirmation (but don't have anyone at your disposal to check your work), but in terms of the mathematics it looks good.

Re: CRLB of Amplitude Estimate in Gamma Noise

Thanks Chiro. I will try simulations. But an efficient estimator is needed to check the CRLB? It's a catch-22 kind of situation unless there is some simulation method that can check the CRLB without really estimating A.

Another concern is, if $\displaystyle k_n < 2 \quad \forall\, n$ then the CRLB will be negative? This doesn't make sense.

Thanks

Re: CRLB of Amplitude Estimate in Gamma Noise

I was just thinking that there may be a problem:

If sn is dependent on w and x then you can't factorize the sn outside of the expectation sign and do what you did.

You are essentially finding E[1/W^2] but the sn dependent on w which means that if you want to take expectations, you will need include the sn term and transform it if taking expectations with respect to some random variable (like W ~ Gamma). So instead of finding the normal mean you are finding the mean of a function of 1/W^2 but since Gamma is defined over the whole real positive line it means that you are going to get singularity blow-ups for various values of kn which means some won't make sense to be used.

So the PDF has to "vanish" quick enough to go to zero when you go to zero for this random variable to have a mean that makes any sense numerically.