# Thread: Application of Bayes Theorem

1. ## Application of Bayes Theorem

This is from a water resources paper that I'm reading that uses Bayesian inference. It is a simple problem, but I don't know if I am missing something.

Assume that you have a vector of imperfect observations Xo of a true unknown variable Xt and that the errors e are additive:

Xo = Xt + e (EQ 1)

I would like to compute the conditional probability p(Xo|Xt) for the case that the variable is exact.

The solution is given in the paper as:

p(Xo|Xt) = e (Xo - Xt) (EQ 2)

So according to Bayes Theorem:

p(Xo|Xt) = p(Xt|Xo) * p(Xo) / p(Xt) (EQ 3)

If the variable is exact I assume that it means that the following conditional probability is equal to one: given an observation Xo, the probability that it is equal to its true value Xt. That is:

p(Xt|Xo) = 1 (EQ 4)

That leaves me with the ratio of the priors:

p(Xo|Xt) = 1 * p(Xo) / p(Xt) (EQ 5)

I can substitute EQ 1 into EQ 5:

p(Xo|Xt) = 1 * p(Xt + e) / p(Xt) (EQ 6)

This is where I have problems. First of all, I know that there is a possibility of observing the Xo that gives as a result the corresponding true value Xt with perfect accuracy since the variable is assumed to be exact. But according to Bayes Theorem also exists the possibility that measuring not Xo (~Xo) would result also in the true value Xt, that is: a false positive? In other examples it is easier for me to understand the concept of ~Xo, but what does it really mean that we are observing ~Xo rather than Xo? Secondly I know that the distribution of the sum of two random variables (such as in the numerator of EQ 6) is a convolution of the individual pdfs and therefore depends on the statistical distribution of the original variables. Is this how this should be solved or the solution is simpler?

2. Originally Posted by hydrodelta
I would like to compute the conditional probability p(Xo|Xt) for the case that the variable is exact.
What does this mean? That e=0?

3. Originally Posted by pedrosorio
What does this mean? That e=0?
Thanks for your interest in this post.

At first, that's what I thought that the term "exact" meant, but note that the solution in the paper is not zero, e is still different than zero. The way I interpret that a variable is exact is that there is a one to one correspondence between certain observed values Xo with their true value Xt, but this doesn't invalidate the fact that there are also certain cases (the probability is not zero) when we observe something different than Xo and we still get the correct aswer Xt. I realize that this second part may sound confusing and perhaps is wrong, but that's how I'm trying to interpret the problem.

4. p(Xt|Xo) = 1 (EQ 4)
Here what you mean is:

$p(Xt|Xo)=\left\{\begin{array}{cc}1,&\mbox{ if }
Xt=Xo\\0,&\mbox{otherwise}\end{array}\right.$

but this is only valid when e=0.

Don't you have any info about the errors?

5. Originally Posted by pedrosorio
Here what you mean is:

$p(Xt|Xo)=\left\{\begin{array}{cc}1,&\mbox{ if }
Xt=Xo\\0,&\mbox{otherwise}\end{array}\right.$

but this is only valid when e=0.

Don't you have any info about the errors?

Yes, the way I understand this is that there is a probability that we observe the value Xo that results in Xo=Xt. Since the variable is exact I would assume that the probability of observing Xo is then equal to the probability that e=0 (which in this case is not 1).

But also there is a posibility that we don't observe Xo, but other value ~Xo. A fraction of the other values not equal to Xo (e.g. ~Xo) result in Xt.

It is easier for me to understand or explain this idea using a conceptual pie chart:
Public - Windows Live
In the chart shown the yellow exploded slice represents the probability that we observe the value Xo that is equal to Xt and therefore for this case e=0. The rest of the pie (red+magenta) is the probability that we observe something different than Xo (~Xo). Within this probability the magenta region corresponds to the probability that we observe ~Xo, but still get the correct answer Xt. The red region therefore represents the probability that we observe ~Xo but we get ~Xt.

As you can see from this graph we can derive another form of the Bayes Theorem:

p(Xo|Xt) = p(Xt|Xo)*p(Xo) / (p(Xt|Xo)*p(Xo)+p(~Xo)*p(Xt|~Xo))

in other words we get the expression that we have before:

p(Xo|Xt) = p(Xt|Xo)*p(Xo) /p(Xt)

The authors mention they selected the additive model thinking in using normally distributed errors (probably with zero mean) later on. However, nothing in their analysis up to this point has mentioned this assumption.

Thanks again.