Results 1 to 6 of 6

Math Help - Comparing Response Rates

  1. #1
    Newbie
    Joined
    Dec 2008
    Posts
    3

    Comparing Response Rates

    I have 2 adverts running.

    The first advert gets 203 views and 4 people respond. That is 1.97% respond.

    The second advert gets 191 views and just 1 person responds. That is 0.52% people respond.

    How confident can you be that the 2 adverts will remain to have significantly different response rates in the future?

    I have spent probably 4 hours searching the net and looking in my stats book for an answer to this. If anyone could explain how to do this in simple english that would be greatly appreciated.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Pim
    Pim is offline
    Member
    Joined
    Dec 2008
    From
    The Netherlands
    Posts
    91
    The distribution that fits with this problem is a binomial one, you get a visitor (trial), who will (or will not) click your ad. (Success/failure)
    There is a formula, which can give you confidence intervals for proportions:

     <br />
P \pm z_c \sqrt{\frac{pq}{n}}<br />

    With P as the sample p, z_c as your confidence level, p as the actual success, which in this case will be approximated by P, q as p -1, failure chance and n as the sample size.

    We'll take 95% as your confidence level, which converts to z_c = 1.96

    Two times computing:

    p_{actual} being the true p

     <br />
n = 203 <br />

    <br />
p = 0.0197<br />
    0.0197 \pm 1.96 \sqrt{\frac{0.0197 * 0.9803}{203}}

    0.0197 \pm 0.01911704079

    0.0005829592104 \leq p_{actual} \leq 0.03881704079



     <br />
n = 191 <br />

    <br />
p = 0.0052<br />
    0.0052 \pm 1.96 \sqrt{\frac{0.0052*0.9948}{191}}

    0.0052 \pm 0.0102002061

    0 \leq p_{actual} \leq 0.01540020612

    (The first part would be smaller than 0, but per defenition, chances cannot be smaller than zero)

    Now, as you can see those intervals overlap for quite a large part, however they are indeed significantly different. What this means is that it's not unlikely for them to remain different. However, if you want more certainity, I suggest you compute this again once you have quite a bit more views, as that means there are more trials and therefore your intervals will get shorter and you will see how much so.
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Dec 2008
    Posts
    3

    Thank you so much!

    Thank you so much!

    I was beginning to think I'd be stuck forever after asking several forums!

    I am humbled by your reply, but would you be so kind as to define how far apart these figures have to be, to be significantly different? Or can you even define it?

    Sorry if I'm asking a stupid question and thanks again!
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Pim
    Pim is offline
    Member
    Joined
    Dec 2008
    From
    The Netherlands
    Posts
    91
    The problem of this lies in mainly this: How significantly different do you need them to be? What this formula does, it gives you intervals for which you can be sure that the average response rate will stay in, with 95% certainity. Now if these intervals do not overlap, they will most likely not get the same any time soon. That is what I would call "significantly different". I hope that you understand how to fill in this formula and I suggest you just try it again at a thousand or so views. Another thing you could do, is compute them again with another certainity rate ( z_c). Here are a few values for which you can be x% certain that the mean will stay within the interval you get as a result:
    50% -- 0.6745
    80% -- 1.28
    90% -- 1.645
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Newbie
    Joined
    Dec 2008
    Posts
    3
    Quote Originally Posted by Pim View Post
    The problem of this lies in mainly this: How significantly different do you need them to be? What this formula does, it gives you intervals for which you can be sure that the average response rate will stay in, with 95% certainity. Now if these intervals do not overlap, they will most likely not get the same any time soon. That is what I would call "significantly different". I hope that you understand how to fill in this formula and I suggest you just try it again at a thousand or so views. Another thing you could do, is compute them again with another certainity rate ( z_c). Here are a few values for which you can be x% certain that the mean will stay within the interval you get as a result:
    50% -- 0.6745
    80% -- 1.28
    90% -- 1.645
    Thanks for your help and yes, I do understand how to fill in the formula with the correct values.
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Member
    Joined
    May 2006
    Posts
    244
    Quote Originally Posted by Pim View Post
    The distribution that fits with this problem is a binomial one, you get a visitor (trial), who will (or will not) click your ad. (Success/failure)
    There is a formula, which can give you confidence intervals for proportions:

     <br />
P \pm z_c \sqrt{\frac{pq}{n}}<br />

    With P as the sample p, z_c as your confidence level, p as the actual success, which in this case will be approximated by P, q as p -1, failure chance and n as the sample size.

    We'll take 95% as your confidence level, which converts to z_c = 1.96

    Two times computing:

    p_{actual} being the true p

     <br />
n = 203 <br />

    <br />
p = 0.0197<br />
    0.0197 \pm 1.96 \sqrt{\frac{0.0197 * 0.9803}{203}}

    0.0197 \pm 0.01911704079

    0.0005829592104 \leq p_{actual} \leq 0.03881704079



     <br />
n = 191 <br />

    <br />
p = 0.0052<br />
    0.0052 \pm 1.96 \sqrt{\frac{0.0052*0.9948}{191}}

    0.0052 \pm 0.0102002061

    0 \leq p_{actual} \leq 0.01540020612

    (The first part would be smaller than 0, but per defenition, chances cannot be smaller than zero)

    Now, as you can see those intervals overlap for quite a large part, however they are indeed significantly different. What this means is that it's not unlikely for them to remain different. However, if you want more certainity, I suggest you compute this again once you have quite a bit more views, as that means there are more trials and therefore your intervals will get shorter and you will see how much so.
    The normal approximations is not valid in this case. What I would suggest is that you take the null hypothesis that there is no difference in the proportions and that the common proportion is about 0.0127, and bootstrap a 95% interval for the difference in the proportions for samples of size 203 and 191.

    Doing this with a bootstrap sample of 50000 gives a 95% interval of about [-0.022,0.024] for the difference in proportions between the two samples.

    As the observed difference is 0.0145 and is well within this interval there is no strong evidence that the response rate is significantly differeent for the two ads. (in fact the observed difference is well inside the 90% interval and only just outside the 75% interval)
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Maximising the response of a system
    Posted in the Advanced Applied Math Forum
    Replies: 2
    Last Post: August 22nd 2010, 03:02 AM
  2. Extended-response question
    Posted in the Pre-Calculus Forum
    Replies: 1
    Last Post: May 18th 2010, 09:11 PM
  3. Linearizing the response
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: March 9th 2009, 04:34 PM
  4. in need of a quick response...cos-->sin equation
    Posted in the Trigonometry Forum
    Replies: 1
    Last Post: June 16th 2008, 07:49 PM
  5. Common Response & Confounding
    Posted in the Statistics Forum
    Replies: 1
    Last Post: March 25th 2007, 08:28 PM

Search Tags


/mathhelpforum @mathhelpforum