Results 1 to 2 of 2

Math Help - searching for an unbiased estimator of the Teachman index

  1. #1
    Newbie
    Joined
    Apr 2008
    Posts
    2

    searching for an unbiased estimator of the Teachman index

    Hi all!

    The Teachman index is often used as an estimator of diversity in the context of social sciences. It is defined as follows:

    TI = - sum[
    pk*ln(pk)]

    where pk is the proportion of group members in the kth category. Therefore, we have maximum diversity if the sample is distributed equally across the k categories.
    Unfortunately, this measure is biased in a way that it underestimates the diversity in smaller sample sizes (n). This is comparable to the biased estimation of the variance if you divide by n and not by (n-1).

    I wonder if there is a correction formula to this equation that yields unbiased estimates of this index. I am not a statistician, so any hints are helpful, even the obvious ones...

    thanks!

    trantor
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Flow Master
    mr fantastic's Avatar
    Joined
    Dec 2007
    From
    Zeitgeist
    Posts
    16,948
    Thanks
    5
    Quote Originally Posted by Trantor75 View Post
    Hi all!

    The Teachman index is often used as an estimator of diversity in the context of social sciences. It is defined as follows:

    TI = - sum[ pk*ln(pk)]

    where pk is the proportion of group members in the kth category. Therefore, we have maximum diversity if the sample is distributed equally across the k categories.
    Unfortunately, this measure is biased in a way that it underestimates the diversity in smaller sample sizes (n). This is comparable to the biased estimation of the variance if you divide by n and not by (n-1).
    I wonder if there is a correction formula to this equation that yields unbiased estimates of this index. I am not a statistician, so any hints are helpful, even the obvious ones...

    thanks!

    trantor
    This index has a very similar form to the Shannon information entropy. So I'm pretty sure that there's no unbiased estimator ..... See p8 of http://www.gatsby.ucl.ac.uk/~pel/pos...tham_sfn00.pdf for example.

    However .... you might have a go at reading through this: Letter to the Editor

    No doubt someone will correct me ......
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Unbiased Estimator
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: April 30th 2011, 11:42 AM
  2. unbiased estimator
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: November 23rd 2010, 07:36 AM
  3. Unbiased Estimator
    Posted in the Advanced Statistics Forum
    Replies: 4
    Last Post: December 2nd 2009, 12:51 AM
  4. unbiased estimator
    Posted in the Advanced Statistics Forum
    Replies: 3
    Last Post: December 7th 2008, 12:41 AM
  5. Unbiased Estimator help
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: December 2nd 2007, 03:38 AM

Search Tags


/mathhelpforum @mathhelpforum