A New Proposed Estimator for Reducing Bias Due to Undetected Species

Abstract

The present paper addresses a new approach to reduce bias when there are undetected species in a plot. Partially density matrix plays essential role in this new proposed estimator. The performance of the new proposed estimator (Ĥ₀) was compared to bias-corrected MLE (MLE_BC), Jackknife (JK) and the proposed estimator of Chao and Shen (Ĥ_cs)  using Principle component analysis (PCA). The result of the first PCA applied to the data including the estimators’ values of the assemblages showed that  Ĥ₀ is located between JK and Ĥ_cs and its’ nearest neighbor becomes JK. The second PCA was applied to the data belonging to the relative estimator values between the pairwise assemblages and, it was found that Ĥ₀ is still located between JK and Ĥ_cs but its’ nearest neighbor becomes Ĥ_cs in this time along the first axis. Those results were evaluated that Ĥ₀ is a better estimator than MLE_BC. Thus the new proposed estimator (Ĥ₀) can also be used as an alternative bias-corrected estimator in addition to the other estimators.

Keywords

• Bias-corrected estimator,Diversity,Entropy,Jackknife,Partially density matrix

References

1. [1] Batten, L. A., “Bird communities of some Killarney woodlands”, Proceedings of the Royal Irish Academy. Section B: Biological, geological, and chemical science. Royal Irish Academy, 76: 285-313, (1976).
2. [2] Bonachela, Juan A., Haye Hinrichsen and Miguel A. Munoz, “Entropy estimates of small data sets”, Journal of Physics A: Mathematical and Theoretical, 41(20): 1-9, (2008).
3. [3] Li, Y. and Busch, P., “Von Neumann entropy and majorization”, Journal of Mathematical Analysis and Applications 408(1): 384–393, (2013).
4. [4] Chao, A. and Lee, S.M., “Estimating the number of classes via sample coverage”, Journal of the American statistical Association, 87(417): 210-217, (1992).
5. [5] Chao, A., Ma, M.C., Yang, M.C., “Stopping rules and estimation for recapture debugging with unequal failure rates”, Biometrika, 80(1): 193-201 (1993).
6. [6] Chao, A., Hwang, W.H., Chen, Y.C., Kuo, C.Y., “Estimating the number of shared species in two communities”, Statistica sinica, 10: 227-246, (2000).
7. [7] Chao, A., Shen, T.J., “Nonparametric estimation of Shannon's index of diversity when there are unseen species in sample”, Environmental and Ecological Statistics, 10(4): 429-443, (2003).
8. [8] Chao, A., Wang, Y.T., Jost, L., “Entropy and the species accumulation curve: a novel entropy estimator via discovery rates of new species”, Methods in Ecology and Evolution , 4(11): 1091-1100, (2013).

9. [9] Chao, A., Chiu, C.H., “Species richness: estimation and comparison”, Wiley StatsRef: Statistical Reference Online, 1-26, (2016).
10. [10] Chiu, C.H., Wang, Y.T., Walther, B.A., Chao, A., “An improved nonparametric lower bound of species richness via a modified Good–Turing frequency formula”, Biometrics, 70: 671-682, (2014).
11. [11] Gorelick, R., “Combining richness and abundance into a single diversity index using matrix analogues of Shannon’s and Simpson’s indices”, Ecography, 29: 525-530, (2006).
12. [12] Grassberger, P., “Finite sample corrections to entropy and dimension estimates”, Physics Letters A, 128(6-7): 369-373, (1988).
13. [13] Grassberger, P., “Entropy Estimates from Insufficient Samplings”, ArXiv Physics e-prints, 0307138, (2003).
14. [14] Holst, L., “Some asymptotic results for incomplete multinomial or Poisson samples”, Scandinavian Journal of Statistics, 8: 243-246, (1981).
15. [15] Holste, D., Grosse, I., Herzel, H., “Bayes' estimators of generalized entropies”, Journal of Physics A: Mathematical and General, 31(11): 2551-2566, (1998).
16. [16] Janzen, D.H., “Sweep samples of tropical foliage insects: description of study sites, with data on species abundances and size distributions”, Ecology, 54(3): 659-686, (1973a).
17. [17] Janzen, D.H., “Sweep samples of tropical foliage insects: effects of seasons, vegetation types, elevation, time of day, and insularity”, Ecology, 54(3): 687-708, (1973b).
18. [18] Miller, G., “Note on the bias of information estimates”, Information Theory in Psychology: Problems and Methods, 95-100, (1955).
19. [19] Schurmann, T., “Bias analysis in entropy estimation”, Journal of Physics A: Mathematical and Theoretical, 37(27): L295-L301, (2004).
20. [20] Shannon, C. E., “A mathematical theory of communication”, Bell System Technical Journal, 27(3): 379-423, (1948).
21. [21] Zahl, S., “Jackknifing an index of diversity”, Ecology, 58(4): 907-913, (1977).
22. [22] Zhang, Z., “Entropy Estimation in Turing's Perspective”, Neural Computation, 24(5): 1368-1389, (2012).