Efficiency of four probability sampling designs: a Monte Carlo simulation study
Keywords:
probability sampling, stratified sampling, cluster sampling, systematic sampling, Monte Carlo simulation, design effectAbstract
Introduction: the choice of sampling design determines the precision and validity of survey inferences, yet differences between designs are rarely quantified on a population with fully known parameters. We compared four probability sampling designs (simple random, stratified, cluster and systematic) for estimating a mean, a standard deviation and a proportion. Method: under specified assumptions, we generated a synthetic population of N = 1,000,000 individuals (age, weight and sex) in 40,000 clusters of 25. For each design we drew 1,000 samples of n = 2,000 and computed estimates and 90%, 95% and 99% confidence intervals. We assessed bias, root mean squared error, relative efficiency, design effect, coverage and interval width, with a sensitivity analysis of standard errors. Results: all designs were practically unbiased (maximum relative bias 0.197%). Age stratification gained only for age, the stratification variable (relative efficiency 10.37 for the mean and 2.94 for the standard deviation), not for weight or the proportion of men; its mean relative efficiency (3.29) falls to 1.52 without mean age. Systematic sampling on a frame ordered by an age-related variable gained moderately (2.01 for mean age). Approximate standard errors produced conservative intervals (99.9% and 99.1% coverage at 95%); design-based standard errors brought them to 93.6% and 95.4%. Cluster sampling was the least efficient (design effects up to 2.85). Conclusions: no design dominated; efficiency depended on how the design relates to the parameter. Because all designs used the same sample size but were not evaluated at equal operational cost, the comparison concerns statistical efficiency, not economic superiority. As the results come from one synthetic population with age-defined strata, the magnitudes should be generalized cautiously.
References
1. Neyman J. On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection. Journal of the Royal Statistical Society. 1934;97(4):558-625. https://doi.org/10.2307/2342192.
2. Horvitz DG, Thompson DJ. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association. 1952;47(260):663-685. https://doi.org/10.1080/01621459.1952.10483446.
3. Cochran WG. Sampling techniques. 3rd ed. New York: John Wiley & Sons; 1977.
4. Särndal CE, Swensson B, Wretman J. Model assisted survey sampling. New York: Springer-Verlag; 1992. https://doi.org/10.1007/978-1-4612-4378-6
5. Lohr SL. Sampling: design and analysis. 3rd ed. Boca Raton: Chapman and Hall/CRC; 2022. https://doi.org/10.1201/9780429298899
6. Kim JK. Statistics in survey sampling. Boca Raton: Chapman and Hall/CRC; 2025. https://doi.org/10.1201/9781003606147
7. Kalton G. Probability vs. nonprobability sampling: from the birth of survey sampling to the present day. Statistics in Transition New Series. 2023;24(3):1-22. https://doi.org/10.59170/stattrans-2023-029.
8. Wu C. Statistical inference with non-probability survey samples. Survey Methodology. 2022;48(2):283-311. https://www150.statcan.gc.ca/n1/pub/12-001-x/2022002/article/00002-eng.htm.
9. Tillé Y. Sampling and estimation from finite populations. Hoboken: John Wiley & Sons; 2020. https://doi.org/10.1002/9781119071259
10. Madow WG, Madow LH. On the theory of systematic sampling, I. The Annals of Mathematical Statistics. 1944;15(1):1-24. https://doi.org/10.1214/aoms/1177731312.
11. Iachan R. Systematic sampling: a critical review. International Statistical Review. 1982;50(3):293-303. https://doi.org/10.2307/1402499.
12. Azeem M, Hussain S, Ijaz M, Salahuddin N, Salam A. An improved version of systematic sampling design for use with linear trend data. Heliyon. 2023;9(6):e17121. https://doi.org/10.1016/j.heliyon.2023.e17121.
13. Kish L. Survey sampling. New York: John Wiley & Sons; 1965.
14. Valliant R, Dever JA, Kreuter F. Practical tools for designing and weighting survey samples. 2nd ed. Cham: Springer; 2018. https://doi.org/10.1007/978-3-319-93632-1
15. Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Statistics in Medicine. 2019;38(11):2074-2102. https://doi.org/10.1002/sim.8086.
16. Chapuis K, Taillandier P, Drogoul A. Generation of synthetic populations in social simulations: a review of methods and practices. Journal of Artificial Societies and Social Simulation. 2022;25(2):6. https://doi.org/10.18564/jasss.4762.
17. Siepe BS, Bartoš F, Morris TP, Boulesteix AL, Heck DW, Pawel S. Simulation studies for methodological research in psychology: a standardized template for planning, preregistration, and reporting. Psychological Methods. 2024. Publicación anticipada en línea. https://doi.org/10.1037/met0000695
18. Kondratev M, Corrao MV, Armstrong R, Smith AMS. Assessing the uncertainty of traditional sample-based forest inventories in mixed and single species conifer systems using a digital forest twin. Forests. 2025;16(11):1617. https://doi.org/10.3390/f16111617.
19. Daraz U, Wu J, Agustiana D, Emam W. Finite population variance estimation using Monte Carlo simulation and real life application. Symmetry. 2025;17(1):84. https://doi.org/10.3390/sym17010084.
20. Alghamdi AS, Almulhim FA. Optimizing finite population mean estimation using simulation and empirical data. Mathematics. 2025;13(10):1635. https://doi.org/10.3390/math13101635.
21. Abbas M, Shehzad MA, Rabia M, Khurram H, Ijaz M. Estimation of finite population mean in a complex survey sampling. PLoS ONE. 2025;20(5):e0324559. https://doi.org/10.1371/journal.pone.0324559.
22. Rehman SA, Al-Essa LA, Shabbir J, Khan Z. On the efficiency of paired ranked set sampling for estimating the population mean in the presence of non-response. Scientific Reports. 2024;14:25334. https://doi.org/10.1038/s41598-024-76072-y.
23. Franceschi S, Fattorini L, Gregoire TG. Exploiting nearest-neighbour maps for estimating the variance of sample mean in equal-probability systematic sampling of spatial populations. Spatial Statistics. 2024;64:100865. https://doi.org/10.1016/j.spasta.2024.100865.
24. Wesołowski J, Wieczorkowski R, Wójciak W. Recursive Neyman algorithm for optimum sample allocation under box constraints on sample sizes in strata. Survey Methodology. 2024;50(2):487-511. https://www150.statcan.gc.ca/n1/pub/12-001-x/2024002/article/00003-eng.htm.
25. Sauer S, Hedt-Gauthier B, Haneuse S. Practical strategies for operationalizing optimal allocation in stratified cluster-based outcome-dependent sampling designs. Statistics in Medicine. 2023;42(7):917-935. https://doi.org/10.1002/sim.9650.
26. Lohmann A, Astivia OLO, Morris TP, Groenwold RHH. It's time! Ten reasons to start replicating simulation studies. Frontiers in Epidemiology. 2022;2:973470. https://doi.org/10.3389/fepid.2022.973470.
27. Pawel S, Kook L, Reeve K. Pitfalls and potentials in simulation studies: questionable research practices in comparative simulation studies allow for spurious claims of superiority of any method. Biometrical Journal. 2024;66(1):2200091. https://doi.org/10.1002/bimj.202200091.
28. Rhoda DA, Soofi SB, Hussain I, Clary CB, Yu M, Khanyile S, et al. Sample size parameters for vaccination coverage cluster surveys: insights from Pakistan. Journal of Survey Statistics and Methodology. 2025;13(5):587-614. https://doi.org/10.1093/jssam/smaf033.
29. Wolter KM. Introduction to variance estimation. 2nd ed. New York: Springer; 2007. https://doi.org/10.1007/978-0-387-35099-8
30. Wilson EB. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association. 1927;22(158):209-212. https://doi.org/10.1080/01621459.1927.10502953.
31. Agresti A, Coull BA. Approximate is better than “exact” for interval estimation of binomial proportions. The American Statistician. 1998;52(2):119-126. https://doi.org/10.1080/00031305.1998.10480550.
32. Brown LD, Cai TT, DasGupta A. Interval estimation for a binomial proportion. Statistical Science. 2001;16(2):101-133. https://doi.org/10.1214/ss/1009213286.
33. Andersson PG. The Wald confidence interval for a binomial p as an illuminating “bad” example. The American Statistician. 2023;77(4):443-448. https://doi.org/10.1080/00031305.2023.2183257.
34. Curto J. Confidence intervals for means and variances of nonnormal distributions. Communications in Statistics - Simulation and Computation. 2023;52(9):4414-4430. https://doi.org/10.1080/03610918.2021.1963448.
35. Bhushan S, Pandey S. Optimality of novel estimators for variance estimation under successive sampling. Communications in Statistics - Theory and Methods. 2025;54(14):4635-4653. https://doi.org/10.1080/03610926.2024.2425746.
36. Dagdoug M, Goga C, Haziza D. Model-assisted estimation through random forests in finite population sampling. Journal of the American Statistical Association. 2023;118(542):1234-1251. https://doi.org/10.1080/01621459.2021.1987250.
37. Deville JC, Särndal CE. Calibration estimators in survey sampling. Journal of the American Statistical Association. 1992;87(418):376-382. https://doi.org/10.1080/01621459.1992.10475217.
38. Heikkinen J, Henttonen HM, Katila M, Tuominen S. Stratified, spatially balanced cluster sampling for cost-efficient environmental surveys. Environmetrics. 2025;36(5):e70019. https://doi.org/10.1002/env.70019.
39. Prentius W, Grafström A. How to find the best sampling design: a new measure of spatial balance. Environmetrics. 2024;35(7):e2878. https://doi.org/10.1002/env.2878.
40. Jauslin R, Panahbehagh B, Tillé Y. Sequential spatially balanced sampling. Environmetrics. 2022;33(8):e2776. https://doi.org/10.1002/env.2776.
41. Yasmeen U, Noor-ul-Amin M, Hanif M. Variance estimation in stratified adaptive cluster sampling. Statistics in Transition New Series. 2022;23(1):173-184. https://doi.org/10.2478/stattrans-2022-0010.
42. Groves RM, Fowler FJ, Couper MP, Lepkowski JM, Singer E, Tourangeau R. Survey methodology. 2nd ed. Hoboken: John Wiley & Sons; 2009.
43. DISPONIBILIDAD DE DATOS Y CÓDIGO
44. No se utilizaron datos primarios procedentes de participantes. Se adjuntan dos scripts Python de comprobación independiente: S1 (Suplementario_S1_simulacion_muestreo.py) implementa el esquema de generación poblacional, selección de muestras y medidas de desempeño descrito en el método; S2 (Suplementario_S2_figuras.py) genera gráficos auxiliares a partir de sus resultados. Ambos archivos son ejecutables y documentan sus parámetros. No sustituyen los scripts originales con los que se produjeron las tablas y figuras del manuscrito, que no estuvieron disponibles para esta revisión; por ello, la reproducción numérica exacta de los resultados publicados continúa pendiente de contrastación por el autor.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Marco Antonio Ramos Espinal
This work is licensed under a Creative Commons Attribution 4.0 International License.
