Calculation of Effect Size in Single-Subject Experimental Studies: Examination of Non-Regression-Based Methods

Year 2019, Volume: 10 Issue: 1, 30 - 48, 29.03.2019

Nihal Şen , Sedat Şen

https://doi.org/10.21031/epod.419625

Cited By: 2

Abstract

It is observed that
meta-analysis studies have not been included in single-subject experimental
studies as much as the experimental studies. In order to overcome this
deficiency in the literature, regression-based and non-regression-based indexes
that can be used as the effect size in single subject experimental studies have
been recently developed. Since most of the regression-based indexes are
affected by the serial dependency of single-subject experimental data,
non-regression-based indexes that were less affected by this dependency and
were preferred more than regression-based indexes were the main subject of this
study. Although there are many indexes that are not based on regression in
single-subject experimental studies, it is observed that most of the
researchers prefer the percentage of non-overlapping data (PND) and percentage
of zero data (PZD). There are many controversies in the literature especially
on the use of the PND index. In the absence of a study describing the
alternative indexes of PND and PZD in Turkey literature, the examination of
non-regression-based indexes makes this study important. The aim of this study
is to examine the non-regression methods used to calculate the effect size in
single-subject experimental studies and to show how these methods will be
applied in single-subject experimental research. In this aim, how to prepare
the data, how to analyze it, how to synthesize it for more than one study and
how to interpret the results are discussed. In this study, suggestions were
made for the researchers based on 10 different indexes.

Keywords

Single-subject experimental research, meta-analysis, effect size, non-regression-based indexes

Tek Denekli Deneysel Çalışmalarda Etki Büyüklüğü Hesaplaması: Regresyona Dayalı Olmayan Yöntemlerin İncelenmesi

Abstract

Tek
denekli deneysel çalışmalarda meta-analiz yöntemlerine grup desenli
çalışmalarda olduğu kadar yer verilmediği gözlenmektedir. Son yıllarda
alanyazındaki bu eksikliği gidermek amacıyla tek denekli deneysel çalışmalarda
etki büyüklüğü olarak kullanılabilecek regresyona dayalı olan ve regresyona
dayalı olmayan indeksler geliştirilmiştir. Regresyona dayalı olan indekslerin
çoğu tek denekli deneysel araştırma verilerindeki zamansal bağımlılıktan
etkilendiği için bu indekslerden daha az etkilenen ve alanyazında daha sıklıkla
tercih edilen regresyona dayalı olmayan indeksler bu çalışmanın ana konusu
olmuştur. Tek denekli deneysel çalışmalarda regresyona dayalı olmayan birçok
indeks üretilmiş olmasına rağmen araştırmacıların çoğunun örtüşmeyen veri
yüzdesi (ÖVY) ve sıfır veri yüzdesi (SVY) indekslerini tercih ettiği
gözlenmiştir. Alanyazında, özellikle ÖVY indeksinin kullanılmasına ilişkin çok
sayıda tartışma bulunmaktadır. Türkiye’deki alanyazında ÖVY ve SVY’ye
alternatif olarak üretilen indeksleri açıklayan mevcut bir çalışma olmaması
regresyona dayalı olmayan indekslerin incelendiği bu çalışmayı önemli
kılmaktadır. Bu çalışmanın amacı tek denekli deneysel çalışmalarda etki
büyüklüğü hesaplamak için kullanılan regresyona dayalı olmayan yöntemlerin
incelenmesi ve bu yöntemlerin tek denekli deneysel araştırmalarda nasıl
uygulanacağının örnek veriler üzerinden gösterilmesidir. Bu amacı
gerçekleştirirken farklı yöntemlerin kullanımında verinin nasıl hazırlanacağı,
nasıl analiz edileceği, birden fazla çalışma için nasıl sentezleneceği ve elde
edilen sonuçların nasıl yorumlanacağından bahsedilmiştir. Bu çalışmada
incelenen 10 farklı indeks temelinde araştırmacılar için önerilerde
bulunulmuştur. 

Keywords

Tek denekli deneysel araştırmalar, etki büyüklüğü, regresyona dayalı olmayan indeksler, meta-analiz

References

• Alresheed, F., Hott, B. L., & Bano, C. (2013). Single subject research: A synthesis of analytic methods. The Journal of Special Education Apprenticeship, 2(1), 1–18.
• Allison, D. B., & Gorman, B. S. (1993). Calculating effect sizes for meta-analysis: The case of the single case∗. Behaviour Research and Therapy, 31(6), 621–631.
• Allison, D. B., & Gorman, B. S. (1994). “Make things as simple as possible, but no simpler.” A rejoinder to Scruggs and Mastropieri. Behaviour Research and Therapy, 32(8), 885–890.
• American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
• Aslan, C., Yalçın, G. & Özdemir, S. (2016). Sosyal öykü tekniğinin etkililiği: Betimsel değerlendirme ve meta-analiz çalışması. Teacher Education in Special Education, Vocational Training and Sports Konferansı (ELMIS).
• Aydın, O. (2017). Otizm spektrum bozukluğu olan bireylere matematik becerilerinin öğretimi: tek-denekli araştırmalarda Betimsel ve meta analiz. (Yayımlanmamış yüksek lisans tezi). Anadolu Üniversitesi, Eğitim Bilimleri Enstitüsü, Eskişehir.
• Bell, R. J., Skinner, C. H., & Fisher, L. A. (2009). Decreasing putting yips in accomplished golfers via solution-focused guided imagery: A single-subject research design. Journal of Applied Sport Psychology, 21(1), 1-14.
• Beretvas, S. N., & Chung, H. (2008). A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2, 129–141.
• Bozkuş-Genç., G. (2017). Otizm spektrum bozukluğu olan çocuklara soru sorarak iletişim başlatmanın kazandırılmasında temel tepki öğretiminin etkileri. Doktora tezi.
• Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp.187–212). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
• Campbell, J. M. (2003). Efficacy of behavioral interventions for reducing problem behavior in persons with autism: A quantitative synthesis of single-subject research. Research in Developmental Disabilities, 24, 120–138.
• Campbell, J. M. (2004). Statistical comparison of four effect sizes for single-subject designs. Behavior Modification, 28(2), 234–246.
• Carr, J. E., Severtson, J. M., & Lepper, T. L. (2009). Noncontingent reinforcement is an empirically supported treatment for problem behavior exhibited by individuals with developmental disabilities. Research in Developmental Disabilities, 30(1), 44-57.
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences. (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
• Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8.
• Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge.
• Heyvaert, M., Saenen, L., Campbell, J. M., Maes, B., & Onghena, P. (2014). Efficacy of behavioral interventions for reducing problem behavior in persons with autism: An updated quantitative synthesis of single-subject research. Research in developmental disabilities, 35(10), 2463–2476.
• Huitema,B.E. (1985). Autocorrelation in applied behavior analysis: A myth. Behavioral Assessment, 7, 107–118.
• Huitema, B. E., & Mckean, J. W. (2000). Design specification issues in time-series intervention models. Educational and Psychological Measurement, 60(1), 38–58.
• Karasu, N. (2009a). Özel eğitimde delile dayalı yöntemlerin belirlenmesi: Tek denekli çalışma analizleri ve karşılaştırmaları. Türk Eğitim Bilimleri Dergisi, 7(1), 143–163.
• Karasu, N. (2009b). Otizmden etkilenmiş bireylerde sosyal ve iletişim becerilerini arttıran yöntemlerin delile dayalı yöntem olarak belirlenmesi: Bir meta-analiz örneği. Türk Eğitim Bilimleri Dergisi, 7(3), 713–739.
• Karasu, N. (2011). Otizmli bireylerin eğitiminde video ile model olma uygulamalarının değerlendirilmesi: Bir alanyazın derlemesi ve meta-analiz örneği. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Özel Eğitim Dergisi, 12(2), 1–12.
• Kavale, K. A. (2001). Decision making in special education: The function of meta-analysis. Exceptionality, 9, 245–268.
• Kaya, F. (2015). Otizm spektrum bozukluğu olan öğrencilere yiyecek-içecek hazırlama becerilerinin öğretiminde sesli anlatım içeren ve içermeyen video ipucunun karşılaştırılması. Yüksek lisans tezi.
• Kırcaali-İftar, G., & Tekin, E. (1997). Tek denekli araştırma yöntemleri. Ankara: Türk Psikologlar Derneği Yayınları.
• Korkmaz, Ö. T., & Diken, İ. H. (2010). Stereotipik davranışların azaltılmasında kullanılan yöntemlerin etkililiği: Betimsel ve meta analizi. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Özel Eğitim Dergisi, 11(2), 1–12.
• Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. What works clearinghouse.
• Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
• Lydon, S., Healy, O., O’Reilly, M., & McCoy, A. (2013). A systematic review and evaluation of response redirection as a treatment for challenging behavior in individuals with developmental disabilities. Research in Developmental Disabilities, 34(10), 3148–3158.
• Ma, H. (2006). An alternative method for quantifying synthesis of single-subject research: Percent of data points exceeding the median. Behavior Modification, 30, 598–617.
• Maggin, D. M., O'Keeffe, B. V., & Johnson, A. H. (2011). A quantitative synthesis of methodology in the meta-analysis of single-subject research for students with disabilities: 1985–2009. Exceptionality, 19(2), 109–135.
• Maggin, D. M., Swaminathan, H., Rogers, H. J., O'keeffe, B. V., Sugai, G., & Horner, R. H. (2011). A generalized least squares regression approach for computing effect sizes in single-case research: Application examples. Journal of School Psychology, 49(3), 301-321.
• Manolov, R., Sierra, V., Solanas, A., & Botella, J. (2014). Assessing functional relations in single-case designs: Quantitative proposals in the context of the evidence-based movement. Behavior modification, 38(6), 878-913.
• Manolov, R., & Solanas, A. (2008). Comparing N=1 effect size indices in presence of autocorrelation. Behavior Modification, 32, 860-875.
• Manolov, R., Solanas, A., & Leiva, D. (2010). Comparing “visual” effect size indices for single-case designs. Methodology, 6, 49–58.
• Manolov, R., Solanas, A., Sierra, V., & Evans, J. J. (2011). Choosing among techniques for quantifying single-case intervention effectiveness. Behavior Therapy, 42(3), 533-545.
• O'Brien, S., & Repp, A. C. (1990). Reinforcement-based reductive procedures: A review of 20 years of their use with persons with severe or profound retardation. Journal of the Association for Persons with Severe Handicaps, 15(3), 148–159.
• Olive, M. L., & Franco, J. H. (2008). (Effect) size matters: And so does the calculation. The Behavior Analyst Today, 9(1), 5–10.
• Olive, M. L., & Smith, B. W. (2005). Effect size calculations and single subject designs. Educational Psychology, 25(2-3), 313–324.
• Parker, R. I., Hagan-Burke, S., & Vannest, K. (2007). Percentage of all non-overlapping data (PAND): An alternative to PND. Journal of Special Education, 40, 194–204.
• Parker, R. I., & Vannest, K. (2009). An improved effect size for single-case research: Nonoverlap of all pairs. Behavior Therapy, 40(4), 357–367.
• Parker, R. I., Vannest, K. J., & Brown, L. (2009). The improvement rate difference for single-case research. Exceptional Children, 75(2), 135–150.
• Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35(4), 303–322.
• Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42(2), 284–299.
• Rakap, S. (2015). Effect sizes as result interpretation aids in single‐subject experimental research: description and application of four nonoverlap methods. British Journal of Special Education, 42(1), 11-33.
• Reichle, J. (2007). Amongst methodologies of functional behavioral assessment, functional analysis yields more effective suppression outcomes. Evidence-Based Communication Assessment and Intervention, 1(4), 153-155.
• Schlosser, R. W., & Koul, R. K. (2015). Speech output technologies in interventions for individuals with autism spectrum disorders: a scoping review. Augmentative and Alternative Communication, 31(4), 285-309.
• Scotti, J. R., Evans, I. M., Meyer, L. H., & Walker, P. (1991). A meta-analysis of intervention research with problem behavior: Treatment validity and standards of practice. American Journal on Mental Retardation, 96, 233–256.
• Scruggs, T. E., & Mastropieri, M. A. (1998). Summarizing single-subject research: Issues and applications. Behavior modification, 22(3), 221–242.
• Scruggs, T. E., & Mastropieri, M. A. (2013). PND at 25: Past, present, and future trends in summarizing single-subject research. Remedial and Special Education, 34(1), 9-19.
• Scruggs, T. E., Mastropieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: Methodology and validation. Remedial and Special Education, 8, 24–33.
• Scruggs, T. E., Mastropieri, M. A., Cook, S. B., & Escobar, C. (1986). Early intervention for children with conduct disorders: A quantitative synthesis of single-subject research. Behavioral Disorders, 11, 260–71.
• Shadish, W. R., Hedges, L. V., & Pustejovsky, J. E. (2014). Analysis and meta-analysis of single-case designs with a standardized mean difference statistic: A primer and applications. Journal of School Psychology, 52(2), 123–147.
• Sönmez, M., & Diken, İ. H. (2010). Problem davranışların azaltılmasında işlevsel iletişim öğretiminin etkililiği: Betimsel ve meta-analiz çalışması. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Özel Eğitim Dergisi, 11(1), 1–16.
• Strain, P. S., Kohler, F. W., & Gresham, F. (1998). Problems in logic and interpretation with quantitative syntheses of single-case research: Mathur and colleagues (1998) as a case in point. Behavioral Disorders, 24, 74–85.
• Swaminathan, H., Rogers, H. J., Horner, R. H., Sugai, G., & Smolkowski, K. (2014). Regression models and effect size measures for single case designs. Neuropsychological rehabilitation, 24(3-4), 554-571.
• Tavil, Y.Z. ve Karasu, N. (2013). Aile eğitim çalışmaları: Bir gözden geçirme ve meta-analiz örneği. Eğitim ve Bilim, 38(168), 85–95.
• Uysal, H. (2017). Zihin yetersizliği olan öğrencilere temel toplama işlemlerinde akıcılık kazandırmada iki farklı uygulamanın karşılaştırılması. Yüksek lisans tezi.
• van den Noortgate, W., & Onghena, P. (2003). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments & Computers, 35, 1–10.
• Vannest, K.J,. & Davis, H. S. (2013). Synthesizing single case research to identify evidence based treatments. In B. G. Cook, M.Tankersley, & T. J. Landrum (Eds.), Evidence-Based practices (pp. 93–119). UK: Emerald Group Publishing.
• Wehmeyer, M. L., Palmer, S. B., Smith, S. J., Parent, W., Davies, D. K., & Stock, S. (2006). Technology use by people with intellectual and developmental disabilities to support employment activities: A single-subject design meta analysis. Journal of Vocational Rehabilitation, 24(2), 81-86.
• White, O. R., & Haring, N. G. (1980). Exceptional teaching (2nd ed.). Columbus, OH: Merrill.
• Wolery, M., Busick, M., Reichow, B., & Barton, E. E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44(1), 18–28.