Data with Hierarchical Structure: Impact of Intraclass Correlation and Sample Size on Type-I Error

Abstract : Least squares analyses (e.g., ANOVAs, linear regressions) of hierarchical data leads to Type-I error rates that depart severely from the nominal Type-I error rate assumed. Thus, when least squares methods are used to analyze hierarchical data coming from designs in which some groups are assigned to the treatment condition, and others to the control condition (i.e., the widely used “groups nested under treatment” experimental design), the Type-I error rate is seriously inflated, leading too often to the incorrect rejection of the null hypothesis (i.e., the incorrect conclusion of an effect of the treatment). To highlight the severity of the problem, we present simulations showing how the Type-I error rate is affected under different conditions of intraclass correlation and sample size. For all simulations the Type-I error rate after application of the popular Kish (1965) correction is also considered, and the limitations of this correction technique discussed. We conclude with suggestions on how one should collect and analyze data bearing a hierarchical structure.
Document type :
Journal articles
Complete list of metadatas

https://hal.univ-rennes2.fr/hal-01958179
Contributor : Laurence Leroux <>
Submitted on : Monday, December 17, 2018 - 5:34:59 PM
Last modification on : Wednesday, May 22, 2019 - 1:30:30 AM

Identifiers

  • HAL Id : hal-01958179, version 1

Citation

Serban Musca, Rodolphe Kamiejski, Armelle Nugier, Alain Méot, Abdelatif Er-Rafiy, et al.. Data with Hierarchical Structure: Impact of Intraclass Correlation and Sample Size on Type-I Error. Frontiers in Psychology, Frontiers, 2011, 2 (74). ⟨hal-01958179⟩

Share

Metrics

Record views

69