Some limitations of chi-square

 

First, chi-square is highly sensitive to sample size. As sample size increases, absolute differences become a smaller and smaller proportion of the expected value. What this means is that a reasonably strong association may not come up as significant if the sample size is small, and conversely, in large samples, we may find statistical significance when the findings are small and uninteresting., i.e., the findings are not substantively significant, although they are statistically significant.

 

 

Chi-square is also sensitive to small frequencies in the cells of tables. Generally when the expected frequency in a cell of a table is less than 5, chi-square can lead to erroneous conclusions. The rule of thumb here is that if either (i) an expected value in a cell is less than 5 or (ii) more than 20% of the expected values in cells are less than 5, then chi-square should not and usually is not computed.