AIR study

The American Institutes for Research (AIR) has released a study: Why Massachusetts Students, the Best in the U.S., Lag Behind 
Best-in-the-World Students of Hong Kong.  The comparison of the two assessments found that Hong Kong items differed from Massachusetts items in several important ways:

Hong Kong items were more concentrated in the number and measurement strands (75 percent), compared with Massachusetts (60 percent). A firm understanding of basic number concepts is essential for doing more-advanced work in fractions and algebra. And a solid understanding of measurement concepts is crucial to handling real-world math and learning geometry.

Hong Kong items were more likely to require students to construct a response (86 percent) than Massachusetts items (29 percent). Constructed-response items tend to be more demanding of students to generate the correct answer by working completely through the problem without the advantage of being able to select a correct answer from a list.

Items in the Hong Kong assessment were more likely to require more than low computational difficulty (37 percent), compared with Massachusetts items (3 percent). In the numbers domain, where computation is an integral component of the solution, 13 out of 15 (87 percent) of Hong Kong items were of higher computational difficulty, whereas only 1 out of 17 (6 percent) of the Massachusetts items in numbers required more than simple computational skills.

Hong Kong items were more likely to fall into the moderate or high cognitive complexity category (55 percent) compared with Massachusetts items (34 percent). Performance on higher cognitively complex items is an indicator of the ability to apply mathematical concepts to solving routine and non routine problems.

“Overall, the comparison revealed that each assessment covered similar mathematics topics. However, the Hong Kong assessment required a greater depth of mathematical understanding required to solve many items,” explained Leinwand. “This expectation of deep understanding of math concepts is a likely contributor to Hong Kong’s achievement as the highest performer on TIMMS in the early grades.”