The team, from the University of Cambridge (UK) and Virginia Commonweath University (US) suggest that the likely cause is that both this scale, and others like it, have been developed using an unrepresentative sample of children.

Researchers have previously pointed out that these assessments tend to be modelled on children who are mostly from comfortable socio-economic settings. By mapping their observed behaviours on to executive functions, they may falsely assume that these behaviours are ‘normal’ markers for any child of the same age. In reality, children’s different backgrounds and lived experiences may mean that executive functions express themselves differently across different groups.

Annie Zonneveld, from the Faculty of Education, University of Cambridge and the study’s first author, said: “There is a big question around how we measure executive functions: are we actually using the right tools? If they are based on white, middle-class students, we cannot be sure that they would actually work for the whole population. We may be seeing evidence of that here.”

Michelle Ellefson, Professor of Cognitive Science at the Faculty of Education, said: “Teachers can provide us with really valuable data about children’s executive functions because they can monitor development in ways we could not possibly replicate in a lab, but they need effective measures to do this. This means the assessments must draw on information about children from different backgrounds.”

According to the Children’s Defense Fund, about 14% of children in the United States live in poverty. While nearly 50% of all children are from ethnic minority families, 71% of those in poverty are from these backgrounds. Most psychometric research on executive functions, however, focuses on white, middle-income, or affluent families. It has never been clear how far its findings can be generalised.

The new study examined the executive function components of two versions of the BASC: the BASC-2 and BASC-3. These ask teachers to observe children’s everyday behaviours and rate, on a scale of ‘never’ to ‘always’, how far they agree with statements such as “acts without thinking”, “is easily distracted”, “cannot wait to take turn”, “is a self-starter” and “argues when denied own way”. They then extrapolate information about the children’s executive functions based on the responses.

The researchers analysed two sample groups of children, aged around nine or 10, all from state schools in high-poverty, urban areas in the United States. In total, 472 children took part. The first sample was assessed using the BASC-2; the other using the BASC-3.

Both groups also completed six computer-based tasks which psychologists and neuroscientists use in lab-based tasks to measure specific executive functions. The researchers looked at how far the scores from these computerised tasks – which are accurate but difficult to run with large groups – corresponded to the measures from the teacher-administered surveys.

The findings indicated that while the BASC-2 provides a reasonable overview of students’ general executive functioning, it does not capture accurate details about specific functions like working memory and self-control. The BASC-3 was far more effective, probably because it uses a different and more focused set of questions.

“The BASC-2 has been used extensively in archived datasets and contributes to academic research about how executive functions develop,” Ellefson said. “It is really important to recognise that without modification, it is not an appropriate basis for making judgements about certain groups of children.”

The assessment is just one of many surveys that measure children’s cognitive development in different countries. “It is important that we know how these tools are establishing their baseline understanding of ‘typical’ development,” Zonneveld said. “If they are based on mostly white populations from affluent suburbs, they won’t necessarily be as representative as we might hope.”

The study is published in Developmental Science.