Ralf St.Clair

Ralf St. Clair is the Dean of Education at the University of Victoria, Canada. Dr. St. Clair is a leading researcher in the areas of adult literacy and community initiatives, the formation of aspirations by First Nations youth, and research patterns in higher education. His research record includes partnerships at the local, national, and international levels, multiple journal publications, and three single authored books on adult education and literacy.

Parsing PIAAC: Accuracy, Analysis, and Application in Adult Competencies

This presentation pulls together three tracks of work centred on the Programme for International Assessment of Adult Competencies. The conclusion of this discussion is that it is critical that our community of people interested in, and committed to, literacy, understand the tools and resources available to us, and how they can contribute most fully to our work.

This presentation pulls together three tracks of work centred on the Programme for International Assessment of Adult Competencies. Based in the International Adult Literacy Surveys of the 1990s, the 2013 PIAAC (and upcoming iteration) are claimed by the Organisation for Economic Cooperation and Development to profile the human capital within an economy and facilitate design and evaluation of educational systems. This important and fascinating enterprise has attracted considerable attention, and three aspects are particularly interesting to consider within the broader literacy context.

Accuracy. The ability of the survey (indeed of any sample-based method) to provide measures of ability with high precision is an open question. PIAAC aims for population based estimates, and the accuracy of individual scores is of less concern. Unfortunately this methodological choice is not always understood by stakeholders, resulting in attempts, for example, to assess individuals against the PIAAC scales. There may also be lack of clarity regarding exactly what is measured, undermining the utility of outcomes.

Analysis. PIAAC devotes considerable effort to ensuring that a wide range of demographic and other background information is collected on respondents. This background data has enormous potential that has not always been well realised. Analysis from this perspective points to different considerations and conclusions from those usually promoted.

Application. The utility of PIAAC is predicated on the existence of a polity to accept, analyse and apply the insights generated. This can be illustrated through discussion of the Canadian case. A key player in the development of the survey, the Canadian government committed to a huge sample size for the PIAAC in 2013. In the meantime, the federal goverment dismantled the organisations and infrastructure that could have made use of the results. The tragic result is a deep and wide dataset with no policy apparatus to make use of it.

The conclusion of this discussion is that it is critical that our community of people interested in, and committed to, literacy, understand the tools and resources available to us, and how they can contribute most fully to our work.”