Longitudinal Study of Skill Dynamics (OECD Steering Group) Meeting 2015

EDUCATION AND SOCIAL PROGRESS SUMMARY RECORD STEERING GROUP MEETING OF THE OECD LONGITUDINAL STUDY OF SKILL DYNAMICS (20 MARCH 2015, OECD HEADQUARTERS, PARIS) Introduction 1. The Steering Group Meeting of the OECD Longitudinal Study of Skill Dynamics was held on 20 March 2015 at the OECD headquarters in Paris. The meeting gathered representatives from OECD member countries: Austria, Canada, Chile, Ireland, Japan, the Netherlands, Norway, the United Kingdom and the United States, as well as partner countries: Argentina, Brazil, Colombia and the Russian Federation. The Secretariat also invited two experts, namely Filip de Fruyt (Ghent University) and Oliver John (University of California at Berkeley), to facilitate the discussions on the social and emotional skills framework. The Annex presents the agenda and complete list of participants. The main objective of the meeting was to gather detailed feedbacks and agreements on: a) b) 2.     the proposed ESP social and emotional skills conceptual framework, and the focus, process and timelines of the proposed Feasibility Study scheduled in 2015-18. The main conclusions from the meeting are as follows: The participants broadly agreed on the proposed conceptual framework of the social and emotional skills constructs as described in the draft circulated to Steering Group members before the meeting. The Secretariat will revise the proposed framework taking into account the Steering Group’s suggestions: 1) the framework should acknowledge the time perspective of skill development; and 2) the framework should be relevant to questions of stakeholders. The Secretariat will prepare a call for tenders for conducting a feasibility study, which will take into account the Steering Group’s suggestions, namely: 1) the survey burden on participants (teachers and parents) can be lowered by keeping the assessment time at an acceptable level; 2) data collection of outcomes should be presented as an option; 3) random sampling of households or students should be an option. The Secretariat will align the proposed framework of social and emotional skills and other elements of the study with other surveys of the Education Directorate including PISA, PIAAC and the Early Learning Outcomes project. Introductory Session: Overview and updates of the ESP longitudinal study 3. Andreas Schleicher (OECD) opened the session by presenting the objectives of the meeting and providing an overview of the OECD’s work on social and emotional skills. He described the different activities under the Education 2030 project that address social and emotional skills, including the ESP 2 Longitudinal Study, PISA 2018 Global Competencies and the Early Learning Outcomes Project. He concluded by highlighting that the task of developing robust methods to study social and emotional skills is not easy; that it is a long-term project; and, that the OECD is convinced that the long-term returns will justify today’s efforts. Session 1: Overview of the ESP longitudinal study 4. Koji Miyamoto (OECD) gave an overview of the proposed longitudinal study and described the different stages of the project. He went over the challenges in conceptualising and measuring these skills, explained the reasons why the Secretariat decided to focus on development of social and emotional skills instruments and to secure a sufficient amount of time for the Feasibility Study. 5.     The Steering Group members: NOTED the updates of the proposed longitudinal study. SUGGESTED the Secretariat to provide concrete information on the Feasibility Study and develop good communication materials to facilitate discussions with local stakeholders. SUGGESTED the Secretariat to consider including cognitive skill measurement in the proposed longitudinal study. Lars Nerdrum (Norway) highlighted the importance of assessing cognitive skills together with social and emotional skills to understand the cross-productivity of these skills; and, hence, better understand student’s overall development. He argued that having an international cognitive assessment is particularly important in countries that don’t administer reliable national cognitive assessments. Astrid Søgnen (Norway) proposed the Secretariat to clarify the linkage between the proposed study and PISA, and Lars Nerdrum (Norway) suggested aligning the proposed study with other works of the OECD. SUGGESTED that the proposal should distinguish more clearly the goals and the outputs of the Feasibility Study and the Main Study. The Steering Group also suggested the proposal to include synthetic information on basic-survey parameters and optional measures to be included in the Feasibility Study and the Main Study. Session 2: Overview of the proposed Social and Emotional Skills Framework 6. Filip de Fruyt (Ghent University) presented a summary of discussions at the Expert Group Meeting held on 2-3 February 2015 in the Hague, and Oliver John (University of California, Berkeley) presented a conceptual framework of the social and emotional skills constructs. The framework includes five broad skill domains: 1) engaging with others, 2) collaboration skills, 3) emotion regulation, 4) task performance, and 5) open-mindedness. He explained the rationale behind choosing these five domains, including cross-cultural and cross-linguistic relevance, capacity to predict important outcomes, replicability and the high degree of independence from cognitive skills. 7.  The Steering Group members: APPROVED the proposed social and emotional skills framework, acknowledging the strengths of the framework in its comprehensiveness and cross-cultural relevance, and its capacity to capture other key constructs of interest to participating countries/cities (e.g. resilience, leadership, civic-mindedness). 3 EDU/CERI/CD/RD(2015)6 EDU/CERI/CD/RD(2015)6  SUGGESTED including developmental pathways of social and emotional skills in the framework, given that skills can be age-specific. Filip de Fruyt nevertheless suggested that measurement should not be limited to age-specific skills, since there is a variation in individuals’ sensitive periods.  SUGGESTED exploring ways to link social and emotional skill measurement of the proposed study with other OECD measurement instruments such as those used in PISA and PIAAC, as well as with local measurement instruments such as standardised achievement tests. Andreas Schleicher (OECD) responded that coherence across projects is important and that any conceptual framework needs to have a convincing validation strategy.  SUGGESTED that the framework should reflect what policy makers expect from the study. Elisabeth Buk-Berge (Norway) pointed out that the framework is scientifically solid but lacks relevance to the real world. Tone Abrahamsen (Norway) suggested facets should be school relevant. Filip de Fruyt responded that the framework is a starting point and should be translated to be linked to specific questions that the study is expected to answer. Session 3: Debriefing of ESP International Report Launching Seminar: “Skills for Social Progress: The Power of Social and Emotional Skills” 8. Dirk Van Damme (OECD) updated the Steering Group with results of the launching seminar of the international report entitled “Skills for Social Progress: The Power of Social and Emotional Skills” held at the OECD headquarters on 10 March 2015. He described the state of the literature on social and emotional skills and the new findings from this OECD study. He also explained that discussions at the seminar highlighted the need for more research to identify skills that need to be prioritised and to understand how learning contexts help develop these skills over time. One of the key messages that emerged from the seminar was the need to improve instruments to measure social and emotional skills so that they are cross-culturally relevant and well validated. This provides an important rationale for launching a new longitudinal study that tracts children’s social and emotional skills over time. 9. The Steering Group members:  NOTED the success of the launching seminar and ACKNOWLEDGED the role that the OECD can play in advancing research in the field by conceptualising and measuring social and emotional skills. Session 4: Process and timelines of the Feasibility Study 10. Maria Huerta (OECD) explained the process and timelines for developing the longitudinal study, focusing on the Feasibility Study. She first presented the different stages of the developmental work, including Feasibility Study (2015-18), Pilot Study (2017-18), Field Trial (2018-19) and Main Study (2019- 20), and explained the main objective of the Feasibility Study as development of robust, valid and reliable international instruments that measure social and emotional skills for Grades 1-12. She then explained the process of the call for tenders that the Secretariat plans to issue in June 2015 to identify one or more contractors who would implement the Feasibility Study. The selected contractor(s) will develop an assessment framework as well as instruments to assess social and emotional skills of children in Grades 1 to 12. The contractors(s) will also validate the instruments based on the data collected during the Feasibility Study. 4 11.   The Steering Group members: APPROVED the focus, process and timelines of the Feasibility Study. SUGESTED the Secretariat to consider the constraints jurisdictions are likely to face during the data collection, especially given the potentially large burden that teachers and schools may face in securing spare time and resources for assessment. Astrid Søgnen (Norway) suggested the burden on schools and parents to be minimised. Jennifer Adams (Canada) suggested the assessment time to be no longer than one lesson unit (e.g. 60 minutes in Canada), and the possibility for the assessments to be shared by more than one teacher. The Secretariat RESPONDED that, while acknowledging the importance of minimising survey burden for schools, teachers and parents, an elaborate validation procedure that involves considerable sample size and assessment time may be necessary to develop cross-culturally validated instruments for the main longitudinal study. The Secretariat SUGGESTED waiting for potential contractors to propose specific ideas to ensure quality validation while minimising survey burden. SUGGESTED the participating cities to be allowed to include locally relevant outcome measures (e.g. bullying) to see how they are related to social and emotional skill measures. The Secretariat RESPONDED that this option could be discussed with contractors and basic outcome measures could be potentially included as an option in the call for tenders. SUGGESTED the Secretariat ensures potential contractors of the Feasibility study take into account the size of cities when defining sample sizes. Tone Abrahamsen, Astrid Søgnen and Elisabeth Buk-Berge (Norway) highlighted the need to seek a survey design that allows the participation of smaller constituencies, such as Oslo. SUGGESTED the longitudinal study to be based on random sampling (representative sampling) of households or individuals, given the potential burden that school-based sampling may impose on teachers. The Secretariat REITERATED the importance of using school-based sampling to efficiently collect information on school learning contexts (including peer interactions) as well as social and emotional skill measures involving multiple informants (i.e., students, teachers and parents). Nevertheless, the Secretariat SUGGESTED that random sampling can be a useful option for the Feasibility Study if contractors come up with viable strategies to validate the pilot instruments under such a setting.    Conclusion and next steps 12. Koji Miyamoto (OECD) summarised the key conclusions from the Steering Group meeting and announced that the Secretariat is planning to organise a series of Foundations’ Forum in Chicago, Zurich and Bogota in May-June 2015. The main objectives of this forum are: to appraise the state of the evidence- base on how to raise social and emotional skills, identify what foundations do in this area of research and practice, and explore ways in which foundations can effectively work with cities, countries and the OECD. 13. The Secretariat encouraged the Steering Group members to provide detailed comments and suggestions on the social and emotional skills framework to the Secretariat by email by 3 April 2015. They also mentioned that the revised report on the Social and Emotional Skills Framework will be circulated to the Steering Group members before the CERI Governing Board meeting on 14-15 April. 5 EDU/CERI/CD/RD(2015)6 EDU/CERI/CD/RD(2015)6 ANNEX 1. AGENDA Steering Group Meeting of the OECD’s Longitudinal Study of Skill Development in Cities Room D, OECD Conference Centre, Paris Friday 20 March 2015 Objectives The Steering Group Members are invited to review and approve (a) the draft conceptual framework of the social and emotional skills constructs and (b) the core components and timelines of the proposed Feasibility Study scheduled in 2015-18. The draft ESP Social and Emotional Skills Framework, prepared by Oliver John (UC-Berkeley) and Filip de Fruyt (U-Ghent), takes into account discussions at the Experts Meeting held in The Hague on 2-3 February 2015. The Secretariat invites the Steering Group Members to reflect on the proposed framework in light of its relevance to education policies and practices in the countries and cities that they represent. The Steering Group’s recommendations will later be endorsed by the Centre for Educational Research and Innovation (CERI) Governing Board. The Secretariat will also present the progress of the Longitudinal Study of Skills Development in Cities with a particular emphasis on the proposed 9:00 – 9:30 9:30 – 9:45 9:45 – 10:45 10:45 – 11:00 11:00 – 12:15 12:15 – 13:30 13:30 – 15:00 15:00 – 16:00 16:00 – 16:30 16:30 – 17:30 17:30 – 18:00 Feasibility Study. Introduction: Objectives of the ESP Steering Group Meeting and OECD-EDU’s work on non-cognitive outcomes Debriefing of ESP International Report Launching Seminar: “Skills for Social Progress: The Power of Social and Emotional Skills” Session 1: Overview of the ESP longitudinal study Coffee break Session 2: Overview of the proposed Social and Emotional Skills Framework Lunch break Session 3: Relevance of the proposed framework given the policy contexts, cross-cultural consideration and state of available instruments. Session 4: Process and timelines of the Feasibility Study Coffee break Session 5: Process and timelines of the Feasibility Study (cont.) Conclusion and next steps Andreas Schleicher, OECD Dirk Van Damme, OECD Koji Miyamoto, OECD Oliver John, UC-Berkeley and Filip de Fruyt, U-Ghent All participants Maria Huerta, OECD All participants Koji Miyamoto, OECD Background documents  Revised proposal of the ESP longitudinal study of skill dynamics (OECD Secretariat)  Social and emotional skills framework (Oliver John, UC-Berkeley and Filip De Fruyt, U-Ghent) 6 First Name: 1 Julia 2 Tatiana 3 Jennifer 4 Patrick 5 Marie-France 6 Juan Ignacio 7 Oscar 8 Peter 9 Takashi 10 Hiromi 11 Andre 12 Tone 13 Elisabeth 14 Lars 15 Astrid 16 Ekaterina 17 Vicky 18 John 19 Oliver 20 Filip 21 Silvia 22 Maria 23 Hiroko 24 Koji 25 Marta 26 Andreas 27 Dirk Last Name: VALSKY FILGUEIRAS ADAMS BUSSIERE CHOUINARD VENEGAS MUGGLI SANCHEZ ARCHER MURAO SASAI DE MOOR ABRAHAMSEN BUK-BERGE NERDRUM SØGNEN OREL PETRIE EASTON JOHN DE FRUYT MONTOYA HUERTA IKESAKO MIYAMOTO RILLING SCHLEICHER VAN DAMME ANNEX 2. LIST OF PARTICIPANTS Representing country Austria Brazil (INEP, Ciera and IAS) Canada Canada Canada Chile Colombia Ireland Japan Japan Netherlands Norway Norway Norway Norway Russian Federation United Kingdom United States (Spencer Foundation) University of California, Berkeley Ghent University UNESCO Institute for Statistics (UIS) OECD OECD OECD OECD OECD OECD 7 Email EDU/CERI/CD/RD(2015)6 julia.valsky@bka.gv.at tatifilgueiras@ias.org.br jennifer.adams@ocdsb.ca Patrick.Bussiere@hrsdc-rhdcc.gc.ca marie-france.chouinard@international.gc.ca juan.venegas@mineduc.cl osanchez@sedbogota.edu.co peter.archer@erc.ie takashi.murao@mofa.go.jp sasai@nier.go.jp a.p.g.demoor@minocw.nl Tone.Abrahamsen@utdanningsdirektoratet.no Elisabeth.Buk-Berge@kd.dep.no Lars.Nerdrum@mfa.no astrid.sognen@ude.oslo.kommune.no ekaterina.orel@gmail.com Vicky.Petrie@education.gsi.gov.uk jqeaston@spencer.org o_johnx5@berkeley.edu Filip.DeFruyt@UGent.be s.montoya@unesco.org Maria.HUERTA@oecd.org Hiroko.IKESAKO@oecd.org Koji.MIYAMOTO@oecd.org Marta.RILLING@oecd.org Andreas.SCHLEICHER@oecd.org Dirk.VANDAMME@oecd.org