Assessing Competencies for Social and Emotional Learning: Conceptualization, Development, and Applications

Description

Assessing Competencies for Social and Emotional Learning explores the conceptualization, development, and application of assessments of competencies and contextual factors related to social and emotional learning (SEL). As programs designed to teach students social and emotional competencies are being adopted at an ever‐increasing rate, new measurements are needed to understand their impact on student attitudes, behaviors, and academic performance. This book integrates standards of fairness, reliability, and validity, and lessons learned from personality and attitude assessment to facilitate the principled development and use of SEL assessments. Education professionals, assessment developers, and researchers will be better prepared to systematically develop and evaluate measures of social and emotional competencies. Cover Half Title Title Page Copyright Page Dedication Table of Contents List of Contributors Foreword The “Learning” Part of SEL Assessing SEL The Need for This Volume References Chapter 1: Introducing Assessing Competencies for Social and Emotional Learning: Conceptualization, Development, and Applications Measuring SEL: Knowing versus Not Knowing Structure of the Book Social and Emotional Learning Assessment: A First Step to Knowing Notes References Part I: Conceptualization Chapter 2: An Integrative Perspective on SEL Frameworks Introduction Some Common Frameworks The CASEL 5 The National Research Council’s 21st-Century Competencies National SEAD Commission Big Five-Based Frameworks The Big Five (and OECD) Model ACT’s Behavioral Skills Framework Behavioral, Emotional, and Social Skills Model Competency Modeling Defining SEL Skills Comparing and Contrasting Frameworks Toward a Transtheoretical Framework Context Relevant Developmentally Appropriate Culturally Relevant Hierarchical Measurable Actionable Parting Thoughts Note References Chapter 3: A Developmental Framework for SEL Assessment: Context, Culture, and Equity Applying a Developmental Perspective to the Assessment of SEL Competencies Change Continuity Contexts that Support SEL Skill Development and Inform Assessment Over Time Assessing Social-Emotional Learning Competencies Through a Developmental Lens Culturally and Linguistically Appropriate SEL Assessment Current Challenges in Equitable SEL Assessment State SEL Standards Summary and Future Directions References Chapter 4: Developing a Validity Argument for Social and Emotional Learning Assessments: Consideration of Uses and Evidence Validity and Validity Theory The History of SEL Assessment Use Use of SEL Assessment in Military Settings Use of SEL Assessment in Workforce Settings Use of SEL Assessment in Academic Settings Formative Use Cases High-Stakes Use Cases Validity Evidence and the Development of a Validity Argument Supporting the Use of SEL Assessments Evidence Based on Test Content Challenges Evidence Based on Response Processes Challenges Evidence Based on Internal Structure Challenges Evidence Based on Relations to Other Variables Challenges Evidence for Validity and Consequences of Testing Challenges Lessons Learned and Recommendations for Future SEL Research Note References Chapter 5: Standards for SEL Assessment We Have Standards The Standards Purposes of the Standards Our Ethical Imperative Enforcement Standards and Assessment Quality Indicators of Quality and Fairness Standards Regarding Fairness Standards Regarding Assessment Specifications Standards and Documentation Documentation Regarding Assessment Interpretation and Use Supporting Intended Interpretations Measurement Invariance Documentation of Assessment Development Documentation of Validity Evidence Validation Samples and Contexts Content Evidence Response Processes Internal Structure Associations with Other Variables Documentation of Score Reliability and Precision Internal Consistency Reliability Test-Retest Reliability Parallel Form Reliability Rater Reliability Standard Error of Measurement for Score Interpretation Latent Trait Precision (Reliability) Factors that Affect Score Reliability Documentation Supporting Assessment Administration, Scoring, and Reporting Assessment Administration Scoring and Reporting Standards and Specific Uses Education Settings Clinical and Private Settings Work Settings Program Evaluation, Policy Studies, and Accountability Systems Standards Supporting Rights and Responsibilities Participant Rights Participant Responsibilities User Rights User Responsibilities Shared Rights and Responsibilities Securing the Substantial Benefits of SEL Assessments References Part II: Development Chapter 6: Principled and Practical Approaches to Developing SEL Assessments Key Considerations in the Development of SEL Assessments Purpose and Target Population Specification of SEL Construct(s) to be Assessed Selection of Assessment Approach Item Generation An Example from the SSIS SEL Brief Scales: Purpose, Approach, and Item Generation Initial Field Test and Standardization Reliability, Validity, and Bias Scores and Score Reporting Facilitating Responsible and Fair Use Commitment to Continuous Improvement Conclusion Note References Chapter 7: Self-report and Observer Ratings: Item Types, Measurement Challenges, and Techniques of Scoring Introduction to Social and Emotional Learning (SEL) Assessment Common Methods of SEL Assessment Graded Response Questions Situational Judgment Tests Multidimensionality of the Self-report Response Process: Defining Different Types of Response Biases Response Styles and Response Sets Self-report Response Processes and Cognitive Biases The Nomological Net of Response Styles Item Designs and Techniques for Improving the Validity of Self-and Observed-report Items Acquiescence Measurement and Control, Positive and Negative Pole Items, and Noise-canceling Mechanisms Social Desirability Control via Experimental Design Controlling Social Desirability Via a Neutralization Procedure Controlling Bias via Forced-choice and Extended Formats Item Scoring and Scaling Methods Via Multidimensional Item Response Theory MIRT for Acquiescence Social Desirability Thurstonian Item Response Theory for Forced-choice Tests Conclusion References Chapter 8: Novel Technology-Based SEL Assessments Core Competencies Examples of Technology-based Assessments to Measure Social and Emotional Competencies Self-awareness Self-management Social Awareness Relationship Skills Responsible Decision-making Multidimensional Skill Assessment Trade-offs Between Paper-based and Technology-based Assessment for Social and Emotional Competence Considerations for Paper-based Assessments Considerations for Technology-based Assessments Usability Security Equity Affordability A Vision for the Future of Technology-based Assessment for Social and Emotional Competence Increasing Ecological Validity Increasing Access Diversity Assessment Purpose Bridging the Gap Between Curricula and Assessment Where Else is the Field Going? Concluding Thoughts References Chapter 9: When Actions Speak Louder than Words: Stealth Assessment of Social and Emotional Skills Introduction Common Approaches Self-report Other-rating Situational Judgment Test Innovative Approaches: Stealth Assessment Design Principles for SEL Stealth Assessment Assessment of Collaboration and Problem-solving Skills Studies: Circuit Runner and Science Fair Design Challenge Study #1: Circuit Runner Methods for Study #1 Findings for Study #1 Discussion for Study #1 Study #2: Science Fair Design Challenge Methods for Study #2 Findings for Study #2 Discussion for Study #2 Conclusions Notes References Chapter 10: Connecting Measurement and Action: Reporting Results in the SEL Context Introduction Purposeful Reporting in the SEL Competency Context Professional Standards for Reporting Activities Report Components and Contents: The Data Story Paradigm The Data of Results Reports Major Components A “Typical” Report Report Dissemination Formats A Report Development Process Phase 1 Phase 2 Phase 3 Phase 4 Challenges, Opportunities, and Conclusions References Part III: Application Chapter 11: Assessment of Social and Emotional Learning for Admissions and Selection Introduction Current SEL Focused Admissions Assessments Letters of Recommendation Personal Statements Interviews and Interview Alternatives Accomplishments and Past Experiences Personality Assessments Situational Judgment Tests Conclusion References Chapter 12: Multi-Informant Formative and Summative Assessments of SEL Skill Development SSIS SEL Assessments: Foci, Functions, Frameworks, and Fairness Foci of Assessments: Behaviors, Skills, and Competencies as Indicators of the SEL Construct Formative and Summative Assessment Functions Interpretive Frameworks The SEL Competency-Referenced Performance Framework (CRPF) A Dual-Factors Mental Health Model that Includes SEL Competencies The Emotional Behavior Criterion-Referenced Concerns Framework (CRCF) Fairness of SEL Assessments Technical Characteristics of the SSIS SEL Assessments SSIS SEL Brief Scales SSIS SEL Brief + Mental Health Scales SSIS SEL Screener/Progress Monitoring Scales Challenges in Using Assessments to Answer Questions and Advance SEL Practices Alignment to Content Standards/Competency Framework Integrating Assessment Evidence from Multi-Informants and/or Multiple Measures Information Needs and Assessment Burden Conclusions Note References Chapter 13: School Climate Assessments: Measuring the Conditions that Support Students’ Social and Emotional Competencies Introduction What is School Climate? School Climate and the Science of Learning and Development School Climate and SEL Measuring School Climate Frameworks for School Climate Surveys National School Climate Council Framework Conditions for Learning Framework Safe and Supportive Schools Framework Effective School Climate Assessment Considerations for Selecting a School Climate Survey Example School Climate Surveys Adapting School Climate Surveys to Changing Needs Turning Research Into Action Moving Forward with School Climate Assessment Conclusion References Chapter 14: Social and Emotional Learning (SEL) Skills Cannot Thrive on Shaky Foundations: Preventing and Detecting Mental Health Problems in School-Based Programs Review of Existing Measures The Well-Being Index Assessing Mental Health and Potential Determinants: Background of our Programmatic Work The High-achieving Schools Survey: Development and Refinement Applications of the HASS to Address Problems in SEL Flexibility in Measurement: Capturing Context-specific Influences COVID-19 and School Closures: Creation of the SRS and FRS Faculty and Staff Resilience Applications of the FRS in Schools Future Directions in Work on SEL Conclusion References Chapter 15: Social and Emotional Learning Assessment: Building a Rigorous Foundation in Support of an Innovative, Equitable Future Conclusion Note References Index