Improving Social Maturity of Cybersecurity Incident Response Teams

Description

FrontCover Acknowledgements Project Team Table of Contents List of Figures List of Tables List of Acronyms Executive Summary: Quick Reference Guide Preface Chapters Chapter One Chapter Two Chapter Three Chapter Four Chapter Five Chapter Six Chapter Seven Chapter Eight Chapter Nine Chapter Ten Chapter Eleven Appendices Appendix A Appendix B Appendix C Appendix D Appendix E Appendix F Appendix G Appendix H Appendix I Figure 1.1 Projects Comprising the CSIRT Effectiveness Research Program Figure 2.1 Elements of CSIRT Social Maturity Figure 2.2 Example of a Cybersecurity Incident Response Multiteam System Figure 2.3 Frequency of all 7 Typical Teams Across 28 CSIRTs Figure 2.4 Challenges in CSIRT MTS Collaboration Figure 4.1 Cybersecurity Incident Response Decision-Making Model Figure 5.1 Communication as a Driver of CSIRT Effectiveness Figure 5.2 Endorsement of Communication Themes by CSIRT Type. Figure 6.1 Cybersecurity Incident Response Information Sharing Model Figure 8.1 Focus Group Support for SKUE Figure 8.2 Example of a Team Knowledge Map Depicting Members of a Team and Their Areas of Expertise (or Specialist Skills) Figure 9.1 Facets of Trust (Sources: Bromily & Harris, 2006; Cook & Wall, 1980; Mayer, Davis, & Schoorman, 1995; McAllister, 1995) Figure 9.2 Focus Group Support for Trust by CSIRT Type. Table 1.1 Collaboration Chillers in CSIRTs: Quotes from the Field Table 1.2 Comparison of Top 20 KSAOs Table 2.1 An Example of an Incident Response MTS Goal Hierarchy (adapted in part from Zaccaro, et al., 2016) Table 2.2 Component Teams in a Typical Large MTS CSIRT Table 3.1 Examples of Proactive and Reactive Performance (Quotes From Analyst Interviews) Table 3.2 Examples of Tasks at Different Levels (Quotes From Analyst Interviews) Table 3.3 Examples of Organizational Citizenship Behavior and Counterproductive Work Behavior That Managers Should Document Table 3.4 Psychological Outcome Categories Assessed Using Self-Ratings Table 4.1 Top Three Decision-Making Knowledge, Skills, Abilities, and Other attributes (KSAOs) Table 5.1 Principles of Communication in Incident Response Table 5.2 Incident Response Cycle Communication Examples Table 5.3 Communication Principles and Strategies to Improve Them Table 7.1 Collaborative Problem-Solving Process Model Table 7.2 CSIRT Reactive Problem-Solving Behaviors from our Focus Groups and Interviews Table 7.3 CSIRT Proactive Problem-Solving Steps and Focus Group Support Table 7.4 CSIRT Staffing Table 10.1 Sample Shift Schedule, Including Breaks, for an Analyst FrontCover FrontCover FrontCover FrontCover 1.0 Introduction to the Handbook 1.0.1 Team Collaboration Failures 1.0.2 The Importance of Social Processes to Cybersecurity Incident Response 1.1 Research Foundation for the Handbook 1.2 Major Research Themes and Findings 1.2.1 Category 1: The Nature and Environment of CSIRT Work Theme 1: Cybersecurity incident responders perform individual and collective knowledge work. Theme 2: Cybersecurity incident responders often need to work within volatile, uncertain, complex, and ambiguous environments (i.e., “VUCA”; Stiehm, 2002; Scott, 2012). Theme 3: Maintaining vigilance (i.e., sustained attention and focus over time) is a substantial problem because of the length of shifts and the nature of CSIRT work. Theme 4: Cybersecurity incident response occurs at multiple levels, including individual, team, and multiteam systems. Theme 5: Incident response collaboration within and between incident responders and teams is typically discretionary. Theme 6: What constitutes good performance among cybersecurity incident responders is not well understood. Performance should be evaluated directly using appropriate metrics—not indirectly (e.g., not only using existing maturity models). 1.2.2 Category 2: Individual and Collective Drivers of CSIRT Effectiveness Theme 7: Four sets of knowledge, skills, abilities, and other attributes (KSAOs) are necessary for effective cybersecurity incident response work. These include: technical skills, cognitive abilities, social skills, and personal character. Theme 8: Team- and MTS-level states and protocols also contribute to CSIRT effectiveness. 1.2.3 Category 3: Fostering Persistent CSIRT Excellence Theme 9: Adaptation and resilience across all levels are vital to effective cybersecurity incident response. Theme 10: Effective cybersecurity incident response work requires continuous learning across all levels. 1.3 Summary FrontCover 2.0 Introduction 2.1 Why are Effective Social Dynamics Important in CSIRTs? 2.1.1 The Elements of Social Maturity in CSIRTs 2.2 CSIRT Multiteam Systems 2.2.1 What is a Multiteam System? 2.2.2 MTS Goal Hierarchies 2.2.3 Internal Versus External MTSs 2.2.4 Forming CSIRT MTSs 2.3 Project Findings: Typical CSIRT MTSs 2.4 Challenges in Managing MTSs 2.5 Strategies for CSIRT MTS Managers 2.6 Summary FrontCover 3.0 Introduction 3.1 Assessing Performance Measurement 3.2 Background 3.2.1 Why is a Comprehensive Approach to Performance Metrics Important for CSIRTs? 3.2.2 Issues with Measurement of Performance Errors of Omission (Measurement Deficiency) Measuring quality versus quantity. Measuring maximum versus typical performance. Measuring proactive versus reactive performance. Measuring performance at different levels of analysis. Errors of Commission (Measurement Contamination) 3.2.3 Performance-related Outcome Categories Performance Outcomes Assessed Using Conventional, Objectively-derived Metrics Performance Outcomes Assessed Using Subjectively-derived Ratings [1] Psychological Outcomes Assessed Using Subjectively-derived Ratings 3.3 Strategies for Designing a More Complete Performance Measurement Program 3.3.1 Strategy 1: Balance Measuring Quantity and Quality 3.3.2 Strategy 2: Measure Maximum Performance in Addition to Typical Performance 3.3.3 Strategy 3: Measure Both Proactive and Reactive Performance 3.3.4 Strategy 4: Determine the Appropriate Level of Measurement 3.3.5 Strategy 5: Create a Balanced Scorecard of Performance Measurement 3.4 Chapter Summary FrontCover 4.0 Introduction 4.1 Assessing Decision-Making Capacity 4.2 Background 4.2.1 The Psychological Process of Incident Response Decision-Making 4.2.2 How Decision-Making Can Go Awry Expert Versus Novice Decision-Making Decision-Making Problems Affecting Both Experts and Novices The Role of Incident Severity 4.3 Strategies for Improved Decision-Making 4.3.1 Strategy 1: Selecting for Decision-Making Skills Problem Sensitivity Critical Thinking Information Ordering 4.3.2 Strategy 2: Training Decision-making Skills Structured Troubleshooting Training Critical Thinking Training Expert Modeling 4.3.3 Strategy 3: Cognitive Prompts to Reduce Overconfidence and Confirmation Bias Five-Why Analysis Premortem 4.3.4 Strategy 4: Using Mnemonics to Capture Necessary Information 4.3.5 Strategy 5: Using Adaptive Case Management 4.4 Chapter Summary FrontCover 5.0 Introduction 5.1 Assessing Communication Skills 5.2 Background Information and Project Findings 5.2.1 Principles of Effective Communication Communication Errors and Team Failure Three Common Challenges to Effective CSIRT Communication Time Urgency Team Dispersion The Impact of National Cultural Differences on Communication 5.2.2 Project Findings 5.3 Developing Communication Skills in CSIRTs 5.3.1 Strategy 1: Require teams or MTSs to complete a team charter to plan how, between whom, and when communication will happen 5.3.2 Strategy 2: Implement checklists and handoff tools to prevent information loss during handoffs 5.3.3 Strategy 3a: Use scenario-based training approaches to engage members in role-play 5.3.4 Strategy 3b: Engage teams and MTSs in simulation-based training 5.3.5 Strategy 4a: Design a virtual display that all team members can use to monitor information 5.3.6 Strategy 4b: Apply best practices to make wikis more effective 5.3.7 Strategy 5: Assign a team member to act as the point of contact for between-team communication in a CSIRT MTS 5.3.8 Strategy 6: Design the work space to increase communication 5.3.9 Strategy 7: Make team staffing decisions by using situational interviews to assess communication skills 5.4 Chapter Summary FrontCover 6.0 Introduction 6.1 An Organizational Science Perspective on CSIRT Information Sharing 6.2 Elements of Information Sharing in CSIRTs 6.2.1. Evidence on Information Sharing from our Study 6.2.2 Degrees of Interaction between Information Sharing Partners Passive Information Sharing Between Partners Information Sharing During Handoffs and Escalation Active Interaction between Information Sharing Partners 6.2.3 Recommendations for Effective Passive and Active Information Sharing 6.2.4 Incident Response Process Requirements Mandatory Information Sharing Discretionary Information Sharing 6.2.5 Recommendations for Effective Mandatory and Discretionary Information Sharing 6.2.6 Information Sharing at Various Levels Dyadic Information Sharing Within-team Information Sharing Multiteam System Information Sharing Intra-Organizational Information Sharing Inter-Organizational Information Sharing 6.2.7 Recommendations for Effective Information Sharing at Various Levels 6.3 Summary FrontCover 7.0 Introduction 7.1 Assessing Collaborative Problem-Solving Capacity 7.2. Background 7.2.1 Collaborative Problem-Solving Processes Shared Situational Awareness Collective Information Processing Collective Solution Forecasting 7.2.2. Adaptive Problem-Solving in CSIRTs 7.3 Project Findings 7.4 Improving CSIRT Collaborative Problem-Solving 7.4.1 Strategy 1: Engage in pre-mission planning (or “pre-briefing”) 7.4.2 Strategy 2: Use a counterfactual thinking approach to get team members to share unique information 7.4.3 Strategy 3: Provide team feedback during structured debriefing 7.4.4 Strategy 4: Develop adaptive thinking by providing exploratory or active learning experiences with wide problem variety 7.4.5 Strategy 5: Train leaders to pre-plan strategies for how multiple teams will work together 7.4.6 Strategy 6: Staff your CSIRT with team members who have a team orientation and teamwork skills 7.5 Chapter Summary FrontCover 8.0 Introduction 8.1 Assessing Shared Knowledge of Unique Expertise 8.2 Background 8.3 Project Findings 8.4 Developing Shared Knowledge of Unique Expertise 8.4.1 Strategy 1: Establish knowledge tools (e.g. information board, knowledge map) that display members' expertise, knowledge, skills, and experiences 8.4.2 Strategy 2: Train team members in areas other than their specialty Lecture/Presentation Job Shadowing Position Rotation 8.5 Summary FrontCover 9.0 Introduction 9.1 Assessing Team Trust 9.2 Background 9.2.1 Swift Trust 9.2.2 Deep Trust 9.2.3 Team Climate Conflict and Trust 9.2.4 Trust between Teams, Organizations and External Parties 9.3 Project Findings 9.4 Developing Team Trust 9.4.1. Strategy 1: Provide structured opportunities for CSIRT members to learn about the expertise, experiences, and functional backgrounds of other members 9.4.2. Strategy 2: Establish clear individual and team goals, roles, and performance standards 9.4.3 Strategy 3: Establish norms for communication transparency in teams and MTSs 9.4.4 Strategy 4: Utilize managerial actions that create a psychologically safe climate in the team and the MTS 9.4.5 Strategy 5: Create opportunities for building strong social connections among CSIRT members to support conflict management 9.4.6 Strategy 6: Increase external connections and social networking to facilitate inter-team and inter-organization trust 9.5 Chapter Summary FrontCover 10.0 Introduction 10.1 Assessing CSIRT Capacity for Sustained Attention 10.2 Background 10.2.1 The Importance of Sustained Attention During Incident Response 10.2.2 Sustained Attention in Relevant Professions 10.3 Project Findings 10.3.1 CSIRT Positions That Require Sustained Attention 10.3.2 Knowledge, Skills, Abilities, and Other Attributes (KSAOs) Relevant to Sustained Attention 10.3.3 Cognitive Abilities Relevant to Sustained Attention 10.4 Improving Sustained Attention and Focus over Time 10.4.1 Strategy 1: Hire Job Applicants Who Display a Capacity For Sustained Attention Working Memory Task Brief Vigilance (i.e., Sustained Attention) Tasks 10.4.2 Strategy 2: Encourage Employees to Incorporate Rest Breaks Into Their Shifts Restorative Settings Socialization 10.4.3 Strategy 3: Shift Design – Create a Shift Plan That Reduces Sleep Disturbances and Maximizes Attentiveness Work Shift Characteristics Shift Length (Eight-Hour Shifts Recommended) Shift Rotation Speed (Rapid Shift Rotations Preferred) Shift Rotation Direction (Forward Shift Rotation Preferred) 10.5 Chapter Summary FrontCover 11.0 Introduction 11.1 Assessing Continuous Learning 11.2 Background 11.2.1 Creativity and Curiosity 11.2.2 Developmental Networks and Networking Behavior 11.2.3 Team Learning Knowledge Sharing Knowledge Storage and Retrieval 11.3 Project Findings 11.3.1 Findings Relevant to Creativity and Curiosity 11.3.2 Findings Relevant to Networking 11.3.3 Findings Relevant to Team Learning 11.4 Strategies and Recommendations 11.4.1 Strategy 1: Selection of individuals who are creative and curious 11.4.2 Strategy 2: Leader behaviors to encourage learning Recommendation 1. Engage employees’ creativity and curiosity Recommendation 2. Facilitate reflection in teams (team reflexivity, or team reflections, and adaptation) Recommendation 3. Provide feedback in debriefings (after-action reviews) Recommendation 4. Promote trust and respect among team members 11.4.3 Strategy 3: Design work to enhance learning and development Recommendation 5: Improve work design to enhance learning Recommendation 6. Put in place mentoring programs 11.4.4 Strategy 4: Training Recommendation 7. Train for networking skills Recommendation 8. Train CSIRT professionals on how to establish a professional developmental network Recommendation 9. Guided discovery learning Recommendation 10. Error management training 11.5 Chapter Summary FrontCover FrontCover A.1 Introduction A.2 Development of Taxonomic Dimensionality A.3 Procedure for Taxon Specification A.3.1 Generation of Taxa A.3.2 Taxon Validation A.4 How CSIRT Managers Can Use the Performance Taxonomy A.4.1 Preparing Position Descriptions A.4.2 Developing Performance Evaluation and Management Tools A.5 Designing Training Programs A.6 Informing CSIRT Process Models References FrontCover B.1 Assessment Exercises and Improvement Strategies by Topic Area FrontCover C.1 Introduction C.2 Hiring Validation Considerations C.2.1 Test Reliability C.2.2 Reliability in CSIRT Applicant Testing C.2.3 Test Validity C.2.4 Further Reading C.3 Training Validation Considerations C.3.1 Increasing the Performance Impact of Training 1. Conduct Training Needs Assessment 2. Ensure Employee Training Readiness 3. Create a Learning Environment 4. Garner Training Support 5. Develop an Evaluation Plan 6. Select Training Method 7. Monitor and Evaluate Training C.3.2 Further Reading FrontCover FrontCover FrontCover F.1 Executive Summary F.2 Introduction F.2.1 Cybersecurity Incident Response Teams (CSIRTs) F.3 Recommendations to Improve CSIRT Effectiveness F.3.1 Military Response Teams F.3.2 Emergency Medical Service Teams F.3.3 Nuclear Power Plant Operating Teams F.4 General Discussion F.4.1 Top Recommendations Overall F.4.2 Top Recommendations for Particular Needs F.4.3 Change Management F.5 Conclusion FrontCover G.1 Introduction G.2 Communication G.2.1 Shared Knowledge of Unique Experience (SKUE) G.2.2 Trust G.3 Identifying Important KSAOs for Cybersecurity Incident Response Team Professionals G.3.1 Initial Identification of KSAOs G.3.2 Reduction of KSAOs G.3.3 Expansion of Sources to Organizational Science Literature G.3.4 Summary of Identification of Relevant KSAOs G.3.5 KSAO Survey G.3.6 Comparison of Importance Ratings for the 20 Most Important KSAOs by Category G.3.7 Comparison of Importance Ratings for the 26 Least Important KSAOs by Category G.4 Summary and Conclusions FrontCover H.1 Acknowledgements H.2 Target Audience H.3 Introduction H.4 Networking Defined H.5 Types of Networks H.6 Development and Maintenance of Networks H.6.1 Network Mapping H.6.2 Networking in Work Teams H.7 Conclusion FrontCover I.1 Purpose I.2 Target Audience I.3 Introduction I.4 Why Resilience Matters in Cybersecurity I.4.1 THE IMPACT OF STRESS The Financial Impact of Stress due to Cyber Attacks I.4.2 SOCIAL RESILIENCE IN CYBERSECURITY INDIVIDUAL RESILIENCE PERSONALITY TRAITS SOCIAL SUPPORT COPING AND PROBLEM-SOLVING I.4.3 COLLECTIVE (TEAM) RESILIENCE TEAM PROCESSES CHARACTERISTICS OF RESILIENT TEAMS I.4.4 HOW TO ENHANCE AND DEVELOP SOCIAL RESILIENCE Battlemind Training and the Comprehensive Soldier Fitness Program Hardiness Training Stress Management and Resiliency Training (SMART) Additional Ways Leaders Can Enhance the Social Resilience of CSIRTs I.5 Conclusion