Evaluating Human Service Outcomes

Description

This all-in-one text assists human service practitioners, and the students of human service educational programs, in the evaluation of their practice with their clients. It takes readers through the entire research process, step by step, starting with the literature review on the nature of the behavior being served, to the development of their study methods, to the statistical analysis of data using the internet and, finally, to the drawing of conclusions based on the outcome study that was conducted. When readers complete this book, they will be prepared to conduct an outcome evaluation study and to present a report to their agencies or instructors.   Key distinctions of this text include:  guides for analysis of data using Excel, the internet or SPSS for statistical analysis of data; the separation of content into basic concepts and intermediate concepts for use in beginning and intermediate courses in human service research methods; an instructor's manual that offers outlines, lists, and test questions additional to those in the text; a student workbook with practice assignments for use in courses as well as a set of checklists that serve as a guide for various tasks in the research process; and objectives, summaries, and tests in all chapters. Evaluating Human Service Outcomes could be used as the basic text for a beginning course in human service research in educational programs in social work, counseling, and psychology where a major goal is to complete a research study. It could also be used as a supplemental text for advanced research courses that include the analysis of data. The text also should be of interest to human service practitioners who are working in programs funded by grants that require outcome evaluation. Preface Contents About the Author Chapter 1: The Essence of Outcome Evaluation Introduction Objectives Types of Human Service Evaluation Outcome Evaluation Other Types of Human Service Evaluation The Evaluation of Human Need The Evaluation of Service Quantity The Evaluation of Service Quality The Evaluation of Service Efficiency The Four Main Purposes of Human Service Research The Evaluation of Services The Description of People The Explanation of Things The Exploration of the Unknown The Process of Outcome Evaluation Step 1: Determine the Research Question and Study Purpose Step 2: Develop a Knowledge Base for the Study Step 3: Design the Evaluative Study Step 4: Collect and Analyze Data Step 5: Draw Conclusions Step 6: Describe the Service that Was Evaluated Don’t Put the Cart Before the Horse! Evidence-Based Practice as a Guide This Book The Organization of This Book Summary Chapter 2: Developing Your Knowledge Base Introduction Objectives Steps in the Process of Developing Your Knowledge Base Step 1: Presenting the Scope of Your Review Step 2: Finding Your Sources Step 3: Reviewing Your Sources Levels of Evidence Step 4: Writing Your Literature Review Summary References Chapter 3: Developing the Methods for Your Outcome Study Introduction Objectives Selecting Your Study Sample and Generalizing Your Study Findings Types of Samples Sampling Error Two Ways to Generalize Your Study Results Measuring Your Study Variables Defining Your Study Variables Qualitative and Quantitative Forms of Measurement Reliability and Validity in the Measurement of Psychosocial Variables Finding a Published Scale Designing Your Own Measurement Scale Defining the Variables You Are Measuring Constructing the Items for the Measurement Device Determining Your Research Design Causes of the Clients’ Measured Growth Group Research Designs One Group Pretest-Posttest Design Comparison Group Design Single-Subject Research Designs Single-Subject Research Designs that Fail to Control for Maturation The Limited AB Single-Subject Design The B Single-Subject Design Single-Subject Research Designs that Control for Maturation The AB Single-Subject Research Design Composing Your Study Hypothesis Summary References Chapter 4: Collecting and Analyzing Your Data Introduction Objectives Collecting Data Collecting Data from Human Subjects in an Ethical Manner Recording Your Data Developing Your Data Plan Selecting the Statistic for Your Outcome Study Preliminary Steps for Testing Your Study Hypothesis The Six Data Situations for Outcome Research Data Situations that Do Not Fit Selecting a Statistic for Describing Clients Common Descriptive Statistics for Data Recorded Numerically Common Descriptive Statistics for Categorical Data Selecting a Statistic for Explaining Client Gain Analyzing Your Data Reporting Your Results Summary Chapter 5: Using the Internet to Analyze Your Outcome Data Introduction Objectives Preliminary Steps Organizing Your Data The Six Data Situations for Outcome Evaluative Research Comparing Matched Pretest and Posttest Scores Example Steps in the Process of Comparing Matched Scores Comparing a Set of Scores to a Single Score Example Steps in the Process of Comparing a Set of Scores to a Single Score Comparing the Gain Scores of Two Groups Example Steps in the Process of Comparing the Scores of Two Groups Comparing Two Groups on the Basis of a Dichotomous Variable Example Comparing Multiple Treatment Scores to a Single Baseline Score for One Client Example Steps in the Comparison of Multiple Scores to a Single Score Comparing Multiple Treatment Scores to Multiple Baseline Scores for a Single Client Example Steps in the Comparison of Multiple Treatment Scores to Multiple Baseline Scores Summary Practice Assignment Research Case Examples Chapter 6: Using SPSS to Analyze Your Outcome Data Introduction Objectives Preliminary Steps Organizing Your Data The Six Data Situations for Evaluative Studies Comparing Matched Pretest and Posttest Scores Example Steps in the Comparison of Matched Scores Step 1: Establishing Your Data File Step 2: Entering Your Data Step 3: Analyzing Your Data Step 4: Reporting Your Results Comparing a Set of Scores to a Single Score for a Group of Clients Example Steps in the Comparison of a Set of Scores to a Single Score Step 1: Establishing Your Data File Step 2: Entering Your Data Step 3: Analyzing Your Data Step 4: Reporting Your Results Comparing the Gain Scores of Two Groups Example Steps in the Comparison of the Scores of Two Groups Step 1: Establishing Your Data File Step 2: Entering Your Data Step 3: Analyzing Your Data Step 4: Reporting Your Results Comparing Two Groups on the Basis of a Dichotomous Variable Example Steps in the Comparison of Two Groups Using a Dichotomous Variable Step 1: Establishing Your Data File Step 2: Entering Your Data Step 3: Analyzing Your Data Step 4: Writing Your Results Comparing Multiple Treatment Scores to a Single Baseline Score for One Client Demonstration Example Steps in the Comparison of a Set of Scores to a Single Score for One Client Step 1: Establishing Your Data File Step 2: Entering Your Data Step 3: Analyzing Your Data Step 4: Reporting Your Results Comparing Multiple Treatment Scores to Multiple Baseline Scores for a Single Client Steps in the Comparison of Multiple Treatment Scores to Multiple Baseline Scores for a Single Client Step 1: Establishing Your Data File Step 3: Analyzing Your Data Step 4: Reporting Your Results Summary Practice Assignment on the Testing of the Evaluative Hypothesis Chapter 7: Using Special Excel Files to Analyze Your Outcome Data Introduction Objectives Preliminary Steps Organizing Your Data The Six Data Situation for Evaluative Studies Comparing Matched Pretest and Posttest Scores Example Steps in Comparing Matched Scores Step 1: Organizing Your Data Step 2: Entering Your Data Step 3: Analyzing Your Data Step 4: Reporting Your Results Comparing a Set of Scores to a Single Score for a Group of Clients Example Steps in Comparing a Set of Scores to a Single Score Step 1: Composing Your Data Step 2: Entering Your Data Step 3: Analyzing Your Data Step 4: Reporting Your Results Comparing the Gain Scores of Two Groups Demonstration Example Steps in Comparing a Set of Scores for Two Groups Step 1: Composing Your Data Step 2: Entering Your Data Step 3: Analyzing the Data Step 4: Reporting Your Results Comparing Two Groups on the Basis of a Dichotomous Variable Demonstration Example Steps in Comparing Two Groups on the Basis of a Dichotomous Variable Step 1: Recording Your Data Step 2: Analyzing the Results Step 3: Reporting Your Results Comparing Multiple Treatment Scores to a Single Baseline Score for One Client Demonstration Example Step 1: Entering the Data into the Excel File Step 2: Analyzing the Data Step 3: Reporting the Results Comparing Multiple Treatment Scores to Multiple Baseline Scores for a Single Client Example Steps in the Comparison of Multiple Treatment Scores to Multiple Baseline Scores Analyzing the Data Step 3: Reporting Your Results Summary Practice Assignment on the Testing of the Evaluative Hypothesis Chapter 8: Describing Clients Introduction Objectives The Process of Descriptive Research Step 1: Determine the Purpose of the Study Step 2: Select the Study Sample Step 3: Decide What to Describe Step 4: Collect Your Data Step 5: Select the Statistic for Each Variable Analyzing Descriptive Data Using the Internet to Compute Descriptive Statistics Step 1: Enter the Calculator Soup Webpage Step 2: Enter Your Data into the Blank Box on the Screen Step 3: Calculate the Descriptive Statistics Reporting the Results of Your Descriptive Study Summary References Chapter 9: Explaining Client Outcomes Introduction Objectives The Issue of Causation The Steps in the Explanatory Research Process Step 1: Determine the Purpose, Research Question, and Knowledge Base Step 2: Develop Your Explanatory Research Hypothesis Step 3: Collect and Record Your Data Step 4: Analyze Your Data Step 5: Report Your Results Analyzing Your Explanatory Data Data Situations Previously Addressed in This Book Additional Data Situations for Explanatory Research Data Situation A: Using the Correlation Coefficient to Examine the Relationship Between Two Interval Variables Example Option 1: Using the Internet as the Mechanism for Computing the Correlation Coefficient Option 2: Using SPSS to Compute the Correlation Coefficient Reporting the Results and Conclusions for the Correlation Example Data Situation B: Using ANOVA When You Are Comparing the Scores of Several Groups Example Option 1: Using the Internet to Test Your Hypothesis Using ANOVA Option 2: Using SPSS to Test Your Hypothesis Using ANOVA Reporting the Results of the Study Using ANOVA Summary Reference Chapter 10: Getting Ideas on How to Improve Service Through Qualitative Surveys Introduction Objectives Quantitative and Qualitative Measurement The Essence of Qualitative Research Exploratory Research and Qualitative Measurement The Nature of Qualitative Data Approaches to Qualitative Research The Social Survey Steps in the Process of Conducting a Social Survey Step 1: Determine the Purpose of the Survey Step 2: Select Your Study Sample Step 3: Design Your Measurement Tool Step 4: Administer the Survey Step 5: Analyze Data Step 6: Draw Conclusions One Model for Content Analysis of Qualitative Data Step 1: First-Level Coding Step 2: Credibility Assessment of First-Level Codes Step 3: Second-Level Coding Step 4: Enumeration of Second-Level Codes Other Steps Drawing Conclusions An Exercise in the Content Analysis of Qualitative Data Description of the Study The 2019 Cohort The 2022 Cohort Tasks in the Content Analysis of These Data Summary Chapter 11: Writing Your Research Report Introduction Objectives Reporting the Purpose of Your Study and the Knowledge Base Reporting Your Study Purpose and Research Question Reporting Your Knowledge Base Reporting Your Study Methods Reporting Your Study Sample Describing Your Measurement Tools Stating Your Study Hypothesis Describing Your Research Design Writing Your Results and Conclusions Describing the Service Being Evaluated Describing the Objectives of the Service Describing the Structure of the Service Describing the Personnel of the Service Describing the Model of the Service Summary of Your Description of the Service Summary References Chapter 12: Facing the Challenges for Outcome Evaluation Introduction How Can We Justify the Resources Expended for Our Services? Is Our Knowledge Base a Sufficient Guide? Can We Generalize Our Findings on a Logical Basis? Does Our Measurement Tool Pass the Test of Face Validity? Do We Know that Our Service Was Delivered According to Promise? Does Our Research Design Have to Control for Normal Growth over Time? Why Should We Be Concerned with Statistical Significance? How Do We Know if We Have Practical Significance? Are Our Conclusions Consistent with Our Data? Did We Put the Cart Before the Horse? Summary References Appendix: Inventory of Critical Research Concepts (N = 111) AB Single-Subject Research Design Alternative Treatment Design Analysis of Variance (ANOVA) B Single-Subject Research Design Bar Chart Baseline Period Causation Chance Cherry Picking Chi Square Code in Content Analysis of Qualitative Data Comparison Group Comparison Group Research Design Content Analysis in Qualitative Research Content Validity Correlation Correlation Coefficient Criterion Validity Data Forms Data Plan Descriptive Study Descriptive Statistics Dichotomous Data Directional Hypothesis Effect Size Empirical Relationship Evaluative Study Evidence-Based Practice Excel Files for Data Analysis Explanatory Study Experimental Group Research Design Face Validity False Positive and False Negative Fisher Exact Test Frequency Generalization of Study Results GraphPad Group Research Designs One-Group Pretest-Posttest Research Design One-Sample t Test Open-Ended Questions on a Survey Outcome Objective History as a Threat to Internal Validity Human Service Outcome Hypothesis Hypothesis Testing Independent t Test Inferential Statistics Institutional Review Board Internal Consistency Reliability Interval Level of Measurement Limited AB Single-Subject Research Design Logical Generalization Logical Justification of a Service Matched Scores Maturation as a Threat to Internal Validity Mean Measurement Error Median Meta-Analysis of Evidence Measurement Error Median Mode Model of a Service Narrative Analysis of Qualitative Data Nominal Level of Measurement Null Hypothesis One-Group Pretest-Posttest Research Design One-Sample t Test One-Tailed Test Ordinal Level of Measurement Paired t Test Pie Chart Posttest-Only Control Group Design Proportion Pretest Scores and Posttest Scores Qualitative Measurement Quantitative Measurement Random Sample Range Ratio Level of Measurement Reliability Research Designs Research Types Sample Sampling Error Sample Types Scientific Generalization Scientific Justification for the Delivery of a Given Service Single-Subject Research Designs Social Desirability Bias Spearman Rank Correlation Coefficient SPSS Standard Deviation Statistics Statistical Significance Structure of the Service Study Population Systematic Review of Evidence Systemic Literature Review Systematic Random Sampling Procedure Test-Retest Reliability Threats to Internal Validity Treatment Group Treatment Period Two-Tailed Test Validity Variable Variance Index