Quantitative Psychology: The 85th Annual Meeting of the Psychometric Society, Virtual

Description

Preface Contents A Rotation Criterion That Encourages a Hierarchical Factor Structure 1 Introduction 2 Methods 2.1 Proposed Rotation Criterion Function 2.2 Computational Techniques 2.3 Simulation Design 3 Results 4 Discussions References Comparison Between Different Estimation Methods of Factor Models for Longitudinal Ordinal Data 1 Introduction 2 Generalized Linear Latent Variable Models for Longitudinal Ordinal Data 3 Model Estimation 3.1 Pairwise Likelihood Approach 3.2 Dimension-Wise Quadrature Method 4 Simulation Study 5 Real Data Analysis 6 Discussion References An Efficient Scheduling Algorithm for Parallel Planar Rotations of Factors 1 Factor Rotation and Planar Rotations 1.1 General Factor Rotation Problem 1.2 Factor Rotation via Component Planar Rotations 2 Scheduling Parallel Orthogonal Planar Rotations 2.1 1-Factorizations of Complete Graphs 2.2 The Proposed Scheduling Algorithm 2.3 Case with More Than s Processing Threads 2.4 Case with Less than s Processing Threads 2.5 Case with an Odd Number of Factors 2.6 Practical Considerations 3 Simulated and Real Examples 3.1 A Simulated Text Mining Example 3.2 A Real Data Example About Human Activity Recognition by Smartphones 4 Conclusions References Explanatory Response Time Models 1 Introduction 2 Explanatory Response Time Models 2.1 Descriptive-Only Response Time Model 2.2 Person-Explanatory Response Time Model 2.3 Item-Explanatory Response Time Model 3 Real Data Example 3.1 Descriptive-Only Model 3.2 Person-Explanatory Model and Covariates 3.3 Item-Explanatory Model and Covariates 3.4 Model Estimation Results 4 Simulation 4.1 Simulation Design 4.2 Simulation Results 5 Discussion References Response Time Relationships Within Examinees: Implications for Item Response Time Models 1 Introduction 2 Method 2.1 Subjects 2.2 Test 2.3 Variables Analyzed 3 Results 3.1 Between Person and Between Item Correlations 3.2 Within Person Correlations 4 Discussion References Nonlinear Latent Effects in Diagnostic Classification Modeling Incorporating Response Times 1 Introduction 1.1 Nonlinear Joint Cognitive Diagnostic Modeling 1.2 Bayesian Parameter Estimation 2 Method 2.1 Real Data Analysis 2.2 Simulation Study 3 Results 3.1 Real Data Analysis 3.2 Simulation Study 4 Discussion References Sequential Monitoring of Aberrant Test-Taking Behaviors Based on Response Times 1 Introduction 2 Sequential Procedures for Detecting Aberrant Test-Taking Behaviors 2.1 CUSUM-Based Approach 2.2 SGLRT-Based Approach 2.3 Moving Sample Strategy 3 Simulation Study 3.1 Data Generation and Simulation Design 3.2 Evaluation 3.3 Results 4 Application to Process Data 4.1 Data 4.2 Results 5 Discussion References Estimating Approximate Number Sense (ANS) Acuity 1 Approximate Number Sense Acuity: Psychophysical Scaling Models as Item Response Models 2 An Application to Data: Magnitudes 2.1 Least Squares Estimation of ANS 2.2 Latent Trait Estimation of ANS 2.3 Results 3 Conclusion References Differences in Symbolic and Non-symbolic Measures of Approximate Number Sense 1 Introduction 2 Method 3 Measurements 4 Results 4.1 Symbolic vs Non-symbolic ANS 4.2 Relationship of ANS to Other Measures of Math Ability 5 Discussion References Formulas of Multilevel Reliabilities for Tests with Ordered Categorical Responses 1 Introduction 2 Reliabilities 2.1 CTT Approach 2.2 CFA Approach 3 Multilevel Reliabilities 3.1 Multilevel Confirmatory Factor Analysis (MCFA) 3.2 Multilevel Reliabilities 4 Multilevel Reliability for Categories Responses 5 Conclusions References Polytomous IRT Models Versus IRTree Models for Scoring Non-cognitive Latent Traits 1 Introduction 2 Traditional Polytomous IRT Models 2.1 The Graded Response Model 2.2 The Generalized Partial Credit Model 3 Non Traditional Polytomous IRT Models 3.1 IRTree Models 4 Application 4.1 Data 4.2 Analyses 4.3 Results 5 Conclusions and Discussion References On the Coefficient Alpha in High-Dimensions 1 Introduction 2 Definitions and Assumptions 3 Theorem 4 Preliminary Results 5 Proof of Theorem 1 6 Illustration with Simulated Data 7 Discussion References IRT Analysis of Dimensional Structure and Item Wording Effects 1 Introduction 1.1 Item Wording: Directionality and Orientation 1.2 Multidimensional Structure 1.3 Graded Response IRT Model 1.4 Summary 2 Method 2.1 Instrument 2.2 Participants 2.3 Analysis 3 Results 3.1 Descriptive Statistics 3.2 Model Fit 3.3 Item Parameter Estimates 3.4 Trait Score Estimates 4 Discussion References Item Level Measurement of Extreme Response Style 1 Introduction 1.1 Anchoring Vignettes 1.2 Item Response Model for Anchoring Vignette Data 2 Methods 3 Results 3.1 IRT Analysis 3.2 Model Comparison Analysis 4 Discussion and Conclusions References On the Marginal Effect Under Partitioned Populations: Definition and Interpretation 1 Introduction 2 Global Marginal Effect 2.1 Definition of the Global Marginal Effect 2.2 Interpretation of the Global Marginal Effect 3 Application 3.1 Results 4 Conclusions and Discussion A.1 Invariant Property of the Global Marginal Effect References Range-Preserving Confidence Intervals and Significance Tests for Scalability Coefficients in Mokken Scale Analysis 1 Introduction 2 Sampling Distribution of Scalability Coefficients 2.1 Wald-Based Methods 2.2 Range-Preserving Methods 2.3 Approximating the Sampling Distribution 3 Simulation Study 4 Implementation in Software 5 Discussion References Equating Nonequivalent Groups Using Propensity Scores: Model Misspecification and Sensitivity Analysis 1 Introduction 2 Kernel Equating 3 Propensity Scores 4 Simulation Study 4.1 Equating Estimators 4.2 Simulation Design 4.3 Evaluation Measures 4.4 Results 5 Discussion References Possible Factors Which May Impact Kernel Equating of Mixed-Format Tests 1 Introduction 2 IRT Kernel Equating 3 Simulation Study 4 Results 5 Discussion References Population Invariance of Equating for Subgroups Differing in Achievement Level 1 Introduction 2 Methodology 2.1 Data and Subgroups 2.2 Equating Analysis 2.3 Comparisons of Equating Results 3 Results 3.1 Comparisons Between Results Based on the Ranked and Random Samples for Equipercentile Equating 4 Conclusions and Discussion References Comparison of Outlier Detection Methods in NEAT Design 1 Introduction 2 Data 3 Methods 3.1 Outlier Manipulation 3.2 Outlier Detection Methods 3.3 Simulation Procedures 3.4 Evaluation Criteria 4 Results 4.1 Baseline Conditions Without Outliers 4.2 Simulated Conditions with Outliers 5 Conclusion and Discussion References An Illustration on the Quantile-Based Calculation of the Standard Error of Equating in Kernel Equating 1 Introduction 2 Equating and the Standard Error of Equating 2.1 Kernel Equating 2.2 SEE in Kernel Equating 3 Quantile-Based Estimation of SEE 3.1 Quantile-Based SEE in KE 4 Illustration 4.1 Data 4.2 Analyses 4.3 Results 5 Discussion References Improving the Measurement Efficiency in Test Construction Related to Cognitive Diagnosis Models 1 Introduction 1.1 CDM 1.2 CDI and ADI 1.3 MCDI and MADI 2 Method 2.1 rCDI and rADI 2.2 Simulation Design 3 Results 4 Discussion References Exploring Temporal Functional Dependencies Between Latent Skills in Cognitive Diagnostic Models 1 Methods 1.1 Dataset Description 1.2 Data Preprocessing/Design 1.3 Parameter Estimation and Model Selection 1.4 Correlation Analysis 2 Results 3 Discussion References Sample Size for Latent Dirichlet Allocation of Constructed-Response Items 1 Introduction 2 Simulations for Latent Dirichlet Allocation 2.1 Latent Dirichlet Allocation (LDA) 2.2 Prior Specification 2.3 Data Generation 2.4 Model Estimation 2.5 Label Switching 2.6 Evaluation 3 Simulation Study 3.1 Design 3.2 Results 4 Discussion References The Asymptotic Power of the Lagrange Multiplier Tests for Misspecified IRT Models 1 Introduction 2 The Lagrange Multiplier Test 2.1 Approximation Procedures for the Asymptotic Power 3 The Generalized Lagrange Multiplier Test 3.1 Estimation Procedure for the Noncentrality Parameter 4 Simulation Study 4.1 Simulation Design 4.2 Results 5 Conclusion References Residual Analysis in Rasch Counts Models 1 Introduction 2 Rasch Counts Model 2.1 Alternative Counts Distribution 2.2 Alternative RC Models 3 Estimation Methods 4 Residual Analysis 5 Application 6 Final Comments References A Bayesian Solution to Non-convergence of Crossed Random Effects Models 1 Introduction 2 Crossed-Random Effects Model (CREMs) 3 Bayesian Approach 4 Simulation 5 Results 5.1 Convergence Rate 5.2 Parameter Recovery 6 Conclusion References Priors in Bayesian Estimation Under the Two-Parameter Logistic Model 1 Introduction 1.1 The 2PL Model and Priors 2 Methods 3 Results 4 Discussion Appendix A: OpenBUGS Code Appendix B: Summary of Priors and Specifications References Increasing Measurement Precision of PISA Through Multistage Adaptive Testing 1 Introduction 2 Experimental Study: Invariance of Item Parameters by Unit Order 2.1 Methods 2.2 Results 3 Simulation Study: Parameter Recovery and Measurement Accuracy 3.1 Methods 3.2 Results 4 Discussion References Simulation Studies of Item Bias Estimation Accuracy 1 Introduction 2 Related Work 3 Data 4 Method 4.1 Bootstrapped Data Set Pairs 4.2 Bootstrap-AB Estimator 4.3 Bagged Parameter Estimates and Bagged-AB Estimator 5 Results 5.1 ECPE Data Set 5.2 TIMSS Data Set 5.3 Social Psychology Data Set 6 Conclusion Appendix A: R Code-Snippets for Bootstrap-AB and Bagged-AB Estimators References Multiple Answer Multiple Choice Items: A Problematic Item Type? 1 Introduction 1.1 Suggestions for MAMC Scoring 1.2 Local Dependence and Gaming Behavior in MAMC Items 1.3 User Perception of MAMC Items 1.4 Examinee Responses to Novel Item Types 2 Method 2.1 Participants 2.2 Instrument 2.3 Analysis Plan and Justification 3 Results 3.1 Latent Class Analysis Results 3.2 Jaccard's Distance Results 4 Conclusions and Next Steps References Modified Method of Drawing Classical ICCs Comparable to IRT-Based ICCs 1 Introduction 1.1 Classical ICCs 1.2 Purpose of Study 2 Proposed Method 2.1 Lateral Coordinates of Group Points 2.2 Score-Based Grouping and Scale-Based Grouping 2.3 Mean Scales of Groups for Scale-Based Grouping 3 Evaluation 3.1 Performance Index 3.2 Simulation 1: Examples of Modified Classical ICCs 3.3 Simulation 2: Evaluation of Proposed Method 3.4 Practical Example 4 Conclusion Appendix References Ontological and Methodological Barriers to the Incorporation of Event Data in Psychometric Models 1 Ontological Difficulties: What Is Meant by ``Process Data?'' 1.1 Terminological Overabundance as Evidence of an Emerging Understanding 1.2 Towards a General Approach 2 Moving Forward: Necessary Conditions for Building Shared Knowledge in Process Data Research 3 Conclusion References Psychometrics for Forensic Fingerprint Comparisons 1 Introduction 2 Data 3 Item Response Trees 3.1 Model for Fingerprint Comparisons 4 Results 4.1 Participant Parameters 4.2 Generating an `Answer Key' from Item Parameters 5 Discussion References After Thematic Analysis: Introducing the Fuzzy Thematic Network Analysis in Psychological Research 1 Introduction 1.1 Structure of a Thematic Network (TN) 1.2 After Thematic Network 2 Fuzzy Thematic Network Analysis (FTNA) 3 Practical Example 4 Conclusion References Psychometric Models for a New State Science Assessment Aligned to the Next Generation Science Standards 1 Introduction 2 Science Pilot Overview 3 Dimensionality Analysis 3.1 Description of Four Datasets and Three IRT Models 3.2 Unidimensional IRT Model (Model 1) 3.3 Three-Dimensional IRT Model (Model 2) 3.4 Testlet Model (Model 3) 3.5 IRT Model-Fit Comparisons 3.6 Item Fit Statistics 3.7 Local Dependency Among Items Within a Task 4 Discussion References Diagnostic Classification Using a Polytomous Measure of Korean Organizational Commitment 1 Introduction 1.1 Recent Polytomous DCMs 2 MORDM 3 Analysis of Data 3.1 Specification of the MORDM 3.2 Fitting the Model 3.3 RMSEA Item-Fit 3.4 Posterior Prediction Model Checking 4 Results 4.1 Item Parameters 4.2 Fit Indices 5 Discussion References An Empirical Study of Developing Automated Scoring Engine Using Supervised Latent Dirichlet Allocation 1 Introduction 1.1 Constructed Response Items and Its Scoring 1.2 Topic Models and Supervised Latent Dirichlet Allocation 2 Methods 2.1 Supervised Latent Dirichlet Allocation 2.2 Data 2.3 Evaluation Criteria 3 Results 3.1 Topic Numbers Selection 3.2 Classification Results 3.3 Unigram Model 4 Discussion and Conclusion References Where the Choice of Model Leads Us: An Empirical Comparison of Dyadic Data Analysis Frameworks 1 Introduction 2 Overview of Dyadic Data Analysis Models 2.1 Actor-Partner Interdependence Model 2.2 Common Fate Model 3 Empirical Analysis 3.1 Research Questions 3.2 Data and Variables 3.3 Analysis Models and Results 4 Discussion References Generalized Additive Modeling for Learning Trajectories in E-Learning Environments 1 Introduction 1.1 Technology-Enhanced E-Learning 1.2 Generalized Additive Mixed Model 1.3 Purpose of Study 2 Application to Real-Life Data 2.1 Statistics Online Data 2.2 Models and Methods 2.3 Results 3 Conclusion and Implication Appendix References Students Ratings Their Open Classroom Discussion 1 Introduction 2 Referent Shift Items 3 Methods 3.1 Selected Data and Measures for Illustrations 4 Problem 1: Wrong Inference Model 5 Problem 2: Students Rating Agreement Variability 6 Conclusion and Discussion References A Generalizability Study of Teach, a Classroom Observation Tool 1 Introduction 1.1 Teach, a Classroom Observation Tool 1.2 Generalizability Theory Framework 2 Methods 2.1 Participants 2.2 Instrument 2.3 Psychometric Model and Data Analysis 3 Results 3.1 Descriptive Statistics 3.2 Generalizability Study 4 Discussion Appendix: Code to Estimate G Models in R Using Teach Data References Index