The Scientific Method: A Guide to Finding Useful Knowledge

Description

The scientific method delivers prosperity, yet scientific practice has become subject to corrupting influences from within and without the scientific community. This essential reference is intended to help remedy those threats. The authors identify eight essential criteria for the practice of science and provide checklists to help avoid costly failures in scientific practice. Not only for scientists, this book is for all stakeholders of the broad enterprise of science. Science administrators, research funders, journal editors, and policymakers alike will find practical guidance on how they can encourage scientific research that produces useful discoveries. Journalists, commentators, and lawyers can turn to this text for help with assessing the validity and usefulness of scientific claims. The book provides practical guidance and makes important recommendations for reforms in science policy and science administration. The message of the book is complemented by Nobel Laureate Vernon L. Smith's foreword, and an afterword by Terence Kealey. CONTENTS List of Tables and Checklists x Foreword by Vernon L. Smith xi Acknowledgments xvi Who Is This Book For? xviii Authors’ Oath for The Scientific Method xx 1 Introduction 1 1.1 Plan of This Book 2 1.2 Scientific Method versus Scientist Opinion 3 1.3 Objective of the Scientific Method 5 1.4 Objectives of Scientific Practice Subverting Science 5 1.5 Operational Guidelines for Scientific Practice 7 2 Defining the Scientific Method 10 2.1 An Aspirational Definition 10 2.2 Criteria for Complying with the Scientific Method 12 3 Checklist for the Scientific Method 24 3.1 Development of the Compliance With Science Checklist 24 3.2 Using the Checklist: For What, How, and by Whom 28 3.3 Not All Checklists Are Useful 29 3.4 Ensuring That Checklists Are Used 30 4 Assessing the Quality of Scientific Practice 31 4.1 How We Reviewed Evidence on Research Practice 32 4.2 Do Journal Papers Help Readers to Make Better Decisions? 33 4.3 More Replications of Important Papers Needed 34 4.4 Are Papers in Scientific Journals Replicable? 35 4.5 How Much of a Concern Is Cheating? 40 4.6 How Efficient Is Research? 42 5 Scientific Practice: Problem of Advocacy 44 5.1 Unnaturalness of Objectivity 44 5.2 Confirmation Bias 47 5.3 How Advocacy Is Practiced 48 5.4 Prevalence and Acceptance of Advocacy 57 6 Scientific Practice: Problem of Journal Reviews 61 6.1 Failure to Find Errors 63 6.2 Advocacy by Reviewers and Editors 63 6.3 Unreliable Reviews 65 6.4 Failure to Improve Papers 65 6.5 Fooled by Fraudulent Reviews and Bafflegab 66 6.6 Distracted by Statistical Significance Tests 68 6.7 Blind to Importance of Authors’ Previous Contributions 70 6.8 Long Delays in Publication 71 7 Scientific Practice: Problem of Government Involvement 72 7.1 Being Useful 72 7.2 Funding 74 7.3 Regulation 83 8 What It Takes To Be a Good Scientist 88 8.1 General Mental Ability 88 8.2 Family Propensity 89 8.3 Early Desire to Do Scientific Research 90 8.4 Personality 91 8.5 Motivated by Intrinsic Rewards 91 8.6 Self-Control 92 8.7 Skepticism 95 8.8 Your Decision 97 8.9 Navigating a Doctoral Program 98 8.10 Acting as a Scientist in all Relevant Roles 98 9 How Scientists Can Discover Useful Knowledge 100 9.1 Identifying Important Problems 100 9.2 Selecting a Problem 105 9.3 Designing a Study 109 9.4 Collecting Data 117 9.5 Analyzing Data 119 10 How Scientists Can Disseminate Useful Findings 121 10.1 Writing a Scientific Paper 121 10.2 Disseminating Findings 128 11 How Stakeholders Can Help Science 139 11.1 Universities 139 11.2 Scientific Journals 147 11.3 Governments, Regulators, and Courts 160 11.4 Media and Interested Groups and Individuals 162 12 Rescuing Science from Advocacy 164 12.1 From Where Should Funding Come? 165 12.2 Advocacy 166 12.3 Bureaucracy and Irrelevant Incentives 166 12.4 How Stakeholders Can Contribute 167 Afterword by Terence Kealey 170 References 176 Index 201