Multimodal AI in Healthcare. A Paradigm Shift in Health Intelligence

Description

Preface Contents Contributors Abbreviations Multimodal Artificial Intelligence: Next Wave of Innovation in Healthcare and Medicine 1 Introduction 2 Clinical and Biomedical Applications of Multimodal AI and Data Science 3 Advances in AI Technologies and Data Analytics in Healthcare References Unsupervised Numerical Reasoning to Extract Phenotypes from Clinical Text by Leveraging External Knowledge 1 Introduction 2 Related Work 3 Methodology 3.1 External Knowledge 3.2 Number and Lexical Candidates Extraction 3.3 Contextualized Embeddings for Numeric Entities and Lexical Candidates 3.4 Embedding Similarity and Deterministic HPO Assignment 4 Experiment Design 4.1 Datasets 4.2 Implementation Details 4.3 Baselines and Evaluation Methods 5 Results and Discussion 5.1 Quantitative Analysis 5.2 Qualitative Analysis 5.3 Ablation Studies 6 Conclusions and Future Works References Domain-specific Language Pre-training for Dialogue Comprehension on Clinical Inquiry-Answering Conversations 1 Introduction 2 Domain-Specific Language Pre-training 2.1 Conversation-based Sample Construction 2.2 Experiment Setup of Pre-training 3 Dialogue Comprehension on Clinical Inquiry-Answering Conversations 3.1 Task Definition 3.2 Clinical Dialogue Corpus 3.3 Baseline Models 3.4 Training Configuration 3.5 Evaluation: Comparison with Baselines 3.6 Evaluation in Low-Resource Scenarios 3.7 Evaluation: Pre-training Scheme Comparison 4 Conclusions References Clinical Dialogue Transcription Error Correction Using Seq2Seq Models 1 Introduction 2 Related Work 3 Clinical Dialogue Transcription 3.1 Gastrointestinal Clinical Dialogue Dataset 4 Methods 4.1 General Purpose Base Language Models 4.2 PubMed Gastrointestinal Dataset 4.3 Fine-Tune Using Self-Supervision 5 Evaluation 5.1 Performance Metric 5.2 Comparison of Base Language Models 5.3 Comparison of Fine-Tuned Language Models 6 Discussion 7 Conclusion References Customized Training of Pretrained Language Models to Detect Post Intents in Online Health Support Groups 1 Introduction 2 Background and Related Work 3 Tweet2Quit Dataset 3.1 Data Collection 3.2 Identification of the Intents to Annotate 3.3 Reliability 3.4 Annotation Process 4 Models 4.1 Random Forest (Baseline) 4.2 Pretrained Language Models 5 Adapting to the Labels' Relationships 5.1 Customized Loss Functions 6 Experiments 6.1 Experimental Setup 6.2 Pretrained Language Models 6.3 Loss Functions and Adjusted Metrics 7 Conclusion and Discussion References EXPECT-NLP: An Integrated Pipeline and User Interface for Exploring Patient Preferences Directly from Patient-Generated Text 1 Introduction 2 Aspect Based Sentiment Analysis 3 Data Model 4 Aspect Extraction Results 5 EXPECT-NLP Interface 6 Real-World Use Cases 7 Conclusion and Future Work References Medication Error Detection Using Contextual Language Models 1 Introduction 2 Proposed Methodology 2.1 Problem Formalization 2.2 Contextual Language Models 3 Experimentation 3.1 Dataset Generation 3.2 ASR Implementation 3.3 Results and Analysis 4 Conclusions References Latent Representation Weights Learning of the Indefinite Length of Views for Conception Diagnosis 1 Introduction 2 Integration and Methods 2.1 Latent Representation Weight Learning 2.2 Optimization 3 Results and Discussion 3.1 Data Collection 3.2 Experimental Setup 3.3 Comparison with the Baseline 4 Conclusion References Phenotyping with Positive Unlabelled Learning for Genome-Wide Association Studies 1 Introduction 2 Related Work 3 Methods 3.1 Problem Formulation 3.2 Anchor Learning 3.3 Using BERT as Anchor Classifier 3.4 Baselines 3.5 Anchor Performance Metrics 3.6 Hyperparameters 4 Experiments and Results 4.1 UK Biobank data 4.2 Anchor Classifier Performance and Robustness to Control Noise 4.3 Evaluating Phenotypes with GWAS 5 Discussion References Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation 1 Introduction 2 Related Work 3 Methods 3.1 Considerations 3.2 Designing OOD Tests 3.3 Dataset 3.4 Models 3.5 AUC Score of OOD Detection 4 Experimental Results 5 Discussion References A Robust System to Detect and Explain Public Mask Wearing Behavior 1 Introduction 2 Background and Related Work 2.1 Face Mask Detection 2.2 Explanation 3 Methodology 3.1 Detection Architecture 3.2 Explanation Architecture 3.3 Transfer Learning 3.4 Individual and Aggregate Explanation 4 Experimentation 4.1 Dataset 4.2 Augmentation 4.3 Experiment Setup 5 Evaluation 5.1 Qualitative Results 5.2 Quantitative Results 5.3 Sanity Check 6 Conclusion References A Federated Cox Model with Non-proportional Hazards 1 Introduction 2 Background and Related Work 3 Model 4 Experiments 4.1 Datasets 4.2 Setup 4.3 Results 5 Conclusion Appendix A Additional Figures References A Step Towards Automated Functional Assessment of Activities of Daily Living 1 Introduction 2 Related Works 3 Dataset 4 Proposed Approach 5 Training and Evaluation 6 Experiments and Results 6.1 Ablation Study 7 Conclusion References The Interpretation of Deep Learning Based Analysis of Medical Images—An Examination of Methodological and Practical Challenges Using Chest X-ray Data 1 Introduction 1.1 Previous Work 2 Performance Quantification 3 Model Training 4 Analysis 4.1 Understanding Data and Findings Interpretation 5 Summary and Conclusions References Predicting Drug Functions from Adverse Drug Reactions by Multi-label Deep Neural Network 1 Introduction 2 Related Work 3 Proposed Methodology and Working Architecture 3.1 Description of the Datasets 3.2 Problem Statement 3.3 Proposed Methodology 4 Performance Evaluation of Experiments 4.1 Performance Measurement 4.2 Experimental Results and Discussions 5 Conclusion References Pattern Discovery in Physiological Data with Byte Pair Encoding 1 Introduction 2 Related Work 3 Method 3.1 PAA Transformation 3.2 Discretization 3.3 Identifying Patterns 3.4 Handling Consecutive Identical Symbols 3.5 Post Processing 3.6 Extension to Multivariate Series 3.7 Classification/Regression 3.8 Hyper-Parameter Tuning 4 Data 5 Results 5.1 Computation Speed 5.2 Interpretability 6 Conclusion References Predicting ICU Admissions for Hospitalized COVID-19 Patients with a Factor Graph-based Model 1 Introduction 2 Model 2.1 Model Overview 2.2 Variable Selection 2.3 Factor Function Construction 2.4 Inference Algorithms 3 Experimental Setup 3.1 Dataset 3.2 Model Evaluation 4 Results 4.1 Predictive Biomarkers 4.2 Model Validation 5 Limitations and Future Work 6 Conclusion References Semantic Network Analysis of COVID-19 Vaccine Related Text from Reddit 1 Introduction 2 Data 3 Methods 4 Results 5 Discussion 6 Conclusion References Towards Providing Clinical Insights on Long Covid from Twitter Data 1 Introduction 2 Related Work 2.1 Social Media Platform for COVID-19 2.2 Clinical Information Extraction 2.3 Interpretability 3 Dataset 3.1 Data Acquisition and De-identification 3.2 Filtering Long COVID Self Reports 4 Methodology 5 Results and Discussion References Predicting Infections in the Covid-19 Pandemic—Lessons Learned 1 Introduction 2 Related Work 3 Learning-based Models for Predicting the Number of Infections 4 Experiments 5 Conclusion References Improving Radiology Report Generation with Adaptive Attention 1 Introduction 2 Related Work 3 Method 3.1 Encoder-Decoder Framework 3.2 Multi-Head Adaptive Attention 3.3 Choice of Pretrained Visual Extractor 4 Experiments and Results 4.1 Datasets 4.2 Model Evaluation 4.3 Model Development and Hyper Parameter Tuning 4.4 Ablation Study 4.5 Quantitative Evaluation 4.6 Qualitative Analysis 5 Conclusion References Instantaneous Physiological Estimation Using Video Transformers 1 Introduction 2 Related Work 2.1 Video Based Physiology Extraction 2.2 Transformers 3 Methods 3.1 Optical Basis of Video-Based Bio-Signal Extraction 3.2 Video Transformer for Physiological Estimation 3.3 Loss Formulation 4 Results 4.1 Implementation Details 4.2 Datasets and Evaluation Protocol 4.3 Heart Rate Estimation Results 4.4 Spatial Attention Mask 4.5 Respiration Rate Estimation Results 5 Conclusion References Automated Vision-Based Wellness Analysis for Elderly Care Centers 1 Introduction 2 Related Work 3 Proposed Wellness Analysis System 3.1 Facial Analysis 3.2 Activity Analysis 3.3 Interaction Analysis 3.4 Analysis of Long-Term Pattern and Trend 4 Evaluation 4.1 Data Collection 4.2 Results 5 Conclusion References Efficient Extraction of Pathologies from C-Spine Radiology Reports Using Multi-task Learning 1 Introduction 2 Datasets 3 Description of the Workflow 4 Methods 5 Results 6 Empirical Evidence Behind MultiTasking Models 7 Conclusion References Benchmarking Uncertainty Quantification on Biosignal Classification Tasks Under Dataset Shift 1 Introduction 2 Related Work 3 Uncertainty Quantification Approaches 4 Bencmark Tasks and Experiemnts 4.1 Biosignal Classification Tasks 4.2 Dataset Shift and Evaluation Protocol 4.3 Metrics 5 Results and Analysis 6 Conclusions References Mining Adverse Drug Reactions from Unstructured Mediums at Scale 1 Introduction 2 Related Work 3 Approach 3.1 Classification 3.2 Named Entity Recognition 3.3 Relation Extraction 4 Experimental Setup 4.1 Datasets 4.2 Experiments 4.3 Results 5 Conclusion References A Graph-based Imputation Method for Sparse Medical Records 1 Introduction 2 Method 3 Results 4 Conclusion References Using Nursing Notes to Predict Length of Stay in ICU for Critically Ill Patients 1 Introduction 2 Related Works 3 Dataset 4 Proposed Architecture of ICU LOS Prediction 4.1 Transformer Based Document Representation 4.2 TF-IDF Vector 4.3 Severity of Illness (SOI) Score 4.4 Training the Model 5 Evaluation of the Proposed Predictive Model 5.1 Results and Discussions 6 Conclusion References Automatic Classification of Dementia Using Text and Speech Data 1 Introduction 2 Related Work 3 Materials and Methods 3.1 Dataset: DementiaBank Pitt and WLS Corpora 3.2 Data Pre-processing 3.3 Feature Extraction 4 Ensemble Model 4.1 Deep Learning Model 4.2 Model Evaluation 5 Discussion 6 Conclusion References Unified Tensor Network for Multimodal Dementia Detection 1 Introduction 2 Background 2.1 Alzheimer's Disease and Dementia 2.2 Multimodal Machine Learning 3 Unified Tensor Analysis Pipeline 3.1 Dataset 3.2 Preprocessing 3.3 Representation 3.4 Fusion 3.5 Evaluation 4 Experiments and Results 5 Conclusion References