JMIR Medical Informatics

A Quantifiable 3D Closed-Loop Framework for Controlled Substance Safety in a HIMSS Electronic Medical Record Adoption Model (EMRAM) Stage 7 Hospital: Pre-Post Quality Improvement Study

2026-09-25T16:00:17-04:00

Background: Controlled substances require strict life cycle management, but manual workflows can lead to documentation errors, traceability gaps, and delays in empty-ampoule recovery. Objective: This study aimed to construct a quantifiable 3D closed-loop framework integrating technology, process redesign, and organizational accountability and to examine whether implementation was temporally associated with changes in controlled substance process indicators. Methods: A single-center pre-post quality improvement study was conducted in a Healthcare Information and Management Systems Society (HIMSS) Electronic Medical Record Adoption Model (EMRAM) stage 7 hospital. The preintervention period was September 2024 to November 2024, and the postintervention period was December 2024 to February 2025. The intervention combined unique drug identifiers, personal digital assistant (PDA)–based bedside verification, smart medication cabinets, predispensing prescription review rules, batch-number traceability, daily reconciliation, tiered alerts, staff training, and anomaly-handling standard operating procedures (SOPs). Outcomes were prescription dispensing record compliance, batch-number management noncompliance, failure to complete empty-ampoule recovery and electronic documentation within 24 hours, and implementation fidelity. Proportions were reported with Wilson 95% CIs. Between-period absolute differences and 95% CIs were calculated using the Wald normal approximation for independent proportions. Comparisons used the chi-square test or Fisher exact test, as appropriate. Results: The analysis included 3264 prescription dispensing records before implementation and 3311 prescription dispensing records after implementation. Prescription dispensing record compliance increased from 94.82% (3095/3264) to 98.7% (3268/3311). Batch-number management noncompliance decreased from 2.4% (24/1000) to 0.6% (6/1000). The rate of failure to complete empty-ampoule recovery and electronic documentation within 24 hours decreased from 5.61% (68/1213) to 0.78% (10/1274). Implementation fidelity indicators generated from operational logs were high across system stability, scanning execution, and anomaly closure. Conclusions: Implementation of the 3D closed-loop framework was temporally associated with improvements in controlled substance process indicators in a digitally mature hospital. The implementation parameters may serve as a reference for local feasibility testing in other wards or institutions, but transferability remains to be evaluated. Multicenter studies are still needed to confirm sustainability and transferability.

SAFE_DTx: Safety-First Framework for AI-Driven Personalization in Digital Therapeutics

2026-09-25T16:00:03-04:00

Digital therapeutics (DTx) are emerging as evidence-based software interventions, but current AI-driven personalization approaches lack dedicated safety-focused frameworks and face challenges due to scarce long-term outcome data and unpredictable model behaviors. We propose SAFE_DTx, a safety-first architectural framework for DTx that integrates well-established principles of predictive modeling and constrained decision-making to prioritize patient safety. SAFE_DTx’s 2-module architecture comprises an AI feedback prediction module that forecasts short-term patient responses and a constrained planning module that selects the next intervention under explicit safety constraints. By decoupling these components and enforcing clear safety guardrails, the framework enables dynamic, real-time adaptation to individual patient feedback while staying within evidence-based safety limits. This modular design also enhances transparency in the decision-making process, and an in silico evaluation demonstrates its preliminary architectural feasibility, showing greater engagement and no safety violations compared to baseline strategies within the simulated environment. SAFE_DTx’s safety-by-design architecture aligns with emerging regulatory emphasis on AI transparency and patient safety. It directly addresses key clinical challenges in AI-driven DTx personalization by ensuring that tailored interventions do not compromise patient safety.

Cloud-Based and Locally Deployed Language Models in Nursing and Health Care: An AI Act–Aligned Framework

2026-09-25T15:30:13-04:00

Background: The integration of large language models (LLMs) into high-risk systems such as health care is accelerating. Rigorous evaluations aligned with emerging legislation are imperative prior to their incorporation into university educational platforms and clinical practice settings. Objective: The study aimed at implementing the first Regulation (European Union [EU]) 2024/1689–aligned methodological framework for a systematic, comprehensive, and dynamically adaptable language model evaluation, supporting decision-making in specialized health care management. Methods: We analyzed 15 LLMs and 2 small language models. A 7-domain, EU AI Act–aligned methodological framework was used. Feasibility was tested with a dataset of 32 multiparametric-engineered clinical prompts to elicit evaluation in 27 items, with Delphi expert responses as ground truth (available in the repository [32 Clinical Engineered Prompts and Delphi Panel's Responses]). Double-blind interdisciplinary evaluation on a 7-point Likert scale achieved high interrater reliability per model (Krippendorff α=.759 on average). A comprehensive analysis identified specific strengths and vulnerabilities. Safety was analyzed as alignment with both evidence-based nursing and novel structured assessments, including ethical resilience testing via progressive “jailbreaking.” Further novel structured assessments included reference classification, automated consistency, and NANDA-I (North American Nursing Diagnosis Association–International) terminology. Results: A stringent “Safety-Gatekeeper” domain immediately classified 11 of 17 language models as unsuitable due to critical failures in evidence-based alignment or ethical resilience. GPT-o1, GPT-4o, Gemini 2.0 Pro Experimental, and 3 Anthropic models surpassed minimum thresholds, permitting evaluation progression. Only Anthropic Sonnet variants achieved uniform “recommended” categorization. For instance, Claude 3.7 Sonnet (extended thinking) produced 75.9% of accurate, focused references, and achieved high average scores both in clinical safety and data security (mean 6.73, SD 0.23 and mean 6.83, SD 0.41, respectively). DeepSeek-R1, Perplexity Sonar, Mistral Large 2, and Qwen2.5-14B-Instruct failed to resist even explicit harmful prompts; Claude 3 Opus resisted both explicit harmful prompts and all jailbreak attempts, while demonstrating null sycophancy. Notably, Qwen2.5-14B-Instruct, operating locally, outperformed 4 of the 15 LLMs in multistep problems in nurse staffing optimization. NANDA-I diagnostic translation capability improved significantly with taxonomy-embedded contexts, with Gemini demonstrating adequate performance (-score=0.59, Mean Absolute Priority Distance=4.0). Conclusions: Regulatory-aligned LLM integration in pilot university hospitals can enhance health care education and decision-making across standardized taxonomy, evidence-based personalized clinical care algorithms, computational tasks, and nondiscrimination policies, under structured interdisciplinary expert oversight. The methodology demonstrates adaptability across various clinical settings. Future advancements should prioritize multimodal capabilities and locally functioning models, addressing resource disparities in line with Sustainable Development Goal 10, alongside operational resilience, and enhanced data protection.

Symptom Terminology Normalization in Traditional Chinese Medicine: Development and Evaluation of a 2-Stage Deep Learning Framework Based on Fine-Grained Semantic Classification

2026-09-25T14:15:11-04:00

Background: Due to the heterogeneity of symptom terminology and the lack of industry standards, the same symptom is often described using multiple expressions. Current normalization approaches struggle to comprehensively retrieve standard terms when a raw term maps to multiple symptoms. Objective: This study aimed to address the lack of industry standards for traditional Chinese medicine (TCM) symptom terminology. This study proposed the split-then-concatenate normalization framework (STC-NF), a novel approach based on fine-grained semantic classification and a 2-stage deep learning architecture that uses electronic medical records (EMRs) as the data source. Methods: This study proposed a 2-stage deep learning framework, “split-then-concatenate.” In the splitting stage, TCM symptom entities were categorized into 12 fine-grained semantic labels, and 3 named entity recognition (NER) models were trained to extract TCM symptom terminology from EMRs. In the concatenation stage, standard terms with the same concept as raw terms were identified using a Bidirectional Encoder Representations from Transformers (BERT)–based binary classification model. The standard terms with specific semantic labels were concatenated and reordered according to predefined rules to output structured text, thereby normalizing TCM symptom terminology. Results: The proposed STC-NF model achieved an accuracy of 91.4% (180/197) and an -score of 360 out of 389 (92.5%) on the single-implication test set. For multi-implication terms, STC-NF achieved an accuracy of 84.3% (311/369) and an -score of 1958 out of 2316 (84.5%), outperforming sequence generation in accuracy by 33.1 percentage points. On the mixed test set containing both single- and multi-implication terms, STC-NF achieved an accuracy of 88.1% (990/1124) and an -score of 3862 out of 4385 (88.1%), exceeding the best-performing baseline model, multi-task candidate generator (MTCG), by 16.7 and 22.2 percentage points in accuracy and -score, respectively. Conclusions: In this study, we verified that the fine-grained semantic classification and the 2-stage “split-then-concatenate” framework effectively improved performance of named entity recognition and entity alignment, providing an improved approach to normalizing TCM symptom terminology.

Longitudinal Digital Phenotyping of Traditional Chinese Medicine Constitution From Routine Health Examination Records: Retrospective Clinical Informatics Study

2026-09-23T16:15:10-04:00

Background: Traditional Chinese medicine (TCM) constitution is a structured health state taxonomy used in preventive care, but its relationship with routinely collected health examination data, disease-related markers, and longitudinal change remains difficult to interpret in clinical informatics settings. Objective: This study aimed to develop and evaluate a longitudinal clinical informatics framework for characterizing TCM constitution as a computable, explainable, record-based phenotype using routine health examination records. Methods: We conducted a retrospective longitudinal analysis of 47,417 examination-constitution records from 11,355 older adults examined between 2017 and 2026. Baseline analyses used the first available record per participant, and longitudinal analyses used 32,648 pairs of adjacent annual visits. The framework included bidirectional disease-constitution mapping, nonoverlapping multimarker burden modeling, lagged next-visit association models, constitution-state transition analysis, and temporal evaluation of routine examination–based label prediction models. Additional sensitivity analyses evaluated participant overlap across calendar-year splits, participant-disjoint temporal evaluation, BMI-only and BMI-plus-waist baselines, exclusion of anthropometric predictors, adiposity adjustment of longitudinal models, and new-onset and persistence outcomes. Results: Phlegm-dampness showed the clearest record-based signature and was associated with higher cardiometabolic marker burden than balanced constitution (incidence rate ratio 1.65, 95% CI 1.60‐1.70). In lagged models, its associations with a subsequent abdominal ultrasound abnormality flag and cardiometabolic risk clustering persisted after BMI adjustment, whereas associations with diabetes-related markers, proteinuria, dyslipidemia, and glucose abnormalities were substantially attenuated. Of 11,355 participants, 8122 appeared in at least 2 original calendar-year splits. In a participant-disjoint temporal sensitivity analysis with 1054 new test participants, extreme gradient boosting (XGBoost) identified phlegm-dampness label presence with an area under the receiver operating characteristic curve of 0.927 (95% CI 0.912‐0.941). A BMI-only model achieved 0.923 (95% CI 0.907‐0.938), whereas XGBoost without anthropometric predictors achieved 0.685 (95% CI 0.653‐0.717). Conclusions: Routine health examination data captured a reproducible, predominantly adiposity-centered phlegm-dampness label phenotype in this cohort of older adults. High discrimination was retained in participant-disjoint evaluation but was nearly matched by the BMI-only model, and several longitudinal associations were explained by adiposity. The models should therefore be interpreted as decision-support and communication aids for an existing constitution assessment process and not as stand-alone diagnostic systems or comprehensive classifiers of TCM constitution.

Multioutput Machine Learning Model for Predicting Postoperative Outcomes After Liposuction: Algorithm Development and Validation Study in a Multicenter Cohort

2026-09-23T16:15:10-04:00

Background: Liposuction is widely performed to remove localized fat deposits and improve body contour, yet individualized prediction of postoperative outcomes remains challenging. Existing machine learning (ML) studies have largely focused on single-outcome prediction, with limited attention to the interdependence between postoperative body weight and circumferential size. Objective: This study aimed to develop and validate a chained multioutput ML framework to jointly predict postoperative body weight and circumferential size after liposuction using a large multicenter cohort from the 365mc network. Methods: We analyzed a multicenter cohort of 7804 individuals who underwent liposuction in 2024 at 20 obesity specialty clinics in the 365mc network across South Korea. Using 15 predictors, we compared 8 individual ML models, an automated ML approach, 2 ensemble approaches, and chained multioutput regression models for predicting postoperative body weight and circumferential size. Models were developed using 5-fold cross-validation and evaluated on an independent test set. Performance was assessed using the coefficient of determination (), root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE), and feature importance was evaluated using Shapley additive explanation (SHAP) values. The selected model was integrated into a web-based clinical decision support system (CDSS). Results: A total of 7804 individuals who underwent liposuction were included; of these, 7612 (97.54%) were female. The chained extra trees regressor model with a weight-to-size prediction order achieved an of 0.98, an RMSE of 2.36, an MAE of 1.24, and a MAPE of 2.19. The SHAP analysis identified preoperative weight as the main predictor of postoperative body weight and preoperative size and liposuction-related factors as key predictors of postoperative circumferential size. The final model was integrated into a web-based CDSS (365mc AI platform). Conclusions: We developed and validated a chained multioutput regression model to predict postoperative body weight and circumferential size after liposuction. Integrated into a web-based CDSS, the model may support patient-specific preoperative counseling and surgical planning.

Development and Validation of Machine Learning Models for Postjudgment Estimation of High Compensation Ratios After Lower Limb Fracture Surgery: Retrospective Study

2026-09-22T18:00:39-04:00

Background: Orthopedic surgery is the second most common subspecialty involved in medical malpractice claims, wherein lower limb surgery carries a higher risk of claims and involves higher compensation amounts. However, effective tools for postjudgment estimation of high compensation ratios and consistency assessment against prior similar cases after lower limb fracture surgery are currently lacking in medicolegal risk management and judicial practice. Objective: This study aimed to develop and validate multiple machine learning (ML) models to estimate a medical malpractice compensation ratio of ≥50% in postjudgment, nonfinalized medicolegal cases, and to systematically evaluate the models’ discriminative ability, stability, calibration performance, and medicolegal utility. Methods: This study developed binary classification models based on 451 medical malpractice cases after lower limb fracture surgery in China from 2004 to 2025. Among these cases, 360 cases from eastern, northeastern, and central China constituted the development dataset and were randomly split at a 7:3 ratio into a training set (n=251) and an internal test set (n=109), whereas 91 cases from western China were held out as an independent geographical external validation set. Least Absolute Shrinkage and Selection Operator (LASSO) was used for feature selection. Logistic regression (LR), k-nearest neighbors (KNN), support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGBoost) were trained using 5-fold cross-validation and grid search. Model performance was assessed using the receiver operating characteristic (ROC) curve, area under the receiver operating characteristic curve (AUROC), bootstrap resampling, calibration curves, and decision curve analysis (DCA). Results: LASSO identified following 8 predictors: inappropriate surgical procedure, age, inadequate medical records, lack of informed consent, disability severity grade 1-4, sex, treatment delay, and inadequate preoperative preparation. In the test set, the LR model achieved an AUROC of 0.933, with recall, precision, accuracy, and F₁-score of 0.810, 0.940, 0.872, and 0.870, respectively. Inappropriate surgical procedure and lack of informed consent were the strongest model-associated factors. Bootstrap analysis showed stable LR discrimination, and calibration was favorable, with a Brier score of 0.105. DCA suggested a potential reference value across threshold probabilities. Considering discrimination, calibration, classification performance, parsimony, and interpretability, the LR model was selected as the final model. External validation provided preliminary support for cross-regional transportability. Conclusions: ML-based models for postjudgment estimation may provide a nonbinding historical benchmark for estimating whether a compensation ratio of ≥50% is broadly consistent with previous similar cases in medical malpractice claims. Notably, the LR model showed the most favorable overall performance among the compared models in terms of discriminative ability, stability, calibration, and medicolegal utility, providing an auxiliary postjudgment reference for hospital risk management and legal departments, legal practitioners, courts, and other judicial professionals before a case becomes final or is practically concluded. Trial Registration: OSF Registries 10.17605/OSF.IO/EMAKT; https://osf.io/emakt/overview

Development and Validation of a Machine Learning–Based Model for Predicting All-Cause Mortality Risk in Patients With Type 2 Diabetes Mellitus Combined With Hypertension: National Cohort Study

2026-09-22T17:45:02-04:00

Background: Type 2 diabetes mellitus (T2DM) combined with hypertension significantly increases mortality risk, yet accurate risk prediction models remain limited. Objective: We aimed to develop and validate machine learning–based models to predict all-cause mortality in patients with T2DM and hypertension. Methods: We analyzed data from the National Health and Nutrition Examination Survey from 1999 to 2018 linked with mortality data up to December 31, 2019. Adult participants (aged ≥20 years) with concurrent T2DM and hypertension were included. Five machine learning algorithms were developed and compared: random forest, light gradient boosting machine, decision tree, extreme gradient boosting, and logistic regression. Model performance was evaluated using area under the curve (AUC), calibration plots, and decision curve analysis. Results: A total of 2428 participants were included (mean age 62.05, SE 0.33 years; n=1218, 50.15% female). During a median follow-up of 6.75 (IQR 4.20-8.90) years, among the 2428 patients, 719 (29.6%) deaths occurred. The random forest model demonstrated superior performance (AUC=0.873, 95% CI 0.856-0.891) compared to light gradient boosting machine (AUC=0.785), decision tree (AUC=0.732), extreme gradient boosting (AUC=0.792), and logistic regression (AUC=0.783). Key predictive features included age, race, chronic kidney disease, BMI, and blood urea nitrogen. The model exhibited excellent calibration and clinical utility across various risk thresholds. Conclusions: Our machine learning–based model provides accurate all-cause mortality prediction for patients with T2DM and hypertension, potentially supporting clinical decision-making and risk stratification in this high-risk population.

A Hierarchical Machine Learning–Based Framework for Clinical Decision Support in Foot Orthosis Prescription: Algorithm Development and Validation Study

2026-09-22T17:00:18-04:00

Background: Foot orthosis prescription is a complex clinical decision-making process that involves selecting and combining multiple structural, functional, and material components based on heterogeneous biomechanical information. In routine outpatient practice, detailed biomechanical assessments are often incomplete, creating substantial variability in prescription decisions, and limiting the applicability of conventional machine learning (ML) models that assume fixed feature availability. Objective: This study aimed to develop and evaluate a hierarchical ML-based clinical decision support framework for foot orthosis prescription that accommodates variable clinical information availability, supports multilabel prescription decisions, and incorporates safety-oriented recommendation strategies for routine outpatient practice. Methods: A retrospective observational study was conducted using 6462 visit-level clinical encounters collected from a single institution between 2015 and 2020. Orthotic prescription was formulated as a multilabel prediction task involving 15 prescription components. A hierarchical modeling framework was implemented, consisting of a basic decision level using routinely available demographic and alignment variables, and an advanced level incorporating subtalar joint inversion and eversion range of motion measurements when available. Tree-based gradient boosting models were evaluated using 5-fold stratified grouped cross-validation. Model performance was assessed using component-wise area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), positive predictive value (PPV), macro-averaged Hamming loss, Top-K hit, and coverage rates based on probabilities calibrated using internal grouped Platt scaling, and component-specific safety-oriented threshold calibration. Results: The hierarchical models demonstrated consistent predictive performance across both decision levels, although precision-recall–based analyses indicated greater variability for low-prevalence prescription components. The Level 4 and Level 8 models achieved mean component-wise AUROC values of 0.792 and 0.816, respectively, with macro-averaged Hamming loss of 0.113 and 0.111. Using component-specific Platt scaling within the 5-fold stratified grouped cross-validation framework, Top-K analysis demonstrated a Top-1 hit rate of 86.7% and a Top-3 hit rate of 97.4%. When all prescribed components were considered, Top-K coverage reached 71.7% at Top-3 and increased to 97.0% at Top-8. Safety-oriented threshold calibration was performed separately for each prescription component, selecting the threshold that achieved the highest sensitivity while maintaining a specificity of at least 95%. Under this component-specific calibration strategy, the aggregated operating profile achieved a microaveraged specificity of 96.00% with a microaveraged sensitivity of 28.71%. Age-stratified analyses revealed distinct feature-importance patterns between pediatric and adult populations. Conclusions: The proposed hierarchical clinical decision support framework supported multicomponent foot orthosis prescription under variable information availability commonly encountered in outpatient practice. By generating ranked component-wise recommendations and high-confidence threshold-calibrated outputs, the framework may support clinician decision-making in routine orthotic practice. Further prospective validation will be required to determine its clinical utility.

Semantic Similarity Search Approach to Extract Exemplars of Stigmatizing and Positive Language in Obstetric Clinical Notes: Exploratory Study

2026-09-22T16:45:10-04:00

Background: Natural language processing can extract meaningful information from clinical notes. However, human annotation is time-consuming and costly, and scarce data poses a challenge. Objective: This study aimed to explore a semantic similarity search approach to extract exemplars of stigmatizing and positive language in obstetric clinical notes. Methods: We used electronic health record data from labor and birth admissions at 2 hospitals in the United States from 2017 to 2019. We used a semantic similarity search approach, which used 200 randomly selected true exemplars, stratified by language categories, as queries to search for similar exemplar candidates. We extracted the top 5 candidates with the highest cosine similarities, which were assessed for accuracy. Results: We retrieved 1000 candidates. An average precision of 0.69 was achieved when candidates with cosine similarity thresholds of 0.75 or higher were included, at which point 68.8% (64/93) of exemplar candidates accurately represented true cases. At the 0.75 threshold, the proportion of true cases was higher for preferred language (41/56, 73.2%) and unilateral/authoritarian decisions (5/7, 71.4%). The proportion of true cases was lower for difficult patients (2/7, 28.6%) and marginalized identities (3/9, 33.3%). Conclusions: The semantic similarity search approach shows promise in efficiently extracting exemplars while reducing the annotation burden, laying the groundwork for future applications in other domains.