Abstract:
Psoriasis is a chronic immune-mediated disorder with diverse manifestations including nail, joint, and skin involvement, often managed with complex therapeutic regimens. Electronic Medical Records (EMRs) support continuity of care but their unstructured free-text format limits systematic analysis. Large language models such as ChatGPT-4 offer a potential solution for transforming narrative records into structured, actionable clinical data.
Objective: To evaluate ChatGPT-4’s ability to extract both disease characterization data (affected body areas, nail and joint involvement) and treatment information from unstructured psoriasis EMRs, and compare its performance against expert analysis.
Methods: We retrospectively analyzed 94 consecutive medical records from patients treated at the Dermatology and Psoriasis Clinic of Sheba Medical Center. Records, written in Hebrew, included anamnesis, physical examination, and treatment plans. ChatGPT-4 was prompted to identify affected body areas (including nails and joints) and all treatments per case. A senior dermatologist served as the expert benchmark. Performance was assessed using sensitivity, specificity, PPV, F1-score, and Cohen’s Kappa.
Results: Of 94 records (55 female, 39 male; age 18.9–86.7 years), ChatGPT-4 identified nail involvement with 90.6% sensitivity/100% specificity, and joint involvement with 96.0% sensitivity/98.6% specificity. Across 479 body areas, accuracy reached 92.9%. For treatment extraction (78 distinct treatments), ChatGPT-4 achieved 84.9% sensitivity, 88.5% PPV, 86.6% F1-score, 99.5% specificity, and strong expert agreement (Cohen’s Kappa = 0.86). Biologics and systemic therapies showed the highest accuracy (F1 = 0.98 and 0.92, respectively).
Conclusion: ChatGPT-4 demonstrated high accuracy in extracting both disease features and treatment data from unstructured psoriasis EMRs written in Hebrew, closely aligning with expert analysis. These findings support LLM-based NLP integration into clinical workflows to enhance EMR usability, automate documentation, and advance AI-assisted medical research.
