Medical code assignment, which predicts medical codes from clinical texts, is a fundamental task of intelligent medical information systems. The emergence of deep models in natural language processing has boosted the development of automatic assignment methods. This project applies deep learning techniques to improve the performance of automatic medical code assignment. Specifically, we use the International Classification of Diseases (ICD) coding system, maintained by the World Health Organization (WHO).
It is challenging to encode lengthy clinical notes with long-term sequential dependency. We propose a Dilated Convolutional Attention Network (DCAN) [1], integrating dilated convolutions, residual connections, and label attention, for medical code assignment. It adopts dilated convolutions to capture complex medical patterns with a receptive field which increases exponentially with dilation size.
Capturing the lengthy and rich semantic information of medical notes and explicitly exploiting the interactions between the notes and codes can help medical text understanding and improve medical code prediction. We propose a novel method, gated convolutional neural networks, and a note-code interaction (GatedCNN-NCI) [2], for automatic medical code assignment. Our methods capture the rich semantic information of the lengthy clinical text for better representation by utilizing embedding injection and gated information propagation in the medical note encoding module. With a novel note-code interaction design and a graph message passing mechanism, we explicitly capture the underlying dependency between notes and codes, enabling effective code prediction. A weight sharing scheme is further designed to decrease the number of trainable parameters.
Unsupervised pretraining is an integral part of many natural language processing systems, and transfer learning with language models has achieved remarkable results in downstream tasks. In the clinical application of medical code assignment, diagnosis and procedure codes are inferred from lengthy clinical notes such as hospital discharge summaries. However, it is not clear if pretrained models are useful for medical code prediction without further architecture engineering. Our third study conducts a comprehensive quantitative analysis [3] of various contextualized language models’ performances, pretrained in different domains, for medical code assignment from clinical notes. We propose a hierarchical fine-tuning architecture to capture interactions between distant words and adopt label-wise attention to exploit label information. Contrary to current trends, we demonstrate that a carefully trained classical CNN outperforms attention-based models on a MIMIC-III subset with frequent codes. Our empirical findings suggest directions for building robust medical code assignment models.
To solve automated medical coding of different coding systems, we propose a multitask recalibrated aggregation network [4]. In particular, multitask learning shares information across different coding schemes and captures the dependencies between different medical codes. Feature recalibration and aggregation in shared modules enhance representation learning for lengthy notes.
Publications
[1] Ji, S., Cambria, E., & Marttinen, P. (2020). Dilated Convolutional Attention Network for Medical Code Assignment from Clinical Text. Proceedings of ClinicalNLP.
[2] Ji, S., Pan, S., & Marttinen, P. (2021). Medical Code Assignment with Gated Convolution and Note-Code Interaction. Findings of ACL-IJCNLP.
[3] Ji, S., Hölttä, M., & Marttinen, P. (2021). Does the Magic of BERT Apply to Medical Code Assignment? A Quantitative Study. Computers in Biology and Medicine.
[4] Sun, W., Ji, S., Cambria, E., & Marttinen, P. (2021). Multitask Recalibrated Aggregation Network for Medical Code Prediction. ECML-PKDD.
[updated on 10 May, 2021.]