Browsing by Author "Dreyer, Andrei Michael"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    Using transformers to assign ICD codes to medical notes
    (Stellenbosch : Stellenbosch University, 2023-03) Dreyer, Andrei Michael; Van der Merwe, Brink; Stellenbosch University. Faculty of Science. Dept. of Computer Science.
    ENGLISH ABSTRACT: International Classification of Disease (ICD) coding plays a significant role in classifying morbidity and mortality rates. Currently, ICD codes are assigned to a patient’s medical record by hand by medical practitioners or specialist clinical coders. This practice is prone to errors, and training skilled clinical coders requires time and human resources. Automatic prediction of ICD codes can help alleviate this burden. In this research, we look at transformer-based architectures for predicting ICD codes. Firstly, we expand the size of an XLNet model with label-wise attention to determine whether an increase in model size leads to a better performing model. We also look at using two transformer-based architectures that are specifically designed to handle long input sequences and compare the results from these architectures based on our best-performing XLNet model. Lastly, we look at the use of different attention mechanisms with our XLNet model to determine which attention mechanism works the best. We found the following three things: an increase in model size does lead to better results, XLNet performs better than the architectures designed for longer sequence lengths, and the label-wise attention used by our XLNet model performs better than the other attention mechanisms.