University courseworkDeep Learning — Final projectGroup project

Medical specialty classification: LSTM vs Bio_ClinicalBERT

NLP on medical transcriptions with a custom BiLSTM and a pre-trained clinical transformer

Problem

Predict which specialty (10 classes) a medical transcription belongs to.

Medical Transcriptions (Kaggle, tboyle10/medicaltranscriptions), ~2,500 transcriptions from the 10 largest specialties.

Text cleaning, tokenisation and padding (tokenizer fitted on train only)
Embedding + bidirectional LSTM with EarlyStopping and ReduceLROnPlateau
Class weights
Frozen Bio_ClinicalBERT as an embedding extractor + light classifier (full fine-tuning did not fit Community Edition)
Per-specialty comparison on the same split

Test (541 transcriptions, 10 specialties): from-scratch bidirectional LSTM 0.3678 accuracy and 0.3600 macro-F1; Bio_ClinicalBERT (frozen extractor + logistic regression) 0.3752 accuracy and 0.3532 macro-F1.
BERT improved recall in 5 of 10 specialties (Cardiovascular/Pulmonary +13 pp); the LSTM kept a more balanced spread. Main limit: ~2,500 training examples.

PythonTensorFlow / KerasHugging Face TransformersPyTorchscikit-learnDatabricks

University work; no claim of sole individual authorship.