Chest X-ray Image Captioning Using Vision Transformer and Biomedical Language Models with GRU and Optuna Tuning

摘要

Chest X-ray (CXR) interpretation is time-intensive and contributes to radiologist
workload and potential diagnostic delays. We propose a multimodal deep learning framework integrating a Vision Transformer (ViT) for global visual feature extraction, a biomedical pre-trained language model (ClinicalBERT) for domain-specific semantic encoding, and a Gated Recurrent Unit (GRU) decoder for sequential report generation. Images from the Indiana University CXR dataset were converted from DICOM to PNG and enhanced with contrast-limited adaptive histogram equalization (CLAHE); reports were cleaned, tokenized, and augmented. Hyperparameters—GRU size, learning rate, and batch size—were optimized using Optuna. On the test set, the ViT + ClinicalBERT + GRU configuration achieved BLEU-4 = 0.278, METEOR = 0.221, ROUGE-L = 0.434, CIDEr = 0.846, and SPICE = 0.530, outperforming CNN–RNN baselines and remaining competitive with transformerbased approaches while being computationally efficient.