Integrated Multimodal Deep Learning Framework for Early Detection of Mouth Cancer Using CT Imaging and Clinical Symptom Analysis

Abbreviations

ROI: Region of Interest; CNN: Convolutional Neural Network; ANN: Artificial Neural Network; CT: Computed Tomography; AI: Artificial Intelligence; MRI: Magnetic Resonance Imaging.

Introduction

Mouth cancer, also referred to as oral cavity carcinoma, is a major global health challenge affecting millions of individuals each year. It includes malignant growths occurring in the lips, tongue, gums, buccal mucosa, palate, and floor of the mouth. According to recent epidemiological studies, the incidence of mouth cancer continues to rise, particularly in developing countries where lifestyle habits such as tobacco chewing, smoking, alcohol consumption, and exposure to environmental carcinogens are highly prevalent [1]. Despite improved medical services and screening technologies, many patients are still present at advanced disease stages because early symptoms are frequently overlooked or misinterpreted. Common early indicators such as persistent mouth ulcers, discomfort, abnormal tissue pigmentation, and difficulty in chewing are often dismissed by the general public, leading to delayed clinical diagnosis [2]. This highlights the critical need for public awareness campaigns and advanced diagnostic systems capable of supporting early detection.

Advancements in artificial intelligence (AI), particularly deep learning, have transformed the landscape of cancer diagnostics. Medical imaging technologies, including computed tomography (CT), magnetic resonance imaging (MRI), and digital intraoral photography, are widely used to identify suspicious lesions. However, raw medical images frequently contain noise that obscures critical details and reduces diagnostic accuracy. For this reason, image preprocessing serves as an essential step in enhancing diagnostic clarity. One of the most commonly used preprocessing methods is the mean filter, which smooths image textures, reduces random noise, and improves the visibility of lesions without distorting structural boundaries [3]. In mouth cancer detection, such enhancement helps clinicians and automated systems better identify early tumour formations and segment the cancer-affected Region of Interest (ROI).

Deep learning models, especially Convolutional Neural Networks (CNNs), have emerged as highly effective tools for medical image classification and segmentation. CNNs are specifically designed to extract hierarchical patterns from image data, enabling them to distinguish between healthy and abnormal tissue regions with high accuracy [4]. The ability of CNNs to learn spatial relationships makes them suitable for detecting tumours of varying sizes, shapes, textures, and contrast levels within mouth images. Recent research has shown that CNN-based approaches outperform traditional machine-learning methods because they automatically identify relevant features rather than relying on handcrafted features designed by experts Litjens Y, et al. [5]. In the context of mouth cancer, CNN algorithms can perform tasks such as tumor localization, lesion boundary extraction, and disease stage classification, which are essential for clinical decision- making.

In addition to imaging data, numerical clinical information plays a critical role in assessing cancer progression. Variables such as patient age, gender, tobacco usage, alcohol consumption, dietary habits, pain severity, lesion duration, and presence of associated symptoms offer valuable insights into the patient’s overall risk profile. Artificial Neural Networks (ANN) are a powerful analytical tool for such numerical datasets because they can model nonlinear relationships and identify hidden associations among clinical features [6]. ANN-based analysis supports risk prediction, symptom clustering, and early-stage assessment in patients who may not yet exhibit observable visual abnormalities. By integrating symptom patterns such as continuous burning sensation, unexplained mouth bleeding, or difficulty moving the tongue ANNs help predict the likelihood of malignancy at an early stage.

Combining medical images with numerical patient data creates a multimodal diagnostic framework that enhances predictive accuracy beyond what either data source can achieve independently. Multimodal deep learning approaches have been shown to significantly improve cancer detection performance across multiple studies because they incorporate complementary information: image data reveal structural abnormalities, while numerical data reveal underlying clinical risk patterns [7]. In the case of mouth cancer, this fusion allows the system to evaluate patients more holistically by considering both visible tumour characteristics and symptom-based indicators. Such integrated systems can classify the disease, determine potential risk levels, and provide stage predictions that support personalised treatment planning.

Early diagnosis is closely linked to improved survival outcomes, as mouth cancer often progresses aggressively once it reaches later stages. Treatment options such as surgery, radiation therapy, and chemotherapy tend to be more effective when initiated at an early phase. Therefore, automated systems that can assist in early detection not only benefit clinicians but also contribute to public health by reducing mortality rates. Public awareness plays an equally vital role, as many individuals do not recognise warning signs until the disease becomes painful or visibly advanced. Integrating educational strategies with emerging computational tools can encourage people to seek clinical evaluation earlier, increasing the probability of successful treatment [8].

This research focuses on developing a deep learning– based diagnostic framework that utilises both image preprocessing and multimodal analysis to detect mouth cancer at its early stages. Mouth cancer images undergo mean filter preprocessing to enhance clarity and ensure accurate ROI segmentation. A CNN model is employed for image-based classification and segmentation, while an ANN model analyses numerical patient data, including symptom characteristics and lifestyle variables. Both models are also evaluated in a fused environment to determine whether multimodal inputs improve predictive accuracy. The study aims to identify the most effective combination of preprocessing and deep learning algorithms for mouth cancer detection, risk assessment, and early-stage prediction. Ultimately, the research emphasizes not only technological innovation but also the importance of public awareness and timely treatment intervention to reduce the global burden of mouth cancer.

Literature Review

Mouth cancer research has expanded significantly in recent years due to rising global incidence and the need for early, accurate diagnosis. Traditional clinical assessment methods, including visual inspection and biopsy, remain the gold standard; however, these approaches often detect cancer only after symptoms become severe or visually prominent. For this reason, researchers have focused on developing automated diagnostic systems capable of identifying early abnormalities from medical images and patient clinical data. Literature in the field highlights several core components of such systems, including symptom analysis, image preprocessing, tumour segmentation, and deep learning– based classification.

Methodology

This study followed a multimodal analytical approach that combines CT scan-based mouth cancer imaging with numerical clinical data to develop a more reliable early detection system. Data collection was carried out through collaboration with radiology departments, oncology units, and diagnostic laboratories in several major hospitals across Tamil Nadu, including centres in Chennai, Coimbatore, Madurai, Trichy, and Salem. These institutions provided a diverse set of patient cases, allowing the dataset to reflect real clinical variations. The final dataset included CT scan images of the oral cavity, digital intraoral photographs showing visible lesions, 500 numerical clinical records containing patient history and symptom information, and 25 images from individuals without oral abnormalities. All records were de-identified to protect patient privacy. The clinical dataset contained essential information such as age, gender, tobacco and alcohol habits, duration of symptoms, pain levels, swallowing difficulties, and the appearance of abnormal tissue changes.

After data collection, all CT scans and photographic images underwent a uniform preprocessing procedure to improve clarity and remove unwanted distortions. CT scan images often contain subtle noise and low contrast regions, while intraoral photographs may present uneven lighting; therefore, a mean filter was applied to all images to smooth variations and reduce pixel-level noise. CT scan slices were also contrast-enhanced to highlight soft-tissue differences in the mouth region. All images were resized to 256 × 256 pixels to maintain consistency during model training. This preprocessing stage ensured that suspicious tissue regions became more distinguishable before segmentation.

Once enhanced, the images proceeded to the Region of Interest (ROI) segmentation stage. Because CT scans reveal deeper structural abnormalities that may not be visible on the surface, a combination of threshold-guided marking and deep learning segmentation was used. Initially, intensity-based cues were applied to highlight unusual tissue densities. These preliminary boundaries were then refined by a Convolutional Neural Network inspired by the U-Net architecture, which was capable of isolating irregular tumour regions in both CT scan images and intraoral photographs. The segmented ROIs provided focused visual samples that contained only the most relevant tissue structures for classification.

In parallel, the numerical clinical data obtained from Tamil Nadu hospitals were cleaned and organised. Missing

values were handled through statistical imputation, and all features were normalised to improve training efficiency. Each patient’s symptoms, including persistent ulcers, colour changes in tissues, bleeding, burning sensation, pain intensity, and functional difficulties, were encoded as numerical variables. Lifestyle risk behaviours such as tobacco chewing, smoking, and alcohol consumption were also incorporated as part of the feature set, providing additional context for cancer risk assessment.

To improve diagnostic accuracy, a multimodal fusion approach was applied. The outputs from the CNN (image- based decision) and ANN (clinical-based decision) were combined to produce a final diagnosis that considered both visual and symptomatic evidence. This approach allowed the system to make decisions based on deep-tissue details from CT scans while also accounting for clinical symptoms that may indicate early-stage disease. Model performance was evaluated using accuracy, sensitivity, specificity, F1- score, and AUC, with the multimodal system demonstrating better consistency and reliability than either model alone. The inclusion of CT scan images significantly strengthened the system by providing a deeper anatomical context that is essential for detecting early or partially hidden tumours (Figure 1).

Two separate deep-learning models were developed. The first model, a Convolutional Neural Network (CNN), was trained using both CT scan ROIs and segmented intraoral images. This model extracted spatial and textural characteristics related to tumour presence and classified each image as normal, benign, or malignant. The second model, an Artificial Neural Network (ANN), used the numerical clinical dataset to learn relationships among symptoms, lifestyle factors, and cancer probability. The ANN produced predictions related to risk level and possible disease stage.

Click to enlarge

Results and Discussion

The proposed multimodal framework for mouth cancer prediction produced strong and clinically meaningful results across all stages of analysis. After collecting CT scan images and numerical clinical data, the preprocessing stage significantly improved the quality of the input data by reducing noise, enhancing contrast, and smoothing irregular lighting variations. These improvements made the anatomical structures clearer and allowed the model to consistently differentiate between healthy and abnormal mouth tissue. The enhanced images strengthened the accuracy of the downstream stages because the CNN received cleaner and more uniform representations of the oral cavity.

Following preprocessing, the ROI segmentation component effectively isolated the suspicious tumour regions from each CT scan. By removing unrelated background structures, the model could concentrate on areas with the highest diagnostic value, such as irregular tissue boundaries, abnormal mass densities, and asymmetric oral structures. The quality of the segmented regions reflected the reliability of the segmentation method, which consistently produced well-defined tumour boundaries that matched the expected clinical appearance of early and advanced lesions. This precision helped the CNN learn discriminative features associated with cancer progression more effectively.

The CNN-based classification model demonstrated strong capability in identifying malignant regions within the segmented images. It successfully captured shape irregularities, texture variations, and density differences that typically appear in mouth cancer cases. The model’s high sensitivity showed that it detected suspicious lesions early, while its overall accuracy confirmed that the extracted visual features were highly relevant for distinguishing between cancerous and non-cancerous tissue. These results validate the effectiveness of deep learning for analysing CT images in the context of oral cancer detection.

In parallel, the Artificial Neural Network that analysed numerical clinical data also produced consistent and meaningful predictions. Symptoms such as persistent ulcers, unexplained bleeding, difficulty chewing or swallowing, and tissue discolouration were strong indicators of risk. The ANN was capable of recognising early-stage patterns even when visual evidence on the CT scan was limited, demonstrating the importance of integrating patient-reported symptoms with medical imaging. This also reflected how real clinical assessments operate, where both imaging and patient history contribute to the diagnosis.

The multimodal fusion stage produced the strongest overall results. By combining the image-based outcomes from the CNN with the symptom-based predictions from the ANN, the framework generated a more complete and reliable assessment of cancer risk and stage. This integrated output reduced the chances of false negatives and provided a more balanced interpretation of the patient’s condition. The fusion method mimicked the decision-making process used by clinicians, who rely on both visual scans and clinical data rather than a single information source. The strong performance of this fused model highlights the advantages of multimodal analysis in early detection and accurate staging of mouth cancer.

Overall, the findings indicate that the proposed system is effective, stable, and capable of supporting clinical decision-making. Each stage of the methodology contributed to improving accuracy, from image enhancement to ROI segmentation, and from feature extraction to multimodal prediction. The combined model offers a promising approach for early diagnosis and personalised treatment planning, ultimately supporting improved patient outcomes and more efficient clinical workflows (Figure 2).

Click to enlarge

Conclusion

This study demonstrates that integrating medical imaging with numerical clinical data provides a powerful and reliable approach for detecting mouth cancer at an early stage. The combination of mean-filter preprocessing, precise ROI segmentation, and deep learning analysis significantly improved diagnostic clarity and model performance. The CNN successfully identified malignant patterns in CT scan images, while the ANN accurately captured risk factors and symptom-based variations that are often overlooked in imaging alone. When the two models were fused, the system produced the highest accuracy and stability, showing its strength in handling diverse patient conditions and variable image quality. The results confirm that multimodal learning offers a more comprehensive evaluation of cancer risk by combining structural visual information with patient-specific clinical attributes. This holistic diagnostic strategy has the potential to support clinicians in early detection, guide timely treatment planning, and ultimately improve patient survival outcomes. The findings highlight that computational techniques, when paired with clinical expertise, can play a crucial role in public health by enabling faster, more accurate, and more accessible mouth cancer screening.