International Journal of Engineering and Management Research

1. Introduction

Breast and lung cancers remain among the most prevalent and deadly malignancies worldwide [1]. Despite advances in screening and diagnostics, reliable early detection and accurate characterization of these tumors are still fraught with challenges. Traditional oncologic diagnostics—ranging from imaging (e.g., mammography, CT scans) to histopathology and serum biomarkers—have notable limitations. Radiological assessment, for instance, often relies on expert visual interpretation, which can suffer from inter-observer variability and missed subtle findings. Tumor heterogeneity and the presence of occult micro-metastases further complicate diagnosis, leading to false negatives or indeterminate results that delay treatment [2]. Even in well-established screening programs, significant fractions of cancers (so-called “interval cancers”) evade detection at initial exams. These shortcomings underscore the need for more robust, sensitive diagnostic approaches in oncology. In recent years, artificial intelligence (AI) and deep learning have been poised to address some of these gaps by augmenting human expertise in medical image analysis [3].

Deep learning, especially convolutional neural networks with deep residual learning architectures (ResNets), has shown remarkable promise in medical imaging for cancer diagnosis. Residual networks enable the training of very deep models by mitigating vanishing-gradient issues, thereby capturing complex visual features of tumors across multiple scales. In both breast and lung cancer domains, AI systems now approach or even surpass human performance in certain tasks. For example, a deep learning model for mammography was able to reduce false negatives and false positives compared to radiologists, effectively surpassing human experts in breast cancer detection [4]. In a recent prospective trial involving over 80,000 women, an AI-assisted screening protocol detected 20% more breast cancers than the standard double-reading by radiologists, without increasing false-positive rates [5]. Notably, the incorporation of AI nearly halved the workload for human readers in this study [5], highlighting how deep learning can enhance efficiency as well as accuracy. Similar success has been observed in lung cancer imaging: modern 3D residual networks can analyze chest CT scans to identify malignant nodules with expert-level sensitivity,

cognitive “chemo-fog,” and the psychological toll of illness – factors that could influence treatment decisions and outcomes if properly recognized [2]. Such findings reinforce calls from experts to integrate the “voice of the patient” into cancer care. Indeed, there is a burgeoning consensus that mixing quantitative measures with narrative evidence yields a fuller understanding of health outcomes [7]. Patient-reported outcome researchers have argued for more narrative collection alongside surveys, noting that narratives allow patients to express the reality behind the numbers [8]. This qualitative dimension is especially important in precision oncology: as treatments become more individualized, understanding each patient’s unique context – their comorbidities, social support, fears, and goals – is vital for truly personalized care.

Given these complementary strengths, there is a compelling rationale to combine deep learning with thematic analysis in a unified framework. We posit that such an integrative approach can leverage the best of both worlds: the pattern-recognition power of AI on high-dimensional data and the contextual understanding afforded by qualitative analysis. Deep residual networks excel at deciphering complex patterns in medical images – for example, detecting a spiculated mass on a mammogram or quantifying irregular nodule growth on serial lung CTs [8]. These models can rapidly distill hundreds of thousands of pixel values into a diagnostic prediction (benign vs. malignant) or even prognostic insights. What they lack is context – a connection to the individual patient behind the image. This is where thematic analysis can fill the gap: by analyzing a patient’s EHR notes, clinic visit transcripts, or survey comments, we can identify salient themes (e.g., “worsening cough and pain,” “anxiety about treatment,” “lack of transportation to clinics”) that provide a narrative backdrop to the imaging findings. Integrating these modalities can yield a more holistic assessment. For instance, in a breast cancer case, an AI may detect a suspicious lesion on imaging with a certain confidence level; simultaneously, thematic analysis of the patient’s history might reveal a strong family cancer history and repeated thematically coded expressions of concern about genetic risk. Taken together, the combined insight could prompt earlier genetic counseling or more aggressive diagnostics than either method alone would suggest.

In image-based oncology, modern CNN models have achieved remarkable success in tasks like digital pathology slide analysis and radiology scan interpretation [9]. The ResNet architecture, with its introduction of residual skip-connections, enables training of very deep networks by mitigating vanishing gradients, thereby improving feature learning in complex medical images [10]. Indeed, pretrained ResNet models originally developed on general imaging datasets have been effectively repurposed for histopathology and radiology, often attaining expert-level accuracy in detecting and classifying tumors [10]. Beyond imaging alone, there is a growing recognition that incorporating clinical context from electronic health records (EHRs) can enhance diagnostic performance. Deep learning models that integrate multimodal inputs—such as radiological images together with patient demographics, lab results, or genomic data—better emulate a physician’s holistic decision-making process and can yield more precise outcomes [11]. Recent surveys underscore this trend: studies fusing medical imaging with EHR data have surged in the past few years, roughly doubling from 2020 to 2021 [12]. By aligning pixel-level features with patient context, these multimodal approaches consistently outperform single-modality models on the same clinical tasks. For example, one scoping review of 34 studies found that the vast majority of multimodal fusion models improved diagnostic accuracy and prognostic prediction compared to image-only or EHR-only baselines. In quantitative terms, combined models have demonstrated improvements in accuracy ranging from ~1–28% over their single-modality counterparts in applications ranging from tumor detection to disease subtype classification [12]. Such gains illustrate the promise of deep residual learning when enriched with complementary clinical data, aligning with the broader goals of precision oncology to tailor diagnoses and risk stratification to the individual patient. Despite these successes, experts note that real-world clinical integration of these advanced algorithms remains in its infancy, with ongoing challenges in data curation, validation, and interpretability that need to be addressed to fully realize patient benefit [13].

In parallel, qualitative research methods have flourished in healthcare as a means to capture patient-centered perspectives and contextual nuances that quantitative data alone may overlook.

There have been exploratory efforts to integrate these traditionally separate domains, recognizing that complex health problems often require both data-driven and human-centered perspectives. One line of research has focused on using machine learning to assist and enhance qualitative analysis itself. For example, Towler et al. employed an unsupervised topic modeling technique to aid thematic analysis of thousands of free-text survey responses during the COVID-19 pandemic. Their machine-assisted approach rapidly identified topics and candidate themes from the large dataset, and when compared to a conventional manual coding process, it reproduced the key themes with a high degree of similarity while cutting the analysis time by more than two-thirds [16]. This demonstrates that natural language processing (NLP) methods can augment human qualitative analysts, especially for scaling up to big textual datasets that would be prohibitively time-consuming to manually code. In another illustrative study, Di Basilio et al. combined traditional thematic analysis with NLP in examining patient-reported outcome measures for individuals with traumatic brain injury [16]. Interview transcripts were analyzed by human researchers to derive themes, while in parallel NLP techniques (including sentiment analysis and keyword extraction) were applied to detect emotional tones and linguistic patterns related to those themes. The integration of findings allowed the researchers to enrich the qualitative themes with quantitative text-derived insights, painting a more comprehensive picture of patient experiences. These early efforts at intertwining machine learning with qualitative methodologies hint at a powerful synergy: machine learning can lend efficiency, consistency, and breadth to qualitative research, whereas qualitative approaches provide interpretive depth and ensure that the outputs of algorithms remain meaningfully connected to human contexts. In the realm of oncology, however, such integrations are only beginning to surface. There is a clear opportunity to unite advanced deep learning systems with qualitative, human-centric analyses to create hybrid frameworks for precision oncology. Prior work has shown, for example, that clinical experts see potential in ML to improve cancer care but emphasize the need for transparency, context, and patient-centered relevance. A framework that weaves together deep residual learning on multimodal biomedical data with thematic analysis

are normalized and encoded into a standardized vector representation. For the unstructured portion—such as physician notes, pathology reports, and patient-generated narratives—an advanced thematic analysis pipeline is employed. The analysis combines human-led coding with natural language processing (NLP)-assisted preprocessing. Initially, texts are cleaned, tokenized, and lemmatized. Using a semi-automated approach, thematic codes are derived through a combination of open coding and clustering techniques, informed by Latent Dirichlet Allocation (LDA) and guided manual validation to ensure conceptual coherence. These themes are then converted into dense embeddings using a transformer-based language model (such as ClinicalBERT), producing fixed-length semantic vectors that preserve both content and contextual nuances.

The integration of imaging and thematic data occurs at the feature fusion stage. Two fusion strategies are evaluated. In the early-fusion strategy, embeddings from the ResNet model and the thematic vectors are concatenated and passed through a shared dense layer followed by classification or regression heads tailored to specific diagnostic or prognostic tasks. This allows the model to learn joint representations from both data types simultaneously. In the late-fusion strategy, each modality is processed independently through separate deep branches and then merged using an attention-based gating mechanism that weighs the importance of visual and narrative inputs dynamically based on task relevance. This mechanism enables the model to adaptively prioritize imaging cues or patient-expressed concerns depending on the clinical context.

The entire system is trained and evaluated within a controlled simulation environment. The synthetic dataset used mimics real-world clinical distributions in terms of cancer prevalence, imaging variability, and EHR documentation patterns. Cross-validation techniques are applied to ensure robustness of performance estimates. The training process incorporates regularization strategies, including dropout and data augmentation, to prevent overfitting and enhance generalizability. Model performance is evaluated on standard metrics such as accuracy, precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC), with particular attention paid to the impact of thematic integration on diagnostic confidence and patient stratification accuracy.

3. EHR Data Processing

a. Structured Data: Data such as age, gender, clinical history, and lab results are vectorized (e.g., one-hot encoding, normalization) and formatted for model input.
b. Unstructured Text Pathway: This text is processed through multiple stages:
(i) Cleaning and Tokenization: Text is prepared using NLP techniques including tokenization, lemmatization, and stopword removal.
(ii) Thematic Analysis: A semi-automated pipeline combines manual open coding with machine assistance using Latent Dirichlet Allocation (LDA) to extract latent topics. Human reviewers validate and refine themes to ensure medical relevance and coherence.
(iii) Embedding Generation: Final themes are embedded using models like ClinicalBERT, which transforms textual themes into dense, numerical representations while preserving semantic context.

4. Fusion and Integration Mechanisms

Two strategies are implemented:

a. Early Fusion
(i) Vectors from ResNet (imaging), structured EHR, and thematic analysis are concatenated.
(ii) The fused vector passes through a dense layer for joint representation learning.
(iii) Final classification or regression layers output predictions (e.g., diagnosis, risk score, treatment stratification).

b. Late Fusion
(i) Each modality is processed independently via separate deep learning branches.
(ii) Outputs are passed to an attention-based gating mechanism, which learns the relative importance of each modality dynamically.
(iii) The aggregated signal is used for final prediction, optimizing context-awareness and adaptability.

5. Evaluation

The fused system is trained and validated in a controlled simulation environment. Model performance is assessed using:
(i) Accuracy, Precision, Recall, F1-score, and AUC.
(ii) Additional evaluation focuses on the model’s confidence and interpretability with and without thematic input.

diagnostics and treatment planning, using synthetically derived yet clinically realistic multimodal data.

The imaging data component consisted of synthetic but high-fidelity breast mammograms and chest CT scan samples generated from publicly available data templates and augmented to reflect a wide range of anatomical and pathological presentations. These images were designed to mimic real patient variability in terms of tumor size, shape, density, and spatial context. To support multi-class diagnostic evaluation, cases were labeled into diagnostic categories such as benign, in situ carcinoma, and invasive carcinoma for breast cancer, and benign nodules, stage I–III malignancies for lung cancer. Standard augmentation techniques—such as rotation, scaling, horizontal flipping, and contrast variation—were applied to improve generalization and replicate image diversity observed across different imaging centers.

Structured clinical data were generated to mirror the demographic and clinical characteristics commonly seen in oncology patients. Each synthetic case included variables such as age, sex, smoking history, genetic markers (e.g., BRCA status), comorbidity indices, and baseline laboratory values (e.g., CA 15-3 for breast cancer, CEA levels for lung cancer). These features were drawn from statistical distributions based on published epidemiological data to ensure representational validity. Conditional correlations between variables (e.g., older age with increased comorbidities, smoking history with nodule malignancy) were embedded into the data generator logic to reinforce clinical realism.

The unstructured text data for thematic analysis were derived from templated clinical notes, simulated patient-reported outcomes, and synthetic interview transcripts created using a rule-based generator trained on linguistic structures found in oncology EHR narratives. These texts incorporated a mix of objective clinical descriptions and subjective patient expressions. Sample excerpts included physician assessments (“Patient reports progressive dyspnea and unintentional weight loss”), psychosocial themes (“Fear of treatment side effects; expresses concern over financial burden”), and lifestyle factors (“Recently quit smoking after 20 years; lives alone”). These narratives were curated to reflect the kinds of concerns and observations typically documented in clinical settings.

Overall, this simulation-based implementation demonstrates the technical feasibility and potential clinical value of the proposed hybrid framework. By closely emulating the multimodal nature of real-world oncology diagnostics, the system provides a foundation for future deployment in empirical clinical studies, where similar architectures can be validated on authentic patient cohorts.

5. Results and Analytical Evaluation

The hybrid framework integrating deep residual learning and thematic analysis was evaluated using a controlled multimodal simulation environment across a variety of clinically relevant diagnostic and prognostic tasks. The analytical evaluation was designed to assess the model’s accuracy, contextual adaptability, and interpretability in comparison to conventional single-modality and dual-modality models. Key findings are organized across classification performance metrics, thematic contribution analysis, attention behavior, and ablation robustness.

The image-only baseline model, utilizing a ResNet-50 architecture trained exclusively on mammograms and chest CT images, achieved strong initial diagnostic performance. For binary classification (malignant vs. benign), the baseline model attained an average accuracy of 87.4%, with an AUC of 0.91 across both breast and lung cancer datasets. Precision and recall were measured at 84.3% and 85.7%, respectively, reflecting competent discrimination capabilities. However, class-specific misclassification analysis indicated vulnerability to false negatives in ambiguous cases (e.g., early-stage tumors with low radiodensity), suggesting the model’s limitations when visual patterns were insufficiently distinct.

The early fusion model, incorporating both structured EHR variables and thematic embeddings alongside imaging features, demonstrated measurable improvements. Average diagnostic accuracy increased to 91.2%, with AUC rising to 0.95. F1-score improved from 84.9% in the baseline model to 89.7% in the fused model. These gains were most notable in borderline imaging cases, where supplementary patient history and psychosocial indicators contributed critical contextual cues.

In several case studies, the fused models provided rationale-consistent predictions, such as suggesting further diagnostic imaging for a lesion when combined with patient-reported “recent weight loss and night sweats.” This narrative-supported inference aligned with oncologic red flags, thus enhancing clinical credibility.

Overall, the analytical evaluation confirms that the proposed hybrid framework outperforms conventional deep learning baselines by integrating contextual awareness through thematic analysis. The inclusion of structured and narrative EHR components not only boosts diagnostic metrics but also brings model decisions closer to patient-centered clinical reasoning. These results underscore the promise of multimodal AI approaches in delivering more nuanced, accurate, and personalized cancer diagnosis and treatment insights.

Figure 2 shows the grouped bar chart that compares five key performance metrics across three models:

1. Image-Only (ResNet) – A baseline model trained solely on imaging data.
2. Early Fusion – Combines image, structured EHR, and thematic data at the feature level.
3. Late Fusion – Integrates the three modalities using an attention-based mechanism to weigh their contributions dynamically.

Metrics Visualized:

(i) Accuracy (%): Reflects the proportion of correctly classified cancer cases. Late Fusion achieved the highest accuracy at 93.1%.
(ii) AUC (Area Under Curve): Measures the model's ability to distinguish between classes. All models perform well, with Late Fusion slightly ahead.
(iii) F1-Score: Balances precision and recall. Again, Late Fusion leads with 91.3%.
(iv) Precision (%): Indicates how many of the positive predictions were correct. Early and Late Fusion significantly improve over the baseline.
(v) Recall (%): Measures how many actual positives were correctly identified. Late Fusion achieves the best recall.

Interpretation:

(i) The addition of structured and thematic data clearly enhances diagnostic accuracy.

Each modality contributes meaningfully to the model's performance, but imaging and thematic analysis are especially critical—highlighting the strength of a truly multimodal AI framework in precision oncology.

This bar chart in Figure 5 shows F1-score comparisons between full and ablated versions of the early and late fusion models:

(i) Removing thematic embeddings caused a noticeable drop in performance for both fusion strategies.
(ii) Early Fusion dropped from 89.7 to 86.2.
(iii) Late Fusion dropped from 91.3 to 87.5.
(iv) These drops confirm that thematic features contribute significantly to diagnostic precision, especially in contextually ambiguous cases.

This pie chart in Figure 6 reflects the impact of thematic embeddings in 21% of total cases where predictions changed due to their inclusion:

(i) 78% of these changes led to correct reclassifications, demonstrating the thematic features' clinical utility.
(ii) 17% had no net effect, suggesting the model's robustness.
(iii) Only 5% led to incorrect shifts, a minimal trade-off for the gains in personalized inference.

This grouped bar chart in Figure 7 illustrates how the late fusion model dynamically reallocates attention based on case characteristics:

(i) In an imaging-dominant case, the model focused primarily on visual features (80% attention).
(ii) In a contextual-dominant case, thematic embeddings received 50% of the attention, outperforming imaging and structured data.
(iii) This adaptability confirms the model’s context-sensitive intelligence—prioritizing patient-reported concerns when imaging is inconclusive.

These figures collectively demonstrate that integrating thematic analysis:

(i) Improves predictive performance,
(ii) Increases diagnostic stability,
(iii) Enhances interpretability, and
(iv) Supports patient-centered, context-aware decision making.

Figure 2: Comparison of Model Performance Across Fusion Strategies

Figure 3: Attention Heatmap Across Modalities in Late Fusion Model

Figure 4: Ablation Analysis — F1-Score Impact of Removing Each Modality

Figure 5: Ablation Study — Impact of Thematic Features on Performance

Figure 9: Model Calibration Curve (Late Fusion)

Figure 10: Reduction in Error Rate with Fusion Strategies

6. Discussion

The results of this study demonstrate that integrating deep residual learning with thematic analysis yields a more effective and context-aware framework for cancer diagnosis and treatment personalization. The hybrid model significantly outperformed conventional image-only architectures across multiple evaluation metrics, including accuracy, F1-score, AUC, and calibration reliability. These performance gains, coupled with improved interpretability and dynamic modality weighting, underscore the potential of multimodal systems in advancing precision oncology.

One of the primary contributions of this work is the successful operationalization of thematic analysis within a deep learning pipeline. While deep neural networks—particularly residual architectures—are highly capable of learning complex patterns in high-dimensional imaging data,

varying lexicons among clinicians, and incomplete records—all of which can impact the reliability of thematic extraction. Similarly, synthetic imaging data, despite high fidelity, may not reflect the full anatomical and pathological variance found in clinical populations. Therefore, external validation using authentic, institutionally diverse clinical datasets is necessary to confirm generalizability and robustness.

Second, while thematic analysis was operationalized using a semi-automated pipeline, its initial setup involved manual theme validation and linguistic curation, introducing a potential subjectivity bias. Future research could explore fully automated, domain-adapted thematic pipelines using more sophisticated NLP models or fine-tuning techniques on specialty-specific corpora. Additionally, longitudinal themes—those evolving across time or treatment stages—were not modeled in this framework but represent a valuable direction for temporal personalization.

Finally, integration of this framework into clinical workflows would require careful consideration of usability, interoperability with health IT systems, and clinical acceptance. The interpretability mechanisms introduced (e.g., attention heatmaps and theme attribution scores) are a step in this direction, but further development is needed to ensure that clinicians can easily understand, trust, and act upon the model's outputs.

In conclusion, this study presents a novel and technically validated approach to precision oncology that combines the pattern-recognition power of deep residual learning with the contextual richness of thematic analysis. The results provide strong theoretical and computational support for the value of hybrid, multimodal AI systems in oncology. With appropriate validation and refinement, such frameworks have the potential to reshape how diagnostic and treatment decisions are made—moving toward a model of care that is not only accurate but also deeply aligned with individual patient realities.

7. Conclusion and Future Work

This study introduced and evaluated a novel hybrid framework that integrates deep residual learning with thematic analysis to improve cancer diagnosis and treatment personalization.

Such validation is critical to ensuring generalizability and translational readiness. Second, future iterations of the thematic analysis pipeline can incorporate fully automated natural language understanding using advanced large language models fine-tuned on oncology-specific corpora. This would reduce manual coding dependencies and enable real-time application in electronic health record systems.

Another promising direction is the integration of temporal modeling. Patient themes and clinical states evolve over time, and the ability to track and integrate longitudinal data could significantly enhance the personalization of treatment pathways. Recurrent neural networks or transformer-based temporal encoders could be incorporated to dynamically update risk profiles and diagnostic confidence as new information is acquired across the patient journey.

Lastly, clinical implementation studies will be essential to explore the human factors associated with model adoption. Questions of interpretability, clinical trust, workflow integration, and ethical transparency must be addressed to facilitate responsible deployment in oncology practices. Interdisciplinary collaborations between data scientists, clinicians, and health informaticians will be key to ensuring that the technology not only performs well but also fits into the nuanced ecology of medical decision-making.

In summary, the proposed hybrid framework represents a significant step toward the next generation of AI-enabled oncology systems—systems that combine the precision of algorithmic inference with the depth and nuance of human narrative understanding. With further development, this approach holds the potential to elevate diagnostic intelligence, support personalized treatment strategies, and ultimately contribute to more humane and effective cancer care.

Numerical Conclusion:

1. Model Performance Improvement:
(i) Accuracy increased from 4% (Image-Only) to 91.2% (Early Fusion) and 93.1% (Late Fusion).
(ii) F1-Score improved from 9% (Image-Only) to 89.7% (Early Fusion) and 91.3% (Late Fusion).
(iii) AUC (Area Under Curve) rose from 91 (Image-Only) to 0.95 (Early Fusion) and 0.96 (Late Fusion).

6. Vanguri, Rami, JianJiong Gao, & Sohrab P. Shah. (2022). Harnessing multimodal data integration to advance precision oncology. Nature Reviews Cancer, 22(2), 114–126. DOI: 10.1038/s41568-021-00408-3.

7. Siam, Mohamed K., et al. (2023). Multimodal deep learning for liver cancer applications: A scoping review. Frontiers in Artificial Intelligence, 6, Article 1247195. DOI: 10.3389/frai.2023.1247195.

8. Meadows, Keith. (2021). Patient-reported outcome measures—A call for more narrative evidence. Journal of Patient Experience, 8, 1–4. DOI: 10.1177/23743735211049666.

9. Nittas, Vasileios, et al. (2025). Realizing the promise of machine learning in precision oncology: Expert perspectives on opportunities and challenges. BMC Cancer, 25, Article 276.

10. Wang, Jiasheng. (2024). Deep learning in hematology: From molecules to patients. Clinical Hematology International, 6(4), 19–42.

11. Mohsen, Farida, et al. (2022). Artificial Intelligence-based methods for fusion of electronic health records and imaging data. Scientific Reports, 12, Article 17981. DOI: 10.1038/s41598-022-22093-7.

12. Huang, Shih-Cheng, et al. (2020). Fusion of medical imaging and electronic health records using deep learning: A systematic review and implementation guidelines. NPJ Digital Medicine, 3, Article 136. DOI: 10.1038/s41746-020-00326-0.

13. Kaptein, Ad A., et al. (2024). Talking cancer—Cancer talking: A linguistic and thematic analysis of patient narratives. Journal of Patient Experience, 11.

14. Khosravi, Mohsen, et al. (2024). Principles and elements of patient-centredness in mental health services: A thematic analysis of a systematic review of reviews. BMJ Open Quality.

15. Towler, Lauren, et al. (2023). Applying machine-learning to rapidly analyze large qualitative text datasets to inform the covid-19 pandemic response: Comparing human and machine-assisted topic analysis techniques. Frontiers in Public Health, 11. Article 1268223. DOI: 10.3389/fpubh.2023.1268223.