Objective To build and merge a diagnostic model called multi-input DenseNet fused with clinical features (MI-DenseCFNet) for discriminating between Staphylococcus aureus pneumonia (SAP) and Aspergillus pneumonia (ASP) and to evaluate the significant correlation of each clinical feature in determining these two types of pneumonia using a random forest dichotomous diagnosis model. This will enhance diagnostic accuracy and efficiency in distinguishing between SAP and ASP.Methods In this study, 60 patients with clinically confirmed SAP and ASP, who were admitted to four large tertiary hospitals in Kunming, China, were included. Thoracic high-resolution CT lung windows of all patients were extracted from the picture archiving and communication system, and the corresponding clinical data of each patient were collected.Results The MI-DenseCFNet diagnosis model demonstrates an internal validation set with an area under the curve (AUC) of 0.92. Its external validation set demonstrates an AUC of 0.83. The model requires only 10.24s to generate a categorical diagnosis and produce results from 20 cases of data. Compared with high-, mid-, and low-ranking radiologists, the model achieves accuracies of 78% vs. 75% vs. 60% vs. 40%. Eleven significant clinical features were screened by the random forest dichotomous diagnosis model.Conclusion The MI-DenseCFNet multimodal diagnosis model can effectively diagnose SAP and ASP, and its diagnostic performance significantly exceeds that of junior radiologists. The 11 important clinical features were screened in the constructed random forest dichotomous diagnostic model, providing a reference for clinicians.