Pneumonia ranks among the most prevalent lung diseases and poses a significant concern since it is one of the diseases that may lead to death around the world. Diagnosing pneumonia necessitates a chest X-ray and substantial expertise to ensure accurate assessments. Despite the critical role of lateral X-rays in providing additional diagnostic information alongside frontal X-rays, they have not been widely used. Obtaining X-rays from multiple perspectives is crucial, significantly improving the precision of disease diagnosis. In this paper, we propose a multi-view multi-feature fusion model (MV-MFF) that integrates latent representations from a variational autoencoder and a $\beta$-variational autoencoder. Our model aims to classify pneumonia presence using multi-view X-rays. Experimental results demonstrate that the MV-MFF model achieves an accuracy of 80.4% and an area under the curve of 0.775, outperforming current state-of-the-art methods. These findings underscore the efficacy of our approach in improving pneumonia diagnosis through multi-view X-ray analysis.