Deep learning (DL) continues to play a pivotal role in a wide range of intelligent systems, including autonomous machines, smart surveillance, industrial automation, and portable healthcare technologies. These applications often demand low-latency inference and efficient resource utilization, especially when deployed on embedded or edge devices with limited computational capacity. As DL models become increasingly complex, selecting the right inference framework is essential to meeting performance and deployment goals. In this work, we conduct a comprehensive comparison of five widely adopted inference frameworks: PyTorch, ONNX Runtime, TensorRT, Apache TVM, and JAX. All experiments are performed on the NVIDIA Jetson AGX Orin platform, a high-performance computing solution tailored for edge artificial intelligence workloads. The evaluation considers several key performance metrics, including inference accuracy, inference time, throughput, memory usage, and power consumption. Each framework is tested using a wide range of convolutional and transformer models and analyzed in terms of deployment complexity, runtime efficiency, and hardware utilization. Our results show that certain frameworks offer superior inference speed and throughput, while others provide advantages in flexibility, portability, or ease of integration. We also observe meaningful differences in how each framework manages system memory and power under various load conditions. This study offers practical insights into the trade-offs associated with deploying DL inference on resource-constrained hardware.
25 September 2022 in Computational and Mathematical Methods in Medicine
Bone marrow transplant (BMT) is an effective surgical treatment for bone marrow-related disorders. However, several associated risk factors can impair long-term survival after BMT. Machine learning (ML) technologies have been proven useful in survival prediction of BMT receivers along with the influences that limit their resilience. In this study, an efficient classification model predicting the survival of children undergoing BMT is presented using a public dataset. Several supervised ML methods were investigated in this regard with an 80-20 train-test split ratio. To ensure prediction with minimal time and resources, only the top 11 out of the 59 dataset features were considered using Chi-square feature selection method. Furthermore, hyperparameter optimization (HPO) using the grid search cross-validation (GSCV) technique was adopted to increase the accuracy of prediction. Four experiments were conducted utilizing a combination of default and optimized hyperparameters on the original and reduced datasets. Our investigation revealed that the top 11 features of HPO had the same prediction accuracy (94.73%) as the entire dataset with default parameters, however, requiring minimal time and resources. Hence, the proposed approach may aid in the development of a computer-aided diagnostic system with satisfactory accuracy and minimal computation time by utilizing medical data records.