This paper aims to explore the possibility of utilizing vision transformers (ViTs) for on-edge medical diagnostics by experimenting with the Kvasir-Capsule image classification dataset, a large-scale image dataset of gastrointestinal diseases. Quantization techniques made available through TensorFlow Lite (TFLite), including post-training float-16 (F16) quantization and quantization-aware training (QAT), are applied to achieve reductions in model size, without compromising performance. The seven ViT models selected for this study are EfficientFormerV2S2, EfficientViT_B0, EfficientViT_M4, MobileViT_V2_050, MobileViT_V2_100, MobileViT_V2_175, and RepViT_M11. Three metrics are considered when analyzing a model: (i) F1-score, (ii) model size, and (iii) performance-to-size ratio, where performance is the F1-score and size is the model size in megabytes (MB). In terms of F1-score, we show that MobileViT_V2_175 with F16 quantization outperforms all other models with an F1-score of 0.9534. On the other hand, MobileViT_V2_050 trained using QAT was scaled down to a model size of 1.70 MB, making it the smallest model amongst the variations this paper examined. MobileViT_V2_050 also achieved the highest performance-to-size ratio of 41.25. Despite preferring smaller models for latency and memory concerns, medical diagnostics cannot afford poor-performing models. We conclude that MobileViT_V2_175 with F16 quantization is our best-performing model, with a small size of 27.47 MB, providing a benchmark for lightweight models on the Kvasir-Capsule dataset.

Pipe leakage in water distribution networks (WDNs) has been an emerging concern for water utilities worldwide due to its public health and economic significance. Not only does it cause significant water losses, but it also deteriorates the quality of the treated water in WDNs. Hence, a prompt response is required to avoid or minimize the eventual consequences. This raises the necessity of exploring the possible approaches for detecting and locating leaks in WDNs promptly. Currently, various leak detection methods exist, but they are not accurate and reliable in detecting leaks. This paper presents a novel GIS-based spatial machine learning technique that utilizes currently installed pressure, flow, and water quality monitoring sensors in WDNs, specifically employing the Geographically Weighted Regression (GWR) and Local Outlier Factor (LOF) models, based on a WDN dataset provided by our partner utility authority. In addition to its ability as a regression model for predicting a dependent variable based on input variables, GWR was selected to help identify locations on the WDN where coefficients deviate the most from the overall coefficients. To corroborate the GWR results, the Local Outlier Factor (LOF) is used as an unsupervised machine learning model to predict leak locations based on spatial local density, where locality is given by k-nearest neighbours. The sample WDN dataset provided by our utility partner was split into 70:30 for training and testing of the GWR model. The GWR model was able to predict leaks (detection and location) with a coefficient of determination (R²) of 0.909. The LOF model was able to predict the leaks with a matching of 80% with the GWR results. Then, a customized GIS interface was developed to automate the detection process in real-time as the sensor’s readings were recorded and spatial machine learning was used to process the readings. The results obtained demonstrate the ability of the proposed method to robustly detect and locate leaks in WDNs.

Vision Transformers (ViTs) have emerged in recent years as a powerful architecture for image classification tasks and beyond. However, due to their back-box nature, our understanding of their learning dynamics is minimal. In this work, we study ViT learning dynamics through the lens of class separation, tracking inter- and intra-class structure, propagating from the patch-embedding layer to the classifier head. Across four datasets and extensive hyper-parameter configurations, we characterize two primary regularities. Firstly, we observe a layer-wise consistency, where separation fuzziness spikes between the input projection and the first transformer block, then decays exponentially with depth. Secondly, provided that the first law holds, fuzziness at the final layer also decays exponentially as a function of the training epochs. The persistence of these laws across datasets and hyper-parameters suggests an intrinsic relationship to ViT training. Our study supplies a quantitative foundation for interpreting the design of transformer-based vision models. Code and reproducibility scripts are available through https://github.com/DaraVaram/vit-data-separation.