Jong-Chan Baik, Youn-Kyung Choi, Yonghun Cho, Yunju Baek, Sung-Hun Kim, Seong-Sik Kim, Soo-Byung Park, Ki Beom Kim, Yong-Il Kim. Korean J Orthod 2024;54:359-73. https://doi.org/10.4041/kjod24.016

Text-dependent speaker verification (TD-SV), which verifies speaker identity using predefined phrases, has gained attention as a reliable contactless biometric authentication method for smart devices, internet of things (IoT), and real-time applications. However, in real-world environments, limited training data, background noise, and microphone channel variability significantly degrade TD-SV performance, particularly on resource-constrained devices that require real-time inference. To address these challenges, we propose a lightweight end-to-end TD-SV model based on a convolution-augmented transformer (Conformer) architecture enhanced with a channel-temporal attention (CTA) module as an input enhancement that specifically targets speaker-discriminative patterns in short, fixed utterances. Unlike existing attention mechanisms (Squeeze-and-Excitation Networks (SENet), Convolutional Block Attention Module (CBAM)) designed for computer vision tasks, our CTA module employs frequency-wise statistical pooling to capture acoustic variability patterns crucial for speaker discrimination within identical phonetic content. Experiments conducted on the challenging far-field and noisy SLR 85 HI-MIA dataset demonstrate that the proposed CTA-Conformer achieves an equal error rate (EER) of 2.04% and a minimum detection cost function (minDCF) of 0.20, achieving competitive performance compared to recent TD-SV approaches. Additionally, INT8 quantization reduces the model size by 75.8%, significantly improves inference speed, and enabling real-time deployment on edge devices. Our approach thus offers a practical solution for robust and efficient TD-SV in embedded internet of things (IoT) environments.

Time Series Forecasting (TSF) is a pivotal capability for critical applications in industrial automation, energy management, and smart infrastructure. While Large Language Models (LLMs) and Transformer-based architectures have set new benchmarks in forecasting accuracy, their exorbitant computational costs and memory requirements prohibit their deployment on resource-constrained Microcontroller Units (MCUs). Lightweight models offer an alternative but often fail to capture complex long-term dependencies. This paper introduces a comprehensive framework to address these interconnected challenges of accuracy and efficiency. We propose a highly efficient Convolutional Neural Network (CNN) architecture, enhanced through a novel heterogeneous knowledge distillation methodology. In our framework, a powerful LLM-based teacher model, utilizing self-attention mechanisms, guides the training of a compact student model that operates solely on lightweight convolutions during inference. This approach is augmented by the Gated Dilated depthwise separable Convolution (GDC) block, which efficiently captures long-range dependencies and multi-scale patterns while drastically minimizing parameter count to meet strict MCU constraints. The robustness and generalizability of our framework are validated through extensive experiments on six long-term and five short-term forecasting benchmarks. Critically, our framework establishes a new state-of-the-art among lightweight architectures, demonstrating consistent performance gains across diverse forecasting horizons. Experimental results confirm that our method reduces Mean Squared Error (MSE) by 11.5% in long-term and 12.4% in short-term forecasting tasks compared to massive LLM-based baselines, all while utilizing less than 0.01% of the parameters. The final, optimized model is successfully deployed on a commercial ESP32-S3 MCU, demonstrating its practical viability for real-time, accurate, and efficient forecasting on edge devices with an inference latency of just 256 ms.