This review provides an in-depth analysis of current hardware acceleration approaches for image processing and neural network inference, focusing on key operations involved in these applications and the hardware platforms used to deploy them. We examine various solutions, including traditional CPU–GPU systems, custom ASIC designs, and FPGA implementations, while also considering emerging low-power, resource-constrained devices.

Unmanned aerial vehicles (UAVs) equipped with thermal cameras are increasingly used in search and rescue (SAR) operations, where low visibility and small human footprints make detection a critical challenge. Existing datasets are mostly limited to urban or open-field scenarios, and our experiments show that models trained on such heterogeneous data achieve poor results. To address this gap, we collected and annotated thermal images in mountainous environments using a DJI M3T drone under clear daytime conditions. This mountain-specific set was integrated with ten existing sources to form an extensive benchmark of over 75,000 images. We then performed a comparative evaluation of object detection models (YOLOv8/9/10, RT-DETR) and semantic segmentation networks (U-Net variants), analyzing accuracy, inference speed, and energy consumption on an NVIDIA Jetson AGX Orin. Results demonstrate that human detection tasks can be accurately solved through both semantic segmentation and object detection, achieving 90% detection accuracy using segmentation models and 85% accuracy using the YOLOv8 X detection model in mountain scenarios. On the Jetson platform, segmentation achieves real-time performance with up to 27 FPS in FP16 mode. Our contributions are as follows: (i) the introduction of a new mountainous thermal image collection extending current benchmarks and (ii) a comprehensive evaluation of detection methods on embedded hardware for SAR applications.

Smart Network Interface Cards (SmartNICs) represent a paradigm shift in system architecture by offloading packet processing and selected application logic from the host CPU to the network interface itself. This architectural evolution reduces end-to-end latency toward the physical limits of Ethernet while simultaneously decreasing CPU and memory bandwidth utilization. The current ecosystem comprises three principal categories of devices: (i) conventional fixed-function NICs augmented with limited offload capabilities; (ii) ASIC-based Data Processing Units (DPUs) that integrate multi-core processors and dedicated protocol accelerators; and (iii) FPGA-based SmartNIC shells—reconfigurable hardware frameworks that provide PCIe connectivity, DMA engines, Ethernet MAC interfaces, and control firmware, while exposing programmable logic regions for user-defined accelerators. This article provides a comparative survey of representative platforms from each category, with particular emphasis on open-source FPGA shells. It examines their architectural capabilities, programmability models, reconfiguration mechanisms, and support for GPU-centric peer-to-peer datapaths. Furthermore, it investigates the associated software stack, encompassing kernel drivers, user-space libraries, and control APIs. This study concludes by outlining open research challenges and future directions in RDMA-oriented data preprocessing and heterogeneous SmartNIC acceleration.

Field-programmable gate arrays (FPGAs) with on-package high-bandwidth memory (HBM) are an attractive substrate for low-latency, precision-customizable AI inference, yet high-level synthesis (HLS) flows expose little control over how tensors are distributed across many independent memory channels. The AMD Alveo V80 spreads its nominal 820 GB/s across 64 pseudo-channels, each capped at 12.8 GB/s, so delivered bandwidth is governed by an explicit tensor-to-channel partitioning decision that existing framework-based flows do not surface. This paper presents a methodology for HLS-based inference on the V80 built on two coupled contributions: a design-time partitioning framework that assigns each tensor to on-chip (BRAM/URAM) storage, a single HBM channel, or a stripe across several channels according to its reuse and access pattern, and a three-stage HLS flow that separates functional baseline, unroll-and-partition throughput extraction, and precision-aware DSP packing. The methodology is developed through five model kernels—a parameterized GEMM, ResNet-18, ViT-Small, a BERT attention block, and GPT-2 Small—and realized as nine engines spanning GEMM (FP32, INT8, INT4), convolution, attention, layer normalization, SoftMax, pooling, and embedding lookup. All nine engines are synthesized in Vitis HLS 2024.2 and placed, routed, and executed on the physical Alveo V80 at 400 MHz (2.5 ns period), and every engine closes timing with positive worst-case slack. We report per-engine cycle counts, post-route utilization, and post-route dynamic-power estimates, and compare engine latency and energy against NVIDIA Titan RTX (GPU) and Intel Xeon W-3223 (CPU) baselines on identical kernels. A central result, confirmed on silicon, is that INT8 roughly halves the GEMM DSP58 footprint relative to FP32 (27 to 14 slices), whereas INT4 yields no further compute reduction and acts purely as a memory-placement lever. The full-model compositions, the roofline classification, and the tensor-to-channel placement framework are analytical, design-time results rather than end-to-end measured performance. The bandwidth-scaling premise underlying the placement framework is confirmed directly on the V80: a 1-to-64 channel sweep shows aggregate HBM bandwidth scaling near-linearly to within 5–13% of the device peak, and placing the GPT-2 LM-head operand on a single channel versus the eight the framework assigns it yields a measured 6.81× speedup—a direct on-board test of the striping decision. The resulting design guidelines target HLS practitioners working with HBM-equipped FPGAs.

Large language models (LLMs) are increasingly used in programming tasks, including code generation, debugging, validation, optimization, test generation, and educational support. Their use as graders of student programming assignments is, however, still not fully understood, especially in object-oriented programming (OOP). In this context, correctness depends not only on the final output of a program but also on class structure, encapsulation, abstraction, and design decisions. This paper introduces a dataset of 271 C++ student solutions to eight OOP examination problems, together with the problem statements and the reference grades produced by the university auto-grader. We evaluate four locally served open-weight models—phi4-mini, gpt-oss:20b, qwen3.6:27b, and qwq:32b—under one endpoint, one prompt, and deterministic decoding, and we report four properties that a deployable grader must have. The first is output-format compliance, which ranges from 88.9% to 98.9% across models: qwq:32b failed to terminate on 30 of 271 submissions even at a doubled generation budget, spending 7.7 MJ, 29.9% of its total energy, to produce nothing. The second is agreement with the reference grades on the 30-point course scale, where qwen3.6:27b performs best (MAE 3.23 points, $r=0.85$) and phi4-mini worst (MAE 10.90, $r=0.17$); all four models grade systematically more strictly than the auto-grader. The third is reproducibility, which fails at two levels. Repeating an identical run, three models returned identical grades at temperature 0 with a fixed seed while gpt-oss:20b varied by up to 21 points on the same submission; restricting the serving daemon to one concurrent request makes it reproducible, locating that failure in request batching rather than in sampling. Grading the same submissions as part of a different set, however, no model is reproducible: agreement with the original run ranges from 13 to 33 of 40 submissions, and a control run with the original client excludes our instrumentation as the cause. The fourth is degeneracy: phi4-mini awarded an identical grade of zero to every submission in three of the eight examinations, 109 submissions in total, so its aggregate agreement statistics describe a model that is not grading. On a subset graded blind by two instructors, each instructor agreed with the auto-grader more closely than the two agreed with each other, so the residual error of the best-performing model is of the same order as ordinary disagreement between human graders. We also report a negative result: a declaration-level reference could be reconstructed for only five of the eight examinations, because the auto-grader merges missing declarations and wrong return values into single checks. Together these results indicate that agreement with a reference is an insufficient basis for choosing a local grading model, and that compliance, determinism, degeneracy, and the cost of failed generations must be measured alongside it.