Edge AI for Real-Time Decision Systems

Abstract

Edge AI -- deploying machine learning inference directly on resource-constrained devices rather than routing data to cloud servers -- is essential for applications requiring low-latency decisions, offline operation, or data privacy guarantees. This study presents a systematic evaluation of five model compression and deployment strategies -- pruning, quantisation (INT8 and INT4), knowledge distillation, neural architecture search (NAS), and combined compression pipelines -- across four edge hardware platforms (NVIDIA Jetson Nano, Jetson Xavier NX, Raspberry Pi 4 with Coral TPU, and STM32 microcontroller) and four real-time decision tasks: object detection for autonomous micro-vehicles (COCO-subset), keyword spotting for voice assistants (Google Speech Commands), predictive maintenance for industrial sensors (NASA C-MAPSS turbofan), and on-device medical triage (ECG arrhythmia classification). A total of 1,920 deployment experiments measured the accuracy-latency-energy trade-off under realistic operating conditions. INT8 quantisation achieved the best efficiency-accuracy balance, reducing latency by a mean of 2.8x with only 0.6 + 0.3% accuracy loss across tasks and platforms. Knowledge distillation from large teacher models recovered 92.4% of the accuracy gap between compact student architectures and full-sized models. NAS-designed architectures achieved 1.4x better accuracy-per-FLOP than manually designed compact models. The STM32 microcontroller platform (256 KB RAM, 1 MB flash) successfully ran keyword spotting (94.2% accuracy, 12 ms latency) and predictive maintenance (RMSE within 8% of cloud baseline) but could not support object detection or ECG classification at acceptable quality. A practical deployment guide mapping task complexity, latency requirements, and hardware constraints to recommended compression strategies is proposed