Edge Computing and AI Integration

Abstract

Edge AI -- the deployment of machine learning inference at or near the data source rather than in centralised cloud infrastructure -- addresses four imperatives that cloud-centric AI cannot satisfy simultaneously: ultra-low latency (sub-10ms for real-time control), data sovereignty and privacy (processing sensitive data without transmission to cloud), resilience under network connectivity loss, and energy efficiency at scale. The ecosystem of edge AI hardware has diversified rapidly -- from microcontroller-class neural processing units handling sub-milliwatt inference to server-grade edge GPUs -- while model compression techniques have enabled deployment of increasingly capable models on increasingly constrained hardware. This study evaluates twelve edge AI deployment configurations across five hardware tiers (microcontroller NPU, mobile SoC, edge GPU, edge server, and federated device swarm) on six application workloads: computer vision for defect inspection, NLP for on-device query answering, real-time anomaly detection, predictive maintenance, video analytics, and federated learning coordination. Hardware platforms include Arduino Nicla Vision, STM32H7 + Edge Impulse, Raspberry Pi 5, Qualcomm QCS8550, NVIDIA Jetson AGX Orin, and Intel NUC with Core Ultra AI PC. Evaluation covers inference latency, throughput, power consumption, model accuracy retention after compression, and total cost of ownership at 5-year horizon. Jetson AGX Orin achieves the best accuracy-latency balance for computer vision (ResNet-50 INT8: 4.2ms, 98.4% accuracy retention). Qualcomm QCS8550 leads on mobile-class energy efficiency (3.8 TOPS/W). Federated learning on 50-device swarms achieves 94.2% of centralised model accuracy with zero raw data leaving devices. A workload-to-hardware matching framework and edge AI system design checklist are proposed