Advances in Deep Neural Network Architectures

Abstract

Deep neural networks have fundamentally altered the landscape of machine learning, yet the architectural decisions underpinning their success remain an active area of investigation. This study presents a systematic empirical evaluation of six prominent deep neural network architectures -- standard feedforward networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), transformer-based models, and graph neural networks (GNNs) -- across four canonical benchmark tasks: image classification (CIFAR-100), sequence modelling (Penn Treebank), tabular regression (UCI Housing), and node classification (Cora citation graph). A total of 2,160 controlled experiments were conducted under standardised hardware and hyperparameter search budgets to isolate the effect of architectural design from implementation variance. Transformer models achieved the highest mean accuracy on sequence tasks (87.3 +- 1.4%) but consumed 3.7x the training FLOPs of LSTMs for equivalent sequence lengths. CNNs retained a significant advantage on image benchmarks (76.8 +- 0.9% top-1 on CIFAR-100) while requiring substantially fewer parameters than vision transformers at comparable resolution. GNNs outperformed all baselines on graph-structured data (node classification F1 = 0.84 +- 0.03) but exhibited pronounced over-smoothing beyond four message-passing layers. The results demonstrate that no single architecture dominates across all task types, and that computational efficiency, data modality, and scaling behaviour should jointly inform architectural selection. A practical decision framework mapping task characteristics to recommended architectures is proposed.