Transfer Learning in Low-Resource Environments

Abstract

Transfer learning -- leveraging knowledge from data-rich source tasks to improve performance on data-scarce target tasks -- is the most practical strategy for deploying machine learning in domains where labelled data is expensive, restricted, or simply unavailable at scale. Yet the conditions under which transfer succeeds or fails remain poorly characterised, and practitioners frequently encounter negative transfer where pre-trained features hurt rather than help target performance. This study presents a controlled evaluation of five transfer learning strategies -- feature extraction (frozen backbone), full fine-tuning, adapter modules, prompt tuning, and few-shot meta-learning (MAML) -- across four low-resource domains: low-resource language NLP (Estonian, Latvian, and Lithuanian NER and sentiment with 500-5,000 labelled examples), medical imaging with scarce annotations (dermatology classification with 200-2,000 images per class), industrial defect detection (semiconductor wafer inspection with 50-500 defect examples), and endangered species identification from camera traps (100-1,000 images per species). A total of 2,400 experiments were conducted varying source model, target data size, and domain distance. Adapter modules achieved the best accuracy-efficiency balance, matching full fine-tuning within 0.8 +- 0.4% while updating only 3.6% of parameters -- critical for deployment where storage of multiple fine-tuned models is impractical. Full fine-tuning achieved the highest absolute accuracy when target data exceeded 1,000 examples but overfitted at smaller sizes. Source-target domain similarity (measured by feature distribution distance) was the strongest predictor of transfer benefit (partial R2 = 0.58), and negative transfer occurred in 14.2% of configurations where domain distance exceeded a quantified threshold. A practical transfer learning decision framework mapping target data size, domain distance, and compute constraints to recommended strategies is proposed