Natural Language Understanding in Conversational Agents

Abstract

Conversational agents -- dialogue systems that interact with users through natural language -- depend critically on natural language understanding (NLU) to interpret user intent, extract relevant entities, manage conversational context, and resolve ambiguity. This study presents a controlled evaluation of five NLU architectures -- joint intent-entity BERT, GPT-2 generative NLU, T5 sequence-to-sequence, retrieval-augmented generation (RAG), and in-context learning with large language models (LLM-ICL) -- across four conversational domains: task-oriented dialogue (MultiWOZ restaurant/hotel booking), customer support (banking FAQ with 77 intents), open-domain chitchat (DailyDialog), and multilingual dialogue (Swedish, German, French customer service). A total of 1,680 experiments evaluated intent accuracy, entity F1, dialogue state tracking accuracy, and response appropriateness. Joint BERT achieved the highest intent accuracy on task-oriented domains (96.4 +- 0.3%) and entity F1 (94.8 +- 0.4). LLM-ICL with 8-shot prompting achieved the strongest zero-shot cross-domain transfer (84.2 +- 1.4% intent accuracy on unseen domains vs. 42.6% for fine-tuned BERT). RAG achieved the highest response factual accuracy (92.4 +- 0.8%) by grounding generation in retrieved knowledge. T5 achieved the best dialogue state tracking (joint goal accuracy = 54.8 +- 0.6% on MultiWOZ) by framing tracking as sequence generation. Multilingual performance dropped 8.4-14.2% relative to English across all models, with the gap smallest for multilingual pre-trained models (mBERT, mT5). The number of in-domain training dialogues was the strongest predictor of NLU quality (partial R2 = 0.56), followed by pre-training corpus size (R2 = 0.24). A practical NLU architecture selection guide mapping dialogue type, language coverage, and training data availability to recommended configurations is proposed.