Complexity aware cascade architecture for improving user satisfaction in conversational AI

Abstract

Conventional task-oriented chatbots frequently suffer from task incompletions and low user satisfaction when handling complex queries. This research intro duces the complexity aware cascade, an adaptive architecture that improves user service quality by dynamically matching query complexity with the appropri ate computational response. The system uses confidence and relevance scores to intelligently route requests through a sequence of a natural language under standing (NLU) model, a retrieval-augmented generation (RAG) pipeline, or a large language model (LLM). The tiered architecture was evaluated via a ran domized controlled trial (RCT) with 150 participants, measuring task success and user satisfaction. The full cascade achieved a 90% journey completion rate, representing a 92.3% improvement over baseline system and substantial gains in SERVQUAL-based service quality scores. The experiment was conducted in a domain-specific knowledge base (essential oils) with a convenience sam ple that does not represent the global population, and no real-time deployment or long-term cost analysis was performed. Accordingly, the findings should be interpreted as evidence of effectiveness in a limited setting rather than as directly scalable to all domains. Even with these limitations, this study provides arigorously tested blueprint for developing more robust and user-centric conversational AI systems.