RouteMoA is a dynamic routing framework for Mixture-of-Agents (MoA) that significantly improves efficiency by avoiding pre-inference. It uses a lightweight scorer (86M parameters) to predict coarse-grained performance scores for each LLM based solely on the query, narrowing the candidate pool to a few high-potential models without running inference. A mixture of judges then refines these scores using self-assessment (confidence scores from active models) and cross-assessment (evaluation by the best model from the previous layer), leveraging posterior knowledge from existing outputs without additional inference cost. Finally, a model ranking mechanism selects models by balancing performance, cost, and latency, with an early-stopping criterion.

Experiments on both small (5 LLMs) and large (15 LLMs) model pools show that RouteMoA matches or surpasses MoA and SMoA in accuracy while drastically reducing cost and latency. On the large pool, it achieves 78.6% average accuracy (vs. MoA's 71.3%) with 89.8% lower cost and 63.6% lower latency. On out-of-distribution tasks (AGIEval-Gaokao), it outperforms SMoA in accuracy while reducing cost by 11.5% and latency by 24.7%. The scorer achieves high hit rates (Top-1: 90.7%, Top-3: 97.9%), and ablation studies confirm the effectiveness of the mixture of judges. RouteMoA provides a practical and scalable solution for multi-agent collaboration, enabling efficient use of diverse LLMs without prohibitive computational overhead.