AutoLLMCompose: QoS-Aware Multi-Agent Framework for API Selection and Composition

Abstract

Modern software systems often rely on multiple external APIs to satisfy complex user requests, but API composition requires more than functional relevance. When several APIs can satisfy the same subtask, response time, throughput, and availability can affect workflow quality. This thesis presents AutoLLMCompose, a QoS-aware multi-agent framework for API discovery, selection, and sequential composition planning. AutoLLMCompose decomposes natural-language requests into ordered API-backed subtasks, retrieves candidates from a shared FAISS-based catalog, applies functional refinement, ranks and selects APIs, and generates planned composition paths. The evaluation compares four strategies: Functional-Only Baseline, LLM-Based QoS Selection, TOPSIS-Based QoS Selection, and Functional-First QoS-Hybrid Selection, implemented as No-QoS, QoS-Pure-LLM, QoS-TOPSIS, and QoS-Hybrid. All strategies share the same queries, subtasks, candidate pools, catalog, and metrics. Results show that QoS-Hybrid achieved the strongest overall performance and was best or tied-best across the evaluation set. QoS-Pure-LLM improved over the baseline and tied with QoS-Hybrid in several cases. These findings indicate that QoS-aware composition is strongest when functional suitability is prioritized before QoS optimization.