المشروع الصيفي
تخصص الذكاء الاصطناعي · دفعة 2024
A Cost-Efficient, LLM Routing System
In this project, we offer a technique for querying LLMs while maximizing quality, speed, while maintaining minimum cost. First, a semantic cache is established to handle frequent prompts, saving time and money. Then, a router directs simple prompts to low-cost local models like llama, further maintaining the cost. Lastly, if the prompt is set to be urgent by the user, it is passed to a powerful, fast, but costly model such as GPT4o. If the request is not urgent, it is routed to a powerful free model, such as Mixture of Agents, which produces high-quality but slower results. In this study, we discuss our system's architecture as well as the results, which demonstrate the tremendous benefit of combining such basic components. Finally, the report concludes with citations to additional resources.
مع أكاديمية كاوست
- دفعة 2024
جامعة الملك عبدالله للعلوم والتقنية (كاوست) · المدرسة الصيفية للذكاء الاصطناعي
هل تريد تحديث بياناتك أو حذفها؟ تواصل معنا.