METHODS AND APPARATUS FOR DISTRIBUTING GENERATIVE ARTIFICIAL INTELLIGENCE TASKS TO ENTERPRISE HARDWARE

Abstract

Methods and apparatus disclosed herein introduce a comprehensive framework for distributing generative-AI workloads across diverse enterprise hardware. Multiple routing strategies disclosed herein include user-controlled, algorithm-controlled, hybrid, and dual-path routing with feedback to accommodate varying user expertise and resource availability. The routing logic is detailed for Question Answering (QA) tasks, Retrieval-Augmented Generation (RAG)-based tasks (e.g., document parsing and retrieval), and agent tasks, including evaluation of resource availability, model complexity, and content characteristics. User feedback can be obtained to continuously refine routing decisions through Large Language Model (LLM) based and traditional machine learning models. Methods and apparatus disclosed herein initiate routing decisions to maintain cost-efficiency, performance optimization, and accuracy across heterogeneous computing environments.

CPC Classifications