SYSTEM AND METHOD FOR DISTRIBUTED AI MODEL PROCESSING USING COMMODITY GRAPHIC ACCELERATORS WITH INTELLIGENT WORKLOAD ORCHESTRATION

Abstract

This present disclosure relates to the field of Artificial Intelligence (AI), in particular to a system and method for distributed AI model processing using commodity graphic accelerators with intelligent workload orchestration. A system and method are provided for registering and utilizing commodity hardware equipped with graphic accelerators for distributed Artificial Intelligence (AI) model inference in both batch and real-time processing environments. The system comprises a hardware registration module for onboarding GPUs and other graphic accelerators, an AI model registration module for cataloguing models and their specifications, and an AI-based orchestration agent that analyses model characteristics, including complexity, data volume, feature set, and lifecycle stage to determine optimal hardware allocation. The orchestration agent, in collaboration with other distributed agents, dynamically assigns workloads to registered hardware resources, prioritizing underutilized legacy devices. This approach enables efficient resource utilization, scalability, and promotes a decentralized, blockchain-like ecosystem for Al processing tasks