Agentic Frameworks Explained: Building a Scalable AI Ecosystem with an LLM Orchestrator in Python
Agentic Frameworks Explained: Building a Scalable AI Ecosystem with an LLM Orchestrator in Python
Introduction
In the realm of Artificial Intelligence, there seems to be a new ‘State of the Art’ every week. From ChatGPT’s ever-evolving model versions to groundbreaking reinforcement-learning techniques implemented in the training of DeepSeek, to the professional coding capabilities of Anthropic’s Claude 3.7 Sonnet — the rapid developments in this field continue to baffle even those who live and breathe data and AI. Talented professionals risk being left behind if they don’t keep up with the latest literature and methodologies behind language models and their applications. Even in my own writings, my hiatus resulted from needing to catch up to a point where I felt comfortable guiding others in their pursuit to build innovative solutions.
In 2025, we have matured from language models that have an almost complete understanding of language to agents that use that understanding to perform dedicated tasks.
To put it simply, agents are applications designed with task-based components, often driven by language models behind the scenes. These task-based components are known as ‘Workflows’. One example of workflow is application that uses LLMs to extract specific information from textual documents and performs some action on that extracted data. Another example is an application that uses provided context to generate a document based on a user’s query.
Example of a 'Form Filling Workflow'
In this article, I aim to lay the foundation for building an agentic framework and demonstrate how we can implement such a system.
Understanding Agentic Frameworks
We’re familiar with the concept of agents, but what exactly is an ‘Agentic Framework’? Think of it like a skill tree in your favorite RPG. The character starts with basic abilities and, over time, learns new skills, gradually building a complex repertoire of capabilities to navigate and the game’s mechanics.
Now, let’s extrapolate this back to agents.
We can create a repository of ‘skills’ — Workflows that each have their own specific function. Perhaps the first skill is to process a particular kind of document, another to code a specific pipeline, and a third to send a daily email with a new joke. For each of these, we can develop a workflow that performs each specific function. These functions don’t necessarily have to utilize AI, but AI opens up numerous possibilities for different kinds of agents.
Once we have these workflows ready, we would typically deploy each to its own server, allowing users to interact with them individually. But now, there’s another way.
Suppose the user simply types in what they want to do, attaches any relevant information, and hits Enter.
For instance, the user types, “I want to build a simple calculator app.” In return, they receive a link to download the repository of the calculator app. Then, in the next message, they upload a document and say, “Extract ABC information from this document.” It sounds like Jarvis from Iron Man, but how do we begin to build something like this?
Introducing the Orchestrator-Agent Framework.
The Orchestrator-Agent Model
High Level Architecture for a simple Agentic Workflow
The diagram above illustrates the high-level architecture of such a system. It consists of an ‘Orchestrator’ agent on the client side, which handles the user query and identifies what the user wants to do. Based on this ‘user intent,’ the orchestrator routes the user’s request to the relevant application in our library of ‘skills’ agents. This orchestrator is powered by a language model with the following system prompt:
"You are orchestrator and are provided with a user query.
Your purpose is to identify the users 'intent'.
Based on this 'intent', determine which of the following services should be utilized to fulfill the users request.
The list of available services is provided below…"
Once the intent is determined and we know which service needs to be enlisted, we simply call the endpoint for that application and provide both user-provided and other relevant context required for that service to function. These services can be deployed wherever needed, but ideally, they are deployed on the ‘server’ side of this system.
This can be further refined through a ‘Planner’ block and a ‘Reflection’ block. These would allow the orchestrator to execute it’s own plan and then reflect on the generated output before returning it to the user.
Intent Classification Mechanism
As mentioned above, we can employ a large language model to choose from a registry of available services. If the user query does not fit the criteria of any available services, we can fall back to a simple ‘chatbot’ service that continues the conversation with the user.
Diagram illustrating the intent classifier
Routing Queries to Service Agents
Once the intent classifier has decided which service to call, we need to connect to that service and make a request. For each service, we can maintain a registry of available endpoints (and their respective authentication mechanisms) to easily route user queries to different services.
The neat part is that we can design all the services to take in the entire conversation and then extract the relevant context for the desired task from that conversation. We can simply pass the conversation in the request body, along with any additional context, and make the request.
Code Snippets: Implementing Intent Classification and Routing
Here is some sample code for a basic intent classifier and routing function.
Intent Classification Function:
The example below shows a rule-based approach to classifying intent. The function determines user intent through a keyword search.
def rule_based_classify_intent(user_query):
intents = {
'calculator': ['calculate', 'compute', 'math'],
'weather': ['weather', 'forecast', 'temperature'],
'news': ['news', 'headlines', 'updates']
}
for intent, keywords in intents.items():
if any(keyword in user_query.lower() for keyword in keywords):
return intent
return 'unknown'
Alternatively, we can use an LLM to make this classification:
from openai import OpenAI
# Ensure you have set your OpenAI v key as an environment variable
# or replace 'YOUR_OPENAI_API_KEY' with your actual API key
client = OpenAI(api_key=YOUR_OPENAI_API_KEY)
def llm_based_classify_intent(user_query):
"""
Classifies the intent of a user's query using OpenAI's GPT-3.5-turbo model.
Parameters:
- user_query (str): The user's input query.
Returns:
- str: The classified intent of the user's query.
"""
# Define the system message to guide the model's behavior
system_message = (
"You are an AI assistant trained to classify user intents. "
"Based on the user's query, identify the most appropriate intent from the following list: "
"1. BuildCalculatorApp "
"2. CheckWeather "
"3. SetReminder "
"4. PlayMusic "
"5. GetNewsUpdate "
"6. UnknownIntent "
"Respond with only the intent name."
)
# Create the messages payload
messages = [
{"role": "system", "content": system_message},
{"role": "user", "content": user_query}
]
try:
# Call the OpenAI API to get the completion
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=messages,
max_tokens=10, # Limit the response length
n=1, # Number of responses to generate
stop=None, # No stop sequence
temperature=0 # Deterministic output
)
# Extract the intent from the response
intent = response.choices[0].message['content'].strip()
# Return the classified intent
return intent
except Exception as e:
# Handle exceptions (e.g., API errors)
print(f"An error occurred: {e}")
return "UnknownIntent"
# Example usage:
user_query = "Can you build me a simple calculator app?"
intent = llm_based_classify_intent(user_query)
print(f"Identified Intent: {intent}")
Routing Function:
Below is a simple routing function that calls specific Workflows based on the user’s query. First, we classify the intent and then make the corresponding service call.
def route_query(user_query):
intent = classify_intent(user_query)
if intent == 'calculator':
return call_calculator_service(user_query)
elif intent == 'weather':
return call_weather_service(user_query)
elif intent == 'news':
return call_news_service(user_query)
else:
return handle_unknown_intent(user_query)
Benefits of the Orchestrator-Agent Framework
This modular approach enables us to scale AI capabilities effectively. By allowing the addition or removal of specialized agents as needed, the system can adjust to varying workloads without disrupting overall operations.
Specialized agents within this framework are designed to handle specific tasks, leading to more effective operations. By delegating responsibilities to agents with targeted expertise, tasks are performed more accurately and swiftly, reducing errors and processing times.
Conclusion
This article presents a simplified version of an agentic framework, focusing on the foundational aspects without delving into the specifics of individual agents. The guidelines provided aim to clarify the concept of AI agents and inspire readers to develop innovative applications. This is just a very basic implementation, but there are many tools out there like LangChain and LangGraph that add a layer of abstraction on the LLMs and make it a lot easier to create these workflows.