Multi AI Agent Systems with CrewAI Framework
author: Marcus A. Lee
published on: July 21, 2024, 4:21 a.m.
updated on: Nov. 27, 2024, 11:25 p.m.
tags: #AI
TL;DR
Multi-AI agents enable automation that thrives on handling fuzzy inputs and outputs, creating a collaborative ecosystem of intelligent systems. This creates an environment where automation works together seamlessly to achieve optimal results.
What is Agentic Automation?
What automation used to be before was more of going from point A to point B with a lot of IF conditional statements that would end up being complex code base. Agentic automation gives a new way to write software by showing the available options to the AI.
Here we have a regular automation process for generating sales leads. This regular automation process can be improved using agentic automation approach to have crew of AI agents doing several tasks to feed the lead generations to sales . In a regular automation process, a website visitor would trigger an action to collect the visitor's data, learn more about the potential customer like their location, company, employee seize etc. Then the information is passed to sales to be categorized appropriately.
To improve this process with an agentic automation approach, we can create a crew of AI agents to do the research and collect the data from the triggered event of the customer visit. The research could be a lot more indept and expanding to provide a score, and even prepare talking points for the sales to engage with potential customers.
Benefits of Multi Agent Systems
Having multiple agents allows us to assign a specific task ensuring that the agent does it well. For example, you can set up 3 agents to help you write a blog content,; Research Agent, Writer Agent and an Editor Agent, all contributing to delivering a well written blog post.
A multiple agent system also enables customization of each agents allow different tools and models to be used for their tasks. The agent can run on different LLMs like Llama 3 70-B, GPT-4, or a trained LLM.
What is CrewAI?
A framework and platform to build a Muti Agent System with the following benefits: - Breaks all these concepts into simple structures - Provides a pattern to put these systems together - Provides tools and skills ready to be used by agents - Gives you a model to build custom tools for agents - Offers a platform for bringing these agents into production
The initial building blocks of CrewAI are Agents, Tasks, and Crews. Lets create a crew of agents to research and write a blog post to better understand how agents, tasks and the crew work.
Creating Agents
Here we will define our agents, and provide them a role, goal and backstory. Research shows that LLMs perform better when they are role playing. Here are some interesting reads on: awesome role playing papers, a survey on role-playing with language models
When sending prompts to an LLM, its best to avoid adding whitespaces. The benefit of using multiple strings versus the triple quote docstring is that it can avoid adding those whitespaces and newline characters.
Double quote strings:
# Double quote Strings
varname = "line 1 of text"
"line 2 of text"
Triple quote strings:
# Triple quote Strings
varname = """line 1 of text
line 2 of text
"""
We can also use the textwrap.dedent python package to remove any common leading whitespaces from every line in text.
For example:
def test():
# end first line with \ to avoid the empty line!
s = '''\
hello
world
'''
print(repr(s)) # prints ' hello\n world\n '
print(repr(dedent(s))) # prints 'hello\n world\n'
For now, we'll stick to using the double quotes.
Agent: Planner
planner = Agent(
role="Content Planner",
goal="Plan engaging and factually accurate content on {topic}",
backstory="You're working on planning a blog article "
"about the topic: {topic}."
"You collect information that helps the "
"audience learn something "
"and make informed decisions. "
"Your work is the basis for "
"the Content Writer to write an article on this topic.",
allow_delegation=False,
verbose=True
)
Agent: Writer
writer = Agent(
role="Content Writer",
goal="Write insightful and factually accurate "
"opinion piece about the topic: {topic}",
backstory="You're working on a writing "
"a new opinion piece about the topic: {topic}. "
"You base your writing on the work of "
"the Content Planner, who provides an outline "
"and relevant context about the topic. "
"You follow the main objectives and "
"direction of the outline, "
"as provide by the Content Planner. "
"You also provide objective and impartial insights "
"and back them up with information "
"provide by the Content Planner. "
"You acknowledge in your opinion piece "
"when your statements are opinions "
"as opposed to objective statements.",
allow_delegation=False,
verbose=True
)
Agent: Editor
editor = Agent(
role="Editor",
goal="Edit a given blog post to align with "
"the writing style of the organization. ",
backstory="You are an editor who receives a blog post "
"from the Content Writer. "
"Your goal is to review the blog post "
"to ensure that it follows journalistic best practices,"
"provides balanced viewpoints "
"when providing opinions or assertions, "
"and also avoids major controversial topics "
"or opinions when possible.",
allow_delegation=False,
verbose=True
)
Creating Tasks
Each Tasks must be provided with a description, expected_output and agent.
Task: Plan
plan = Task(
description=(
"1. Prioritize the latest trends, key players, "
"and noteworthy news on {topic}.\n"
"2. Identify the target audience, considering "
"their interests and pain points.\n"
"3. Develop a detailed content outline including "
"an introduction, key points, and a call to action.\n"
"4. Include SEO keywords and relevant data or sources."
),
expected_output="A comprehensive content plan document "
"with an outline, audience analysis, "
"SEO keywords, and resources.",
agent=planner,
)
Task: Write
write = Task(
description=(
"1. Use the content plan to craft a compelling "
"blog post on {topic}.\n"
"2. Incorporate SEO keywords naturally.\n"
"3. Sections/Subtitles are properly named "
"in an engaging manner.\n"
"4. Ensure the post is structured with an "
"engaging introduction, insightful body, "
"and a summarizing conclusion.\n"
"5. Proofread for grammatical errors and "
"alignment with the brand's voice.\n"
),
expected_output="A well-written blog post "
"in markdown format, ready for publication, "
"each section should have 2 or 3 paragraphs.",
agent=writer,
)
Task: Edit
edit = Task(
description=("Proofread the given blog post for "
"grammatical errors and "
"alignment with the brand's voice."),
expected_output="A well-written blog post in markdown format, "
"ready for publication, "
"each section should have 2 or 3 paragraphs.",
agent=editor
)
Creating the Crew
To create the crew, we pass in tasks and agents. In this example, the tasks will be performed sequentially, so the order of the task in the list matters.
Note:verbose=2 allows you to see all the logs of the execution
crew = Crew(
agents=[planner, writer, editor],
tasks=[plan, write, edit],
verbose=2
)
Running the Crew
We use kickoff and assign the topic parameter to run the crew. We can see the result in Markdown:
topic = "YOUR TOPIC HERE"
result = crew.kickoff(inputs={"topic": topic})
from IPython.display import Markdown
Markdown(result)
Custom LLMs for your Agents
Hugging Face (HuggingFaceHub endpoint)
from langchain_community.llms import HuggingFaceHub
llm = HuggingFaceHub(
repo_id="HuggingFaceH4/zephyr-7b-beta",
huggingfacehub_api_token="<HF_TOKEN_HERE>",
task="text-generation",
)
Passing "llm" to our agent function:
editor = Agent(
role="Editor",
goal="Edit a given blog post to align with "
"the writing style of the organization. ",
backstory="You are an editor who receives a blog post "
"from the Content Writer. "
"Your goal is to review the blog post "
"to ensure that it follows journalistic best practices,"
"provides balanced viewpoints "
"when providing opinions or assertions, "
"and also avoids major controversial topics "
"or opinions when possible.",
allow_delegation=False,
verbose=True
llm=llm # passing "llm" to our agent function
)
Key Elements of AI Agent
Role playing
Be mindful when setting the role for the agent as it will determine how the agent response to a question. The more specific with the role the better results we will get.
- "Give me an analysis on Tesla stock"
- "You are a Financial Analyst, give me an analyst on Tesla stock"
- "You are a FINRA approved Financial Analyst, give me an analysts on Tesla stock"
Note: the last question we used FINRA approved Financial Analyst versus just a Financial Analyst.
Focus
The ability for agents to focus reduces the probability to [hallucinate](https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence). We can keep our agent focus by reducing the amount of context they consume, limiting the number of tools. Provide only key tools to do a specific task Having too many tools available for agents can lose focus
Tools
Overloading an agent with too many tools would slow down the process of an agent to do their task, because they will not be able to decide which tools to use. Rule of thumb is to choose the tool we will give our agent is a key tool to do the their job.
Different Ways to Give Agents Tools:
- Agent Level: The Agent can use the tools on any Task it performs.
- Task Level: The Agent will only use the tools when performing that specific Task.
Note: Task Tools override the Agent Tools.
Cooperation
Ability to cooperate and bounce ideas of each other makes a big difference on the outcomes. Cooperation enables feedback and task delegation which creates better results.
Guardrails
Guardrails are important because we are dealing with fuzzy inputs, fuzzy transformation and fuzzy outputs which means we might necessarily get strong results which part of the application. However, this would sometime lead to hallucinations, hanging on using the same tools, or taking too long create an answer, so guardrails help agents avoid those things .
Memory
Memory is the ability for agents to remember what it has done. it also uses the memory to decide on future executions. CrewAI gets 3 types of memory out of the box; Long-term memory, short-term memory, and entity memory.
Short tem memory is a memory that it only needs during the crew task execution. This memory is shared across all agents, such as knowledge, activities, and learning. So short term memory helps share context during the crew execution for better outcomes.
Long term memory is stored in the database locally, allowing agents to tap into it to self-improve and learn from previous tasks. Every time an agent completes a task, it self critique itself what it could have done better or what should not be in the answer, and store the information to be tapped in the future, which results in better produced outcomes.
Entity memory is short living and is stored to help the agent understand the subjects of discussion.
Mental Framework for Agent Creation
To get the most and the best results when creating a multi AI agent system is to think like a Manager. Apparently there is a high correlation of being a good manager and being able to create a great multi agent system, because managers are conditioned to think about what is the goal and what is the process.
Questions like: - What are the people I would hire to get a job done? - What should be their roles, backstory, and goals?
The more we are specific with the roles, this will help us create the best results.
Poor Examples: - Researcher - Writer - Financial Analyst
Great Examples: - HR Research Specialist - Senior Copywriter - FINRA Approved Analyst
Key Elements of Agent Tools
There are 3 things that makes a great tool. Tools that are versatile, fault tolerant, and implement caching.
Versatile
A tool needs to be able to accept different kinds of requests because the tool is the connecting between the AI App and the external world that have strong typed inputs. So the tool needs to be versatile enough that it can handle the different things an LLM might send to them. CrewAI supports converting this arguments into the correct types and can handle the different nuances coming out of the agents.
Fault Tolerant
The second important principle of building a great tool is that it can handle errors properly without stopping the AI agent. We want to make sure that our tools can fail gracefully and try again. CrewAI implements this by default.
Caching
Caching is key to making tools worth it and usable in the real world. In most times, tools are used to communicate with the web using API requests. Caching layer would prevent unnecessary requests for having an optimal crew. CrewAI provides cross-agent caching, which allows the agents within the crew to cross check other agent's tools and previous tasks that no calls are repetitive. This prevents unnecessary requests, that prevents hitting rate limits.
Key Elements of Well Defined Tasks
Tasks are a cornerstone of multi ai agent system. When creating tasks, it is important to have the following things in mind, (1) having a clear description of the task and (2) setting a clear and concise expectation. CrewAI makes it mandatory that we have at least 2 attributes on every tasks, the description and expected_output. However, CrewAI provides a list of attributes that allows you to have more control of the task.
Pydantic Model
Pydantic Allows you to create models and work with strong typed variables in a very easy way. It is used within multi ai agent systems to help transform fuzzy outputs into strongly typed outputs that we can use in code. We can see how it works within a Crewai of agents for automating event planning.
Defining the Pydantic model:
from pydantic import BaseModel
# Define a Pydantic model for venue details
# (demonstrating Output as Pydantic)
class VenueDetails(BaseModel):
name: str
address: str
capacity: int
booking_status: str
Agents will populate this object with information about different venues gathered from a research, this is done by creating different instances of it. We would then assign a task's output_json to our pydantic model VenueDetails:
- By using
output_json, you can specify the structure of the output you want. - By using
output_file, you can get your output in a file. - By setting
human_input=True, the task will ask for human feedback (whether you like the results or not) before finalizing it.
venue_task = Task(
description="Find a venue in {event_city} "
"that meets criteria for {event_topic}.",
expected_output="All the details of a specifically chosen"
"venue you found to accommodate the event.",
human_input=True,
output_json=VenueDetails,
output_file="venue_details.json",
# Outputs the venue details as a JSON file
agent=venue_coordinator
)
The output of the agent's research would be structured like so:
// Example of Agent's output in station
{
"name": "Main Hall",
"address": "Trellis, San Francisco",
"capacity": 200,
"booking_status": "available"
}
This then makes it easier for us to use this data, like storing it in a database or saving it into a file, bridging the gap between AI agent applications and web applications.
Agentic Collaboration
Allowing agents to collaborate with each other requires a few setting adjustments. We can see how its done in a crew of agents created to perform financial analysis on stocks. In this case here we will be using the Hierarchical Process:
Creating the Agent
import os
from utils import get_openai_api_key, get_serper_api_key
from crewai_tools import ScrapeWebsiteTool, SerperDevTool
# Getting keys for tools
openai_api_key = get_openai_api_key()
os.environ["OPENAI_MODEL_NAME"] = 'gpt-3.5-turbo'
os.environ["SERPER_API_KEY"] = get_serper_api_key()
# Setting up tools
search_tool = SerperDevTool()
scrape_tool = ScrapeWebsiteTool()
# Data Analyst Agent
data_analyst_agent = Agent(
role="Data Analyst",
goal="Monitor and analyze market data in real-time "
"to identify trends and predict market movements.",
backstory="Specializing in financial markets, this agent "
"uses statistical modeling and machine learning "
"to provide crucial insights. With a knack for data, "
"the Data Analyst Agent is the cornerstone for "
"informing trading decisions.",
verbose=True,
allow_delegation=True,
tools = [scrape_tool, search_tool]
)
# Trading Agnet
trading_strategy_agent = Agent(
role="Trading Strategy Developer",
goal="Develop and test various trading strategies based "
"on insights from the Data Analyst Agent.",
backstory="Equipped with a deep understanding of financial "
"markets and quantitative analysis, this agent "
"devises and refines trading strategies. It evaluates "
"the performance of different approaches to determine "
"the most profitable and risk-averse options.",
verbose=True,
allow_delegation=True,
tools = [scrape_tool, search_tool]
)
## Execution Agent
execution_agent = Agent(
role="Trade Advisor",
goal="Suggest optimal trade execution strategies "
"based on approved trading strategies.",
backstory="This agent specializes in analyzing the timing, price, "
"and logistical details of potential trades. By evaluating "
"these factors, it provides well-founded suggestions for "
"when and how trades should be executed to maximize "
"efficiency and adherence to strategy.",
verbose=True,
allow_delegation=True,
tools = [scrape_tool, search_tool]
)
# Risk Management Agent
risk_management_agent = Agent(
role="Risk Advisor",
goal="Evaluate and provide insights on the risks "
"associated with potential trading activities.",
backstory="Armed with a deep understanding of risk assessment models "
"and market dynamics, this agent scrutinizes the potential "
"risks of proposed trades. It offers a detailed analysis of "
"risk exposure and suggests safeguards to ensure that "
"trading activities align with the firm’s risk tolerance.",
verbose=True,
allow_delegation=True,
tools = [scrape_tool, search_tool]
)
Creating the Tasks
IF we are working in a Sequential process, we may want a specific tasks to run only after receiving outputs from other tasks. By doin this we can set the context attribute to the specific task(s).
context=[research_task, profile_task],
Since this example, we are using hierarchical process so we do not need that .
# Task for Data Analyst Agent: Analyze Market Data
data_analysis_task = Task(
description=(
"Continuously monitor and analyze market data for "
"the selected stock ({stock_selection}). "
"Use statistical modeling and machine learning to "
"identify trends and predict market movements."
),
expected_output=(
"Insights and alerts about significant market "
"opportunities or threats for {stock_selection}."
),
agent=data_analyst_agent,
)
# Task for Trading Strategy Agent: Develop Trading Strategies
strategy_development_task = Task(
description=(
"Develop and refine trading strategies based on "
"the insights from the Data Analyst and "
"user-defined risk tolerance ({risk_tolerance}). "
"Consider trading preferences ({trading_strategy_preference})."
),
expected_output=(
"A set of potential trading strategies for {stock_selection} "
"that align with the user's risk tolerance."
),
agent=trading_strategy_agent,
)
# Task for Trade Advisor Agent: Plan Trade Execution
execution_planning_task = Task(
description=(
"Analyze approved trading strategies to determine the "
"best execution methods for {stock_selection}, "
"considering current market conditions and optimal pricing."
),
expected_output=(
"Detailed execution plans suggesting how and when to "
"execute trades for {stock_selection}."
),
agent=execution_agent,
)
# Task for Risk Advisor Agent: Assess Trading Risks
risk_assessment_task = Task(
description=(
"Evaluate the risks associated with the proposed trading "
"strategies and execution plans for {stock_selection}. "
"Provide a detailed analysis of potential risks "
"and suggest mitigation strategies."
),
expected_output=(
"A comprehensive risk analysis report detailing potential "
"risks and mitigation recommendations for {stock_selection}."
),
agent=risk_management_agent,
)
Creating the Crew
The Process class helps to delegate the workflow to the Agents (kind of like a Manager at work). In the crew we will create below, it will run this hierarchically. manager_llm lets you choose the "manager" LLM you want to use.
By setting the process to Process.hierarchical, CrewAI creates a manager agent by default to delegate the work to the agents to perform their tasks.
Note: However, either manager_agent or manager_llm must be set when using the hierarchical process.
from crewai import Crew, Process
from langchain_openai import ChatOpenAI
# Define the crew with agents and tasks
financial_trading_crew = Crew(
agents=[data_analyst_agent,
trading_strategy_agent,
execution_agent,
risk_management_agent],
tasks=[data_analysis_task,
strategy_development_task,
execution_planning_task,
risk_assessment_task],
manager_llm=ChatOpenAI(model="gpt-4-turbo",
temperature=0.7),
process=Process.hierarchical,
verbose=True
)
In cases that we want to specify the specific agent to be a manager, we can set the manager_agent attribute to that agent:
manager_agent=None, # Optional: explicitly set a specific agent as manager instead of the manager_llm planning=True, # Enable planning feature for pre-execution strategy
Running the Crew
# Example data for kicking off the process
financial_trading_inputs = {
'stock_selection': 'AAPL',
'initial_capital': '100000',
'risk_tolerance': 'Medium',
'trading_strategy_preference': 'Day Trading',
'news_impact_consideration': True
}
# this execution will take some time to run
result = financial_trading_crew.kickoff(inputs=financial_trading_inputs)
# Print it out in markdown
from IPython.display import Markdown
Markdown(result)