Single-prompt LLMs fail when tasked with multifaceted engineering projects like market research, code auditing, and documentation drafting. A single model prompt frequently skips verification steps, hallucinates dependencies, or loses focus.
CrewAI provides a production-grade multi-agent orchestration framework. By breaking complex workflows into specialized autonomous agents - each with distinct roles, backstories, memory, and executable tools - CrewAI coordinates collaborative problem solving with deterministic task pipelines.
Table of Contents
Step 1: Environment Setup & Virtual Environment
CrewAI requires Python 3.10 to 3.12. Ensure you have an active OpenAI API key or a local Ollama instance configured.
Create an isolated virtual environment and install the required dependencies:
# Create virtual environment
python -m venv crewai-env
# Activate virtual environment
# Windows (PowerShell): .\crewai-env\Scripts\Activate.ps1
# Linux / macOS: source crewai-env/bin/activate
# Install project dependencies
pip install "crewai[tools]" langchain-community duckduckgo-search
Step 2: Architecture & Multi-Agent Collaboration Architecture
In CrewAI, workflows operate as a coordinated crew composed of Agents, Tasks, Tools, and Processes.
Agents communicate sequentially or hierarchically: the output of a research task automatically feeds into the context of the drafting and auditing agents.
CrewAI Multi-Agent Role Breakdown
| Agent Role | Primary Goal | Toolset Assigned | Task Output |
|---|---|---|---|
| Senior Tech Researcher | Scrape and synthesize security advisories | DuckDuckGoSearchTool | Structured factual vulnerability report |
| Lead Backend Developer | Write hardened Python API code | CodeInterpreterTool | Production-ready Python modules |
| Security & QA Auditor | Audit code for OWASP Top 10 vulnerabilities | Linter / AST Analyzer | Security verification sign-off |
Step 3: Step-by-Step Code Component Breakdown (Component Breakdown)
Step 3.1: Configuring Tools and Agents
We initialize the DuckDuckGo search tool and define specialized agents. Each agent receives a clear role, a specific goal, and a detailed backstory that constrains its behavior and ensures deterministic outputs:
from crewai import Agent, Task, Crew, Process
from crewai_tools import DuckDuckGoSearchTool
# 1. Initialize Tools
search_tool = DuckDuckGoSearchTool()
# 2. Define Researcher Agent
researcher = Agent(
role="Senior Cloud Security Analyst",
goal="Discover top 3 critical FastAPI security misconfigurations in 2026",
backstory="You are a veteran cybersecurity researcher specializing in cloud API defenses and OWASP vulnerabilities.",
tools=[search_tool],
verbose=True,
memory=True
)
# 3. Define Writer Agent
writer = Agent(
role="Lead Technical Documentation Engineer",
goal="Transform technical security audit findings into a clear, actionable developer guide",
backstory="You are an expert technical author who writes concise, practical developer tutorials without jargon.",
verbose=True
)
Step 3.2: Defining Tasks with Contextual Handoffs
Tasks represent actionable units of work. The research task gathers raw facts, and its expected output is defined strictly. The writing task takes the research output and compiles a structured guide:
# 4. Define Granular Tasks
task_research = Task(
description="Search the web for recent FastAPI security vulnerabilities (CORS, dependency injection, and JWT leaks). Output 3 concrete findings.",
expected_output="A bulleted technical report detailing each vulnerability, its CVE reference, and mitigation strategy.",
agent=researcher
)
task_write = Task(
description="Draft a 500-word developer guide based on the researcher's findings, with Python code snippets showing safe vs unsafe configurations.",
expected_output="A complete Markdown article with code blocks and hardening recommendations.",
agent=writer
)
Step 3.3: Assembling the Crew and Execution Loop
The Crew object brings together our agents and tasks. Using Process.sequential executes tasks in chronological order, automatically passing previous outputs as context to subsequent agents:
# 5. Form Crew and Execute
crew = Crew(
agents=[researcher, writer],
tasks=[task_research, task_write],
process=Process.sequential,
verbose=True
)
Step 4: How to Run and Verify Output
Execute the application from your terminal:
python main.py
Expected terminal output verification:
Initiating Multi-Agent Crew Workflow...
[Senior Cloud Security Analyst] Searching DuckDuckGo: 'FastAPI security misconfigurations 2026'...
[Senior Cloud Security Analyst] Observation: Found CORS wildcard risks, unhandled OAuth2 exceptions...
[Lead Technical Documentation Engineer] Drafting technical documentation...
[Workflow Completed] Final Synthesis Output:
# FastAPI Security Hardening Guide
...
Step 5: Complete, Working Final Code
Here is the complete, consolidated implementation ready for production testing. Save this script as main.py:
import os
from crewai import Agent, Task, Crew, Process
from crewai_tools import DuckDuckGoSearchTool
def run_security_research_crew():
# 1. Initialize Tools
search_tool = DuckDuckGoSearchTool()
# 2. Define Autonomous Agents
researcher = Agent(
role="Senior Cloud Security Analyst",
goal="Discover top 3 critical FastAPI security misconfigurations in 2026",
backstory="You are a veteran cybersecurity researcher specializing in cloud API defenses and OWASP vulnerabilities.",
tools=[search_tool],
verbose=True,
memory=True
)
writer = Agent(
role="Lead Technical Documentation Engineer",
goal="Transform technical security audit findings into a clear, actionable developer guide",
backstory="You are an expert technical author who writes concise, practical developer tutorials without jargon.",
verbose=True
)
# 3. Define Tasks
task_research = Task(
description="Search the web for recent FastAPI security vulnerabilities (CORS, dependency injection, and JWT leaks). Output 3 concrete findings.",
expected_output="A bulleted technical report detailing each vulnerability, its CVE reference, and mitigation strategy.",
agent=researcher
)
task_write = Task(
description="Draft a 500-word developer guide based on the researcher's findings, with Python code snippets showing safe vs unsafe configurations.",
expected_output="A complete Markdown article with code blocks and hardening recommendations.",
agent=writer
)
# 4. Assemble and Kickoff Crew
crew = Crew(
agents=[researcher, writer],
tasks=[task_research, task_write],
process=Process.sequential,
verbose=True
)
result = crew.kickoff()
return result
if __name__ == "__main__":
print("Starting CrewAI Autonomous Research Pipeline...")
final_report = run_security_research_crew()
print("\n================ FINAL REPORT ================\n")
print(final_report)
Step 6: Production Hardening, Edge Cases & Security
- Memory Consumption in Long Runs: Enable memory=True only on agents that require multi-turn historical recall to prevent unbounded token context growth.
- Tool Execution Timeouts: Wrap external API tools in try/except blocks and specify max_execution_time=60 to prevent agents from getting stuck on unresponsive servers.
- Local Model Execution: To run 100% offline without OpenAI costs, pass llm='ollama/llama3.1' directly to the Agent constructor.
Step 7: Frequently Asked Questions & Troubleshooting
Q: Can CrewAI agents use custom Python functions as tools?
A: Yes. Decorate any Python function with @tool from crewai_tools to allow agents to execute your internal database queries or API calls.
Q: How does CrewAI handle tool execution errors?
A: CrewAI intercepts exceptions and returns the error message directly to the agent, allowing the LLM to adjust its parameters and self-correct.
0 Comments