SRK.AI
Published on

News Agent using Crew AI

Summary

In this blog post, we will build an agentic workflow to get the latest news on a given topic and then summarize the main points from the news as the output. We will use duckduckgo for fetching the latest news and CrewAI for the agentic framework.

Multi Agent Systems

Multi-agent systems are becoming more common these days. Instead of relying on one LLM to get the output, agentic systems make use of multiple LLMs or tools to get the desired output. In most cases, this is achieved by breaking down a complex task into simpler tasks, which are then solved by individual agents..

Agentic Frameworks

As agentic systems are gaining more steam, agentic frameworks that help us build these systems are also on the rise. Some of the notable ones are AutoGen, crewAI, LangGraph, phidata.

In this blog post, we will use crewAI to build our agent system. There is a great course by deeplearning.ai on Multi AI Agent Systems with crewAI in case anyone is interested in learning more about it.

News Agent

Suppose say, we are interested in a particular stock or topic and we want to know the latest news on a daily basis. We can go and search for the news daily, try to read all the articles and then understand the core information.

Some challenges in this are

  • It is time consuming to check through each article manually.
  • The same news will be repeated multiple times.
  • It becomes a repetitive task if we are interested in multiple topics.

Instead, if we can build an agent system that searches for the latest news and summarizes the important points for us, it will be helpful. Let us try to build a simple agent system for that in this post.

Setting up the environment

We will need the following packages and so let us install them.

crewai
crewai[tools]
duckduckgo-search
openai

Let us start with importing the necessary modules.

import os
from crewai import Agent
from crewai import Task
from crewai_tools import BaseTool
from duckduckgo_search import DDGS

from dotenv import load_dotenv

load_dotenv()

We will use OpenAI gpt-3.5-turbo LLM model for our system and so let us set it up.

os.environ["OPENAI_MODEL_NAME"] = "gpt-3.5-turbo"

Latest News Search Tool

We need to get the latest news on our topic of interest. So, let us create a 'News Search Tool' for that. A tool generally helps with a specific capability like web scraping, searching the internet, file reading etc. crewAI itself has a lot of inbuilt tools, including a search tool using Serper.

Instead of using the Serper tool, which requires an API key we will make use of DuckDuckGo search. So, let us create a custom tool for the same.

class NewsSearchTool(BaseTool):
    name: str = "Latest News Search tool"
    description: str = "News search tool that searches the latest news on the given topic from internet and provides the results."

    def _run(self, topic: str) -> str:
        # Using the duckduckgo python module
        ddgs = DDGS()
        # Fetching the latest news from last 1 day on a given topic
        results = ddgs.news(keywords=topic, timelimit="d", max_results=20)
        result_string=""
        # Combine the news topic and description
        for result in results:
            result_string += f"Title: {result['title']} \n Description: {result['body']}\n\n"
        # In case if there are no news
        if len(result_string.strip()) < 2:
            result_string="No news found for the given keyword"
        return result_string.strip()

news_search_tool = NewsSearchTool()

Here, we are subclassing the BaseTool in crewAI to create our own tool. We need to provide a clear name and description so that the LLM agent can make use of the tool effectively. We are using DDGS().news() from DuckDuckGo to get the latest 20 news articles and then combining their titles and descriptions to get the final result.

Agents

Now let us create agents for our task. We will create two agents:

  • Reasearcher agent: This LLM agent will use the news search tool to research about a specific topic and provide the results.
  • Summarizer agent: This LLM agent will summarize the news provided by the researcher agent.

Researcher Agent

In crewAI, we can use the Agent to create the agents. We need to provide the role, goal, backstory and tools to the agent.

# Creating a researcher agent with memory and verbose mode
researcher = Agent(
  role='Researcher',
  goal='Get the latest news in {topic}',
  verbose=True,
  memory=True,
  backstory=(
      "As a knowledge enthusiast, you always look for new happenings on a given topic "
      "and share the knowledge with broader community."
  ),
  tools=[news_search_tool],
)

Summarizer Agent

Similarly let us now create a summarizer agent by providing its role, goal, backstory and tools.

# Creating a summarizer agent with custom tools and delegation capability
summarizer = Agent(
  role='Summarizer',
  goal='Summarize the latest news in {topic}',
  verbose=True,
  memory=True,
  backstory=(
    "With a flair for simplifying complex topics, you "
    "remove redundant information and summarize the "
    "discoveries in an accessible manner."
  ),
  allow_delegation=False
)

Tasks

Now we need to specify the tasks that each of these agents need to perform. We will have two tasks

  • Research Task: This task uses the news search tool to get the latest news.
  • Summarize Task: This task summarizes the news into 2 or 3 bullet points.
# Research task
research_task = Task(
    description=(
    "Get the latest news in {topic}."
    ),
    expected_output='Points covering the latest news on {topic}.',
    tools=[news_search_tool],
    agent=researcher,
)

# Summarizer task 
summarize_task = Task(
    description=(
      "Given all the news about the {topic}, summarize them to 2 or 3 bullet points."
      "Discard the news which do not have central focus on {topic}."
      "If no news is relevant, return that no news found for the {topic}"
    ),
    expected_output=(
    "2 or 3 summarized bullet points, each one focussing on one specific news "
    "about the {topic}. If no news found, return no news found"
    ),
    agent=summarizer,
    output_file='news_summary.md',  # Example of output customization
)

As seen above, we need to specify the description and the expected_output of the task and the agent to which the task is assigned. We can also optionally specify the output_file to save the output of the task.

Kickoff

Now let us form the crew (agentic system) using the created agents and tasks, and start the process.

# Forming the tech-focused crew with some enhanced configurations
crew = Crew(
  agents=[researcher, summarizer],
  tasks=[research_task, summarize_task],
  process=Process.sequential,  # Optional: Sequential task execution is default
)

result = crew.kickoff(inputs={'topic': 'HDFC Bank'})
print(result)

In this example, we have searched for HDFC Bank topic and the crew started working.

crew_start
The results are crew_result

Conclusion

In this blog post, we saw how to build a simple news agent for our topic of interest using crewAI. Let's explore more agentic systems in our future blogs.

Code used for this blog post is present in this notebook.