Tavily Web Eval Generator

Tavily Real-Time Dataset Generator for RAG Evals

[](https://opensource.org/licenses/Apache-2.0) [](http://makeapullrequest.com) [](https://www.linkedin.com/in/eyal-ben-barouch-007-1a2b3c4d5/)

The **Real-Time Dataset Generator** is an advanced agent designed to **automate the creation of datasets** for evaluating **web-augmented AI agents**. By **generating domain-specific queries**, **collecting real-time web data**, and **filtering results**, it streamlines the evaluation process for **LLM-based agents**.

---


🌟 Key Features

1. `generate_search_queries_on_subject`

This node dynamically generates targeted web search queries for each subject in the provided list to ensure that subsequent retrieval processes focus on the most relevant information.

2. `web_search`

Leverages the Tavily Search API, together with optional additional search providers, to retrieve web pages and documents for each subject. Using multiple providers helps reduce bias and improves coverage. This node serves as the foundation of the retrieval process, delivering the context needed to generate high-quality question-answer pairs.

3. `generate_qa`

Processes each retrieved web page to generate question-answer pairs. Using a map-reduce paradigm, it extracts key insights from the content and synthesizes comprehensive QA items for each document.

4. `save`

Ensures that the generated question-answer pairs are saved in langsmith or locally. (Based on user input)

---

🚀 Getting Started with Tavily Real-Time Dataset Generator

Input

Parameters

• num_qa: Specifies the number of question-answer items to generate. For example, setting this to 100 will produce 100 QA items.
• qa_subjects: A list of sub