Anthropic Retrieval Demo

Claude Search and Retrieval Demo [Experimental]

Introduction

Lightweight demo using the Anthropic Python SDK to experiment with Claude's Search and Retrieval capabilities over a variety of knowledge bases (Elasticsearch, vector databases, web search, and Wikipedia). In this demo, we explore an alternative to tradional retrieval-augmented generation (RAG) techniques.

The repository contains the following directories:

Directory	Description
claude_retriever	Contains the core search and retrieval logic using Claude's API.
examples	Provides example notebooks and scripts showing how to use claude_retriever with various search tools.
tests	Includes unit and integration tests for the embedders, utils, and search tools.

Table of Contents

• Setup
  • General Environment Variables
• How it works
  • Explanation of core methods
  • A peek under the hood
• Examples
  • Wikipedia Search
  • Vector Database Search
  • Web Search (Brave)
  • Enterprise Search (Elasticsearch)

Setup

This retrieval demo uses Python 3.10. Install it if you don't already have it.

Clone the repository:

git clone https://github.com/anthropic/claude-retriever-demo.git

Navigate into the demo directory:

cd claude-retriever-demo

It's recommended to use a virtual environment. Create and activate one:

python3 -m venv venv
source venv/bin/activate

Install the demo:

pip install -e .

For conda or other virtual environment managers, once you've cloned the repo, create a new environment:

conda create --name retrieval
conda activate retrieval

Install requirements:

conda install pip 
pi