Generative Ai Toolkit

Generative AI Toolkit

The **Generative AI Toolkit** is a lightweight library for building, testing and evaluating AI agents in Python, using any of the LLMs supported by the Amazon Bedrock Converse API.

Compared to other libraries out there, the Generative AI Toolkit indexes heavily on production observability, tracing, testing and evaluation, and simplicity of deployment on AWS. A typical production-grade deployment uses just AWS Lambda (or ECS, EKS), Amazon DynamoDB and Amazon CloudWatch.

**Sample usage**

from generative_ai_toolkit.agent import BedrockConverseAgent
from generative_ai_toolkit.context import AgentContext
from generative_ai_toolkit.test import Case, Expect

agent = BedrockConverseAgent(
    model_id="eu.amazon.nova-micro-v1:0",
    system_prompt="You are a helpful assistant. Use your tools to help the user.",
)


def weather_report(city_name: str) -> str:
    """
    Gets the current weather report for a given city

    Parameters
    ------
    city_name: string
      The name of the city
    """

    # Example of how to add tracing to your tool implementations
    tracer = AgentContext.current().tracer

    with tracer.trace("inside-weather-report") as span:
        span.add_attribute(
            "attribute_name", {"Attribute values": ["can be any Python object"]}
        )

        # Tool response
        return "Sunny"


agent.register_tool(weather_report)

# Send a message to the agent and have it stream its response back:
for chunk in agent.converse_stream("What's the weather like right now in Amsterdam?"):
    print(chunk, end="")

# Assert that the weather_report tool was used (raises an error if not):
Expect(agent.traces).tool_invocations.to_include("weather_report")

# Similar, but using test case with 2 turns:
test_case = Case(user_inputs=["What's the weather like right now?", "In Amsterdam"])
traces = test_case.run(agent)
Expect(traces).tool_invocations.to_include("weather_report").with_output("Sunny")

# Start a new conv