Agent Evaluation

   [](https://github.com/PyCQA/bandit) [](https://github.com/psf/black) [](https://squidfunk.github.io/mkdocs-material/)

Agent Evaluation

Agent Evaluation is a generative AI-powered framework for testing virtual agents.

Internally, Agent Evaluation implements an LLM agent (evaluator) that will orchestrate conversations with your own agent (target) and evaluate the responses during the conversation.

✨ Key features

• Built-in support for popular AWS services including Amazon Bedrock, Amazon Q Business, and Amazon SageMaker. You can also bring your own agent to test using Agent Evaluation.
• Orchestrate concurrent, multi-turn conversations with your agent while evaluating its responses.
• Define hooks to perform additional tasks such as integration testing.
• Can be incorporated into CI/CD pipelines to expedite the time to delivery while maintaining the stability of agents in production environments.

📚 Documentation

To get started, please visit the full documentation [here](https://awslabs.github.io/agent-evaluation/). To contribute, please refer to [CONTRIBUTING.md](./CONTRIBUTING.md)

👏 Contributors

Shout out to these awesome contributors: