Garak (NVIDIA)

garak, LLM vulnerability scanner

*Generative AI Red-teaming & Assessment Kit*

`garak` checks if an LLM can be made to fail in a way we don't want. `garak` probes for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks, and many other weaknesses. If you know `nmap` or `msf` / Metasploit Framework, garak does somewhat similar things to them, but for LLMs.

`garak` focuses on ways of making an LLM or dialog system fail. It combines static, dynamic, and adaptive probes to explore this.

`garak`'s a free tool. We love developing it and are always interested in adding functionality to support applications.

[](https://opensource.org/licenses/Apache-2.0) [](https://github.com/NVIDIA/garak/actions/workflows/test_linux.yml) [](https://github.com/NVIDIA/garak/actions/workflows/test_windows.yml) [](https://github.com/NVIDIA/garak/actions/workflows/test_macos.yml) [](http://garak.readthedocs.io/en/latest/?badge=latest) [](https://arxiv.org/abs/2406.11036) [](https://discord.gg/uVch4puUCs) [](https://github.com/psf/black) [](https://pypi.org/project/garak) [](https://badge.fury.io/py/garak) [](https://pepy.tech/project/garak) [](https://pepy.tech/project/garak)

Get started

> See our user guide! [docs.garak.ai](https://docs.garak.ai/)

> Join our [Discord](https://discord.gg/uVch4puUCs)!

> Project links & home: [garak.ai](https://garak.ai/)

> Twitter: [@garak_llm](https://twitter.com/garak_llm)

> DEF CON [slides](https://garak.ai/garak_aiv_slides.pdf)!

LLM support

currently supports:

• hugging face hub generative models
• replicate text models
• openai api chat & continuation models
• aws bedrock foundation models
• litellm
• pretty much anything accessible via REST

...