Open-Prompt-Injection

Open-Prompt-Injection

Introduction

This repository is an open-source toolkit for prompt injection attacks and defenses. It enables implementation, evaluation, and extension of attacks, defenses, and LLM-integrated applications and agents. For a deeper dive into prompt injection, see [these slides](https://people.duke.edu/~zg70/code/PromptInjection.pdf), an extended version of a presentation given at the Safer with Google Summit 2025.

Required Python packages

Pre-requisite: [conda](https://www.anaconda.com/docs/getting-started/miniconda/install)

Install the environment using the following command:

conda env create -f environment.yml --name my_custom_env

Then activate the environment:

conda activate my_custom_env

Usage

A simple demo

Before you start, go to './configs/model\_configs/palm2\_config.json' and replace the API keys with your real keys. Please refer to Google's official site for how to obtain an API key for PaLM2. For Meta's Llama models and OpenAI's GPT models, please also refer to their websites for registration details.

The following code snippet creates a model and queries the model with the prompt "Write a poem about monkeys":

import OpenPromptInjection as PI
from OpenPromptInjection.utils import open_config

model_config_path = './configs/model_configs/palm2_config.json'
model_config = open_config(config_path=model_config_path)
model = PI.create_model(config=model_config)
model.print_model_info()

msg = "Write a poem about monkeys"
print(model.query(msg))

Combined attack

The following code snippet evaluates the ASV of the scenario where the target task is sentiment analysis (i.e., the target data is sst2), the injected task is spam detection (i.e., the injected data is spam detection), the model is PaLM2, and no defense is applied:

import OpenPromptInjection as PI
from OpenPromptInjection.utils import open_config

# Create the target task

...