Genai Security Demo

Security Chatbot


A chatbot implementation that demonstrates security defenses for generative AI. Created for the talk: [Hacking and securing your GenAI applications](https://docs.google.com/presentation/d/18FUV5O1mfQEeSth6F8JlLV7fsSmDTGnG1OCgTOg8urU/edit#slide=id.p1)

Table of Contents

• 1. Installation
• 2. LLM Providers
• 3. Running the Chatbot
• 4. Testing
• 5. Format and Linter (with auto-fix)

1. Installation

make install

1.1 Security Features

This project integrates two Meta Llama security tools:

• Llama PromptGuard
• Purple-Llama CodeShield

**Important**: These tools are part of the [Llama 3.1 license](https://www.llama.com/llama3_1/license/), which has restrictions on commercial use.

• You can disable security features by modifying the LLAMA_SECURITY_FAMILY variable in `src/config/llm_config.py`.

Accessing Security Models

1. Request access to models on HuggingFace (e.g., meta-llama/Prompt-Guard-86M)

1. Configure your HF_TOKEN environment variable with your HuggingFace personal token

2. LLM Providers

The chatbot supports three LLM providers:

• Amazon Bedrock
• Groq
• OpenAI
• Anthropic

Configure your preferred provider in [`config/llm_config.py`](config/llm_config.py).

2.1 Provider Setup

2.1.1 Amazon Bedrock

Configure AWS SSO profile:

PROFILE_NAME="data-dev" 
ACCOUNT_NUMBER="123456789123"
ROLE="power"
SSO_START_URL="https://YOUR_ORGANIZATION.awsapps.com/start"
REGION="eu-west-1"

aws configure set sso_start_url $SSO_START_URL --profile $PROFILE_NAME
aws con