ml-model-data-leak-layer

Layer For Detecting LLM Inference PII Leakage

Introduction

This project introduces a robust layer designed to be placed in front of your machine learning (ML) model inference endpoints. Its primary function is to scan for potential leaks of Personally Identifiable Information (PII) as the model generates content. This is particularly crucial for projects utilizing Large Language Models (LLMs), which, due to their extensive training data, might inadvertently generate sensitive information.

What are LLMs?

Large Language Models (LLMs) like GPT (Generative Pre-trained Transformer) and BERT (Bidirectional Encoder Representations from Transformers) are advanced AI models capable of understanding and generating human-like text. They are trained on vast datasets from the internet, enabling them to perform a wide range of language tasks, from translation and summarization to question-answering and content generation.

What Does Leaking Data Mean in the Context of LLMs?

Data leakage in the context of LLMs refers to the models generating output that includes sensitive or personal information. This can happen when the models, trained on large datasets containing real-world information, reproduce patterns or data points that should be confidential. Such leaks can pose significant privacy risks, especially when the models are used in applications that handle user data.

How This Project Works

The project employs a two-pronged approach to detect potential data leaks:

undefined

...