Ai Qe Agent banner
vijayarjun7 vijayarjun7

Ai Qe Agent

Testing community
SOURCE https://github.com/vijayarjun7/ai-qe-agent

Description

AI QE Agent with LLM Evaluation Layer — catches hallucinations, monitors chain consistency, self-heals Playwright selectors. Claude + TruLens + LangSmith + Pinecone

Installation

This entry records only its repository, not the path inside it, so there is no exact command to give. Open the source below and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

README

AI QE Agent

**End-to-End AI-Driven Quality Engineering Pipeline — Powered by Playwright + Claude AI**

![Claude](https://img.shields.io/badge/Claude-Sonnet--4--6-orange?logo=anthropic) ![TypeScript](https://img.shields.io/badge/TypeScript-E2E%20Tests-blue?logo=typescript) ![Python](https://img.shields.io/badge/Python-MLOps%20Stack-green?logo=python) ![TruLens](https://img.shields.io/badge/TruLens-Monitoring-purple) ![LangSmith](https://img.shields.io/badge/LangSmith-Tracing-red) ![MLflow](https://img.shields.io/badge/MLflow-Experiments-blue) ![Pinecone](https://img.shields.io/badge/Pinecone-Vector%20Search-teal) ![HuggingFace](https://img.shields.io/badge/🤗%20Demo-Live-yellow) ![Stars](https://img.shields.io/github/stars/vijayarjun7/ai-qe-agent?style=social)

AI QE Agent is a TypeScript automation framework that demonstrates a complete AI-driven QE pipeline: **AI writes the app → AI writes & reviews manual tests → AI generates & reviews automation scripts → Self-healing on selector/requirement changes.**

🧪 LLM Evaluation Layer

Built a custom **LLM-as-Judge eval suite** on top of the pipeline — Claude evaluating Claude's own outputs across 4 dimensions.

Eval Results (First Production Run)

Agent Quality Faithfulness Hallucination Chain Compat
ManualTestGenerator 0.87 ✅ 0.0 ⚠️ None 0.0 ⚠️
QAReviewAgent 0.80 ✅ 1.0 ✅ None 0.0 ⚠️
AutomationScriptGenerator 0.94 ✅ 1.0 ✅ Detected 🔴 —
SelfHealingAgent 1.0 ✅ 0.0 ⚠️ None —

**Avg Quality Score: 0.902 | Hallucinations caught: 2 | Pipeline breaks: 2**

Observability Stack

Tool Purpose
TruLens Real-time eval dashboard (4 metrics per agent)
LangSmith Production API tracing (latency, tokens, cost)
MLflow Experiment tracking + run comparison
Pinecone Semantic test deduplication (0.85 threshold)

Run the eval suite

npm run eval                    # TypeScrip

Related Skills

Spec Kit preview

Spec Kit

💫 Toolkit to help you get started with Spec-Driven Development

github Testing
Webapp Testing preview

Webapp Testing

Test local web applications using Playwright for UI verification and debugging

Anthropic Testing
#29 preview

#29

, [#52](https://github.com/affaan-m/everything-claude-code/issues/52), [#103](https://github.com/affaan-m/ever

affaan-m Testing
Fix Issue preview

Fix Issue

by metabase - Addresses GitHub issues by taking issue number as parameter, analyzing context, implementing sol

metabase Testing
Pypict Test Design preview

Pypict Test Design

Design comprehensive test cases using PICT (Pairwise Independent Combinatorial Testing) for optimized test sui

omkamal Testing
gstack preview

gstack

| 15,000+ | Garry Tan's exact Claude Code setup: 6 opinionated tools that serve as CEO, Eng Manager, Release M

garrytan Testing