Generate More Training Data

---

name: Generate More Training Data allowed-tools: Bash, Read, Write description: Analyze data patterns and generate additional synthetic training data

Generate More Training Data

This command analyzes patterns in existing data files (CSV, JSONL) and generates additional synthetic training data based on those patterns. Uses bash commands or inline uv python to append data efficiently without loading large files into memory.

Instructions

• IMPORTANT: You can use inline astral uv python code (with any libraries you need) for data processing. Use uv run python --with pandas --with -c "import pandas as pd; print(\"whatever you want here\")"
  • Example: uv run --with pandas --with faker python -c "import pandas as pd; from faker import Faker; fake = Faker(); print(fake.name())"
• IMPORTANT: Both bash commands and uv python are acceptable - choose the most efficient approach for each task
• IMPORTANT: When generating synthetic data, ensure variety and realistic patterns

Variables

DROPPED_FILE_PATH: [[FILE_PATH]] DROPPED_FILE_PATH_ARCHIVE: agentic_drop_zone/training_data_zone/drop_zone_file_archive/ DATA_OUTPUT_DIR: agentic_drop_zone/training_data_zone/data_output// - This is the directory where all generated data will be saved - The date_time is the current date and time in the format YYYY-MM-DD_HH-MM-SS NUM_NEW_ROWS: 25 - Default number of new data rows to generate - Can be overridden if specified in the dropped file SAMPLE_SIZE: 50 - Number of rows to sample for pattern analysis (keeps context window small) - Use Read with only a specific number of rows to keep the context window small

Workflow

• Create output directory: DATA_OUTPUT_DIR//
• Determine file format by extension (.csv or .jsonl)
• Copy the original file to output directory: cp DROPPED_FILE_PATH DATA_OUTPUT_DIR//original_

Pattern Analysis Phase

• Extract a sample for analysis (to keep context window small):

  **Option 1: Using bash commands**

  • For CSV: head -n SAMPLE_SIZE DROPPED_FILE_PATH > DATA_OUTPUT_DIR//sample.csv
  • For JSONL: head -n SAMPLE_SIZE DROPPED_FILE_PATH > DATA_OUTPUT_DIR//sample.jsonl

  **Option 2: Using uv python**

  # For CSV
  uv run --with pandas python -c "
  import pandas as pd
  df = pd.read_csv('DROPPED_FILE_PATH', nrows=100)
  df.to_csv('DATA_OUTPUT_DIR//sample.csv', index=False)
  print(f'Sampled {len(df)} rows for analysis')
  "
  
  # For JSONL
  uv run --with pandas python -c "
  import pandas as pd
  df = pd.read_json('DROPPED_FILE_PATH', lines=True, nrows=100)
  df.to_json('DATA_OUTPUT_DIR//sample.jsonl', orient='records', lines=True)
  print(f'Sampled {len(df)} rows for analysis')
  "

• Read and analyze ONLY the sample file to determine:

  • Data schema/structure
  • Field types and patterns
  • Value distributions and constraints
  • Any relationsh