alicankiraz1

Gemma 4 31B MTP MLX — Testing skill for Claude Code

Testing community

Local MLX gateway for running Gemma 4 31B with the Gemma 4 MTP assistant drafter.

How to install Gemma 4 31B MTP MLX

This entry records only its repository, not the path inside it, so there is no exact command to give. Open alicankiraz1/Gemma-4-31B-MTP-MLX and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Gemma 4 31B MTP MLX does

Local MLX gateway for running Gemma 4 31B with the Gemma 4 MTP assistant drafter. This is a text-only local gateway alpha. It supports Claude Code and Kilo Code protocol smoke testing, but it is not a full agent-ready gateway until tool/function calling, true token streaming, and stronger runtime concurrency are implemented.

Alternatives in Testing

Webapp Testing — Test local web applications using Playwright for UI verification and debugging 94.1k ★
Marketing Skills Versions — Current versions of all skills 13.8k ★
AI Contribution Skills (Engram) — branch-pr: clean branch and PR workflow 2.5k ★

README

Gemma 4 31B MTP MLX

Local MLX gateway for running Gemma 4 31B with the Gemma 4 MTP assistant drafter.

Current status: this is a text-only local gateway alpha. It supports Claude Code and Kilo Code protocol smoke testing, but it is not a full agent-ready gateway until tool/function calling, true token streaming, and stronger runtime concurrency are implemented.

Architecture Overview

flowchart LR
    clients["OpenAI / Anthropic compatible clients"]
    gateway["FastAPI local gateway"]
    guardrails["Auth, limits, rate limit, CORS"]
    selector["Server engine selector"]
    serialized["serialized engine
mlx_vlm.generate"]
    batched["batched-mtp engine
experimental ResponseGenerator"]
    upstream["mlx-vlm runtime"]
    target["Gemma 4 31B target model"]
    drafter["Gemma 4 MTP assistant drafter"]
    diagnostics["/health, /readyz, /metrics"]

    clients --> gateway
    gateway --> guardrails
    guardrails --> selector
    selector --> serialized
    selector --> batched
    serialized --> upstream
    batched --> upstream
    upstream --> target
    upstream --> drafter
    gateway --> diagnostics

The default profile is built for 64 GB Apple Silicon machines:

Target: mlx-community/gemma-4-31b-it-8bit
Draft: mlx-community/gemma-4-31B-it-assistant-bf16
Draft kind: mtp
Server draft block size: 4
CLI draft block size: 6

`mlx-vlm` is pinned to a post-`v0.5.0` upstream `main` snapshot:

d85ca4d0f06abc55be5b359f2bf1d33c53be2608

`v0.5.0` includes the initial Gemma 4 MTP drafter and server support. This newer SHA additionally includes later upstream fixes for Gemma 4 MTP server batching and speculative-decoding utility refactors. It is a reproducibility pin for the tested upstream snapshot, not the original MTP feature merge commit.

Install

This project targets macOS on Apple Silicon with enough unified memory for the selected profile. The default `safe64` profile is intended for 64 GB machines.