Req Llm Gateway

ReqLLMGateway

OpenAI-compatible HTTP proxy for LLM providers. Implemented as a Plug that can be mounted in any Phoenix or Plug-based application. Routes requests to multiple providers through ReqLLM.

Status

Metric	Value
Version	0.1.0
Elixir	>= 1.14
Runtime deps	10
Dev/test deps	5
Modules (.ex)	12
Test files	5
CI	No dedicated CI workflow (last run was a devcontainer update, failed)
Hex published	No (package metadata is configured in mix.exs)

What it does

• Exposes an OpenAI-compatible /v1/chat/completions endpoint.
• Delegates to ReqLLM for provider routing. Provider is selected by model string prefix (e.g., openai:gpt-4, anthropic:claude-3-sonnet).
• Tracks usage in ETS (no database required).
• Emits telemetry events for request lifecycle.
• Includes a LiveDashboard page for usage stats.
• Rate limiting via Hammer.
• Model name parsing and cost estimation via built-in pricing tables.

Setup

git clone https://github.com/jmanhype/req_llm_gateway
cd req_llm_gateway
mix deps.get
mix test --no-start

Mount in a Phoenix app

# router.ex
scope "/v1" do
  forward "/chat/completions", ReqLLMGateway.Plug
end

# Optional: LiveDashboard page
live_dashboard "/dashboard",
  additional_pages: [req_llm: ReqLLMGateway.LiveDashboard]

Standalone demo

export OPENAI_API_KEY="sk-..."
mix run -e "ReqLLMGateway.Application.start(:normal, [])"
# or
mix demo

Modules

Module	Purpose
ReqLLMGateway.Plug	Plug endpoint, request handling
ReqLLMGateway.LLMClient	ReqLLM wrapper
ReqLLMGateway.ModelParser	Parses provider:model strings
ReqLLMGateway.Pricing	Cost calculation per model
ReqLLMGateway.Usage	ETS-based usage tracking
ReqLLMGateway.Telemetry	Telemetry event definitions
ReqLLMGateway.LiveDashboard	Phoenix LiveDashboard page
ReqLLMGateway.Application	OTP appl