azrtydxb/Fastllm-proxy

Manage and invoke A2A agents behind FastLLM — register, patch, delete and list agents on the control plane, list them through the gateway, fetch an agent card, and invoke an agent by name.

azrtydxb/Fastllm-proxy

Run and troubleshoot the inference backends on the DGX Spark pair that FastLLM proxies to — starting or stopping models with vLLM, SGLang or sparkrun, choosing memory and speculative-decoding…

azrtydxb/Fastllm-proxy

Manage FastLLM prompt classes for semantic routing — create classes and their example prompts, list or delete them, and evaluate how a given prompt would be classified.

azrtydxb/Fastllm-proxy

Inspect and control the running FastLLM deployment — read effective configuration and deployment settings, force a snapshot rebuild, fetch the snapshot the proxies consume, and check liveness and…

azrtydxb/Fastllm-proxy

Send inference requests through the FastLLM OpenAI-compatible gateway — chat completions, completions, embeddings, rerank, score, responses, moderations, audio speech and transcription, image…

azrtydxb/Fastllm-proxy

Manage and use MCP servers behind FastLLM — register, patch, delete and list MCP servers on the control plane, and list or call their tools through the gateway.

azrtydxb/Fastllm-proxy

Register and maintain the models FastLLM can serve — create, patch or delete a model, attach backends to it, remove a backend, and set the deployment-wide fallback model.

azrtydxb/Fastllm-proxy

Read what FastLLM has been doing — usage records, time-series aggregates, the configuration audit trail, Prometheus metrics, control-plane health, and per-replica fleet status.

azrtydxb/Fastllm-proxy

Manage FastLLM principals (API clients and users) and their API keys — create or delete principals, issue and revoke keys, attach roles, and set per-principal budgets and rate limits.

azrtydxb/Fastllm-proxy

Define FastLLM roles and their permissions — create or delete roles, grant and revoke individual permissions, and inspect what a role allows.

azrtydxb/Fastllm-proxy

Route requests across models in FastLLM — create, inspect or change frontend models, their default targets, weighted splits, and routing rules (by caller, prompt size, streaming, headers, budget…

azrtydxb/Fastllm-proxy

Authenticate to the FastLLM control plane and manage sessions — log in for a session cookie, log out, set or reset a principal's password, and revoke every session at once.

azrtydxb/Fastllm-proxy

Inspect and reconcile FastLLM spend and rate limits — read global budgets and limits, trigger limit reconciliation across replicas, and sync provider prices.

azrtydxb/Fastllm-proxy

Operate and diagnose the FastLLM deployment on the kw Kubernetes cluster — check whether it is running, bring it back after a shutdown, deploy manifest changes, and work out why the gateway is…

azrtydxb/Fastllm-proxy