Your LLM gateway should be administered like a database

Every team that ships on LLM APIs eventually rebuilds the same layer: retries, failover, a cache, some rate limits, a bit of routing, and a dashboard to see what's going on. Databases solved this decades ago. ProxySQL, in particular, gave MySQL a runtime you could query and reconfigure live — no restarts, no redeploys.

MindBalancer takes that operating model and points it at AI traffic. You point your OpenAI SDK at one endpoint, and you administer the gateway with a MySQL client.

The shape of it

Connect to the admin plane on port 6032 with any MySQL client and the gateway looks like a small database of your AI infrastructure:

$ mysql -h127.0.0.1 -P6032 -uadmin -p

SELECT hostgroup, name, provider, status FROM ai_servers;
INSERT INTO ai_servers (hostgroup, name, provider, weight)
  VALUES (0, 'gpt-primary', 'openai', 100);
LOAD AI SERVERS TO RUNTIME;
SHOW HOSTGROUPS;
SHOW HEALTH STATUS;
CACHE CLEAR;

Servers live in ai_servers. Routing rules live in ai_routing_rules. Tunables live in global_variables. Stats live in stats_* tables you can SELECT from. You change the running system by writing to these tables and calling LOAD ... TO RUNTIME — the same two-phase (write, then load) model ProxySQL uses.

Hostgroups: the idea worth stealing

A hostgroup is a logical tier. Put premium models in hostgroup 0 and economy models in hostgroup 1, then route by model pattern — gpt-* to 0, everything else to 1 — with priority-ordered rules. Weighting inside a hostgroup handles capacity and cost. None of this lives in your application code.

Why YAML redeploys hurt at 3am

A provider starts throwing 500s. With a config-file gateway, the fix is: edit YAML, get it reviewed, redeploy the proxy, wait for it to roll. With a runtime you administer with SQL, the fix is one statement:

UPDATE ai_servers SET status = 'OFFLINE' WHERE name = 'gpt-primary';
LOAD AI SERVERS TO RUNTIME;

The change is live, with zero downtime, and it's auditable — it's just SQL. Health checks would shun a hard-down upstream on their own, but the point is that intervention is immediate and reversible, not a deploy.

What it doesn't do

Honesty keeps this credible: rate limiting and routing are global and model-based today; per-user limits and guardrails are on the roadmap. There's no TLS termination at the proxy — run it behind nginx, Caddy or your load balancer. And we don't publish benchmarks we haven't measured.

Try it

Clone, build, point your SDK at localhost:6034, and open a MySQL client on 6032. If you've ever run ProxySQL, you already know the workflow.