Back to Docs

gRPC Gateway

Strongly-typed Connect-RPC service over HTTP/1.1 with JSON wire format. Same authoritative gateway logic as the OpenAI-compatible HTTP route; different contract shape for clients that prefer proto-defined services.

Endpoint: https://evalguard.ai/api/grpc/<package>.<Service>/<Method>

Foundation today, expanding through the next sprint

All three methods are live: Health wraps the dependency-probe logic of the HTTP /api/v1/gateway/health route; ListModels returns the structured model catalog (overlay of MODEL_PRICING_DB + the seeded model_catalog table, optional provider filter); and Chat wraps the existing OpenAI-compatible gateway proxy at /api/v1/gateway/proxy/v1/chat/completions — same authoritative path, so firewall, BYOK lookup, cost tracking, audit, and budget all run on the gRPC path unchanged. The proto schema is the contract authority; adding a method is a single PR.

Proto schema

Lives at proto/evalguard/gateway/v1/gateway.protoin the repo. Codegen is optional — Connect's JSON transport is the authoritative wire format, so customers can hit the endpoint with any HTTP client that can JSON-encode requests.

Request shape

Connect-RPC over HTTP/1.1 + JSON
POST /api/grpc/evalguard.gateway.v1.GatewayService/Health HTTP/1.1
Host: evalguard.ai
Authorization: Bearer eg_live_…
Content-Type: application/json
Connect-Protocol-Version: 1

{}

Note: Connect requires the Connect-Protocol-Version: 1 header on every request. Missing it returns a 400 with { code: "invalid_argument", message: "..." }.

Health response

200 OK
{
  "status": "healthy",
  "components": {
    "db": "healthy",
    "redis": "healthy",
    "queue": "healthy"
  },
  "timestamp": "2026-05-22T20:30:00.000Z"
}

Error shape (Connect Codes)

501 Unimplemented
{
  "code": "unimplemented",
  "message": "Chat is not implemented yet — use POST /api/v1/gateway/proxy/v1/chat/completions"
}

We follow the standard Connect-RPC code → HTTP-status mapping: invalid_argument=400, not_found=404, permission_denied=403, unimplemented=501, internal=500, unavailable=503, etc.

TypeScript client

@connectrpc/connect-web
import { createPromiseClient } from "@connectrpc/connect";
import { createConnectTransport } from "@connectrpc/connect-web";
import { GatewayService } from "./gen/evalguard/gateway/v1/gateway_connect"; // protoc-gen-connect output

const transport = createConnectTransport({
  baseUrl: "https://evalguard.ai/api/grpc",
  interceptors: [
    (next) => (req) => {
      req.header.set("Authorization", `Bearer ${process.env.EVALGUARD_API_KEY}`);
      return next(req);
    },
  ],
});

const client = createPromiseClient(GatewayService, transport);
const health = await client.health({});
console.log(health.status);

Raw HTTP example (no proto codegen)

curl
curl -X POST https://evalguard.ai/api/grpc/evalguard.gateway.v1.GatewayService/Health \
  -H "Connect-Protocol-Version: 1" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${EVALGUARD_API_KEY}" \
  -d '{}'

Vs the competition

  • Portkey — ships gRPC support in beta. Ours is wire-compatible via the Connect protocol.
  • Promptfoo / DeepEval — REST-only.
  • LangSmith / Langfuse — REST + OTLP only; no gRPC for LLM gateway calls.

Our gRPC surface is a strict superset: same JSON request bodies as REST callers send today, but with the Connect-RPC contract shape baked in. HTTP/2 + binary protobuf is a follow-up — the wire is spec-compatible across all three transports.