API reference

Overview

The gateway exposes an OpenAI-compatible REST API. Requests are authenticated with a bearer token, metered per model, and routed to the healthiest upstream provider for that model. Responses follow the upstream schema, so existing parsers keep working.

Authentication

Send your API key in the Authorization header on every request. Keys are issued to your account after an order is provisioned, and can be rotated from your account page at any time.

Authorization: Bearer $MCLY_API_KEY
Content-Type: application/json

Quickstart

Install nothing new — every official OpenAI client accepts a base URL override.

curl https://api.mclytechnology.com/v1/chat/completions \
  -H "Authorization: Bearer $MCLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Explain vector databases in one paragraph."}
    ]
  }'

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MCLY_API_KEY"],
    base_url="https://api.mclytechnology.com/v1",
)

resp = client.chat.completions.create(
    model="claude-3-5-sonnet",
    messages=[{"role": "user", "content": "Summarize this RFC."}],
)
print(resp.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.MCLY_API_KEY,
  baseURL: "https://api.mclytechnology.com/v1",
});

const resp = await client.chat.completions.create({
  model: "gemini-1.5-pro",
  messages: [{ role: "user", content: "Draft a release note." }],
});
console.log(resp.choices[0].message.content);

Chat completions

The workhorse endpoint. Accepts the standard messages array, sampling parameters, tool definitions and JSON-mode response formats.

Response

{
  "id": "chatcmpl-9xK2pQ...",
  "object": "chat.completion",
  "created": 1774220400,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "..." },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 58, "completion_tokens": 214, "total_tokens": 272 },
  "provider_routed": "openai/global",
  "latency_ms": 384
}

provider_routed and latency_ms are gateway extensions — useful for logging, safe to ignore.

Streaming

Set "stream": true to receive server-sent events. Chunks arrive as data: lines and terminate with data: [DONE].

stream = client.chat.completions.create(
    model="llama-3.1-70b",
    messages=[{"role": "user", "content": "Count to ten slowly."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

const stream = await client.chat.completions.create({
  model: "llama-3.1-70b",
  messages: [{ role: "user", content: "Count to ten slowly." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Embeddings

Batch text into vectors. Vector width depends on the embedding model — check the catalog entry before you size a store.

curl https://api.mclytechnology.com/v1/embeddings \
  -H "Authorization: Bearer $MCLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "text-embedding-3-large", "input": ["first document", "second document"]}'

List models

Returns every model currently routable on your account. Cache the response and refresh hourly — the list grows as we onboard providers.

curl https://api.mclytechnology.com/v1/models \
  -H "Authorization: Bearer $MCLY_API_KEY"

GPU compute

Compute plans are delivered as dedicated instances. Once an instance is provisioned your account page lists the access host; connect over SSH with the key you registered.

ssh -i ~/.ssh/mcly_gpu deploy@<instance-host>

# NVIDIA driver and CUDA are preinstalled
nvidia-smi
python -c "import torch; print(torch.cuda.device_count())"

# Expose any OpenAI-compatible server on the instance, then
# point it in your gateway config so it appears as a model:
python -m vllm.entrypoints.openai.api_server \
    --model meta-llama/Llama-3.1-70B-Instruct \
    --port 8000

Credits & metering

Token plans are prepaid: credits are drawn down per request using the upstream provider's own token accounting, priced at the rate card attached to your plan. Nothing is billed twice, and there is no minimum spend on pay-as-you-go packs.

• Usage is metered per request, per model, per key — sufficient for chargeback to internal teams.
• Unused credits remain on the account for the validity window stated on the plan.
• Approaching your credit ceiling? Email us for a top-up or a larger pack; limits are raised manually to keep spend predictable.

Errors & limits

Errors use standard HTTP status codes with a JSON body carrying a human-readable message. Retry 429 and 5xx with exponential backoff.

Default limits: 600 requests/minute per key, 1,000,000 tokens/minute, 32 concurrent streams. Raise these with a note to [email protected].