Skip to content
InferMesh

SDK migration

Bring your existing code

InferMesh implements the OpenAI Chat Completions and Anthropic Messages protocols, so official SDKs work unchanged. Swap the base URL and key; keep your prompts, tools and streaming code.

OpenAI SDK

Python

from openai import OpenAI

# Before: client = OpenAI()
client = OpenAI(base_url="https://infermesh.dev/v1", api_key="sk-infer-YOUR_KEY")

stream = client.chat.completions.create(
    model="openai/gpt-6.1-sol",          # or any catalog model, e.g. "anthropic/claude-sonnet-5.5"
    messages=[{"role": "user", "content": "Write a haiku about Solana."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

TypeScript

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://infermesh.dev/v1", apiKey: process.env.INFERMESH_API_KEY });

const completion = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-5.5",
  messages: [{ role: "user", content: "Summarize this repository's README." }],
  tools: [/* your existing tool definitions */],
});

Any model in the catalog can be called through Chat Completions — including Claude and Gemini — so one client covers every lab.

Anthropic SDK

import Anthropic from "@anthropic-ai/sdk";

// The SDK appends /v1/messages itself, so omit /v1 from the base URL.
const anthropic = new Anthropic({ baseURL: "https://infermesh.dev", apiKey: process.env.INFERMESH_API_KEY });

const message = await anthropic.messages.create({
  model: "anthropic/claude-sonnet-5.5",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Hello, Claude" }],
});
  • /v1/messages and /v1/messages/count_tokens serve Claude models natively, including prompt caching (cache writes and reads are billed at Anthropic's cache rates) and anthropic-beta headers.
  • max_tokens is required on /v1/messages, as with Anthropic's API.

Vercel AI SDK

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { streamText } from "ai";

const infermesh = createOpenAICompatible({
  name: "infermesh",
  baseURL: "https://infermesh.dev/v1",
  apiKey: process.env.INFERMESH_API_KEY,
  includeUsage: true,
});

const result = streamText({
  model: infermesh("anthropic/claude-sonnet-5.5"),
  prompt: "Explain time-weighted staking in two sentences.",
});

For Claude-specific features through the AI SDK you can also use @ai-sdk/anthropic with baseURL: "https://infermesh.dev/v1".

Compatibility

FeatureStatus
Chat Completions (streaming & non-streaming)Supported
Tool / function calling, JSON mode, vision inputsSupported where the model supports it (see capabilities)
Anthropic Messages, token counting, prompt cachingSupported for Claude models
Model listing — GET /v1/modelsSupported
Embeddings, images, audio, Responses API, AssistantsNot supported — returns 404 with an error body

Errors keep their shape

Errors use the native error envelope of the endpoint you called (OpenAI or Anthropic style) with InferMesh codes such as insufficient_credits added, so existing retry and error handling keeps working.