Developer Documentation & API

BatchIn Developer Documentation

Complete docs organized around quick start, API reference, tutorials, and cookbook content.

Get API Key Explore Models

Runtime Contract & Specifications

Production posture

Built for stable production-scale text and multimodal traffic.

Capability mix

Text and multimodal traffic share one Model API contract.

Agent protocols

OpenAI-compatible access, MCP, and llms.txt are exposed together.

Traffic policy

TTFT, backpressure, slow-consumer isolation, and regional rate limits.

Public Addresses & Protocols

OpenAPI

https://api.batchin.tech/openapi.json

API base

https://api.batchin.tech/v1

MCP

https://api.batchin.tech/v1/mcp

Public endpoints

/v1/chat/completions · /v1/responses · /v1/embeddings · /v1/images · /v1/audio/* · /v1/videos

Concurrency & Billing Semantics

Global and regional endpoints serve their audiences with a unified API experience.

Scoped rate limits and timeout controls apply by key, workspace, and region with clear failure classification.

Standard enterprise traffic uses the high-performance shared core; dedicated capacity routes to isolated pools.

Human billing and agent settlement stay separated, backed by Ed25519 cryptographic receipts.

Quickstart Guide

Quick Start (3 Languages)

1) Install

pip install openai

2) First Request

from openai import OpenAI

client = OpenAI(
  base_url="https://api.batchin.tech/v1",
  api_key="YOUR_API_KEY"
)

resp = client.chat.completions.create(
  model="deepseek-v4-pro",
  messages=[{"role": "user", "content": "Hello from BatchIn"}]
)
print(resp.choices[0].message.content)

3) Streaming

stream = client.chat.completions.create(
  model="deepseek-v4-pro",
  messages=[{"role": "user", "content": "Write a fast python async worker"}],
  stream=True
)

for chunk in stream:
  delta = chunk.choices[0].delta.content
  if delta:
    print(delta, end="")

Platform Playbooks

Dedicated Endpoints & Concurrency Pools

See the self-serve path for dedicated endpoints, private throughput usage, billing, and settlement.

Open

Dedicated Capacity Path

Understand dedicated endpoints, private deployment boundaries, key formats, and usage separation before you start a capacity rollout.

Open

Route Control + Failover Path

Review how BatchIn positions route control, batch lanes, and fallback behavior before moving production traffic.

Open

Audit Trace Verification Path

Open the public verification flow, inspect a signed evidence pack, and reproduce the trust check in the browser.

Open

OTLP Observability Path

Review the reserved OTLP ingest/export surface and the planned bridge from native traces into external observability tools.

Open

API Reference

GET/v1/users/me

Bearer

Get current user profile for authenticated API key.

Code Samples

curl -X GET https://api.batchin.tech/v1/users/me \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

POST/v1/keys

Bearer

Create a new API key with optional rate limit and monthly budget.

Code Samples

curl -X POST https://api.batchin.tech/v1/keys \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Production Key",
  "rate_limit_rpm": 120,
  "monthly_budget_cents": 500000
}'

Parameters

Name	Type	Required	Description
name	string	No	Key name
rate_limit_rpm	number	No	Requests per minute limit
monthly_budget_cents	number	No	Monthly budget in cents

Open Try it

GET/v1/keys

Bearer

List all API keys for current user.

Code Samples

curl -X GET https://api.batchin.tech/v1/keys \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

DELETE/v1/keys/{id}

Bearer

Delete one API key by id.

Code Samples

curl -X DELETE https://api.batchin.tech/v1/keys/{id} \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	API key id

Open Try it

POST/v1/chat/completions

Bearer

Text and multi-turn chat completion with SSE streaming support.

Code Samples

curl -X POST https://api.batchin.tech/v1/chat/completions \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen3.5-27b",
  "messages": [{"role":"user","content":"Explain attention in 3 bullets"}],
  "temperature": 0.7,
  "stream": false
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Model ID
messages	array	Yes	Chat message array
temperature	number	No	Sampling temperature
max_tokens	number	No	Max output tokens
stream	boolean	No	Enable streaming

Open Try it

POST/v1/responses

Bearer

Responses API compatibility layer with non-streaming response objects, SSE streaming events, and runtime receipt headers.

Code Samples

curl -X POST https://api.batchin.tech/v1/responses \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen3.5-27b",
  "input": "Explain batch inference in 3 bullets",
  "max_output_tokens": 256
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Model ID
input	string\|array	Yes	Input text or message array
max_output_tokens	number	No	Max output tokens
stream	boolean	No	Return Responses events as text/event-stream.

Open Try it

POST/v1/completions

Bearer

OpenAI-compatible text completions with prefix/suffix FIM-style passthrough.

Code Samples

curl -X POST https://api.batchin.tech/v1/completions \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "qwen3.5-27b",
  "prompt": "Complete this sentence: Batch processing helps",
  "suffix": "for large workloads.",
  "max_tokens": 128
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Completion model id
prompt	string\|array	Yes	Prompt or prefix content
suffix	string	No	FIM suffix
max_tokens	number	No	Max completion tokens

Open Try it

POST/v1/embeddings

Bearer

Convert text into embeddings.

Code Samples

curl -X POST https://api.batchin.tech/v1/embeddings \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "bge-m3",
  "input": ["batch inference", "retrieval augmented generation"]
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Embedding model id
input	string\|array	Yes	Text input to embed

Open Try it

POST/v1/audio/speech

Bearer

Convert text into synthesized speech with standard TTS-compatible speech engines.

Code Samples

curl -X POST https://api.batchin.tech/v1/audio/speech \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "tts-1",
  "input": "Welcome to BatchIn",
  "voice": "alloy",
  "response_format": "mp3"
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Speech model id
input	string	Yes	Text to synthesize
voice	string	No	Voice preset
response_format	string	No	Audio response format

Open Try it

POST/v1/audio/transcriptions

Bearer

Transcribe speech or audio files into text, with compact or verbose segment-aware output.

Code Samples

curl -X POST https://api.batchin.tech/v1/audio/transcriptions \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -F "file=@meeting.wav" \
  -F "model=whisper-large-v3-turbo" \
  -F "response_format=verbose_json" \
  -F "language=en"

Parameters

Name	Type	Required	Description
file	binary	Yes	Audio file upload
model	string	Yes	Transcription model id
language	string	No	Language code
prompt	string	No	Transcription prompt
response_format	string	No	Response format
temperature	number\|string	No	Sampling temperature

Open Try it

POST/v1/videos

Bearer

Submit video generation jobs through the unified Model API video entry.

Code Samples

curl -X POST https://api.batchin.tech/v1/videos \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "wan-2.2",
  "prompt": "Create a product teaser for an AI cloud launch",
  "duration_seconds": 6
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Video model id
prompt	string	Yes	Video prompt
duration_seconds	number	No	Video duration in seconds
aspect_ratio	string	No	Aspect ratio

Open Try it

POST/v1/images

Bearer

Generate images from text prompts through the unified Model API image entry.

Code Samples

curl -X POST https://api.batchin.tech/v1/images \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "flux-schnell",
  "prompt": "A futuristic city at sunrise",
  "size": "1024x1024"
}'

Parameters

Name	Type	Required	Description
model	string	Yes	Image model id
prompt	string	Yes	Image prompt
size	string	No	Output size

Open Try it

GET/v1/models

Public

Returns model status, pricing, context length, and license metadata.

Code Samples

curl -X GET https://api.batchin.tech/v1/models \

Open Try it

POST/v1/batches

Bearer

Submit batch jobs with mixed-model tasks and priorities.

Code Samples

curl -X POST https://api.batchin.tech/v1/batches \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "priority": "fill",
  "webhook_url": "https://your-domain.com/webhooks/batchin",
  "tasks": [
    {"custom_id":"task-1","model":"qwen3.5-27b","request_body":{"messages":[{"role":"user","content":"translate this"}]}},
    {"custom_id":"task-2","model":"glm-5.1","request_body":{"messages":[{"role":"user","content":"summarize this"}]}}
  ]
}'

Parameters

Name	Type	Required	Description
tasks	array	Yes	Task array (each task can set its own model)
priority	string(high\|low\|fill)	No	Scheduling priority
webhook_url	string(url)	No	Webhook callback URL

Open Try it

GET/v1/batches

Bearer

List batch jobs for current user.

Code Samples

curl -X GET https://api.batchin.tech/v1/batches \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/batches/{id}

Bearer

Get batch detail including task results.

Code Samples

curl -X GET https://api.batchin.tech/v1/batches/{id} \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Batch id

Open Try it

POST/v1/batches/{id}/cancel

Bearer

Cancel a pending or running batch job.

Code Samples

curl -X POST https://api.batchin.tech/v1/batches/{id}/cancel \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Batch id

Open Try it

POST/v1/topup/alipay/checkout

Bearer

Create an Alipay checkout session for Chinese-site credit top-up.

Code Samples

curl -X POST https://api.batchin.tech/v1/topup/alipay/checkout \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "amount_cents": 5000,
  "currency": "cny",
  "locale": "zh-CN"
}'

Parameters

Name	Type	Required	Description
amount_cents	number	Yes	Top-up amount in CNY cents
locale	string	No	Chinese locale, zh-CN or zh-HK

Open Try it

POST/v1/topup/stripe/checkout

Bearer

Create Stripe checkout session for credit top-up.

Code Samples

curl -X POST https://api.batchin.tech/v1/topup/stripe/checkout \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "amount_cents": 5000
}'

Parameters

Name	Type	Required	Description
amount_cents	number	Yes	Top-up amount in cents

Open Try it

GET/v1/topup/history

Bearer

Get top-up transaction history.

Code Samples

curl -X GET https://api.batchin.tech/v1/topup/history \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/topup/usdc/chains

Bearer

Get supported USDC chains and deposit addresses.

Code Samples

curl -X GET https://api.batchin.tech/v1/topup/usdc/chains \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/usage/logs?limit=50&offset=0

Bearer

List request-level usage logs.

Code Samples

curl -X GET https://api.batchin.tech/v1/usage/logs?limit=50&offset=0 \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
limit	query:number	No	Page size (1-500)
offset	query:number	No	Offset

Open Try it

GET/v1/usage/summary

Bearer

Get total requests, tokens, and cost summary.

Code Samples

curl -X GET https://api.batchin.tech/v1/usage/summary \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/usage/by-model

Bearer

Aggregate usage by model dimension.

Code Samples

curl -X GET https://api.batchin.tech/v1/usage/by-model \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

POST/v1/vaas

Bearer

Create verifiable audit record and signature.

Code Samples

curl -X POST https://api.batchin.tech/v1/vaas \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "input_text": "What is a zero-knowledge proof?",
  "output_text": "A cryptographic method...",
  "model_id": "qwen3.5-27b",
  "gpu_id": "execution-a19c-42ef"
}'

Parameters

Name	Type	Required	Description
input_text	string	Yes	Input text
output_text	string	Yes	Output text
model_id	string	Yes	Model id
gpu_id	string	No	Masked execution environment identifier

Open Try it

GET/v1/vaas/{id}

Public

Fetch audit details by audit_id.

Code Samples

curl -X GET https://api.batchin.tech/v1/vaas/{id} \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Audit id

Open Try it

GET/v1/vaas/{id}/verify

Public

Verify audit signature validity.

Code Samples

curl -X GET https://api.batchin.tech/v1/vaas/{id}/verify \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Audit id

Open Try it

GET/v1/vaas/{id}/evidence

Public

Return the audit evidence pack used for browser verification and export workflows.

Code Samples

curl -X GET https://api.batchin.tech/v1/vaas/{id}/evidence \

Parameters

Name	Type	Required	Description
id	path:uuid	Yes	Audit id

Open Try it

GET/v1/vaas/pubkey/current

Public

Get public key used for browser-side signature verification.

Code Samples

curl -X GET https://api.batchin.tech/v1/vaas/pubkey/current \

Open Try it

GET/v1/traces?limit=20

Bearer

List native runtime traces with route, prompt-version, and audit linkage.

Code Samples

curl -X GET https://api.batchin.tech/v1/traces?limit=20 \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
limit	query:number	No	Page size (1-100)

Open Try it

GET/v1/traces/{trace_id}

Bearer

Fetch one native runtime trace by trace_id.

Code Samples

curl -X GET https://api.batchin.tech/v1/traces/{trace_id} \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Parameters

Name	Type	Required	Description
trace_id	path:string	Yes	Trace ID

Open Try it

GET/v1/prompts

Bearer

Return prompt registries with promoted versions, latest observed versions, and version details.

Code Samples

curl -X GET https://api.batchin.tech/v1/prompts \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/prompt-regressions

Bearer

Return automatic sampled-replay regression runs with sample counts, similarity, and latency summaries.

Code Samples

curl -X GET https://api.batchin.tech/v1/prompt-regressions \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/v1/experiments

Bearer

Return automatic experiments for candidate prompt versions, including replay state and auto-promotion results.

Code Samples

curl -X GET https://api.batchin.tech/v1/experiments \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

POST/v1/otlp/traces

Bearer

Accept OTLP/HTTP JSON or protobuf traces and merge spans into the unified trace view.

Code Samples

curl -X POST https://api.batchin.tech/v1/otlp/traces \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "resourceSpans": []
}'

Open Try it

GET/v1/otlp/export

Bearer

Export spans from the unified trace view as OTLP protobuf or JSON.

Code Samples

curl -X GET https://api.batchin.tech/v1/otlp/export \
  -H "Authorization: Bearer $BATCHIN_API_KEY" \

Open Try it

GET/health

Public

Service health probe endpoint.

Code Samples

curl -X GET https://api.batchin.tech/health \

Open Try it

Tutorials

Batch Tutorial

Upload a JSONL input file (one task per line).
Create a batch job and store batch_id.
Poll status, then download output file.

Responses Tutorial

Send a single input to /v1/responses when you want a lighter response object instead of Chat Completions.
Inspect traceparent and X-BatchIn-* receipt headers to capture route, prompt, and run context.
Follow the trace into /console/traces and open the linked audit proof in /verify.

VaaS Tutorial

Send text to /v1/vaas.
Define policy thresholds using risk_score.
Connect final decisions to audit logs.