Claude Batch API: Patterns for 50% Cost Reduction

Not every LLM request needs a real-time response. For the workloads that don't — data enrichment, content moderation, document classification, bulk analysis — Anthropic's Message Batches API offers a 50% cost reduction. The trade-off is latency: results come back within 24 hours instead of seconds. Here's how to architect systems that exploit this trade-off systematically.

When batch makes sense

The decision framework is simple:

• Use real-time API when a human is waiting for the response (chat, interactive features, search)
• Use batch API when results can be consumed asynchronously (overnight processing, report generation, data pipelines, content enrichment)

In our systems, roughly 60% of Claude API spend was on workloads that didn't need real-time responses. Moving those to batch immediately cut our monthly bill by 30% — and the batch results were identical in quality.

Architecture: the batch processing pipeline

Job orchestration

The core pattern is a job queue that batches individual requests, submits them to the Message Batches API, polls for completion, and routes results back to their consumers.

import anthropic
import json
import time
from dataclasses import dataclass, field
from typing import Optional
from datetime import datetime
from pathlib import Path

@dataclass
class BatchJob:
    job_id: str
    requests: list[dict]
    batch_id: Optional[str] = None
    status: str = "pending"
    created_at: datetime = field(default_factory=datetime.utcnow)
    completed_at: Optional[datetime] = None
    results: list[dict] = field(default_factory=list)

class ClaudeBatchProcessor:
    def __init__(self, api_key: str, max_batch_size: int = 10_000):
        self.client = anthropic.Anthropic(api_key=api_key)
        self.max_batch_size = max_batch_size

    def submit_batch(self, job: BatchJob) -> str:
        """Submit a batch of requests to the Message Batches API."""
        requests = []
        for i, req in enumerate(job.requests):
            requests.append({
                "custom_id": f"{job.job_id}_{i:06d}",
                "params": {
                    "model": req.get("model", "claude-sonnet-4-20250514"),
                    "max_tokens": req.get("max_tokens", 1024),
                    "system": req.get("system", ""),
                    "messages": req["messages"],
                },
            })

        batch = self.client.messages.batches.create(requests=requests)
        job.batch_id = batch.id
        job.status = "processing"
        return batch.id

    def poll_until_complete(
        self, batch_id: str, poll_interval: int = 60
    ) -> dict:
        """Poll batch status until completion."""
        while True:
            batch = self.client.messages.batches.retrieve(batch_id)

            if batch.processing_status == "ended":
                return self._collect_results(batch_id)

            time.sleep(poll_interval)

    def _collect_results(self, batch_id: str) -> dict:
        """Stream and collect batch results."""
        results = {}
        for result in self.client.messages.batches.results(batch_id):
            custom_id = result.custom_id

            if result.result.type == "succeeded":
                message = result.result.message
                text = "".join(
                    block.text
                    for block in message.content
                    if block.type == "text"
                )
                results[custom_id] = {
                    "status": "success",
                    "text": text,
                    "usage": {
                        "input_tokens": message.usage.input_tokens,
                        "output_tokens": message.usage.output_tokens,
                    },
                }
            else:
                results[custom_id] = {
                    "status": "failed",
                    "error": str(result.result),
                }

        return results

Intelligent request batching

Not all requests should go to the same batch. Group them by priority, model, and deadline:

interface BatchRequest {
  id: string;
  priority: "high" | "medium" | "low";
  deadline: Date;
  params: {
    model: string;
    maxTokens: number;
    system: string;
    messages: Array<{ role: string; content: string }>;
  };
}

class BatchScheduler {
  private queues: Map<string, BatchRequest[]> = new Map();
  private readonly MAX_BATCH_SIZE = 10_000;
  private readonly FLUSH_INTERVAL_MS = 300_000; // 5 minutes

  enqueue(request: BatchRequest): void {
    const key = `${request.params.model}:${request.priority}`;
    if (!this.queues.has(key)) {
      this.queues.set(key, []);
    }
    this.queues.get(key)!.push(request);

    // Flush if batch is full
    if (this.queues.get(key)!.length >= this.MAX_BATCH_SIZE) {
      this.flush(key);
    }
  }

  private flush(key: string): void {
    const requests = this.queues.get(key) || [];
    if (requests.length === 0) return;

    // Submit batch to API
    this.submitBatch(requests);
    this.queues.set(key, []);
  }

  private submitBatch(requests: BatchRequest[]): void {
    // Partition into chunks of MAX_BATCH_SIZE
    for (let i = 0; i < requests.length; i += this.MAX_BATCH_SIZE) {
      const chunk = requests.slice(i, i + this.MAX_BATCH_SIZE);
      // Submit chunk via ClaudeBatchProcessor
      console.log(
        `Submitting batch of ${chunk.length} requests, ` +
        `model: ${chunk[0].params.model}`
      );
    }
  }

  // Run periodic flush for time-based batching
  startPeriodicFlush(): void {
    setInterval(() => {
      for (const key of this.queues.keys()) {
        this.flush(key);
      }
    }, this.FLUSH_INTERVAL_MS);
  }
}

Cost optimization strategies beyond batch

Batch processing is the biggest lever, but combine it with these strategies for maximum savings:

1. Model routing

Not every request needs Sonnet. Build a complexity classifier that routes simple tasks to Haiku (95% cheaper):

Task type	Model	Cost per 1K requests
Simple classification	Haiku	$0.25
Content moderation	Haiku	$0.30
Complex analysis	Sonnet	$3.00
Document summarization	Sonnet (batch)	$1.50

2. Deduplication

Before submitting a batch, deduplicate identical or near-identical requests. In our content moderation pipeline, 15% of requests were duplicates within the same batch window.

3. Result caching

Cache batch results by input hash. If the same document comes through again within the cache TTL, serve from cache instead of reprocessing.

Production cost benchmarks

After implementing the full batch optimization pipeline:

Metric	Before	After	Change
Monthly Claude API spend	$28,400	$11,200	-61%
Requests processed/day	145,000	152,000	+5%
Avg cost per request	$0.0065	$0.0025	-62%
Batch vs real-time ratio	0:100	62:38	—
Processing SLA met	99.2%	99.7%	+0.5%

The 61% total reduction came from three sources: 50% discount on batch API (30% savings), model routing to Haiku (18% savings), and deduplication + caching (13% savings).

Handling batch failures gracefully

Batch processing introduces new failure modes:

1. Partial failures — some requests in a batch fail. Always process results individually and retry only the failed ones.
2. Timeout concerns — batches can take up to 24 hours. Build your SLAs around this.
3. Rate limit on batch creation — you can only have a limited number of concurrent batches. Queue accordingly.

Build an exponential backoff retry mechanism specifically for batch: if a batch partially fails, extract the failed request IDs, resubmit them in a new batch, and merge results once both complete.

Key takeaways

1. Audit your workloads for batch eligibility. Most teams have 40-70% of requests that don't need real-time responses.
2. The 50% batch discount is just the start. Combine with model routing, deduplication, and caching for 60%+ total savings.
3. Build for partial failure. Batch systems need per-request error handling and retry logic.
4. Use time-based and size-based flush triggers. Don't wait for full batches — set a maximum wait time too.
5. Monitor batch latency distributions. 24 hours is the maximum, but most batches complete in 1-6 hours. Understand your actual patterns.

Batch processing is the single highest-ROI optimization for any team spending more than $5K/month on Claude. The engineering investment is modest — mostly plumbing — and the savings are permanent.