GCP Spanner Global Consistency: TrueTime, Latency Trade-offs, and Production Patterns

Cloud Spanner is the only production database I've used that delivers on the promise of global strong consistency without making you choose between consistency and availability. After running Spanner for 18 months across three continents serving a financial services platform, I want to share what actually happens when you depend on TrueTime for transaction ordering at global scale.

The Problem: Global Consistency Without Compromise

Our platform processes currency exchange transactions across US, EU, and APAC regions. The requirements were non-negotiable: strong consistency (no stale reads for balances), sub-100ms read latency for local users, and five-nines availability. Traditional databases force you to pick two. Spanner gives you all three — with caveats that matter in production.

How TrueTime Actually Works

TrueTime isn't magic — it's infrastructure. Google's data centers contain a combination of GPS receivers and atomic clocks that provide a time API with bounded uncertainty.

The key insight: TrueTime doesn't give you exact time. It gives you an interval [earliest, latest] where the true time definitely falls. The uncertainty is typically 1-7ms.

When a transaction commits, Spanner waits out the uncertainty interval before making the commit visible. This "commit wait" is what makes external consistency possible:

Commit timestamp = TrueTime.now().latest
Wait duration = TrueTime.now().latest - TrueTime.now().earliest (uncertainty)

In practice, this adds 2-7ms to every write transaction. For our workload, this was invisible — network latency to the nearest Spanner node dominated.

Multi-Region Configuration

We deployed a multi-region instance with the following topology:

gcloud spanner instances create finance-global \
  --config=nam-eur-asia1 \
  --description="Global financial platform" \
  --processing-units=3000

The nam-eur-asia1 configuration places leader replicas in us-central1 with read replicas in europe-west1 and asia-southeast1.

Critical design decision: Leader placement determines write latency. All writes route to the leader region. Reads can be served from any replica depending on staleness tolerance.

Latency Data From Production

Here's what we measured over 90 days across regions:

Operation	Same Region (us-central1)	Cross-Region EU	Cross-Region APAC
Strong read (single row)	4.2ms p50 / 8.1ms p99	82ms p50 / 124ms p99	168ms p50 / 215ms p99
Stale read (10s staleness)	3.8ms p50 / 7.2ms p99	5.1ms p50 / 9.8ms p99	5.4ms p50 / 11.2ms p99
Write transaction (single)	6.8ms p50 / 14.2ms p99	89ms p50 / 142ms p99	172ms p50 / 228ms p99
Write transaction (batch 10)	9.1ms p50 / 18.6ms p99	94ms p50 / 151ms p99	178ms p50 / 241ms p99

The stale read row is the revelation. With just 10 seconds of bounded staleness, cross-region reads drop from 82ms to 5.1ms — served from the local replica.

Read Patterns for Global Applications

Pattern 1: Strong Reads for Critical Data

Account balances must always be strongly consistent:

func GetAccountBalance(ctx context.Context, client *spanner.Client, accountID string) (int64, error) {
    var balance int64
    
    // Strong read - routes to leader, globally consistent
    row, err := client.Single().ReadRow(
        ctx,
        "Accounts",
        spanner.Key{accountID},
        []string{"balance_micros"},
    )
    if err != nil {
        return 0, fmt.Errorf("reading account %s: %w", accountID, err)
    }
    
    if err := row.Columns(&balance); err != nil {
        return 0, fmt.Errorf("parsing balance: %w", err)
    }
    
    return balance, nil
}

Pattern 2: Stale Reads for Analytics and Dashboards

Transaction history doesn't need real-time consistency:

func GetTransactionHistory(ctx context.Context, client *spanner.Client, accountID string) ([]*Transaction, error) {
    // Bounded staleness - can be served from local replica
    readOptions := spanner.ExactStaleness(15 * time.Second)
    
    stmt := spanner.Statement{
        SQL: `SELECT transaction_id, amount_micros, currency, created_at
              FROM Transactions
              WHERE account_id = @accountID
              ORDER BY created_at DESC
              LIMIT 100`,
        Params: map[string]interface{}{
            "accountID": accountID,
        },
    }
    
    iter := client.Single().WithTimestampBound(readOptions).Query(ctx, stmt)
    defer iter.Stop()
    
    var transactions []*Transaction
    for {
        row, err := iter.Next()
        if err == iterator.Done {
            break
        }
        if err != nil {
            return nil, fmt.Errorf("iterating transactions: %w", err)
        }
        
        var txn Transaction
        if err := row.ToStruct(&txn); err != nil {
            return nil, fmt.Errorf("parsing transaction: %w", err)
        }
        transactions = append(transactions, &txn)
    }
    
    return transactions, nil
}

Schema Design for Global Performance

Spanner's interleaved tables are essential for co-locating related data:

CREATE TABLE Accounts (
    account_id STRING(36) NOT NULL,
    owner_id STRING(36) NOT NULL,
    currency STRING(3) NOT NULL,
    balance_micros INT64 NOT NULL,
    created_at TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp=true),
    updated_at TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp=true),
) PRIMARY KEY (account_id);

-- Interleaved: Transactions are physically co-located with their Account
CREATE TABLE Transactions (
    account_id STRING(36) NOT NULL,
    transaction_id STRING(36) NOT NULL,
    amount_micros INT64 NOT NULL,
    currency STRING(3) NOT NULL,
    counterparty_id STRING(36),
    created_at TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp=true),
) PRIMARY KEY (account_id, transaction_id),
  INTERLEAVE IN PARENT Accounts ON DELETE CASCADE;

Interleaving ensures that reading an account and its recent transactions requires hitting a single split — eliminating distributed reads for the most common access pattern.

Hotspot Avoidance

Spanner splits data based on key ranges. Sequential keys create hotspots:

// BAD: Auto-incrementing IDs create write hotspots
id := fmt.Sprintf("%d", atomic.AddInt64(&counter, 1))

// GOOD: UUIDs distribute writes across splits
id := uuid.New().String()

// ALSO GOOD: Bit-reversed sequential IDs
id := bitReverse(atomic.AddInt64(&counter, 1))

We learned this the hard way when a batch import job created 2M rows with timestamp-prefixed keys, causing a single split to absorb all writes and throttling the entire instance.

Cost Considerations

Spanner's pricing is based on processing units, not queries. One node (1000 processing units) costs approximately $0.90/hour. Our 3000 PU multi-region instance costs roughly $6,500/month.

The economics work when:

• You need global strong consistency (not eventually consistent)
• Your dataset exceeds what a single-region PostgreSQL can handle
• You can't afford planned maintenance windows
• Write throughput needs to scale horizontally

Key Takeaways

1. TrueTime's commit wait adds 2-7ms, not seconds. In practice, network latency dominates over consistency overhead.
2. Stale reads are your best friend. 10-15 seconds of staleness drops cross-region latency by 90%+ for read-heavy workloads.
3. Leader placement is your most important configuration decision. Writes always route to the leader — put it where your writes originate.
4. Interleave aggressively. Parent-child data access patterns that would require joins in PostgreSQL become single-split reads in Spanner.
5. Never use sequential keys. The hotspot behavior is immediate and severe. UUIDs or hash-prefixed keys distribute load correctly.

Spanner isn't cheap, but for workloads that genuinely need global consistency, it eliminates an entire class of distributed systems problems that would otherwise require custom conflict resolution logic.