Scaling Reads on RDS: Auto Scaling Groups, Redis, and CDN Caching in Front of Your Replicas

Databases are where scaling problems go to become expensive. At Rafeeq, our RDS cluster sits under a platform doing 16M+ orders a year, with peak-hour read traffic around 38,000 queries per second. Eighteen months ago, that load was crushing us: three db.r6g.2xlarge read replicas running at 74% CPU during dinner rush, p99 API latency at 420 ms, and the on-call channel lighting up every weekend.

The instinct in that situation is to add a fourth replica. We did the opposite: we made sure most reads never reach RDS at all. Today, two replicas idle at 28% CPU while serving more traffic than three did before.

Here's the architecture, layer by layer.

The layered read path

Each layer exists to protect the one below it. The design rule we follow: a request should be answered at the cheapest layer that can answer it correctly.

Layer 1: CDN caching (not just for images)

Everyone puts static assets behind CloudFront. The unlock for us was caching API responses. A delivery app's read traffic is dominated by data that is identical for thousands of users: restaurant lists, menus, prices, banners, service areas.

• GET /stores?zone=X: same response for every user in the zone. Cached 60 s.
• GET /stores/{id}/menu: identical for everyone. Cached 5 min, invalidated on change.
• Personalized and transactional endpoints: never cached.

Two things made this safe:

1. Cache keys designed on purpose: vary on zone and locale headers only, never on cookies or auth headers for cacheable routes.
2. Event-driven invalidation: when a partner edits a menu, we push a CloudFront invalidation for exactly those paths. Median staleness after an edit: under 10 seconds.

Result: 54% of all read requests terminate at the edge. They never touch a load balancer, an EC2 instance, or a database. Edge latency for those requests: ~30 ms globally instead of a full round trip to the region.

Layer 2: The app tier scales, the database doesn't

Behind the CDN sits an ALB and an EC2 Auto Scaling Group that grows from 6 to 22 instances across the day. Two decisions matter more than the instance type:

• Scale on RequestCountPerTarget, not CPU. Our workload is I/O-bound; CPU lags the real signal. Request count per instance tracks load directly and scales before latency degrades.
• Scheduled scaling for known peaks. Dinner rush happens at the same time every day. We pre-warm the group 20 minutes ahead of it. Reactive scaling alone always arrives a few minutes late, and those few minutes are your worst user experience of the day.

But here's the trap: an Auto Scaling Group multiplies pressure on your database. Twenty-two app servers with connection pools can open more connections than RDS comfortably holds. Two protections:

1. RDS Proxy between the ASG and the database: connection multiplexing means 22 instances share a small, stable pool.
2. The cache layers below: scaling the app tier only works if added instances don't translate 1:1 into added database queries.

Layer 3: Redis, the shield in front of the replicas

Everything that misses the CDN hits application logic, and application logic checks ElastiCache Redis before touching RDS:

• Hot entities (store profiles, menus, pricing rules): cache-aside with 60–300 s TTLs, explicit invalidation on write.
• Sessions and driver-location snapshots: Redis is the primary store; this data never belonged in Postgres.
• Computed aggregates (ratings, delivery-time estimates): recomputed by a background job every minute, served from Redis alone.

Our sustained hit rate is 89%. Combined with the CDN, that means another 31% of total reads are answered from memory in under a millisecond.

Two rules we learned the hard way:

• TTL + invalidation, always both. Invalidation events get lost; a TTL caps how stale anything can go. A missing TTL turned a pricing bug into a two-hour incident instead of a 60-second one.
• Protect against stampedes. When a hot key expires, a thousand requests race to rebuild it against RDS. Per-key mutex locking on rebuild (one request regenerates, the rest wait 50 ms) flattened those spikes completely.

What actually reaches the database

After both cache layers, RDS sees only 15% of read traffic (cache misses, personalized queries, and transactional reads):

The replicas felt it immediately. We rolled out Redis in week 2 and CDN API caching in week 5:

The results, in numbers

| Metric | Before | After | |---|---|---| | Read replicas | 3 × db.r6g.2xlarge | 2 × db.r6g.2xlarge | | Replica CPU (peak hours) | 74% | 28% | | Reads reaching RDS | 100% | 15% | | p99 API latency | 420 ms | 95 ms | | Redis hit rate | n/a | 89% | | Monthly database spend | baseline | −31% |

And the number that doesn't fit in a table: replica lag stopped being a problem. At 74% CPU our replicas would fall seconds behind the primary during spikes, causing stale-read bugs (an order marked paid that a replica didn't know about yet). At 28% CPU, lag sits under 100 ms all day.

Where I'd start if I were doing it again

1. Measure your read repetition first. One day of query logs told us ~80% of our reads were the same few hundred shapes. That number is your ceiling for how much caching can save you.
2. Redis before CDN. It's application-internal (no cache-key design, no invalidation infrastructure at the edge) and it caught our worst offenders in two weeks.
3. CDN API caching second. Bigger architectural payoff (it also absorbs regional latency and traffic spikes before they enter your VPC), but it demands discipline on cache keys and invalidation.
4. Only then decide how many replicas you need. Sizing your database for uncached traffic means paying for the cache you didn't build.

Scaling the database is sometimes the right answer. But most of the time, the cheapest replica is the query you never send.