When the War Reached Our Cloud: Evacuating an AWS Region in 6 Hours
The attack that took down AWS Bahrain forced an emergency migration: our DR plan under real fire, and how Kiro moved 63 services in 6 hours, not 3 weeks.

Every disaster-recovery plan I've ever written had a quiet assumption baked into it: the disaster would be technical. A bad deploy, a failed disk, maybe a whole availability zone going dark. I never wrote the sentence "a regional war disables our primary cloud region". And then this summer, that's exactly what happened.
When the conflict between the US and Iran escalated across the Gulf, the AWS Bahrain data center was hit, and me-south-1 (the region closest to our users in Qatar, the region running nearly everything Rafeeq) degraded from "our home" to "unreliable" within the hour. This is the story of the six hours that followed, and what I'd tell any CTO in a region where geopolitics is part of your threat model.
Hour zero: the call you can't prepare for emotionally
You can prepare for it operationally, though. And barely, we had.
Our disaster-recovery posture before the crisis:
- Backups: automated cross-region copies to
ap-south-1(Mumbai). RDS snapshots every 6 hours, continuous point-in-time recovery, S3 replication on critical buckets. We had deliberately chosen a DR region outside the Gulf. RPO on paper: 15 minutes for databases, near-zero for object storage. - A warm skeleton, not a warm standby: our Terraform could theoretically build everything in another region, but it had never been fully exercised. VPCs and IAM existed in
ap-south-1; almost no compute did. - One cross-region RDS read replica for the orders database, the one system we had decided we could never lose. That decision saved us.
Here's the geography of the problem, and the escape route:
The decision: don't wait, evacuate
For the first 45 minutes we did what everyone does: watched the AWS status page, retried failures, hoped. Capacity in me-south-1 was intermittent: some instances healthy, new capacity impossible to launch, latency spiking 20×. Hope is not a strategy, and in a war, "degraded" doesn't trend back toward "fine."
Forty-five minutes in, we made the call: full evacuation to ap-south-1. Not a partial failover, not "wait and see": move everything, permanently, starting with the order path. And crucially: get out of the Gulf entirely, because in a regional conflict, the region is the blast radius.
Our estimate for that migration, made calmly a year earlier during DR planning: about three weeks. We had 63 production services, hundreds of Terraform modules with me-south-1 assumptions baked in, region-specific configs scattered across 40+ repositories, and a database tier serving 34,000 queries per second.
We did it in six hours. Honestly, I still find that number hard to believe, and a big part of it was that we pointed Kiro at the mechanical work.
Where Kiro changed the math
Kiro had been our engineers' agentic IDE for months, mostly for feature work. During the evacuation it became something else: the tireless engineer who does the boring, error-prone 80% while humans make the calls.
What we actually delegated:
- Region-proofing the infrastructure code. Kiro swept our Terraform and found every hardcoded region, AZ name, AMI ID, and endpoint (across all repositories) and produced the
ap-south-1equivalents as reviewable pull requests. A human approved every change; nobody had to find every change. That sweep alone was budgeted at four days in the old plan. It took under an hour. - Config archaeology. Connection strings, SDK clients with implicit regions, queue ARNs, KMS key references buried in services nobody had touched in a year. We gave Kiro the migration spec; it worked through the repos as a task list and flagged the judgment calls it couldn't make alone.
- DMS migration configs. For the databases without cross-region replicas, Kiro drafted the AWS DMS task definitions, table mappings, and validation queries. Engineers reviewed, corrected two, and ran them.
- Runbooks under pressure. Every cutover got a generated, service-specific runbook: steps, health checks, rollback procedure. Four hours into an overnight sprint, tired people don't improvise well. Checklists saved us more than once.
Of the 63 services, Kiro generated the region-ready code and configuration for 54. The other nine (the payment integrations, anything touching money or compliance) we did by hand, on purpose.
The database story: promote, drain, verify
Data is where migrations die, so this is where we were most conservative.
- Hour 1: promote the orders replica. The cross-region read replica in
ap-south-1became the new primary. Because it was continuously replicating before the attack, we lost essentially nothing. Effective RPO on the order path: under 30 seconds. - Everything else via snapshot + DMS. Secondary databases restored from the cross-region snapshots, then DMS replayed changes captured while
me-south-1limped along. Two low-traffic services accepted a 4-hour data gap; we chose availability and reconciled later from event logs. - Drain, don't flip. We shifted traffic in stages (10%, 25%, 50%, 100%), watching database load on both sides. The new primary had to earn each increment.
The scariest moment wasn't the promotion. It was hour three, both regions at half load, when a replication lag alarm fired. It peaked at 1.8 seconds and recovered. Total customer-visible downtime across the entire evacuation: 41 minutes, most of it in two planned cutover windows announced in the app beforehand.
The scoreboard
| Metric | Plan (peacetime estimate) | Reality | |---|---|---| | Full region migration | ~3 weeks | 6 hours | | Services moved | 63 | 63 | | Customer-visible downtime | "hours, maybe days" | 41 minutes | | Data loss (order path) | RPO 15 min | < 30 s | | Data loss (worst service) | RPO 6 h | 4 h, reconciled | | Peak replica lag during shift | n/a | 1.8 s | | Latency cost for Doha users | n/a | +24 ms round-trip |
What I'd tell every CTO in this region
- Geopolitics belongs in your threat model. Not as a slide, but as an architecture requirement. If your DR region shares a body of water with your primary, it shares the same risks. Ours being 2,700 km away in Mumbai is why it was still standing when we needed it.
- A backup you haven't restored is a hope, not a backup. Our snapshots worked because we'd tested restores quarterly. The Terraform "skeleton region" half-worked because we'd never fully exercised it. That gap is where our three-week estimate came from.
- Pay for one real cross-region replica on the system you can't lose. Ours cost a few hundred dollars a month for years. It repaid that in the first hour of the crisis.
- Agentic tooling changes DR math. The reason region evacuation plans take weeks is thousands of small, mechanical, region-specific edits. That is exactly the work an agent like Kiro does relentlessly and a human verifies quickly. Our new DR plan officially assumes it.
- People first, always. Our team executed this while some of them worried about family across the region. It was one intense overnight sprint: rotations were short, meals were ordered, and nobody was allowed to be a hero alone at 2 a.m. The plan matters less than the people running it.
We're staying in ap-south-1. And the new DR plan has a sentence the old one lacked: "Assume the region is gone and will not come back." I hope we never use it again. We rehearse it quarterly anyway.
Recommended reading
RDS Proxy in Production: What the Docs Don't Tell You
A year of RDS Proxy under 16M orders: multiplexing that works, the pinning trap that silently disables it, and the failover win nobody markets.
Real-Time Order Tracking with AWS AppSync: How We Cut Update Latency by 94%
How we replaced REST polling with AWS AppSync subscriptions at Rafeeq: live order tracking and partner alerts in under a second instead of 45.
Scaling Reads on RDS: Auto Scaling Groups, Redis, and CDN Caching in Front of Your Replicas
How we took our RDS read replicas from 74% CPU to 28% while traffic grew, by letting CloudFront and Redis absorb 85% of reads before they ever reach the database.

Comments
No comments yet. Be the first to share your thoughts.