ML-Driven Autoscaling: Predicting Traffic 30 Minutes Ahead of the Spike

Reactive autoscaling has a fundamental flaw: by the time CPU hits 70% and the scaling policy triggers, your users have already experienced degradation for 3-5 minutes while new instances boot, pass health checks, and join the load balancer. For our e-commerce platform handling 14,000 requests/second at peak, those 3-5 minutes mean $12K in lost revenue per incident. We built a predictive scaling system that forecasts traffic 30 minutes ahead and pre-provisions capacity before demand arrives. It has eliminated scaling-related latency events entirely.

Why Reactive Scaling Fails at Scale

CloudWatch-triggered autoscaling responds to metrics that already indicate stress. The timeline of a traffic surge under reactive scaling:

1. T+0: Traffic starts increasing
2. T+60s: CloudWatch alarm evaluates (1-minute datapoints)
3. T+120s: Alarm breaches threshold after 2 consecutive periods
4. T+130s: Auto Scaling launches new instances
5. T+250s: Instances pass health checks, join target group
6. T+310s: New capacity actually serving traffic

Over 5 minutes of degradation. With predictive scaling, we shift the entire process left:

1. T-30min: Model predicts incoming surge
2. T-28min: Pre-scaling action launches instances
3. T-26min: Instances boot and pass health checks
4. T-24min: Full capacity ready, waiting for traffic
5. T+0: Traffic arrives, absorbed by pre-provisioned capacity

The Forecasting Model

We use a hybrid approach: Facebook Prophet for capturing seasonality and trend, combined with a gradient-boosted model that incorporates external signals (marketing campaigns, partner traffic agreements, known events):

import numpy as np
import pandas as pd
from prophet import Prophet
from sklearn.ensemble import GradientBoostingRegressor
from datetime import datetime, timedelta

class TrafficForecaster:
    """Hybrid forecasting model combining Prophet seasonality with 
    external signal boosting for 30-minute-ahead predictions."""
    
    def __init__(self, service_name: str):
        self.service_name = service_name
        self.prophet_model = None
        self.booster = GradientBoostingRegressor(
            n_estimators=200,
            max_depth=6,
            learning_rate=0.05,
            subsample=0.8,
        )
        self.is_fitted = False
    
    def train(self, 
              traffic_history: pd.DataFrame,
              external_signals: pd.DataFrame) -> dict:
        """
        Train on 90 days of 1-minute traffic data.
        
        Args:
            traffic_history: DataFrame with columns ['ds', 'y'] 
                           (timestamp, request_count)
            external_signals: DataFrame with columns for campaigns,
                            partner_traffic, holidays, etc.
        """
        # Stage 1: Prophet captures base seasonality
        self.prophet_model = Prophet(
            changepoint_prior_scale=0.05,
            seasonality_prior_scale=10,
            yearly_seasonality=True,
            weekly_seasonality=True,
            daily_seasonality=True,
        )
        
        # Add custom seasonalities
        self.prophet_model.add_seasonality(
            name='hourly', period=1/24, fourier_order=8
        )
        
        self.prophet_model.fit(traffic_history)
        
        # Stage 2: Get Prophet's residuals
        prophet_pred = self.prophet_model.predict(traffic_history[['ds']])
        residuals = traffic_history['y'].values - prophet_pred['yhat'].values
        
        # Stage 3: Train booster on residuals + external signals
        features = self._build_features(traffic_history['ds'], external_signals)
        self.booster.fit(features, residuals)
        
        self.is_fitted = True
        
        # Return training metrics
        combined_pred = prophet_pred['yhat'].values + self.booster.predict(features)
        mape = np.mean(np.abs(
            (traffic_history['y'].values - combined_pred) / traffic_history['y'].values
        )) * 100
        
        return {'mape': mape, 'samples': len(traffic_history)}
    
    def predict(self, 
                horizon_minutes: int = 30,
                external_signals: pd.DataFrame = None) -> pd.DataFrame:
        """Generate traffic forecast for the next N minutes."""
        if not self.is_fitted:
            raise RuntimeError("Model not trained. Call train() first.")
        
        # Generate future timestamps at 1-minute intervals
        now = datetime.utcnow()
        future_dates = pd.date_range(
            start=now, 
            periods=horizon_minutes, 
            freq='1min'
        )
        future_df = pd.DataFrame({'ds': future_dates})
        
        # Prophet base prediction
        prophet_forecast = self.prophet_model.predict(future_df)
        
        # Booster correction
        features = self._build_features(future_dates, external_signals)
        booster_correction = self.booster.predict(features)
        
        # Combined forecast with uncertainty bounds
        forecast = pd.DataFrame({
            'timestamp': future_dates,
            'predicted_rps': prophet_forecast['yhat'].values + booster_correction,
            'upper_bound': prophet_forecast['yhat_upper'].values + booster_correction,
            'lower_bound': prophet_forecast['yhat_lower'].values + booster_correction,
        })
        
        # Ensure non-negative predictions
        forecast['predicted_rps'] = forecast['predicted_rps'].clip(lower=0)
        forecast['upper_bound'] = forecast['upper_bound'].clip(lower=0)
        
        return forecast
    
    def _build_features(self, timestamps, external_signals) -> np.ndarray:
        """Build feature matrix from timestamps and external signals."""
        ts = pd.DatetimeIndex(timestamps)
        features = pd.DataFrame({
            'hour': ts.hour,
            'minute': ts.minute,
            'day_of_week': ts.dayofweek,
            'is_weekend': (ts.dayofweek >= 5).astype(int),
            'hour_sin': np.sin(2 * np.pi * ts.hour / 24),
            'hour_cos': np.cos(2 * np.pi * ts.hour / 24),
        })
        
        if external_signals is not None:
            features = features.join(external_signals, how='left').fillna(0)
        
        return features.values

The Scaling Controller

The forecast feeds a controller that translates predicted RPS into required instance counts and triggers scaling actions:

import math
from dataclasses import dataclass

@dataclass
class ScalingDecision:
    current_instances: int
    target_instances: int
    action: str  # 'scale_up', 'scale_down', 'maintain'
    predicted_peak_rps: float
    confidence: float
    lead_time_minutes: int

class PredictiveScalingController:
    """Converts traffic forecasts into scaling decisions."""
    
    def __init__(self, config: dict):
        self.rps_per_instance = config['rps_per_instance']  # Measured capacity
        self.headroom_factor = config.get('headroom_factor', 1.3)
        self.min_instances = config.get('min_instances', 3)
        self.max_instances = config.get('max_instances', 200)
        self.scale_up_threshold = config.get('scale_up_threshold', 0.7)
        self.scale_down_threshold = config.get('scale_down_threshold', 0.4)
        self.cooldown_minutes = config.get('cooldown_minutes', 5)
    
    def decide(self, 
               forecast: 'pd.DataFrame',
               current_instances: int,
               last_scale_time: 'datetime') -> ScalingDecision:
        """Determine scaling action from traffic forecast."""
        
        # Use upper bound for scaling up (conservative)
        peak_rps = forecast['upper_bound'].max()
        
        # Required instances = peak RPS / capacity per instance * headroom
        required = math.ceil(
            (peak_rps / self.rps_per_instance) * self.headroom_factor
        )
        required = max(self.min_instances, min(required, self.max_instances))
        
        # Current utilization projected
        current_capacity = current_instances * self.rps_per_instance
        projected_utilization = peak_rps / current_capacity if current_capacity > 0 else 1.0
        
        # Determine action
        if projected_utilization > self.scale_up_threshold:
            action = 'scale_up'
            target = required
        elif projected_utilization < self.scale_down_threshold:
            action = 'scale_down'
            # Scale down uses predicted (not upper bound) for less aggressive reduction
            predicted_peak = forecast['predicted_rps'].max()
            target = math.ceil(
                (predicted_peak / self.rps_per_instance) * self.headroom_factor
            )
            target = max(self.min_instances, target)
        else:
            action = 'maintain'
            target = current_instances
        
        # Confidence based on forecast uncertainty
        uncertainty_range = (
            forecast['upper_bound'] - forecast['lower_bound']
        ).mean()
        confidence = max(0, 1 - (uncertainty_range / peak_rps))
        
        return ScalingDecision(
            current_instances=current_instances,
            target_instances=target,
            action=action,
            predicted_peak_rps=peak_rps,
            confidence=confidence,
            lead_time_minutes=30,
        )

Integration with AWS Auto Scaling

The controller outputs feed AWS predictive scaling policies and custom ASG adjustments:

# CloudFormation snippet for predictive scaling integration
PredictiveScalingPolicy:
  Type: AWS::AutoScaling::ScalingPolicy
  Properties:
    AutoScalingGroupName: !Ref ServiceASG
    PolicyType: PredictiveScaling
    PredictiveScalingConfiguration:
      MetricSpecifications:
        - TargetValue: 70
          PredefinedMetricPairSpecification:
            PredefinedMetricType: ASGCPUUtilization
          CustomizedCapacityMetricSpecification:
            MetricDataQueries:
              - Id: capacity
                MetricStat:
                  Metric:
                    Namespace: Custom/PredictiveScaling
                    MetricName: PredictedCapacityNeeded
                    Dimensions:
                      - Name: ServiceName
                        Value: !Ref ServiceName
                  Stat: Maximum
                  Period: 60
      Mode: ForecastAndScale
      SchedulingBufferTime: 300  # 5 minutes pre-provisioning buffer

Production Results: 6 Months of Data

Metric	Reactive Only	Predictive + Reactive	Improvement
Scaling-related P99 spikes/month	23	0	-100%
Revenue lost to scaling delays	$12K/incident	$0	-100%
Average instance utilization	34%	62%	+82%
Monthly compute cost	$89K	$71K	-20%
False positive scale-ups/month	0	4.2	+4.2
Forecast MAPE (30min horizon)	N/A	8.3%	—

The 8.3% mean absolute percentage error means our predictions are within ±8% of actual traffic 30 minutes ahead. This is sufficient for scaling decisions because we maintain a 30% headroom buffer.

The False Positive Problem

Predictive scaling over-provisions 4.2 times per month — scaling up for surges that never materialize. Each false positive costs approximately $40 in wasted compute (extra instances running for 30 minutes). At $168/month in false positive cost versus $12K/month in prevented degradation, the ROI is clear.

We accept false positives because the asymmetry is extreme: over-provisioning costs dollars, under-provisioning costs thousands in lost revenue plus customer trust.

External Signals That Improved Accuracy by 34%

The Prophet-only model achieved 12.4% MAPE. Adding external signals via the gradient booster dropped it to 8.3%:

Signal	MAPE Improvement
Marketing email sends (time + list size)	-1.8%
Partner API traffic agreements	-1.1%
Scheduled promotional events	-0.7%
Social media mention velocity	-0.3%
Mobile push notification sends	-0.2%

Marketing email sends were the single biggest predictor of traffic surges. A campaign email hitting 500K inboxes generates a predictable traffic spike 8-12 minutes later with a distinct ramp shape.

Lessons Learned

Train on failures. Our best model improvements came from analyzing the 23 monthly scaling incidents we had before the system. Each incident became a labeled training example of "traffic pattern the reactive system missed."

Never disable reactive scaling. Predictive scaling is a layer on top of reactive, not a replacement. Unpredictable events (viral social posts, DDoS, partner misbehavior) cannot be forecast. Keep reactive policies as a safety net.

Retrain weekly. Traffic patterns drift. A model trained on 90 days of data degrades noticeably after 2 weeks without retraining. We run automated retraining every Sunday at 3 AM UTC when traffic is lowest.

Separate models per service. One global model performed 40% worse than per-service models. Each service has distinct traffic patterns, and a shared model regresses to the mean.

Conclusion

Predictive autoscaling is not about replacing reactive policies — it is about buying yourself time. Thirty minutes of lead time transforms scaling from a reactive scramble into a controlled, graceful operation. The ML component is simpler than most teams expect: time-series decomposition plus a few external signals gets you 90% of the way there. The engineering challenge is the integration: feeding predictions into ASG APIs, handling model uncertainty safely, and maintaining the feedback loop that keeps forecasts accurate as traffic patterns evolve. Start with one high-value service, prove the model works, then expand.