Why your website goes down when marketing sends an email

Corporate mail scanners prefetch every link the moment a campaign lands, and thousands of HEAD requests reach your origin in seconds. How to confirm it from the logs and fix it in layers, from the web server to the WAF.

The pattern is specific enough to diagnose from the description. Marketing sends a campaign. Within a minute or two the website stalls. Dashboards are green, every host reports healthy, there are no 5xx errors, and a few minutes later it clears on its own. Then it happens again with the next send. If that is your site, this is almost certainly what is going on, and here is how to fix it in layers.

What is actually hitting you

Corporate mail systems run link-protection scanners that prefetch every URL in an email the moment it lands. A blast to a large list produces thousands of HEAD requests to the same handful of links within seconds. They spoof a browser user agent, they come from hundreds of cloud IP addresses, and they are not malicious. They are Outlook and Google keeping your recipients safe.

The damage happens when those links point at pages your application renders in full. A checkout or event page marked no-store reaches the origin on every prefetch. If the framework runs sessions, route-model binding, and the page's data loading before it notices the request is a HEAD, every prefetch costs a worker for the full page's work. The worker pool fills, the whole site stalls, and if the health check also runs through the application, the load balancer starts killing healthy instances.

The first incident we worked through this way was about 14,000 prefetches on one checkout URL in seven minutes. Zero 5xx errors. The only trace was the load balancer's 460 code, which means a client gave up before the origin ever answered.

How to confirm it

  1. Pull the load balancer logs for the last campaign window and count requests by method. A spike of HEAD requests on a few URLs, timed to the send, is the signature.
  2. Count distinct client addresses on those requests. Hundreds or more means a scanner fleet, and it means per-IP rate limiting will never catch it.
  3. Look for 460 or client-timeout codes rather than 5xx. That is what saturation looks like from the edge.

Fix it at the cheapest layer first

The principle that drives every fix below: the cheapest place to answer a request is the one closest to the client. A HEAD answered at the web server costs microseconds. Answered at the application it costs a worker.

  • Answer HEAD before the application exists. A web-server rule that returns a 200 to HEAD on the affected paths before the request touches the application runtime. Under a load test at twice the incident's volume, 92,000 requests, this short-circuited 100 percent of them at about 2 milliseconds with zero reaching the application. Fixing the application's own HEAD handling first, which we tried, was not enough, because framework middleware still ran before the controller.
  • Split the health probes. Liveness should be a static web-server response so a saturated instance is never mistaken for a dead one. Readiness should treat "busy" as healthy so load cannot eject the instances that are still serving.
  • Block at the edge by method and path. Real buyers never send HEAD. A web application firewall rule scoped to method plus path has zero false positives and returns a 200 that looks like the origin's answer. Do not key it on user agent; the scanner fleet rotates them. Do not key it on IP; the fleet spans hundreds of addresses a day, and a per-IP limit will eventually block your own office.
  • Throttle the GETs that must reach the application. The same scanners also send GETs, which real users need. A throttle keyed on user agent with a 429 and a Retry-After header, set well above the busiest human-plausible client, handles those. A 30-day analysis of 3,684 distinct user agents put the busiest legitimate one at 111 requests a minute; the throttle went in at 200.

The scanner will move, so fix the general case

After the checkout pages were protected, the next campaign pointed the scanner at event pages, then at the footer links in every email: unsubscribe and subscription pages at 461 and 1,225 times their baseline. Path-specific rules were catching under 2 percent of the damage by then. One HEAD-by-path block rule covering the whole application replaced the whack-a-mole.

Two problems that hide behind the first one

Synchronized cache expiry. If your CDN caches pages with a fixed TTL and no stale-while-revalidate, then at expiry a single blocking revalidation hits an origin that may be saturated, and every viewer behind it times out. Add stale-while-revalidate to every cached page so expiry serves stale and revalidates in the background.

Request cost, not request count. One outage arrived at only 600 requests a minute. A broad crawl hit the expensive uncached pages, feeds at 3.6 seconds, landing pages at 6, search at 4, and enough of them in flight exhausted the pool. Autoscaling never fired because it watched request count. Cache the feeds at the edge, return cached 404s for dead URLs, and set a timeout cascade so the application reclaims a stuck worker before the web server gives up, and the web server before the CDN does. Ours was 20, 25, and 30 seconds. With that in place a storm that pins the pool clears in about two minutes instead of fifteen.

Alarm on the right signal

Every existing alarm on that site watched signals that stay green during a storm. The alarm that works is a composite on the client-gave-up count plus p99 latency. Each half fires alone briefly under normal load; together they paged zero times in the first week and would have caught every incident above.

What it looked like afterwards

The next storm was four times bigger than the one that caused a 15-minute stall, and it did nothing: p99 held at 1.13 seconds with seven dropped requests. A real campaign at 423 requests a second at the edge from 15,834 addresses reached the origin at 87 a second with zero 5xx errors, and the origin never scaled. Between 300,000 and 450,000 scanner requests a day now stop at the edge. Nobody rebuilt the application, and nobody had to. The defences were a few web-server lines, a handful of firewall rules, cache headers, and three timeouts in the right order.

The full story

All five rounds, including the ones that were not enough, are in the email blast case study. The same lessons applied to a new site before launch are in the headless CMS migration case study.


Does your site fall over when marketing hits send?

We'll pull your load balancer and CDN logs for the last campaign, show you exactly which requests pinned the origin, and tell you which layer should be answering them.

Schedule a call