Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations

Meta Engineering describes a production system that harvests pockets of idle inference capacity across a globally constrained fleet and spends them on better ads recommendations. The core problem is familiar to anyone running large inference: the fleet can be short overall while regions, accelerator types, or time-of-day leave meaningful capacity stranded.

Two elasticity levers do the spending. Rank More Ads (RMA) expands how many candidates deeper ranking stages evaluate when both CPU and accelerator surplus exist. Serve Better Models (SBM) routes a larger share of requests to a more powerful model when accelerator-only surplus shows up. Physical and logical isolation, PID-style latency feedback, value-aware routing, and millisecond load shedding are meant to ensure that when opportunistic capacity vanishes, optional work shrinks instead of crushing baseline serving. Meta says the approach drives advertiser value and capacity efficiency without new capacity orders.

Our take: this is the grown-up version of “use spare GPUs.” Most shops either leave idle silicon on the floor or let optional jobs collide with latency SLOs. Meta’s contribution is treating idle capacity as a product surface with two different resource shapes and a fail-closed contract—lose the free capacity, lose only the incremental ranking quality, never the baseline path.

RMA and SBM are complementary for a reason. Ranking more candidates eats CPU and accelerators; upgrading the model mostly eats accelerators. A fleet that strands one resource or the other can still monetize the leftover. Value-aware routing and shedding matter here: when capacity snaps back, Meta tries to drop the lowest-value optional work first rather than randomly thinning traffic. That is the difference between opportunistic inference and a polite denial-of-service against yourself.

Useful for advertisers and for people who see more relevant ads if the quality claims hold. The open-eyed part is concentration of power. A system that safely converts stranded capacity into recommendation lift is a moat. Competitors without Meta’s fleet diversity and control loops cannot copy the upside as easily, and the same machinery that improves ad match also intensifies the attention economy’s optimization loop. Meta is careful not to publish concrete lift percentages in this post, so we treat “significant advertiser value” as a company claim, not a measured public result.

I’m watching the next levers Meta names—precomputation elasticity, cross-region inference, and automated allocation between RMA and SBM—because those decide whether opportunistic capacity stays an ads trick or becomes a general pattern for any elastic AI workload.

The post sits in Meta’s broader ads ML engineering series, which has covered multi-stage ranking architectures, foundation-model training efficiency, and kernel scheduling for ads services. Authors include engineers across ranking, capacity, and ML applications teams.