How would you stop one slow dependency or tenant from consuming all of an API's capacity?

Isolate dependency failures with bulkheads and bounded concurrency

Uses per-dependency concurrency limits, queues, pools, and workload partitions to stop one failing path from exhausting the whole API.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

I would put a concurrency limit around calls to the risky dependency and keep any waiting queue bounded. When the limit and queue are full, new work should fail quickly, be delayed durably, or use a defined fallback instead of waiting without bound. The limit protects request capacity and the downstream service. I would also partition work where necessary, for example per dependency, endpoint class, tenant tier, or background workload, so one hot path cannot starve unrelated critical traffic.

Full answer below

Practice focus

Bulkhead isolation, bounded concurrency, queue limits, fairness, resource pools, rejection behavior, and saturation telemetry.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Why this question matters
Bulkhead isolation reserves failure boundaries inside a service so one slow provider, expensive feature, or noisy tenant cannot consume every shared resource.
I would put a concurrency limit around calls to the risky dependency and keep any waiting queue bounded. When the limit and queue are full, new work should fail quickly, be delayed durably, or use a defined fallback instead of waiting without bound. The limit protects request capacity and the downstream service. I would also partition work where necessary, for example per dependency, endpoint class, tenant tier, or background workload, so one hot path cannot starve unrelated critical traffic.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
The isolation boundary should match the constrained resource. A semaphore around HTTP calls can protect provider concurrency, but it does not fix a shared database connection pool or an unbounded in-memory queue. Critical and optional traffic may need separate budgets, while fairness prevents one tenant from holding every permit. Limits should account for expected latency and downstream capacity; a queue that is much larger than useful throughput only converts immediate rejection into later timeout and memory pressure.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
In production I would measure active work, queued work, wait duration, rejection count, completion rate, timeout rate, and downstream saturation per partition. Limits can adapt through configuration, but raising them during an incident is safe only when the downstream bottleneck has spare capacity. I would test slow dependencies, hot tenants, queue overflow, cancellation while waiting, and recovery after the backlog clears. Bulkheads work with load shedding and priority rules, and the user contract explains whether rejected work can be retried, queued, or must be submitted again.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Using an unbounded queue in front of the concurrency limit.
Sharing one pool across critical and optional work.
Raising concurrency when the downstream system is already saturated.
Ignoring cancellation while callers wait for a permit.
Applying an HTTP semaphore to a database-pool problem.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

One tenant can consume every permit.
Rejected work has no stable API response.
Queue wait time is not measured.
The chosen partition does not match the actual bottleneck.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

How would you choose the initial concurrency limit?
What should happen when the queue is full?
Would you partition by tenant?
How is a bulkhead different from a circuit breaker?

Continue the section