API Rate limiting
Inspired by Design a Distributed Rate Limiter w/ a Ex-Meta Staff Engineer: System Design Breakdown
Django rate limiter? In memory per server. You don't get a global picture tho
Or separate service but extra latency
Most common is to have it with the api gateway
Rate limiting within the app allows complex domain rules like allowing premium users
Gateway you can just do by numbers. But at least you don't contact the service at all if you don't need to, which protects the service resource more
How to identify users? Authenticated users or just ip adresse. Maybe authed users can do more than anonymous
Common algorithms: n requests per time window. Fixed window = key value user 1pm: counter. It's not sliding so you can do a lot in the edges
Sliding window: keep previous minute and current minute. If it remains 30% of time in the current minute, use 30% of the previous minute in the calculation
Token bucket is most popular
Remove token when using a request. There's a refill rate. Bucket capacity + steady refill removes any approximation simply. What about cold starts?
Beware of rate conditions
On each request, you get the time it was last refilled and refill based on theoretical refill rate
Can the Django rate limiter use a shared storage?
Fail open Vs fail close.
When it's down, either allow or disallow all requests
To handle failovers better, have an async replica for each shard. Redis clusters handle that for you