Rate Limiting in Axum and Tower

Introduction

Rate limiting is how a service protects itself from being overwhelmed — whether by an abusive client, a runaway script, a scraper, or an outright attack. It caps how many requests any given caller can make in a window of time, returning a polite 429 Too Many Requests when the cap is exceeded. In this article we build rate limiting into an Axum 0.7 service, first from scratch with a Tower middleware so you understand the mechanics, then with the tower_governor crate for production use.

Why rate limit at all?

• Protect the origin. A single client hammering an expensive endpoint can exhaust your database or CPU. A limit keeps one bad actor from degrading service for everyone else.
• Blunt abuse and attacks. Credential-stuffing, scraping, and application-layer floods all rely on high request volume. Rate limiting caps the damage each source can do.
• Fair sharing. On a multi-tenant service, per-client limits stop one heavy user from starving the rest.
• Cost control. Fewer wasted requests means less compute and bandwidth spent on traffic you did not want.

Algorithms in brief

Algorithm	How it works	Trade-off
Fixed window	Count requests per calendar window (e.g. per minute); reset at the boundary.	Simple, but allows a burst of up to 2× the limit straddling the boundary.
Sliding window	Weight the previous window's count into the current one.	Smoother, slightly more state and computation.
Token bucket	Tokens refill at a steady rate; each request spends one; empty bucket = reject.	Allows controlled bursts up to the bucket size. The usual default.

Token bucket is the most common choice because it permits short, legitimate bursts while still enforcing a long-run rate.

Identifying the caller

Before you can limit a client you have to identify it. The usual key is the client IP address. If your service is the true edge, that is the connection's peer address via ConnectInfo (remember to opt in with into_make_service_with_connect_info). If you sit behind another proxy or a load balancer, the peer address is that device, and you must instead read the real client from the X-Forwarded-For header — and only trust it when the request comes from a known upstream, or an attacker can simply spoof the header to dodge your limits.

A rate-limiting middleware from scratch

Axum middleware written with axum::middleware::from_fn is the most readable way to start. We keep a per-IP token bucket in a DashMap so lookups stay concurrent, refill based on elapsed time, and reject when the bucket is empty:

use std::net::IpAddr;
use std::sync::Arc;
use std::time::Instant;

use axum::{
    extract::{ConnectInfo, State},
    http::StatusCode,
    middleware::Next,
    response::{IntoResponse, Response},
    extract::Request,
};
use dashmap::DashMap;

#[derive(Clone)]
struct Bucket {
    tokens: f64,
    last_refill: Instant,
}

#[derive(Clone)]
pub struct RateLimiter {
    buckets: Arc<DashMap<IpAddr, Bucket>>,
    capacity: f64,     // max tokens (burst size)
    refill_per_sec: f64, // sustained requests/second
}

impl RateLimiter {
    pub fn new(capacity: f64, refill_per_sec: f64) -> Self {
        RateLimiter {
            buckets: Arc::new(DashMap::new()),
            capacity,
            refill_per_sec,
        }
    }

    /// Returns true if the request is allowed.
    fn check(&self, ip: IpAddr) -> bool {
        let now = Instant::now();
        let mut bucket = self.buckets.entry(ip).or_insert(Bucket {
            tokens: self.capacity,
            last_refill: now,
        });

        // Refill based on time elapsed since we last saw this IP.
        let elapsed = now.duration_since(bucket.last_refill).as_secs_f64();
        bucket.tokens = (bucket.tokens + elapsed * self.refill_per_sec).min(self.capacity);
        bucket.last_refill = now;

        if bucket.tokens >= 1.0 {
            bucket.tokens -= 1.0;
            true
        } else {
            false
        }
    }
}

pub async fn rate_limit(
    State(limiter): State<RateLimiter>,
    ConnectInfo(addr): ConnectInfo<std::net::SocketAddr>,
    req: Request,
    next: Next,
) -> Response {
    if limiter.check(addr.ip()) {
        next.run(req).await
    } else {
        (
            StatusCode::TOO_MANY_REQUESTS,
            [("Retry-After", "1")],
            "rate limit exceeded",
        )
            .into_response()
    }
}

Wire it into the router as a layer, carrying the limiter as state:

use axum::{routing::get, Router};

let limiter = RateLimiter::new(20.0, 10.0); // burst 20, sustain 10 req/s

let app = Router::new()
    .route("/", get(|| async { "hello" }))
    .layer(axum::middleware::from_fn_with_state(limiter.clone(), rate_limit))
    .with_state(limiter);

Note the Retry-After header on the 429 response: it tells well-behaved clients how long to wait before retrying, which is both good manners and part of the HTTP spec. Don't forget a background sweep (or size bound) on the bucket map, or it will accumulate an entry per unique IP forever — the same eviction concern as any in-memory store.

Using tower_governor in production

For real deployments you usually don't want to hand-maintain bucket state and edge cases. The tower_governor crate wraps the well-tested governor rate-limiting library as a Tower layer, with per-IP keying, burst support, and correct 429 responses out of the box:

[dependencies]
tower_governor = "0.4"

use std::sync::Arc;
use axum::{routing::get, Router};
use tower_governor::{governor::GovernorConfigBuilder, GovernorLayer};

let config = Arc::new(
    GovernorConfigBuilder::default()
        .per_second(10)  // sustained rate
        .burst_size(20)  // allowed burst
        .finish()
        .unwrap(),
);

let app = Router::new()
    .route("/", get(|| async { "hello" }))
    .layer(GovernorLayer { config });

tower_governor defaults to keying on the peer IP and can be configured to extract the client from X-Forwarded-For when you run behind a trusted proxy. It handles the token-bucket accounting, cleanup, and Retry-After headers for you.

Where to enforce limits

Rate limiting inside your application protects that application, but by the time a request reaches your handler it has already consumed a connection, TLS, and routing work. Enforcing limits at the edge — in the reverse proxy in front of your origin — rejects abusive traffic earlier and cheaper, and does it across your whole fleet rather than one instance at a time. This is why ARC2 Proxy offers per-domain rate limiting as a built-in feature: you set a limit once and the edge absorbs the excess before it ever touches your servers, which also makes it a core part of application-layer DDoS defence.

Conclusion

Rate limiting is a small investment with an outsized payoff in resilience. Understand the token-bucket model, key on a client identity you can actually trust, and return a proper 429 with Retry-After. Build it by hand with a Tower middleware and a concurrent map when you want full control, or reach for tower_governor when you want a battle-tested layer — and remember that the earlier in the request path you enforce the limit, the more it protects.