In-Memory Caching in Rust with DashMap

Introduction

The fastest request is the one you never compute. For a reverse proxy, an API, or any high-throughput Rust service, an in-memory cache is often the single biggest performance lever available — but only if the cache itself does not become a bottleneck. This article is about DashMap, a concurrent hash map for Rust, why it beats the obvious Mutex<HashMap> approach under concurrency, and how to build a practical in-memory cache with expiry on top of it.

The problem with a single lock

The naive way to share a map across async tasks or threads is to wrap the standard library's HashMap in a lock:

use std::collections::HashMap;
use std::sync::{Arc, Mutex};

let cache: Arc<Mutex<HashMap<String, Vec<u8>>>> = Arc::new(Mutex::new(HashMap::new()));

This is correct, but it serialises access: every read and every write has to acquire the one global lock, so no matter how many CPU cores you have, only one task can touch the cache at a time. For a proxy fielding tens of thousands of concurrent requests that all want to read the cache, that single lock becomes a chokepoint — threads spend their time waiting on each other instead of doing work. An RwLock<HashMap> helps a little by allowing concurrent readers, but a single writer still blocks every reader, and write-heavy workloads gain nothing.

How DashMap solves it

DashMap presents the same familiar API as HashMap — insert, get, remove, the entry API — but internally it is sharded. The keyspace is split across many independent segments, each with its own lock. A key is hashed to pick its shard, so two operations on two different keys almost always touch different shards and proceed fully in parallel. Contention only happens when two operations genuinely collide on the same shard, which is rare with a good hash distribution.

The result is a map you can share directly, with no outer lock, and read from many tasks at once:

[dependencies]
dashmap = "6"

use dashmap::DashMap;
use std::sync::Arc;

let cache: Arc<DashMap<String, Vec<u8>>> = Arc::new(DashMap::new());

// No .lock() anywhere — just use it.
cache.insert("key".to_string(), b"value".to_vec());

if let Some(entry) = cache.get("key") {
    // `entry` derefs to the value; the shard's read lock is held
    // for as long as `entry` is alive.
    println!("{} bytes cached", entry.len());
}

Building a cache with expiry

A cache without expiry is just a memory leak. To make entries expire, store the value alongside the instant it becomes stale, and check it on read. Here is a small, practical TTL cache:

use dashmap::DashMap;
use std::sync::Arc;
use std::time::{Duration, Instant};

#[derive(Clone)]
struct Entry {
    value: Arc<Vec<u8>>,
    expires_at: Instant,
}

#[derive(Clone)]
pub struct Cache {
    map: Arc<DashMap<String, Entry>>,
    ttl: Duration,
}

impl Cache {
    pub fn new(ttl: Duration) -> Self {
        Cache { map: Arc::new(DashMap::new()), ttl }
    }

    pub fn get(&self, key: &str) -> Option<Arc<Vec<u8>>> {
        let entry = self.map.get(key)?;
        if entry.expires_at > Instant::now() {
            Some(entry.value.clone())
        } else {
            // Lazily evict the stale entry. Drop the read guard first
            // to avoid holding a lock on the shard we are about to write.
            drop(entry);
            self.map.remove(key);
            None
        }
    }

    pub fn insert(&self, key: String, value: Vec<u8>) {
        self.map.insert(key, Entry {
            value: Arc::new(value),
            expires_at: Instant::now() + self.ttl,
        });
    }
}

Wrapping the value in an Arc means get hands back a cheap clone of a shared pointer rather than copying the whole payload — important when cached responses are large and read often.

Bounding memory: eviction

Lazy expiry removes entries only when they are next requested, so keys that are written once and never read again linger forever. For a long-running service you want an upper bound on memory. Two common strategies:

• Background sweeper. A periodic Tokio task iterates the map and removes expired entries. Because DashMap supports retain, this is a one-liner run off the hot path: map.retain(|_, e| e.expires_at > now).
• Size-bounded / LRU. If you need a hard cap on entry count, reach for a crate built for it — for example moka, which offers an async, concurrent cache with size-based and time-based eviction and TinyLFU admission. DashMap is the right primitive when you want direct control; a dedicated cache crate is better when you want eviction policy handed to you.

// Off-the-hot-path sweep, e.g. every 30 seconds.
pub fn spawn_sweeper(cache: Cache) {
    tokio::spawn(async move {
        let mut tick = tokio::time::interval(Duration::from_secs(30));
        loop {
            tick.tick().await;
            let now = Instant::now();
            cache.map.retain(|_, e| e.expires_at > now);
        }
    });
}

The one gotcha: don't deadlock a shard

The most common DashMap mistake is holding a reference into the map while trying to modify the same shard. Because each shard is guarded by a lock, code like this can deadlock:

// BAD: the read guard from get() may still hold the shard lock
// that insert() needs, if both keys land on the same shard.
if let Some(v) = map.get("a") {
    map.insert("b".into(), v.clone()); // risk of self-deadlock
}

The fix is simple and shown in the cache above: end the borrow first (let the guard drop, or clone out the value you need) before you write back. Keep the window in which you hold a guard as short as possible, and never call another mutating map method while one is alive.

Why this matters for a proxy

A caching reverse proxy lives or dies on this data structure. Every request computes a cache key, looks it up, and — on a hit — returns the stored response without ever touching the origin. With a global lock that lookup would serialise all traffic; with a sharded concurrent map it stays O(1) and genuinely parallel across cores, which is exactly why ARC2 Proxy keeps its hot-path cache in a DashMap. The lookup cost stays flat as concurrency climbs, and combined with Rust's lack of a garbage collector, the latency stays predictable even under heavy load.

Conclusion

For shared, concurrent, read-heavy state in Rust, DashMap gives you the ergonomics of HashMap without the single-lock bottleneck of Mutex<HashMap>. Add a stored expiry instant for TTL, an Arc around large values to make reads cheap, and a background retain sweep to bound memory, and you have a fast, correct in-memory cache. Mind the same-shard borrow-then-write pitfall, and reach for a dedicated cache crate when you need a strict eviction policy — otherwise DashMap is an excellent, direct foundation.