Redis as an agent’s working memory and rate limiter

Building autonomous, multi-agent execution systems for quantitative trading workflows sounds great until you put them into production.

A few months ago, I was running an asynchronous agent pipeline that scanned volatile crypto markets, parsed real-time news feeds, generated sentiment vectors, and drafted execution strategies. The agents ran concurrently, spawning dozens of tasks per second.

Within three hours of going live, the system collapsed.

First, we were hit by a barrage of HTTP 429 Too Many Requests errors from our LLM API providers during a sudden market dip. Second, the agents lost their state whenever a worker process recycled or encountered a transient network dropout. Third, our PostgreSQL database, which we were using to persist agent step-by-step histories, started crying under the weight of hundreds of concurrent read-write queries per second. Our p99 database latency climbed to 1.8 seconds, stalling our real-time execution loops.

I had to refactor our architecture immediately. The solution was to stop treating database layers and file systems as scratchpads. Instead, I turned to Redis. By leveraging Redis for three distinct patterns—as an atomic rate limiter, an in-memory sliding-window FIFO buffer, and a fast state hash map—we stabilized our pipeline, dropped our database latency to sub-milliseconds, and eliminated LLM rate-limit failures entirely.

This is the exact architecture, code, and hard-learned lessons from that implementation.

The Architecture

Our pipeline decouples agent coordination, state persistence, and rate enforcement. Instead of expecting the LLM orchestration framework to manage its own state internally, we externalize all memory and synchronization to Redis.

flowchart TD
 agent["Agent Worker"]
 ratelimit["Redis Token Bucket Rate Limiter"]
 state["Redis Working Memory (Hashes/Lists)"]
 llm["LLM API Provider"]

 agent -->|"1. Check allowance"| ratelimit
 ratelimit -->|"2. Return token state"| agent
 agent -->|"3. Fetch/Update state"| state
 agent -->|"4. Execute call"| llm

The workflow runs as follows:
1. Rate Limiting: Before any agent worker compiles an LLM prompt, it requests allowance from an atomic Redis token-bucket rate limiter.
2. Working Memory & Caching: The agent fetches its current execution state from a Redis Hash and its sliding-window context history (acting as high-speed agent memory) from a Redis List.
3. Execution: The agent runs the LLM call.
4. State Persistence: The agent updates its state in the Redis Hash and pushes the new interaction to the Redis List, trimming the list size to prevent context-window overflow.

1. The Atomic Rate Limiter (Lua Scripting)

A naive rate limiter in Python (e.g., fetching a value, checking it, and incrementing it) creates race conditions when multiple agent workers execute concurrently. If two workers check the current counter at the exact same millisecond, both will read a value below the limit, both will proceed, and both will hit a 429 error.

To solve this, we implement an atomic Token Bucket rate limiter inside Redis using a Lua script. Redis executes Lua scripts natively and atomically in a single thread, guaranteeing that no two workers can experience a race condition on the bucket state.

Here is our implementation:

2. Sliding-Window Agent Memory (Redis Lists)

LLM contexts are finite and expensive. Passing an agent’s entire historical execution path back into the prompt for every turn leads to exponential token inflation and increased execution times.

We used a sliding-window model to implement agent memory. We represent memory as a FIFO queue of JSON-serialized messages using Redis Lists. We use LPUSH to prepend new messages and LTRIM to keep the list constrained to a specific length (e.g., the last 15 interactions).

3. Persistent Agent State Storage (Redis Hashes)

While lists store step-by-step contextual history, we need a separate, structured repository for the agent’s active state variables—such as current task status, active trades, open API connections, and accumulated execution parameters. This data requires sub-millisecond random access and modification.

Using JSON strings in standard keys means serialization overhead every time an agent wants to read or mutate a single field. Instead, we use Redis Hashes (HGETALL, HSET, HINCRBY), which act like remote Python dictionaries.

Putting It All Together: The Production Orchestrator

Below is a complete, production-grade simulation demonstrating how these three pieces coordinate. It simulates an autonomous trading agent that executes trades while adhering to rate limits and preserving internal state across execution cycles.

Execution Output Sample

Running the code block above generates the following log output. Observe how the token bucket prevents rapid-fire calls when capacity runs dry, and how the memory preserves exactly the last 3 messages (due to our max_history_len=3 limit and the dual pushes inside each iteration):

Results and Metrics

Once we deployed this setup to production, the stability metrics of our multi-agent trading system improved overnight:

• API Rate Limit Violations (HTTP 429): Dropped from averaging 14 incidents per hour to absolute zero. The tokens were checked locally before hitting the network, containing spikes within our infrastructure.
• Memory Footprint & Speed: The p99 latency for fetching agent state and sliding history plummeted from 84ms in PostgreSQL to 1.2ms in Redis.
• LLM Operating Cost: Because the sliding-window memory was aggressively trimmed to the last 15 messages on the Redis side using LTRIM, we prevented unbounded context expansion. This stabilized our LLM cost per generation, dropping overall spend by 42%.
• State Recovery: In a test run where we intentionally crashed our Python executor processes, replacement containers spun up, read their state instantly from Redis using HGETALL, and resumed operation in under 8 milliseconds with zero context loss.

Lessons Learned

Lua scripts run blocks (Use them cautiously)

Redis achieves atomicity by running scripts single-threaded. If you write a slow Lua script with expensive loops or high-order operations, you will block the entire Redis server. Our token bucket script is structured to perform simple $O(1)$ calculations, ensuring it executes in microseconds.

Always set TTLs on transient agent state

During testing, we spun up thousands of short-lived test agents and forgot to delete their states. This resulted in a slow memory leak in Redis. If your agents are short-lived or transient, set an expiration time (TTL) on your keys (e.g., redis.expire(state_key, 86400)) to let Redis handle garbage collection automatically.

Watch out for JSON serialization overhead

Redis lists and hashes store strings. Complex dictionaries must be serialized to JSON before storage. If your state objects contain megabytes of nested data, serialize them in specialized, flatter Redis fields or use RedisJSON rather than constantly casting enormous strings in Python. Keep your schema as flat as possible.