Distributed Locking with Redis to Orchestrate Access for a Shared Resource

When multiple servers compete to update the same record, send the same notification, or run the same scheduled job — race conditions happen. Distributed locks with Redis let you enforce mutual exclusion across any number of processes or machines without a dedicated coordination service.

The Problem: Race Conditions in Distributed Systems

In a single-process app, a mutex handles concurrency. In a distributed system with multiple servers, threads, or workers, you need a lock that all participants can see — stored in a shared location like Redis.

Common scenarios requiring distributed locks:

Preventing duplicate email sends when multiple workers pick up the same job
Ensuring only one server runs a scheduled cron job at a time
Serializing access to a third-party API with strict rate limits
Coordinating inventory deductions to prevent overselling

The SET NX EX Pattern

The foundation of Redis locking is a single atomic command: SET key value NX EX seconds. NX means "only set if Not eXists", and EX sets an expiry — preventing deadlocks if the lock holder crashes.

# Acquire lock (returns OK on success, nil on failure)
SET lock:inventory:item42 "worker-uuid-abc123" NX EX 30

# Check if lock is held
GET lock:inventory:item42   # "worker-uuid-abc123"

# Release lock (must verify owner before deleting)
# Use a Lua script for atomic check-and-delete:
EVAL "
  if redis.call('get', KEYS[1]) == ARGV[1] then
    return redis.call('del', KEYS[1])
  else
    return 0
  end
" 1 lock:inventory:item42 "worker-uuid-abc123"

The unique value (UUID) is critical — it ensures only the lock owner can release it, preventing a slow worker from accidentally releasing a lock that was re-acquired after its TTL expired.

Python Implementation

import redis
import uuid
import time

r = redis.Redis(host='localhost', port=6379, decode_responses=True)

RELEASE_SCRIPT = """
if redis.call('get', KEYS[1]) == ARGV[1] then
    return redis.call('del', KEYS[1])
else
    return 0
end
"""

def acquire_lock(resource, ttl_seconds=30, retry=3, retry_delay=0.1):
    lock_key = f"lock:{resource}"
    lock_value = str(uuid.uuid4())

    for attempt in range(retry):
        acquired = r.set(lock_key, lock_value, nx=True, ex=ttl_seconds)
        if acquired:
            return lock_key, lock_value
        time.sleep(retry_delay * (attempt + 1))  # backoff

    return None, None

def release_lock(lock_key, lock_value):
    r.eval(RELEASE_SCRIPT, 1, lock_key, lock_value)

# Usage
lock_key, lock_value = acquire_lock("inventory:item42", ttl_seconds=10)
if lock_key:
    try:
        # Critical section — only one worker runs this at a time
        update_inventory(42)
    finally:
        release_lock(lock_key, lock_value)
else:
    print("Could not acquire lock — another worker is processing")

The Redlock Algorithm

For stronger guarantees against Redis node failures, Redis creator Salvatore Sanfilippo proposed Redlock: acquire the lock on a majority of N independent Redis instances (typically 5). The lock is valid only if acquired on at least ⌊N/2⌋+1 instances within a time window.

Tolerates up to ⌊N/2⌋ Redis node failures
Uses the redlock-py or redlock (Node.js) libraries
Overkill for most applications — single-node locking is sufficient when Redis has persistence and replication

Lock TTL Best Practices

TTL must exceed max expected work time — if your critical section can take up to 5 seconds, set TTL to 15+ seconds
Keep critical sections short — long-held locks create contention; do minimal work inside the lock
Never extend locks without re-acquiring — a lock that expires while held is safer than a runaway lock
Log failed acquisitions — contention spikes indicate a bottleneck in your design

Key Takeaways

SET NX EX atomically acquires a lock with automatic expiry — no deadlocks
Store a unique value as the lock owner to safely release only your own lock
Use Lua scripts for atomic check-and-release operations
Retry with backoff when acquisition fails — don't spin-wait
Redlock provides stronger guarantees in multi-node setups but adds complexity