Skip to main content

Deep Dive into Redis VI: Redis vs ZooKeeper vs etcd

· 4 min read

A Deep, Conceptual Comparison for Distributed Systems​


1. High-Level Positioning​

SystemCore IdentityPrimary Purpose
RedisIn-memory data platformFast data access, caching, counters, queues
ZooKeeperDistributed coordination serviceStrong consistency, coordination, metadata
etcdDistributed key-value storeCluster configuration & service discovery

One Sentence Summary​

  • Redis: “Give me data now.”
  • ZooKeeper: “Coordinate distributed systems safely.”
  • etcd: “Store and watch cluster state consistently.”

2. Fundamental Design Philosophy​

Redis Philosophy​

Speed > Everything
Memory-first
Eventual consistency (cluster)
Application-driven logic

Redis optimizes for latency and throughput.


ZooKeeper Philosophy​

Correctness > Performance
Strong consistency
Coordination first
Centralized control plane

ZooKeeper optimizes for correct distributed behavior.


etcd Philosophy​

Simplicity + Consistency
Raft-based consensus
Cloud-native control plane
Observability-friendly

etcd optimizes for operational clarity and reliability.


3. Core Architecture Comparison​

Redis Architecture (Cluster Mode)​

Client
|
┌───────────┴────────┐
│ Redis Cluster │
└───────────┬────────┘
┌───────────┴───────────┐
┌────┴────┐ ┌────┴────┐ ┌────┴────┐
│ Master1 │ │ Master2 │ │ Master3 │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
Replicas Replicas Replicas

Key points:

  • Hash slots (16,384)
  • Sharding for scale
  • Replication for availability
  • Not consensus-based

ZooKeeper Architecture​

Client
|
┌────────┴──────────┐
│ ZooKeeper Quorum │
└────────┬──────────┘
┌──────────┼──────────┐
┌────┴────┐┌────┴────┐┌────┴────┐
│ Leader ││Follower ││Follower │
└─────────┘└─────────┘└─────────┘

Key points:

  • Single leader
  • Zab protocol
  • Linearizable writes
  • Hierarchical namespace

etcd Architecture​

Client
|
┌────────┴────────┐
│ etcd Cluster │
└────────┬────────┘
┌──────────┼──────────┐
┌────┴────┐┌────┴────┐┌────┴────┐
│ Leader ││ Follower││ Follower│
└─────────┘└─────────┘└─────────┘

Key points:

  • Raft consensus
  • Strong consistency
  • Flat key-value space
  • Watch-based design

4. Consistency Model (Critical Difference)​

FeatureRedisZooKeeperetcd
ConsistencyEventualStrongStrong
Consensus❌ZabRaft
Split-brainPossiblePreventedPrevented
Linearizable writes❌✅✅

Visual Consistency Comparison​

Redis:
Write → Master → Async replicas
(possible stale reads)

ZooKeeper / etcd:
Write → Leader → Majority ACK → Commit
(always consistent)

5. Data Model Comparison​

Redis Data Model​

Key → Value
├── String
├── Hash
├── List
├── Set
├── ZSet
└── Stream

Optimized for application data structures.


ZooKeeper Data Model​

/
├── app
│ ├── leader
│ ├── config
│ └── locks
  • Tree-like (filesystem style)
  • Znodes (small metadata)
  • Ephemeral nodes

etcd Data Model​

/services/app1/instances/1
/services/app1/instances/2
/config/db/timeout
  • Flat KV
  • Prefix-based grouping
  • First-class WATCH support

6. Failure Handling​

Redis Failure Model​

Master crashes
↓
Replica promoted
↓
Possible data loss window
  • Fast
  • Not transactional
  • Best-effort durability

ZooKeeper Failure Model​

Leader crashes
↓
Election
↓
New leader with consistent state
  • No data loss (committed)
  • Slower failover
  • Strong correctness

etcd Failure Model​

Leader crashes
↓
Raft election
↓
New leader
  • Guaranteed state safety
  • Designed for automation

7. Persistence & Storage​

AspectRedisZooKeeperetcd
Primary storageRAMDiskDisk
PersistenceRDB / AOFWAL + snapshotWAL + snapshot
Data volumeLargeSmallMedium
Restart recoveryFastModerateModerate

Redis Persistence​

RAM
│
├─ RDB (snapshot)
└─ AOF (append)

ZooKeeper / etcd Persistence​

Client writes
↓
WAL (log)
↓
Snapshot

8. Typical Use Cases (MOST IMPORTANT)​


Redis – When You SHOULD Use It​

✔ Caching
✔ Rate limiting
✔ Distributed counters
✔ Session storage
✔ Queues / Streams
✔ Real-time analytics
✔ Leaderboards

Diagram:

User → App → Redis → Response (sub-ms)

ZooKeeper – When You SHOULD Use It​

✔ Leader election
✔ Distributed locks
✔ Metadata management
✔ Kafka / HBase coordination
✔ Strong ordering guarantees

Diagram:

Services → ZooKeeper → Coordination

etcd – When You SHOULD Use It​

✔ Kubernetes state
✔ Service discovery
✔ Feature flags
✔ Distributed config
✔ Cloud-native control planes

Diagram:

Nodes → etcd → Watch → React

9. Watch / Notification Capability​

FeatureRedisZooKeeperetcd
WatchPub/SubWatcherNative Watch
ReliabilityBest-effortStrongStrong
Replay❌❌✅

etcd Watch Model​

Client sets watch
↓
Key changes
↓
Guaranteed event delivery

10. Scalability Characteristics​

AspectRedisZooKeeperetcd
Horizontal scaleExcellentPoorLimited
Data sizeVery largeVery smallSmall-medium
ClientsMillionsThousandsThousands

Why ZooKeeper Does NOT Scale Like Redis​

All writes → Leader
All coordination → Leader

This is intentional.


11. Operational Complexity​

AspectRedisZooKeeperetcd
SetupEasyHardMedium
DebuggingMediumHardMedium
TuningMany knobsMany knobsFew knobs
Cloud-nativePartialNoYes

12. Security & Access Control​

FeatureRedisZooKeeperetcd
AuthACL / AUTHACLTLS + RBAC
EncryptionTLSTLSTLS
Multi-tenancyLimitedLimitedStrong

13. What NOT to Do (Very Important)​

❌ Do NOT use Redis for:​

  • Distributed locks requiring strict correctness
  • Leader election with safety guarantees
  • Source of truth metadata

❌ Do NOT use ZooKeeper for:​

  • Large datasets
  • High-QPS application reads
  • Caching

❌ Do NOT use etcd for:​

  • Hot data paths
  • Large object storage
  • High-frequency counters

14. Decision Matrix (Quick Guide)​

Need speed? ───────────▶ Redis
Need correctness? ─────▶ ZooKeeper
Need cloud control? ───▶ etcd

15. Mental Model Summary​

Redis = Data Plane
ZooKeeper = Coordination Plane
etcd = Control Plane

They complement, not replace, each other.


16. Final Takeaway​

  • Redis is fast but not authoritative
  • ZooKeeper is authoritative but heavy
  • etcd is authoritative and cloud-native

The best systems often use all three, each for what it does best.