whereq.com 开源了:地图、爬虫,以及它读到的所有数据
一枚图钉,其实只是一句很小的话
站在多伦多的一条街上,看着眼前这个门牌号,地图只会告诉你一句话:这里有一枚图钉。一家咖啡馆,一间医院,一所学校,也许还有一个停车场。
但一个地方本身是一大堆话,而图钉是其中最不重要的那句。
这个地址属于哪个学区 catchment?走路能到图书馆吗,是哪一家?街角加油站今天收 1.42 还是 1.79?这栋楼落在 ward 里、neighbourhood 里,还是两者都算?上周这条街上出过什么事?
站在多伦多的一条街上,看着眼前这个门牌号,地图只会告诉你一句话:这里有一枚图钉。一家咖啡馆,一间医院,一所学校,也许还有一个停车场。
但一个地方本身是一大堆话,而图钉是其中最不重要的那句。
这个地址属于哪个学区 catchment?走路能到图书馆吗,是哪一家?街角加油站今天收 1.42 还是 1.79?这栋楼落在 ward 里、neighbourhood 里,还是两者都算?上周这条街上出过什么事?
Stand on a street in Toronto and look at the address in front of you. A map gives you one claim about it: there is a pin here. A coffee shop. A hospital. A school. Maybe a parking garage.
But the place itself is a pile of claims, and the pin is the least interesting one of them.
Which school catchment is that address in? Can I walk to a library, and which one? Is the fuel station on the corner charging 1.79? Is that building inside a ward, or inside a neighbourhood, or in a ward and a neighbourhood? Did anything happen on this street last week?
The WhereQ Universe is not a single product — it is a living ecosystem of independent applications that share one identity layer, one set of engineering conventions, and an increasingly open set of APIs. This post walks through how the pieces fit together: why authentication lives in a single place, how the apps expose data to customers, and where LLM agents and MCP fit into the picture.
Hermes
vs🦞 OpenClaw从所有维度进行全面深度对比 — 架构、安全、记忆、技能、平台、成本等
更新于 2026 年 4 月
OpenClaw 定义了个人 AI 智能体品类。最初名为 Clawdbot(2025 年 11 月),因商标问题先后改名为 Moltbot 和 OpenClaw。创始人 Peter Steinberger 于 2026 年 2 月加入 OpenAI,项目转由非营利基金会社区治理。
Hermes Agent 两个月后由 Nous Research 发布 — 该实验室是 Hermes、Nomos、Psyche 模型家族的创造者。OpenClaw 优化的是连接尽可能多的平台和工具,而 Hermes 优化的是深度:让智能体通过使用真正学习和进化。
| 维度 | 🦞 OpenClaw | |
|---|---|---|
| 核心 | AIAgent 循环(Python,约 9,200 行) | Gateway 控制平面(Node.js/TS) |
| 入口 | hermes(CLI)+ hermes gateway | openclaw CLI + Gateway 守护进程 |
| 状态存储 | SQLite + FTS5 + 扁平文件 | JSON 配置 + 工作区文件 |
| 协议 | 直接 LLM API 调用 | WebSocket 控制平面 |
| 插件模型 | Python 工具 + MCP 服务器 | Node.js 技能 + ClawHub 注册表 |
| 技术栈 | Python | Node.js / TypeScript |
这是最大的差异化特性。Hermes 拥有其他智能体框架无法复制的闭环学习循环。OpenClaw 的技能是静态的 — 你写了它们,除非手动更新,否则永远不变。
| 能力 | 🦞 OpenClaw | |
|---|---|---|
| 技能创建 | 复杂任务后自主创建(5+ 工具调用) | 仅手动(人工编写 SKILL.md) |
| 技能改进 | 使用过程中自我改进 | 无(静态文件) |
| 用户建模 | Honcho 辩证建模,跨会话积累 | SOUL.md 静态身份文件 |
| 周期性提示 | 提示 Agent 反思并持久化有用信息 | 无等效机制 |
| 跨会话召回 | FTS5 搜索 + LLM 摘要 | 按助手持久记忆 |
| 实测改进 | 重复任务速度提升 40% | 无复利效果 |
| 层级 | 🦞 OpenClaw | |
|---|---|---|
| 持久事实 | MEMORY.md(约 2,200 字符,会话开始冻结) | 按助手存储记忆 |
| 用户画像 | USER.md(偏好、风格) | SOUL.md(身份和人格) |
| 程序性知识 | 技能库(自动创建、可搜索) | 技能(人工编写、静态) |
| 情景记忆 | SQLite + FTS5 全文搜索,无限量 | 按助手的会话持久化 |
| 摘要 | LLM 驱动的历史会话摘要 | 无等效功能 |
| 外部提供商 | 7 个可插拔提供商(Hindsight、PLUR 等) | 无可插拔记忆系统 |
| 上下文压缩 | 有损摘要引擎(可插拔) | 通过 Gateway 管理上下文 |
| 提示词缓存 | Anthropic 前缀缓存,冻结快照 | 特定提供商缓存 |
| 团队共享 | PLUR 记忆碎片跨实例共享 | 按助手隔离(内置) |
Hermes 将记忆视为分层系统:始终在上下文中的事实、按需加载的程序性技能、可搜索的情景历史。OpenClaw 的记忆更简单但对团队隔离有效 — 每个助手有自己独立的存储,防止多代理设置中的交叉污染。
| 维度 | 🦞 OpenClaw | |
|---|---|---|
| 技能总数 | 70+ 内置,社区增长中 | ClawHub 13,700+(报称 44K) |
| 技能格式 | agentskills.io 开放标准(Markdown) | 自定义 SKILL.md 格式(Markdown + 脚本) |
| 技能来源 | Agent 自动创建 + 社区 + Skills Hub | 人工编写 + ClawHub 市场 |
| 加载方式 | 渐进式披露(L0/L1/L2) | 启动时加载 |
| 自我改进 | 技能在使用中演化 | 写完就是静态的 |
| 供应链风险 | Tirith 预扫描 + 第三方需 --force | 高 — 发现 1,467+ 恶意技能(ClawHavoc) |
| 审查 | 扫描判定默认阻止危险技能 | VirusTotal 扫描(危机后添加),早期审查极少 |
OpenClaw 的生态系统在绝对数量上远超 Hermes。ClawHub 有数千个社区技能,覆盖从 Gmail 到 Kubernetes 的各种场景。但这种规模带来了严重的安全代价 — ClawHavoc 供应链攻击和 1,467+ 确认的恶意技能代表严重的信任问题。Hermes 用广度换安全:更少的技能,但有预执行扫描和更安全的默认姿态。
| 平台 | 🦞 OpenClaw(25+) | |
|---|---|---|
| 核心平台 | Telegram、Discord、Slack、WhatsApp、Signal | Telegram、Discord、Slack、WhatsApp、Signal |
| 企业通讯 | 企业微信 (WeCom)、钉钉、飞书 | 企业微信、飞书、Microsoft Teams、Google Chat |
| 中国生态 | 企业微信、微信(HermesClaw 桥接)、钉钉、飞书 | 微信(官方腾讯插件)、飞书 |
| 日韩东南亚 | 无 | LINE、Zalo、Zalo Personal |
| 开源/极客 | Matrix、Mattermost | Matrix、Mattermost、IRC、Nostr、Tlon、Twitch |
| 苹果生态 | BlueBubbles (iMessage) | BlueBubbles + iMessage (legacy) + macOS App |
| 物联网 | Home Assistant | Synology Chat、Nextcloud Talk |
| 邮件/短信 | Email (IMAP/SMTP)、SMS (Twilio) | WebChat(内置) |
| 原生应用 | 无 | macOS 菜单栏、iOS App、Android App |
| 跨平台连续性 | 语义上下文恢复(核心优势) | 基于 Gateway 的会话持久化 |
OpenClaw 支持 LINE、Google Chat、Microsoft Teams、IRC、Twitch、Nostr、Zalo 以及原生 iOS/Android/macOS 配套应用 — Hermes 目前均不支持。Hermes 的优势在于跨平台对话连续性(语义上下文恢复)和更深的单平台集成(Discord 语音频道、Telegram Webhook 模式、企业微信 WebSocket)。
| 维度 | 🦞 OpenClaw | |
|---|---|---|
| 提供商数量 | 18+ 内置提供商 | 主要提供商(Claude、GPT、Gemini、Groq、Mistral、xAI) |
| OpenRouter | 200+ 模型统一计费 | 支持 |
| 本地模型 | Ollama、vLLM、SGLang、llama.cpp | Ollama + 自定义端点 |
| 中国提供商 | MiniMax、Kimi/月之暗面、智谱 GLM、DashScope、小米 MiMo | 有限支持 |
| MiniMax | 一流合作伙伴(M2.7 直接集成) | 通过 OpenRouter |
| 切换 | hermes model — 零代码更改 | 编辑配置文件 + 重启 |
| 故障转移 | 错误时自动切换备用模型(会话内) | 无自动故障转移 |
| 每代理路由 | 单模型 + 备用 | 不同代理可用不同模型 |
| 最低上下文 | 要求 64K tokens | 无强制最低要求 |
| 辅助 LLM | 独立模型用于视觉、摘要 | 使用主模型 |
这是对比中最触目惊心的部分。OpenClaw 经历了 2026 年最严重的 AI 安全危机之一。
| 安全维度 | 🦞 OpenClaw | |
|---|---|---|
| CVE 数量 | 0 个智能体特定 CVE | 138+ 已追踪(7 个严重,49 个高危) |
| 最严重 CVE | 无 | CVE-2026-32922(CVSS 9.9)— 权限提升 |
| 供应链 | Tirith 预执行扫描,第三方需 --force | ClawHavoc:1,467+ 恶意技能,AMOS 窃取器分发 |
| 默认绑定 | 安全默认值 | 早期版本绑定到 0.0.0.0(所有网络接口) |
| 认证默认 | 需要允许列表/配对 | 部分版本无认证运行 |
| 容器加固 | 只读根 FS、丢弃 caps、命名空间隔离(默认) | 可选 Docker 沙箱 |
| 文件系统保护 | 自动检查点 + 回滚 | 基于工作区的隔离 |
| 命令扫描 | Tirith 预执行扫描器 | Exec 审批系统(CVE 后添加) |
| 记忆注入 | 条目扫描威胁模式 | 无等效扫描 |
| 遥测 | 零 — 架构级设计 | 零 — 本地优先设计 |
| 政府警告 | 无 | 中国、比利时、韩国发布限制令 |
重要背景:Hermes 更年轻,攻击面更小(64K vs 345K Stars)。它还没有经受同等级别的审查。但其安全优先的架构决策 — Docker 加固作为默认而非选项、第三方技能强制预扫描、记忆注入扫描 — 代表了一种从根本上更注重安全的设计。
| 维度 | 🦞 OpenClaw | |
|---|---|---|
| 终端后端 | 6 种:Local、Docker、SSH、Daytona、Singularity、Modal | Local、Docker |
| Serverless | Daytona + Modal(空闲近零成本) | 无原生 Serverless |
| 安装 | 一行 curl 脚本 | npm install -g openclaw(需 Node.js 22+) |
| OS 支持 | Linux、macOS、WSL2、Termux (Android) | Linux、macOS、Windows(原生) |
| Windows 原生 | 不支持(仅 WSL2) | 原生支持 |
| HPC 集群 | Singularity/Apptainer 支持 | 无 HPC 支持 |
| 托管服务 | 社区提供商涌现中 | DigitalOcean 一键部署($24/月)、OpenClaw VPS、OneClaw |
| 服务安装 | hermes gateway install(systemd/launchd) | Systemd / LaunchAgent |
| 配套应用 | 仅 CLI(ACP 用于编辑器) | macOS 菜单栏、iOS App、Android App |
| 功能 | 🦞 OpenClaw | |
|---|---|---|
| CLI/TUI | 完整 TUI,多行编辑、自动补全、流式传输 | CLI + 引导向导 |
| 斜杠命令 | CLI 和网关共享 | 跨频道共享 |
| MCP 支持 | 原生(stdio + HTTP、重连、采样) | 原生 MCP 支持 |
| IDE 集成 | ACP 服务器(VS Code、Zed、JetBrains) | 无原生 IDE 集成 |
| 浏览器控制 | 网页搜索 + 浏览器自动化 + 视觉 | CDP 浏览器控制(Chrome/Chromium) |
| 语音模式 | CLI 麦克风、TTS 回复、Discord 语音频道 | Voice Wake、推说话、Talk Mode 浮层 |
| 编程 API | from run_agent import AIAgent | WebSocket API + REST 端点 |
| 配置文件 | hermes profile create — 隔离环境 | 通过 Gateway 多代理路由 |
| 诊断 | hermes doctor | openclaw doctor |
| 组成 | 🦞 OpenClaw | |
|---|---|---|
| 框架 | $0(MIT) | $0(MIT) |
| 托管 | $5/月 VPS 或约 $0 Serverless(Modal/Daytona) | $5/月 VPS 或 $24/月 托管(DigitalOcean) |
| LLM API | $15–80/月 典型 | $15–80/月 典型 |
| 每次调用开销 | 约 73% 固定开销(工具定义) | 类似开销比例 |
| 空闲成本 | Modal/Daytona 休眠约 $0 | 全额 VPS 费用,即使空闲 |
| 典型总计 | $10–25/月(个人) | $20–40/月(个人,无托管) |
Hermes 的 Serverless 后端(Modal、Daytona)对于不需要 24/7 运行智能体的用户来说具有显著成本优势。如果你的智能体每天运行 8 小时,你只为 8 小时付费,而不是 24 小时。
| 指标 | 🦞 OpenClaw | |
|---|---|---|
| GitHub Stars | 64K+(2 个月) | 345K+(5 个月) |
| 贡献者 | 142+ | 数千 |
| 技能市场 | Skills Hub(增长中) | ClawHub(13,700+ 技能) |
| 企业支持 | 无(仅社区) | NVIDIA NemoClaw 合作 |
| YouTube 内容 | 增长中 | 大量教程和评测 |
| 治理 | Nous Research(活跃实验室) | 非营利基金会(创始人已加入 OpenAI) |
| 增长轨迹 | v0.8.0 后单日 6,400+ Stars | 初始病毒式传播 72 小时 60K+ Stars |
OpenClaw 的社区是 AI 智能体领域最大的。大量的教程、第三方托管提供商、技能和集成形成了网络效应。但 Hermes 的增长速度表明它可能在一年内挑战 OpenClaw — 尤其是随着开发者因安全问题而迁移。
| 功能 | 🦞 OpenClaw | |
|---|---|---|
| 多代理 | 子代理并行工作 | Gateway 多代理路由(按工作区隔离) |
| 团队聊天 | 共享 Discord/Slack 频道 | Gateway 路由到按频道/账户隔离的代理 |
| 数据隔离 | 按 Profile 隔离 | 按助手隔离存储(内置) |
| NVIDIA 支持 | 无 | NemoClaw 合作 |
| 托管服务 | 社区提供商 | DigitalOcean、OpenClaw VPS、多个托管方 |
| 配套应用 | 无 | macOS、iOS、Android 配套应用 |
| 可视化工作区 | 无 | Live Canvas(代理驱动的可视化工作区) |
| 能力 | 🦞 OpenClaw | |
|---|---|---|
| 批量轨迹 | 内置并行轨迹生成 | 无等效功能 |
| RL 训练 | Atropos RL 环境 | 无 RL 基础设施 |
| 轨迹导出 | 压缩 + 导出用于微调 | 无训练数据导出 |
| 可配置工作者 | 批量大小、工具集分布 | 无批处理 |
| 构建者 | Nous Research(模型训练实验室) | 社区(创始人已加入 OpenAI) |
Hermes 由模型训练实验室构建,从一开始就设计为研究基础设施 — 不仅仅是面向用户的产品。OpenClaw 没有等效的研究能力。
OpenClaw → Hermes:内置迁移工具
hermes claw migrate 可导入你的人格(SOUL.md)、记忆、技能(自动转换为 agentskills.io 格式)、消息平台配置、API 密钥和 TTS 资源。支持交互式预览和 dry-run。技能冲突通过 skip/overwrite/rename 选项处理。可以在过渡期间同时运行两个代理。
无内置迁移工具。需要手动映射记忆文件、技能格式转换和频道重新配置。自我进化技能系统在 OpenClaw 中没有等效物 — 你会失去复利自动化价值。预估工作量:12–21 小时。
选择 Hermes 当你...
由于 Hermes 使用 Python 而 OpenClaw 使用 Node.js,它们在运行时不冲突。你可以将不同频道路由到不同代理,用 OpenClaw 覆盖广泛的消息平台,同时用 Hermes 处理受益于其学习循环的深度工作任务。hermes claw migrate 工具支持你准备好时的渐进式迁移。
Hermes
vs🦞 OpenClawA comprehensive comparison from every perspective — architecture, security, memory, skills, platforms, cost, and more.
Updated April 2026
OpenClaw defined the personal AI agent category. Originally called Clawdbot (Nov 2025), it was renamed Moltbot and then OpenClaw after trademark concerns. Creator Peter Steinberger joined OpenAI in February 2026, and the project moved to community governance under a non-profit foundation.
Hermes Agent arrived two months later from Nous Research — the lab behind the Hermes, Nomos, and Psyche model families. Where OpenClaw optimized for connecting to as many platforms and tools as possible, Hermes optimized for depth: making the agent genuinely learn and improve through use.
| Aspect | 🦞 OpenClaw | |
|---|---|---|
| Core | AIAgent loop (Python, ~9,200 lines) | Gateway control plane (Node.js/TS) |
| Entry points | hermes (CLI) + hermes gateway | openclaw CLI + Gateway daemon |
| State storage | SQLite + FTS5 + flat files | JSON config + workspace files |
| Protocol | Direct LLM API calls | WebSocket control plane |
| Plugin model | Python tools + MCP servers | Node.js skills + ClawHub registry |
| Codebase | Python (primary) | Node.js / TypeScript |
This is the single biggest differentiator. Hermes has a closed learning loop that no other agent framework replicates. OpenClaw skills are static — you write them, they stay the same forever unless you manually update them.
| Capability | 🦞 OpenClaw | |
|---|---|---|
| Skill creation | Autonomous after complex tasks (5+ tool calls) | Manual only (human writes SKILL.md) |
| Skill improvement | Self-improving during use | None (static files) |
| User modeling | Honcho dialectic modeling across sessions | SOUL.md static identity file |
| Periodic nudges | Agent prompted to reflect and persist | No equivalent |
| Cross-session recall | FTS5 search + LLM summarization | Persistent memory per assistant |
| Reported improvement | 40% speed improvement on repeated tasks | No compounding effect |
| Layer | 🦞 OpenClaw | |
|---|---|---|
| Persistent facts | MEMORY.md (~2,200 chars, frozen at session start) | Per-assistant memory storage |
| User profile | USER.md (preferences, style) | SOUL.md (identity & personality) |
| Procedural | Skills library (auto-created, searchable) | Skills (human-authored, static) |
| Episodic | SQLite + FTS5 full-text search, unlimited | Session persistence per assistant |
| Summarization | LLM-powered summarization of past sessions | No equivalent |
| External providers | 7 pluggable providers (Hindsight, PLUR, etc.) | No pluggable memory system |
| Context compression | Lossy summarization engine (pluggable) | Context management via Gateway |
| Prompt caching | Anthropic prefix caching, frozen snapshot | Provider-specific caching |
| Team sharing | PLUR engrams across instances | Per-assistant isolation (built-in) |
Hermes treats memory as a layered system: always-in-context facts, on-demand procedural skills, and searchable episodic history. OpenClaw's memory is simpler but effective for team isolation — each assistant has its own separate storage, preventing cross-contamination in multi-agent setups.
| Aspect | 🦞 OpenClaw | |
|---|---|---|
| Total skills | 70+ bundled, growing community | 13,700+ on ClawHub (44K reported) |
| Skill format | agentskills.io open standard (Markdown) | Custom SKILL.md format (Markdown + scripts) |
| Skill source | Auto-created by agent + community + Skills Hub | Human-authored + ClawHub marketplace |
| Loading | Progressive disclosure (L0/L1/L2) | Loaded at startup |
| Self-improvement | Skills evolve during use | Static once written |
| Supply chain risk | Tirith pre-scan + --force for third-party | High — 1,467+ malicious skills found (ClawHavoc) |
| Vetting | Scan verdicts block dangerous skills by default | VirusTotal scanning (added post-crisis), minimal early vetting |
OpenClaw's ecosystem is far larger in raw numbers. ClawHub has thousands of community skills covering everything from Gmail to Kubernetes. But that scale came with severe security costs — the ClawHavoc supply chain attack and 1,467+ confirmed malicious skills represent a serious trust problem. Hermes trades breadth for safety: fewer skills, but with pre-execution scanning and a safer default posture.
OpenClaw has LINE, Google Chat, Microsoft Teams, IRC, Twitch, Nostr, Zalo, and native iOS/Android/macOS companion apps — none of which Hermes currently supports. Hermes's advantage is cross-platform conversation continuity with semantic context restore, and deeper per-platform integration (Discord voice channels, Telegram webhook mode, WeCom WebSocket).
| Aspect | 🦞 OpenClaw | |
|---|---|---|
| Provider count | 18+ built-in providers | Major providers (Claude, GPT, Gemini, Groq, Mistral, xAI) |
| OpenRouter | 200+ models via unified billing | Supported |
| Local models | Ollama, vLLM, SGLang, llama.cpp | Ollama + custom endpoints |
| China providers | MiniMax, Kimi/Moonshot, z.ai/GLM, DashScope, Xiaomi | Limited |
| MiniMax | First-class partner (M2.7 direct integration) | Via OpenRouter |
| Switching | hermes model — zero code change | Config file edit + restart |
| Fallback model | Auto-fallback mid-session on error | No automatic fallback |
| Per-agent routing | Single model + fallback | Different models per agent |
| Minimum context | 64K tokens required | No enforced minimum |
| Auxiliary LLM | Separate model for vision, summarization | Uses primary model |
This is where the comparison becomes most stark. OpenClaw has faced one of the most significant AI security crises of 2026.
| Security Aspect | 🦞 OpenClaw | |
|---|---|---|
| CVE count | 0 agent-specific CVEs | 138+ tracked (7 critical, 49 high) |
| Worst CVE | None | CVE-2026-32922 (CVSS 9.9) — privilege escalation |
| Supply chain | Tirith pre-execution scan, --force required | ClawHavoc: 1,467+ malicious skills, AMOS stealer distribution |
| Default binding | Safe defaults | Early versions bound to 0.0.0.0 (all interfaces) |
| Auth default | Allowlist/pairing required | Some versions ran without auth |
| Container hardening | Read-only root FS, dropped caps, namespace isolation (defaults) | Optional Docker sandboxing |
| Filesystem protection | Automatic checkpoints + rollback | Workspace-based isolation |
| Command scanning | Tirith pre-execution scanner | Exec approval system (added post-CVE) |
| Memory injection | Entries scanned for threat patterns | No equivalent scanning |
| Telemetry | Zero — architectural design | Zero — local-first design |
| Government warnings | None | China, Belgium, South Korea issued restrictions |
Important context: Hermes is younger and has a smaller attack surface (64K stars vs 345K). It hasn't faced the same level of scrutiny. However, its safer-by-default architectural decisions — Docker hardening as defaults rather than options, mandatory pre-scan of third-party skills, memory injection scanning — represent a fundamentally more security-conscious design.
| Aspect | 🦞 OpenClaw | |
|---|---|---|
| Terminal backends | 6: Local, Docker, SSH, Daytona, Singularity, Modal | Local, Docker |
| Serverless | Daytona + Modal (near-zero idle cost) | No native serverless |
| Installation | One-line curl script | npm install -g openclaw (Node.js 22+) |
| OS support | Linux, macOS, WSL2, Termux (Android) | Linux, macOS, Windows (native) |
| Windows native | Not supported (WSL2 only) | Supported natively |
| HPC clusters | Singularity/Apptainer support | No HPC support |
| Managed hosting | Community providers emerging | DigitalOcean 1-Click ($24/mo), OpenClaw VPS, OneClaw |
| Service install | hermes gateway install (systemd/launchd) | Systemd / LaunchAgent |
| Companion apps | CLI only (ACP for editors) | macOS menu bar, iOS app, Android app |
| Feature | 🦞 OpenClaw | |
|---|---|---|
| CLI/TUI | Full TUI with multiline, autocomplete, streaming | CLI with onboarding wizard |
| Slash commands | Shared across CLI and gateway | Shared across channels |
| MCP support | Native (stdio + HTTP, reconnection, sampling) | Native MCP support |
| IDE integration | ACP server (VS Code, Zed, JetBrains) | No native IDE integration |
| Browser control | Web search + browser automation + vision | CDP browser control (Chrome/Chromium) |
| Voice mode | CLI mic, TTS replies, Discord voice channels | Voice Wake, Push-to-Talk, Talk Mode overlay |
| Programmatic API | from run_agent import AIAgent | WebSocket API + REST endpoints |
| Profiles | hermes profile create — isolated environments | Multi-agent routing via Gateway |
| Diagnostics | hermes doctor | openclaw doctor |
| Component | 🦞 OpenClaw | |
|---|---|---|
| Framework | $0 (MIT) | $0 (MIT) |
| Hosting | $5/mo VPS or ~$0 serverless (Modal/Daytona) | $5/mo VPS or $24/mo managed (DigitalOcean) |
| LLM API | $15–80/mo typical | $15–80/mo typical |
| Overhead per call | ~73% fixed overhead (tool definitions) | Similar overhead profile |
| Idle cost | ~$0 with Modal/Daytona hibernation | Full VPS cost even when idle |
| Typical total | $10–25/mo (personal) | $20–40/mo (personal, no managed) |
Hermes's serverless backends (Modal, Daytona) give it a significant cost edge for users who don't need their agent running 24/7. If your agent runs 8 hours/day, you pay for 8 hours instead of 24.
| Metric | 🦞 OpenClaw | |
|---|---|---|
| GitHub stars | 64K+ (2 months) | 345K+ (5 months) |
| Contributors | 142+ | Thousands |
| Skills marketplace | Skills Hub (growing) | ClawHub (13,700+ skills) |
| Enterprise support | None (community only) | NVIDIA NemoClaw partnership |
| YouTube content | Growing | Extensive tutorials and reviews |
| Governance | Nous Research (active lab) | Non-profit foundation (founder left for OpenAI) |
| Growth trajectory | 6,400+ stars in single day (post v0.8.0) | 60K+ stars in 72 hours (initial viral wave) |
OpenClaw's community is the largest in the AI agent space. The sheer volume of tutorials, third-party hosting providers, skills, and integrations creates a network effect. However, Hermes is growing at a pace that suggests it could challenge OpenClaw within a year — especially as developers migrate due to security concerns.
| Feature | 🦞 OpenClaw | |
|---|---|---|
| Multi-agent | Subagents for parallel work | Multi-agent routing via Gateway (per-workspace isolation) |
| Team chat | Shared Discord/Slack channels | Gateway routes to isolated agents per channel/account |
| Data isolation | Per-profile isolation | Per-assistant isolated storage (built-in) |
| NVIDIA support | No | NemoClaw partnership |
| Managed hosting | Community providers | DigitalOcean, OpenClaw VPS, multiple managed providers |
| Companion apps | None | macOS, iOS, Android companion apps |
| Visual workspace | No | Live Canvas (agent-driven visual workspace) |
| Capability | 🦞 OpenClaw | |
|---|---|---|
| Batch trajectories | Built-in parallel trajectory generation | No equivalent |
| RL training | Atropos RL environments | No RL infrastructure |
| Trajectory export | Compression + export for fine-tuning | No training data export |
| Configurable workers | Batch sizes, toolset distributions | No batch processing |
| Built by | Nous Research (model training lab) | Community (founder joined OpenAI) |
Hermes was built by a model training lab. It's designed from the ground up as research infrastructure — not just a user-facing product. OpenClaw has no equivalent research capabilities.
OpenClaw → Hermes: Built-in Migration Tool
hermes claw migrate imports your persona (SOUL.md), memory, skills (auto-converted to agentskills.io), messaging configs, API keys, and TTS assets. Interactive preview with dry-run support. Skill conflicts are handled with skip/overwrite/rename options. You can run both agents side by side during transition.
No built-in migration tool. Requires manual mapping of memory files, skill format conversion, and channel reconfiguration. The self-improving skills system has no OpenClaw equivalent — you lose compounding automation value. Estimated effort: 12–21 hours.
Choose Hermes When You...
Since Hermes uses Python and OpenClaw uses Node.js, they don't conflict at runtime. You can route different channels to different agents, use OpenClaw for broad messaging coverage while Hermes handles deep-work tasks that benefit from its learning loop. The hermes claw migrate tool supports gradual migration when you're ready.
| System | Core Identity | Primary Purpose |
|---|---|---|
| Redis | In-memory data platform | Fast data access, caching, counters, queues |
| ZooKeeper | Distributed coordination service | Strong consistency, coordination, metadata |
| etcd | Distributed key-value store | Cluster configuration & service discovery |
Speed > Everything
Memory-first
Eventual consistency (cluster)
Application-driven logic
Redis optimizes for latency and throughput.
Correctness > Performance
Strong consistency
Coordination first
Centralized control plane
ZooKeeper optimizes for correct distributed behavior.
Simplicity + Consistency
Raft-based consensus
Cloud-native control plane
Observability-friendly
etcd optimizes for operational clarity and reliability.
Client
|
┌───────────┴────────┐
│ Redis Cluster │
└───────────┬────────┘
┌───────────┴───────────┐
┌────┴────┐ ┌────┴────┐ ┌────┴────┐
│ Master1 │ │ Master2 │ │ Master3 │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
Replicas Replicas Replicas
Key points:
Client
|
┌────────┴──────────┐
│ ZooKeeper Quorum │
└────────┬──────────┘
┌──────────┼──────────┐
┌────┴────┐┌────┴────┐┌────┴────┐
│ Leader ││Follower ││Follower │
└─────────┘└─────────┘└─────────┘
Key points:
Client
|
┌────────┴────────┐
│ etcd Cluster │
└────────┬────────┘
┌──────────┼──────────┐
┌────┴────┐┌────┴────┐┌────┴────┐
│ Leader ││ Follower││ Follower│
└─────────┘└─────────┘└─────────┘
Key points:
| Feature | Redis | ZooKeeper | etcd |
|---|---|---|---|
| Consistency | Eventual | Strong | Strong |
| Consensus | ❌ | Zab | Raft |
| Split-brain | Possible | Prevented | Prevented |
| Linearizable writes | ❌ | ✅ | ✅ |
Redis:
Write → Master → Async replicas
(possible stale reads)
ZooKeeper / etcd:
Write → Leader → Majority ACK → Commit
(always consistent)
Key → Value
├── String
├── Hash
├── List
├── Set
├── ZSet
└── Stream
Optimized for application data structures.
/
├── app
│ ├── leader
│ ├── config
│ └── locks
/services/app1/instances/1
/services/app1/instances/2
/config/db/timeout
Master crashes
↓
Replica promoted
↓
Possible data loss window
Leader crashes
↓
Election
↓
New leader with consistent state
Leader crashes
↓
Raft election
↓
New leader
| Aspect | Redis | ZooKeeper | etcd |
|---|---|---|---|
| Primary storage | RAM | Disk | Disk |
| Persistence | RDB / AOF | WAL + snapshot | WAL + snapshot |
| Data volume | Large | Small | Medium |
| Restart recovery | Fast | Moderate | Moderate |
RAM
│
├─ RDB (snapshot)
└─ AOF (append)
Client writes
↓
WAL (log)
↓
Snapshot
✔ Caching
✔ Rate limiting
✔ Distributed counters
✔ Session storage
✔ Queues / Streams
✔ Real-time analytics
✔ Leaderboards
Diagram:
User → App → Redis → Response (sub-ms)
✔ Leader election
✔ Distributed locks
✔ Metadata management
✔ Kafka / HBase coordination
✔ Strong ordering guarantees
Diagram:
Services → ZooKeeper → Coordination
✔ Kubernetes state
✔ Service discovery
✔ Feature flags
✔ Distributed config
✔ Cloud-native control planes
Diagram:
Nodes → etcd → Watch → React
| Feature | Redis | ZooKeeper | etcd |
|---|---|---|---|
| Watch | Pub/Sub | Watcher | Native Watch |
| Reliability | Best-effort | Strong | Strong |
| Replay | ❌ | ❌ | ✅ |
Client sets watch
↓
Key changes
↓
Guaranteed event delivery
| Aspect | Redis | ZooKeeper | etcd |
|---|---|---|---|
| Horizontal scale | Excellent | Poor | Limited |
| Data size | Very large | Very small | Small-medium |
| Clients | Millions | Thousands | Thousands |
All writes → Leader
All coordination → Leader
This is intentional.
| Aspect | Redis | ZooKeeper | etcd |
|---|---|---|---|
| Setup | Easy | Hard | Medium |
| Debugging | Medium | Hard | Medium |
| Tuning | Many knobs | Many knobs | Few knobs |
| Cloud-native | Partial | No | Yes |
| Feature | Redis | ZooKeeper | etcd |
|---|---|---|---|
| Auth | ACL / AUTH | ACL | TLS + RBAC |
| Encryption | TLS | TLS | TLS |
| Multi-tenancy | Limited | Limited | Strong |
Need speed? ───────────▶ Redis
Need correctness? ─────▶ ZooKeeper
Need cloud control? ───▶ etcd
Redis = Data Plane
ZooKeeper = Coordination Plane
etcd = Control Plane
They complement, not replace, each other.
The best systems often use all three, each for what it does best.
Organized by usage frequency and modern patterns
Purpose: Store string value with optional expiration
Syntax:
SET key value [EX seconds|PX milliseconds] [NX|XX]
Real-life Scenarios:
Best Practices:
# Cache with 60-second expiration
SET user:123:profile "{name: 'John', email: 'john@email.com'}" EX 60
# Set only if key doesn't exist (for locks)
SET lock:order:456 "processing" EX 30 NX
# Set only if key exists (for updates)
SET user:123:status "active" XX
Diagram:
┌─────────────────────────────────────────────┐
│ SET Command Flow │
├─────────────────────────────────────────────┤
│ Client Redis Server │
│ │ │ │
│ │ SET key value EX 30 │ │
│ │──────────────────────────>│ │
│ │ │ │
│ │ "OK" │ │
│ │<──────────────────────────│ │
│ │ │ │
│ │ (30 seconds later) │ │
│ │ GET key │ │
│ │──────────────────────────>│ │
│ │ (nil) │ │
│ │<──────────────────────────│ │
└─────────────────────────────────────────────┘
Purpose: Get value of a key
Syntax: GET key
Real-life Scenarios:
Examples:
# Basic get
GET user:456:profile
# Use with EXISTS check first
EXISTS user:456:profile
GET user:456:profile
# Pattern for cache miss handling
GET product:789
# If nil, fetch from database and SET with EX
Purpose: Remove one or more keys
Syntax: DEL key [key ...]
Best Practices:
# Delete single key
DEL user:123:session
# Delete multiple keys
DEL user:123:profile user:123:preferences user:123:cart
# Safer deletion with EXISTS check
EXISTS important:config
DEL important:config
# Use UNLINK for large keys in production (non-blocking)
UNLINK large:dataset:456
Purpose: Increment/decrement integer values atomically
Real-life Scenarios:
Examples:
# Basic counter
SET page:views 0
INCR page:views # Returns 1
INCR page:views # Returns 2
# Rate limiting pattern
INCR rate:limit:192.168.1.1
EXPIRE rate:limit:192.168.1.1 60
# Decrement inventory
SET product:789:stock 100
DECR product:789:stock # Returns 99
# Increment by specific amount
INCRBY user:123:score 50
DECRBY user:123:score 20
Rate Limiting Diagram:
┌─────────────────────────────────────────────────┐
│ Rate Limiting Pattern │
├─────────────────────────────────────────────────┤
│ Minute 1 │ Minute 2 │ Minute 3 │
│ ┌─────────┐ │ ┌─────────┐ │ ┌─────────┐ │
│ │Request 1│ │ │Request 1│ │ │Request 1│ │
│ │INCR key │ │ │INCR key │ │ │INCR key │ │
│ │EXPIRE │ │ │EXPIRE │ │ │EXPIRE │ │
│ └─────────┘ │ └─────────┘ │ └─────────┘ │
│ │Request 2│ │ │Request 2│ │ │
│ │INCR→2 │ │ │INCR→2 │ │ │
│ └─────────┘ │ └─────────┘ │ │
│ │Request 3│ │ │Request 3│ │ │
│ │INCR→3 │ │ │INCR→3 │ │ │
│ └─────────┘ │ └─────────┘ │ │
│ │Request 4│ │ │Request 4│ │ │
│ │INCR→4 │ │ │GET→4 │ │ │
│ │IF >3 │ │ │IF >3 │ │ │
│ │Reject │ │ │Reject │ │ │
│ └─────────┘ │ └─────────┘ │ │
└─────────────────────────────────────────────────┘
Purpose: Set timeout on key
Syntax: EXPIRE key seconds
Modern Alternatives:
# Set with expiration in one command (preferred)
SET session:abc123 "user_data" EX 3600
# Update existing key's TTL
EXPIRE cache:key 300
EXPIREAT cache:key 1672531200 # Unix timestamp
# Check remaining TTL
TTL session:abc123
# Returns: -2 (key doesn't exist)
# Returns: -1 (key exists, no expire)
# Returns: >0 (seconds remaining)
# Remove expiration
PERSIST important:config
Purpose: Store field-value pairs within a key
Real-life Scenarios:
Examples:
# Store user profile
HSET user:123 name "John Doe" email "john@email.com" age 30
# Get specific fields
HGET user:123 name # "John Doe"
HGET user:123 age # "30"
# Get all fields
HGETALL user:123
# Update single field
HSET user:123 age 31
# Check if field exists
HEXISTS user:123 email
# Increment numeric field
HINCRBY user:123 visits 1
# Get multiple fields
HMGET user:123 name email phone
Hash vs String Storage:
┌─────────────────┐ ┌─────────────────┐
│ String Storage │ │ Hash Storage │
├─────────────────┤ ├─────────────────┤
│ │ │ │
│ user:123 │ │ user:123 │
│ ──────────── │ │ ──────────── │
│ { │ │ name: "John" │
│ "name": │ │ email: "john@" │
│ "John", │ │ age: "30" │
│ "email": │ │ │
│ "john@...", │ │ │
│ "age": 30 │ │ │
│ } │ │ │
│ │ │ │
│ Memory: 100B │ │ Memory: 80B │
│ │ │ (more efficient│
│ │ │ for many flds)│
└─────────────────┘ └─────────────────┘
Purpose: Add elements to lists
Real-life Scenarios:
Examples:
# Message queue pattern
LPUSH messages:queue "{job: 'process', data: {...}}"
LPUSH messages:queue "{job: 'email', data: {...}}"
# Worker processing
RPOP messages:queue # Gets oldest item
# Recent activities
LPUSH user:123:activity "Logged in"
LPUSH user:123:activity "Viewed product:789"
LPUSH user:123:activity "Purchased item"
# Get range (last 10 activities)
LRANGE user:123:activity 0 9
# Fixed-length list (news feed)
LPUSH feed:global "News item 1"
LTRIM feed:global 0 99 # Keep only 100 items
Message Queue Pattern:
┌─────────────┐ LPUSH ┌─────────────┐ RPOP/BLPOP ┌─────────────┐
│ Producer │───────────>│ Redis │<───────────────│ Consumer │
│ │ │ List │ │ │
│ │ │ ┌───────┐ │ │ │
│ │ │ │ Msg 3 │ │ │ │
│ │ │ │ Msg 2 │ │ │ │
│ │ │ │ Msg 1 │ │ │ │
│ │ │ └───────┘ │ │ │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
│ │ │
│ (FIFO Queue) │
└──────────────────────────────────────────────────────────┘
Purpose: Store unique unordered elements
Real-life Scenarios:
Examples:
# Store tags
SADD article:456:tags "redis" "database" "nosql" "cache"
# Check membership
SISMEMBER article:456:tags "redis" # Returns 1 (true)
# Get all members
SMEMBERS article:456:tags
# Set operations (powerful!)
SADD users:online "user1" "user2" "user3"
SADD users:active "user2" "user3" "user4"
# Intersection (users both online AND active)
SINTER users:online users:active # ["user2", "user3"]
# Union (all unique users)
SUNION users:online users:active # ["user1","user2","user3","user4"]
# Difference (online but not active)
SDIFF users:online users:active # ["user1"]
# Random element (lottery, sampling)
SRANDMEMBER users:online
SPOP users:online # Removes and returns random
Purpose: Store sorted unique elements with scores
Real-life Scenarios:
Examples:
# Leaderboard
ZADD leaderboard 1000 "player1"
ZADD leaderboard 850 "player2"
ZADD leaderboard 920 "player3"
# Get top 3 players
ZREVRANGE leaderboard 0 2 WITHSCORES
# Get player rank (0-indexed from top)
ZREVRANK leaderboard "player2"
# Increment score
ZINCRBY leaderboard 50 "player1"
# Range queries (scores 800-900)
ZRANGEBYSCORE leaderboard 800 900
# Time-series pattern
ZADD temperature:nyc 1672444800 "38.5" # timestamp, value
ZADD temperature:nyc 1672448400 "39.2"
ZRANGEBYSCORE temperature:nyc 1672444800 1672448400
Leaderboard Architecture:
┌─────────────────────────────────────────────────┐
│ Redis Sorted Set Leaderboard │
├─────────────────────────────────────────────────┤
│ Score │ Member │ Operations │
├──────────┼─────────────────┼───────────────────┤
│ 1050 │ player1 │ ZADD score │
│ 920 │ player3 │ ZINCRBY +50 │
│ 850 │ player2 │ ZREVRANGE 0 2 │
│ 720 │ player4 │ ZREVRANK player2 │
│ 650 │ player5 │ ZRANGEBYSCORE │
│ │ │ 800 1000 │
└──────────┴─────────────────┴───────────────────┘
▲
│
└─── Scores automatically sorted,
O(log N) operations
Purpose: Incrementally iterate keys without blocking
Why not KEYS?
KEYS * blocks Redis (use in production with extreme caution)SCAN is non-blocking, production-safeExamples:
# Iterate through all keys matching pattern
SCAN 0 MATCH user:* COUNT 100
# Iterate specific type
SCAN 0 MATCH product:* COUNT 50 TYPE string
# Hash scan
HSCAN user:123 0 MATCH *name* COUNT 10
# Set scan
SSCAN tags:article:456 0 COUNT 20
# Sorted set scan
ZSCAN leaderboard 0 COUNT 15
SCAN Iteration Pattern:
┌─────────────────────────────────────────────┐
│ SCAN Iteration Process │
├─────────────────────────────────────────────┤
│ Initial: SCAN 0 MATCH user:* COUNT 5 │
│ Returns: │
│ 1) "5" (next cursor) │
│ 2) ["user:1", "user:2", "user:3"] │
│ │
│ Next: SCAN 5 MATCH user:* COUNT 5 │
│ Returns: │
│ 1) "12" (next cursor) │
│ 2) ["user:4", "user:5"] │
│ │
│ Next: SCAN 12 MATCH user:* COUNT 5 │
│ Returns: │
│ 1) "0" (iteration complete) │
│ 2) ["user:6"] │
│ │
│ Cursor 0 = Done │
└─────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Choosing the Right Data Structure │
├──────────────┬────────────────┬─────────────┬───────────────┤
│ Use Case │ Structure │ Best For │ Example │
├──────────────┼────────────────┼─────────────┼───────────────┤
│ Simple Cache │ String │ Single vals │ SET/GET │
│ │ │ With TTL │ user:session │
├──────────────┼────────────────┼─────────────┼───────────────┤
│ Object Store │ Hash │ Multiple │ HSET/HGETALL │
│ │ │ fields │ user:profile │
├──────────────┼────────────────┼─────────────┼───────────────┤
│ Queue │ List │ FIFO/LIFO │ LPUSH/RPOP │
│ │ │ │ jobs:queue │
├──────────────┼────────────────┼─────────────┼───────────────┤
│ Unique Items │ Set │ Membership │ SADD/SISMEMBER│
│ │ │ Operations │ article:tags │
├──────────────┼────────────────┼─────────────┼───────────────┤
│ Rankings │ Sorted Set │ Scores │ ZADD/ZRANGE │
│ │ │ Ranges │ leaderboard │
├──────────────┼────────────────┼─────────────┼───────────────┤
│ Time Series │ Sorted Set │ Timestamps │ ZADD ts:data │
│ │ or Stream │ Sequences │ XADD log:stream│
└──────────────┴────────────────┴─────────────┴───────────────┘
Purpose: Reduce round-trip time
# Without pipeline (3 round trips)
SET user:1:name "John"
SET user:1:email "john@email.com"
SET user:1:age 30
# With pipeline (1 round trip)
MULTI
SET user:1:name "John"
SET user:1:email "john@email.com"
SET user:1:age 30
EXEC
MULTI
GET account:123:balance
DECRBY account:123:balance 100
INCRBY account:456:balance 100
EXEC
EVAL "local current = redis.call('GET', KEYS[1])
if tonumber(current) < tonumber(ARGV[1]) then
return redis.call('INCR', KEYS[1])
else
return nil
end" 1 rate:limit:ip 10
# Producer
XADD mystream * sensor-id 1234 temperature 19.8
# Consumer group
XGROUP CREATE mystream mygroup 0
# Consumer
XREADGROUP GROUP mygroup consumer1 COUNT 1 BLOCK 5000 STREAMS mystream >
AUTH your_strong_password_here
# Better: Use Redis 6+ ACL
ACL SETUSER alice on >password ~cache:* +get +set
# Get configuration
CONFIG GET maxmemory
CONFIG GET requirepass
# Set configuration (prefer redis.conf for persistence)
CONFIG SET maxmemory 2gb
CONFIG SET save "900 1 300 10 60 10000"
# Never expose these in production:
CONFIG SET requirepass "" # DON'T!
CONFIG SET protected-mode no # DON'T!
# Get all info (verbose)
INFO
# Get specific sections
INFO memory
INFO stats
INFO clients
INFO replication
# Critical metrics to monitor:
INFO memory # used_memory, maxmemory, evictions
INFO stats # instantaneous_ops_per_sec, keyspace_hits/misses
INFO persistence # rdb_last_save_time, aof_rewrite_in_progress
MEMORY USAGE large:key
MEMORY STATS
MEMORY PURGE # Attempts to defragment (Redis 4+)
# Use appropriate data types
# Strings under 44 bytes use EMBSTR encoding
SET small:key "short" # More efficient
# Hashes with <= 512 fields use ziplist
HSET config:app field1 "val1" ... field500 "val500"
# Lists with small elements use ziplist
LPUSH recent:items "item1" "item2" # Check list-max-ziplist* configs
┌─────────────┐ ┌─────────────────┐ ┌─────────────┐
│ App │ │ Connection │ │ Redis │
│ Server │─────▶│ Pool │─────▶│ Server │
│ │ │ │ │ │
│ Request 1 │ │ ┌───────────┐ │ │ │
│ ───────── │ │ │ Connection│──┼─────▶│ Process │
│ GET key │ │ │ 1 │ │ │ Request │
│ │ │ └───────────┘ │ │ │
│ Request 2 │ │ ┌───────────┐ │ │ │
│ ───────── │ │ │ Connection│──┼─────▶│ Process │
│ SET key │───┬─▶│ │ 2 │ │ │ Request │
│ │ │ │ └───────────┘ │ │ │
│ Request 3 │ │ │ ┌───────────┐ │ │ │
│ ───────── │ └─▶│ │ Connection│──┼─────▶│ Process │
│ INCR key │ │ │ 3 │ │ │ Request │
│ │ │ └───────────┘ │ │ │
└─────────────┘ └─────────────────┘ └─────────────┘
# Configure slow log
CONFIG SET slowlog-log-slower-than 10000 # 10ms
CONFIG SET slowlog-max-len 128
# View slow logs
SLOWLOG GET 10
SLOWLOG LEN
SLOWLOG RESET
# 1. KEYS * in production (use SCAN instead)
KEYS * # BLOCKS SERVER!
# 2. Large values in single key
SET huge:key "[...10MB of data...]" # Blocks during serialization
# 3. Not using TTL for cache
SET cache:key "data" # Might fill memory
# 4. Using Redis as primary database
# Redis is in-memory - data can be lost
# 5. Not monitoring memory usage
# Leads to OOM (Out Of Memory) errors
# 1. Use pipelining for bulk operations
MULTI
SET key1 "val1"
SET key2 "val2"
EXEC
# 2. Use appropriate data structures
# For object with many fields: Hash, not multiple String keys
# 3. Always set TTL for cache
SET session:abc "data" EX 3600
# 4. Use Lua scripts for complex atomic operations
# 5. Monitor with INFO and SLOWLOG
# On replica
REPLICAOF master-ip 6379
REPLICAOF NO ONE # Promote to master
# Check replication status
INFO replication
# Monitor replication lag
redis-cli --latency
# Sentinel
SENTINEL MASTERS
SENTINEL GET-MASTER-ADDR-BY-NAME mymaster
# Cluster
CLUSTER NODES
CLUSTER INFO
CLUSTER SLOTS
# Manual save (blocks)
SAVE # Synchronous, blocks
BGSAVE # Asynchronous, forks
# Check persistence status
INFO persistence
LASTSAVE # Unix timestamp of last save
# AOF operations
BGREWRITEAOF
# Flush data (CAUTION!)
FLUSHDB # Current database
FLUSHALL # All databases
# Debug object
DEBUG OBJECT keyname
# Monitor commands in real-time
MONITOR # Use sparingly, verbose
# Client management
CLIENT LIST
CLIENT KILL addr ip:port
CLIENT PAUSE timeout_ms # Pause clients
┌─────────────────────────────────────────────────────┐
│ Redis Health Dashboard │
├─────────────────┬─────────────────┬─────────────────┤
│ Memory Usage │ Command Stats │ Connections │
├─────────────────┼─────────────────┼─────────────────┤
│ Used: 1.2GB/2GB│ Ops/sec: 5,000 │ Connected: 45 │
│ 60% │ Hits: 98% │ Max: 10,000 │
│ Peak: 1.8GB │ Misses: 2% │ Rejected: 0 │
│ Evictions: 0 │ Cmd/sec: 8,000 │ Timeout: 0 │
├─────────────────┼─────────────────┼─────────────────┤
│ Replication │ Persistence │ Keyspace │
├─────────────────┼─────────────────┼─────────────────┤
│ Role: Master │ RDB: Enabled │ DB0 Keys: 45K │
│ Slaves: 2 │ Last Save: 2m │ Expires: 12K │
│ Lag: 0.5s │ AOF: Enabled │ Avg TTL: 300s │
│ Status: OK │ Size: 500MB │ Hit Rate: 99% │
└─────────────────┴─────────────────┴─────────────────┘
# Cache-aside pattern
1. GET cache:key
2. If nil, fetch from DB
3. SET cache:key value EX 300
# Write-through pattern
1. Update DB
2. SET cache:key new_value EX 300
# Write-behind pattern
1. SET cache:key new_value EX 300
2. Async update to DB
Good: Bad:
user:123:profile user_profile_123
session:abc123 abc123_session
product:789:views product_views_789
geo:nyc:users newyork_users_list
Pattern: object_type:id:field
# Cache: 5 minutes - 1 hour
SET cache:key value EX 300
# Session: 30 minutes - 24 hours
SET session:key data EX 1800
# Rate limit: 1 second - 1 minute
SET rate:limit:ip 1 EX 60 NX
# Leaderboard: 1 day - 7 days
EXPIRE leaderboard:daily 86400
# Check if Redis is alive
PING # Should return PONG
# Check latency between commands
redis-cli --latency
# Monitor commands in real-time (debug)
MONITOR
# Check memory fragmentation
INFO memory | grep fragmentation
# List connected clients
CLIENT LIST
# Check slow queries
SLOWLOG GET 5
# Benchmark tool (outside redis-cli)
redis-benchmark -t set,get -n 100000 -q
# Pipeline benchmark
redis-benchmark -t set,get -n 100000 -q -P 16
String:
┌────────┬────────┬────────┐
│ Header │ Data │ Null │
│ Type │ "value"│ term │
└────────┴────────┴────────┘
Hash (ziplist encoding):
┌────┬────┬────┬────┬────┬────┐
│ ZL │ F1 │ V1 │ F2 │ V2 │ ZL │
│bytes│"name"│"John"│"age"│"30"│end │
└────┴────┴────┴────┴────┴────┘
List (quicklist):
┌─────────┬─────────┬─────────┐
│ Node │ Node │ Node │
│ [ziplist│ [ziplist│ [ziplist│
│ "item1" │ "item4" │ "item7" │
│ "item2" │ "item5" │ │
│ "item3" │ "item6" │ │
└─────────┴─────────┴─────────┘
maxmemory and eviction policyRemember: Redis is single-threaded for commands. One slow command blocks all others. Always test commands in staging before production, and monitor performance metrics regularly.
Kubernetes chose etcd because it provides strong consistency, fault tolerance, watchability, and simplicity—exactly what a distributed control plane needs.
Before understanding why etcd, we must understand what Kubernetes fundamentally is.
A distributed state machine that continuously reconciles desired state with actual state
That sentence alone explains why etcd matters.
User declares desired state
↓
Kubernetes stores desired state
↓
Controllers compare desired vs actual
↓
Actions taken to reconcile
↓
State updated
👉 Everything depends on a reliable, consistent source of truth
The data store must hold:
This data store is the single source of truth.
If the data store is wrong, Kubernetes is wrong.
Wrong state → Wrong scheduling
Wrong locks → Split brain
Wrong config → Cluster outage
So Kubernetes needs a store that is:
✔ Always correct
✔ Always consistent
✔ Always available (with quorum)
✔ Observable (watchable)
✔ Deterministic
Let’s eliminate obvious candidates first.
Fast
In-memory
Eventual consistency
No consensus protocol
Best-effort durability
Redis write → Master
Replica lag possible
Failover may lose writes
APIServer → Redis
↓
Possible stale read
🚫 Unacceptable for Kubernetes Kubernetes requires linearizable writes, not eventual correctness.
ZooKeeper was a strong candidate.
Strong consistency
Leader election
Distributed locks
Proven correctness
| Issue | Explanation |
|---|---|
| Operational complexity | Hard to operate at scale |
| Watch semantics | One-time triggers |
| API ergonomics | Not cloud-native |
| Data model | Hierarchical, not flexible |
| Tooling | Heavy JVM ecosystem |
ZooKeeper was correct but heavy.
Distributed
Strongly consistent
Key-value store
Raft consensus
Watch-first design
┌───────────────┐
│ Kubernetes │
│ Control Plane │
└───────┬───────┘
│
┌───────▼───────┐
│ etcd │
│ (Source of │
│ Truth) │
└───────────────┘
Write acknowledged
↓
Majority of nodes committed
↓
All future reads see it
Client
↓
Leader
↓
Follower A
↓
Follower B
(majority ACK)
✔ No stale reads ✔ No lost writes ✔ No split brain
State S₀
↓
Event E₁
↓
State S₁
↓
Event E₂
↓
State S₂
etcd guarantees:
This is hugely important.
Client sets watch
↓
Key changes
↓
Guaranteed event stream
etcd change
↓
APIServer
↓
Controller
↓
Action
Controllers are event-driven, not polling-based.
| System | Watch Reliability |
|---|---|
| Redis Pub/Sub | Best effort |
| ZooKeeper | One-shot |
| etcd | Continuous, ordered, resumable |
Only etcd matches Kubernetes needs.
/registry/pods/ns1/podA
/registry/services/ns1/svcA
/registry/nodes/node1
✔ Easy versioning
✔ Easy compaction
✔ Easy snapshot
✔ Easy migration
ZooKeeper’s tree model complicates this.
etcd supports:
IF version == X
THEN update
ELSE fail
Kubernetes uses this for:
Controller A
Controller B
Controller C
Only ONE should be active.
Lease → TTL → automatic expiry
Diagram:
Controller A holds lease
↓
Crash
↓
Lease expires
↓
Controller B elected
Crash
↓
Restart
↓
State recovered
etcd durability is mandatory for cluster recovery.
Writes → WAL
↓
Periodic snapshot
Perfect for Kubernetes.
This is HUGE.
Nodes die
Pods die
APIServer restarts
As long as etcd survives, Kubernetes reconstructs everything.
| Feature | Why It Matters |
|---|---|
| gRPC API | Performance & tooling |
| TLS everywhere | Security |
| Small binary | Easy deployment |
| Raft | Simple mental model |
Low write volume
Small objects
High read consistency
etcd is not designed for:
Which aligns perfectly.
3–5 nodes
Quorum based
Simple backups
Predictable behavior
Kubernetes admins can reason about failures.
❌ Pod logs
❌ Metrics
❌ Application data
❌ High-frequency events
etcd is control plane only.
| Criteria | ZooKeeper | etcd |
|---|---|---|
| Consistency | Strong | Strong |
| Watch model | Weak | Excellent |
| API | Heavy | Clean |
| Cloud-native | No | Yes |
| Operational burden | High | Medium |
| Fit for Kubernetes | ❌ | ✅ |
etcd = Brain
Kubernetes = Nervous System
Nodes = Muscles
If the brain is wrong, the body fails.
Kubernetes chose etcd because:
✔ Strong consistency
✔ Event-driven watches
✔ Simple, correct semantics
✔ Cloud-native design
✔ Perfect data profile match
etcd is not fast—but Kubernetes does not need fast. Kubernetes needs correct.
Short answer up front:
ZooKeeper is chosen over etcd when absolute correctness, strict ordering, and coordination semantics matter more than cloud-native ergonomics and operational simplicity.
This goes straight to the philosophy and historical trade-offs of distributed systems.
When people say “ZooKeeper is more correct”, they usually mean coordination correctness, not storage correctness.
| Type | Meaning |
|---|---|
| Data correctness | Reads return the latest committed value |
| Coordination correctness | Ordering, fencing, session semantics are strictly enforced |
Both etcd and ZooKeeper are data-correct. ZooKeeper is exceptionally strong at coordination correctness.
ZooKeeper was not designed as a KV store.
It was designed to solve distributed coordination problems:
“If coordination is wrong, the entire distributed system collapses.”
This assumption drove extremely conservative design choices.
ZooKeeper has sessions, etcd does not (in the same way).
Client connects
↓
Session established
↓
Ephemeral nodes tied to session
↓
Client dies → session expires → nodes deleted
Lock correctness
Leader correctness
Membership correctness
Client A holds lock
↓
Client A crashes
↓
ZooKeeper deletes ephemeral node
↓
Lock immediately released
This is automatic fencing, not optional logic.
ZooKeeper guarantees:
But more importantly:
ZooKeeper exposes ordering as a first-class primitive
ZooKeeper can create nodes like:
/lock/node-00000001
/lock/node-00000002
/lock/node-00000003
This enables:
etcd cannot do this natively.
You might hear:
“etcd has better watches”
This is true for data propagation, but false for coordination logic.
Why?
Because ZooKeeper forces you to reason about race conditions explicitly.
This reduces false assumptions in coordination logic.
ZooKeeper assumes:
Network partitions WILL happen
Clients WILL crash
GC pauses WILL happen
So it prefers:
etcd is conservative too — but ZooKeeper is paranoid.
These systems care about:
✔ Strict leader fencing
✔ Lock correctness
✔ No split-brain EVER
✔ Predictable failure modes
| Dimension | ZooKeeper | etcd |
|---|---|---|
| Primary role | Coordination service | Consistent KV store |
| Session semantics | First-class | Lease-based |
| Lock safety | Extremely strong | Strong but simpler |
| Ordering primitives | Built-in | Limited |
| API philosophy | Explicit, low-level | Simple, declarative |
| Cloud-native | ❌ | ✅ |
| Operational simplicity | ❌ | ✅ |
Now the flip side.
Kubernetes does not need:
Kubernetes needs:
✔ State storage
✔ Event watching
✔ Simple transactions
✔ Declarative reconciliation
ZooKeeper would have been overkill and harder to operate.
ZooKeeper wins when:
Examples:
In these systems:
A single incorrect lock is worse than total outage
ZooKeeper optimizes for never being wrong, even if that means being unavailable.
If unsure → stop
If quorum exists → continue
Both are correct — but ZooKeeper is stricter.
ZooKeeper = Distributed coordination kernel
etcd = Distributed state store
ZooKeeper acts like a kernel enforcing rules. etcd acts like a database holding truth.
ZooKeeper is chosen over etcd when:
etcd is chosen when:
If you fear split-brain more than downtime → ZooKeeper If you fear operational complexity more than coordination edge cases → etcd
Before diving into details, here's the complete Redis ecosystem at a glance:
graph TB
subgraph "Application Layer"
App[Application]
end
subgraph "Redis Layer"
subgraph "Single Instance"
Master1[Master]
Replica1[Replica]
Master1 --> Replica1
end
subgraph "Redis Cluster (Sharded)"
M1[Master A<br/>Slots 0-5460]
M2[Master B<br/>Slots 5461-10922]
M3[Master C<br/>Slots 10923-16383]
R1[Replica A]
R2[Replica B]
R3[Replica C]
M1 --> R1
M2 --> R2
M3 --> R3
end
subgraph "High Availability"
S1[Sentinel 1]
S2[Sentinel 2]
S3[Sentinel 3]
end
end
subgraph "Persistence Layer"
RDB[(RDB Snapshot)]
AOF[(AOF Log)]
end
App --> Master1
App --> M1
App --> M2
App --> M3
S1 -.Monitor.-> Master1
S2 -.Monitor.-> Master1
S3 -.Monitor.-> Master1
Master1 --> RDB
Master1 --> AOF
M1 --> RDB
M2 --> RDB
M3 --> RDB
Key Components:
| Component | Purpose | When to Use |
|---|---|---|
| Single Master | Basic Redis instance | Small datasets, simple caching |
| Master + Replica | Read scaling + failover | Read-heavy workloads |
| Sentinel | Automatic failover | High availability without sharding |
| Redis Cluster | Horizontal scaling (sharding) | Large datasets exceeding single node |
| RDB | Point-in-time snapshots | Fast restarts, backups |
| AOF | Write operation log | Maximum data durability |
Decision Tree:
flowchart TD
Start{Data fits in<br/>single instance?}
Start -->|Yes| HA{Need high<br/>availability?}
Start -->|No| UseCluster[Use Redis Cluster]
HA -->|No| SingleMaster[Single Master<br/>+ optional replicas]
HA -->|Yes| UseSentinel[Master + Replicas<br/>+ Sentinel]
UseCluster --> Persist{Need persistence?}
UseSentinel --> Persist
SingleMaster --> Persist
Persist -->|Fast restart| UseRDB[Enable RDB]
Persist -->|Max durability| UseAOF[Enable AOF]
Persist -->|Best of both| UseHybrid[Enable Both<br/>Recommended]
Redis stands for REmote DIctionary Server.
At its heart, Redis is:
Redis is a single-threaded, in-memory data structure server optimized for speed, simplicity, and predictable behavior.
flowchart LR
Client --> Redis
Client --> DB[(Database)]
Redis --> DB
Redis often sits between your application and your database to:
Redis makes several strong design choices:
| Principle | Meaning |
|---|---|
| In-memory | Data lives in RAM for speed |
| Single-threaded | No locks, no race conditions |
| Simple data structures | Predictable performance |
| Explicit durability | You choose persistence level |
sequenceDiagram
participant Client1
participant Client2
participant Redis
Client1->>Redis: INCR counter
Client2->>Redis: INCR counter
Redis-->>Client1: OK
Redis-->>Client2: OK
Because Redis processes one command at a time:
INCR are naturally atomicRedis favors:
Consistency per key, not global consistency
Key Guarantees:
Why Redis is fast:
graph TD
RAM --> NoDiskIO
SingleThread --> NoLocks
SimpleCommands --> PredictableLatency
Typical Latency:
| Operation | Latency |
|---|---|
| GET | ~1µs |
| INCR | ~1–2µs |
| Network RTT | Dominant cost |
Redis trades features for speed.
What Redis Is Great At:
What Redis Is Not Good At:
Redis is not a table store.
Think of Redis as a huge hash map in memory:
graph TD
Redis --> Key1["user:1"]
Redis --> Key2["order:991"]
Redis --> Key3["counter:login"]
Each key:
Redis supports rich native data types:
| Type | Think of it as |
|---|---|
| String | Basic value / counter |
| Hash | Object |
| List | Queue / Stack |
| Set | Unique collection |
| Sorted Set | Ranking system |
| Stream | Append-only log |
Example Mental Mapping:
graph LR
AppObject --> RedisHash["user:42 (hash)"]
Queue --> RedisList["jobs:list"]
Ranking --> ZSet["leaderboard"]
INCR)Counters are one of Redis's killer features.
Why INCR Is Special:
How INCR Works Internally:
sequenceDiagram
participant App1
participant App2
participant Redis
App1->>Redis: INCR page_views
App2->>Redis: INCR page_views
Redis->>Redis: page_views = page_views + 1
Redis-->>App1: 101
Redis-->>App2: 102
Because Redis is single-threaded:
Important Clarification:
Redis does NOT understand { "user": "whereq" } as a data structure by default. Redis only sees bytes.
The application must decide how to store data:
Option 1: Simple key-value
SET user whereq
Stored as:
"user" -> "whereq"
Option 2: Namespaced key
SET user:whereq ""
Stored as:
"user:whereq" -> ""
Option 3: JSON serialized
SET user:whereq '{"user":"whereq"}'
Stored as:
"user:whereq" -> '{"user":"whereq"}'
Option 4: Redis Hash (logical structure)
HSET user:whereq user whereq
Stored internally as:
"user:whereq" -> hash(user=whereq)
Redis is fundamentally an in-memory database, but it provides optional durability through two complementary persistence mechanisms:
graph LR
Memory["In-Memory Data<br/>(Source of Truth)"] --> RDB["RDB<br/>(Point-in-Time Snapshot)"]
Memory --> AOF["AOF<br/>(Write Command Log)"]
Key Principle:
Memory is always the source of truth. RDB and AOF are derived artifacts for recovery.
| Aspect | RDB | AOF |
|---|---|---|
| What it stores | Binary snapshot of all data | Sequential log of write commands |
| When it writes | Periodic (configurable intervals) | After every write (configurable) |
| Recovery speed | Fast (binary load) | Slower (replay commands) |
| Data loss risk | Higher (between snapshots) | Lower (per-write durability) |
| File size | Compact | Larger (needs rewriting) |
| Best for | Backups, replica sync | Maximum durability |
Data Flow Through Persistence:
sequenceDiagram
participant Client
participant Redis Memory
participant AOF
participant RDB
Client->>Redis Memory: SET key value
Redis Memory->>Client: OK
Redis Memory->>AOF: Append command (immediate)
Note over Redis Memory,RDB: Later (based on save intervals)
Redis Memory->>RDB: Fork & snapshot all data
Note over Redis Memory: On restart:
alt AOF exists
AOF->>Redis Memory: Replay commands
else RDB only
RDB->>Redis Memory: Load snapshot
end
Relationship Between Components:
┌───────────────────────────────────────────────┐
│ │
│ WRITES FLOW (Write Command) │
│ │
│ Client → Memory → AOF (append) │
│ ↓ │
│ (periodic) │
│ ↓ │
│ RDB (snapshot) │
│ │
└───────────────────────────────────────────────┘
┌───────────────────────────────────────────────┐
│ │
│ RECOVERY FLOW (Redis Restart) │
│ │
│ 1. Check if AOF exists? │
│ ├─ Yes → Load AOF (replay all commands) │
│ └─ No → Load RDB (restore snapshot) │
│ │
│ 2. Memory rebuilt │
│ 3. Ready to serve │
│ │
└───────────────────────────────────────────────┘
What RDB Really Is:
RDB = Redis DataBase file
It is:
Think of RDB as:
📸 "A photograph of all Redis data at a specific moment in time"
What is inside an RDB file?
graph TD
RAM["Redis In-Memory Data"] --> Snapshot["RDB File on Disk"]
Is RDB a runtime state?
No. RDB exists outside Redis memory.
Redis runtime state is:
RDB is only used for:
How RDB is created (conceptually):
Redis:
sequenceDiagram
Redis->>OS: fork()
Child->>Disk: write snapshot
Parent->>Clients: continue serving
✅ No blocking of the main Redis thread ❗ Extra memory during fork (copy-on-write)
When Redis uses RDB:
| Situation | Use RDB? |
|---|---|
| Redis restart | ✅ |
| Full data reload | ✅ |
| Replica sync (initial) | ✅ |
| Incremental updates | ❌ |
Characteristics:
timeline
t1 : Snapshot
t2 : Snapshot
t3 : Crash (data lost between t2 and t3)
What AOF Is:
flowchart LR
Write --> AppendLog --> Disk
Key clarification:
There is NO extra daemon, NO sync process, NO background service.
Everything is handled by the Redis server process itself.
RDB vs AOF – different purposes:
| RDB | AOF |
|---|---|
| Snapshot | Command log |
| Periodic | Continuous |
| Fast restore | Durable |
| Compact | Larger |
They do not sync with each other.
How Redis actually works with both enabled:
graph TD
Write --> Memory
Write --> AOF
Timer --> RDB
Important insight:
Restart logic (very important):
When Redis restarts:
flowchart TD
Start --> AOFExists{AOF exists?}
AOFExists -->|Yes| LoadAOF
AOFExists -->|No| LoadRDB
Why no syncing is needed:
Because:
Memory → Persistence Not persistence → memory → persistence
Production Reality:
Most systems use:
graph TD
Redis --> RDB
Redis --> AOF
Snapshots for speed + AOF for safety.
When slot migration occurs:
If a master crashes after migration but before snapshot:
Redis handles this safely:
Key Principle:
Cluster metadata (slot ownership) overrides RDB contents
Will snapshot happen automatically after migration?
No. Slot migration does NOT automatically trigger an RDB snapshot.
Redis deliberately decouples:
Otherwise, resharding would cause massive disk IO spikes and latency stalls.
Important:
This section explores the internal structure of Redis persistence files, how they are organized on disk, and the engineering trade-offs made in their design.
What's Inside an RDB File?
An RDB file is a binary serialization of Redis's entire in-memory dataset at a specific point in time.
graph TD
RDB[RDB File] --> Header[Header: Magic + Version]
RDB --> DB0[Database 0]
RDB --> DB1[Database 1]
RDB --> DBN[Database N]
RDB --> EOF[EOF Marker + Checksum]
DB0 --> Meta0[Metadata: DB selector]
DB0 --> Keys0[Key-Value Pairs]
DB0 --> Expire0[Expiry Information]
RDB File Structure (Sequential Layout):
┌─────────────────────────────────────────────────────────┐
│ MAGIC NUMBER: "REDIS" (5 bytes) │
├─────────────────────────────────────────────────────────┤
│ VERSION: "0009" (4 bytes, current version) │
├─────────────────────────────────────────────────────────┤
│ AUXILIARY FIELDS (optional metadata) │
│ - Redis version │
│ - Creation timestamp │
│ - Used memory │
│ - Replication info │
├─────────────────────────────────────────────────────────┤
│ DATABASE SELECTOR (e.g., DB 0) │
├─────────────────────────────────────────────────────────┤
│ RESIZE DB (hash table size hints) │
├─────────────────────────────────────────────────────────┤
│ KEY-VALUE PAIRS │
│ ┌──────────────────────────────────────┐ │
│ │ Type byte (string/list/hash/etc.) │ │
│ │ Key (encoded string) │ │
│ │ Value (type-specific encoding) │ │
│ │ Expiry (optional, millisecond TTL) │ │
│ └──────────────────────────────────────┘ │
│ ... (repeat for all keys in DB) │
├─────────────────────────────────────────────────────────┤
│ DATABASE SELECTOR (e.g., DB 1) │
│ ... (more key-value pairs) │
├─────────────────────────────────────────────────────────┤
│ EOF MARKER (0xFF) │
├─────────────────────────────────────────────────────────┤
│ CRC64 CHECKSUM (8 bytes) │
└─────────────────────────────────────────────────────────┘
Key-Value Encoding:
Each data type has optimized encoding:
| Data Type | Encoding Strategy | Example |
|---|---|---|
| String | Raw bytes, integer encoding for numbers | "123" → 3-byte integer vs 3-byte string |
| List | Ziplist (compact) or LinkedList | Small lists: compressed format |
| Hash | Ziplist or Hash Table | Depends on size thresholds |
| Set | IntSet or Hash Table | Integer-only sets: compact array |
| Sorted Set | Ziplist or Skip List | Hybrid encoding based on size |
| Stream | Radix tree nodes | Specialized compact format |
Example: String Encoding
Small integers (0-9999):
Type: 1 byte (REDIS_ENCODING_INT)
Value: variable length integer encoding
Short strings (<= 20 bytes):
Type: 1 byte
Length: 1 byte
Data: N bytes
Long strings:
Type: 1 byte
Length: 4 bytes (encoded)
Data: N bytes
LZF Compressed strings:
Type: 1 byte + compression flag
Compressed length: variable
Uncompressed length: variable
Compressed data: N bytes
Does RDB Have Indexes?
❌ No — RDB has NO internal indexes.
RDB is designed for:
How RDB Restore Works:
sequenceDiagram
participant Redis
participant RDB File
participant Memory
Redis->>RDB File: Open and validate checksum
RDB File->>Redis: Read header
Redis->>RDB File: Read DB selector
loop For each key-value pair
RDB File->>Redis: Read type + key + value
Redis->>Memory: Insert into hash table
Redis->>Memory: Set expiry if present
end
RDB File->>Redis: EOF marker
Redis->>Redis: Validate CRC64
Redis->>Memory: Rebuild indexes (if needed)
Key Point:
RDB is a dump, not a database. It's optimized for bulk I/O, not random access.
RDB Trade-offs:
| Advantage ✅ | Disadvantage ❌ |
|---|---|
| Compact binary format | No indexes = can't query RDB directly |
| Fast to write (sequential I/O) | Must load entire file to access any data |
| Fast to read (sequential load) | Memory spike during fork (copy-on-write) |
| Built-in compression (LZF) | Periodic snapshots = potential data loss |
| CRC64 integrity check | Large datasets = long snapshot time |
| Portable across architectures | Not human-readable |
RDB Creation Process (Technical Details):
flowchart TD
Start[Write command triggers save] --> Fork[fork() creates child process]
Fork --> COW[Parent uses Copy-on-Write]
Fork --> Child[Child has memory snapshot]
COW --> Parent[Parent continues serving requests]
Child --> Iterate[Iterate all databases]
Iterate --> Encode[Encode each key-value]
Encode --> Write[Sequential write to temp file]
Write --> Rename[Atomic rename to dump.rdb]
Rename --> Exit[Child exits]
Parent --> Continue[Continues normally]
Exit --> Continue
Why RDB Uses fork():
Storage Organization:
Redis Data Directory:
/var/lib/redis/
├── dump.rdb ← Current RDB snapshot
├── temp-<pid>.rdb ← Temporary file during save (deleted after rename)
└── appendonly.aof ← AOF file (if enabled)
RDB Configuration:
# Automatic snapshots
save 900 1 # After 900 sec (15 min) if at least 1 key changed
save 300 10 # After 300 sec (5 min) if at least 10 keys changed
save 60 10000 # After 60 sec if at least 10000 keys changed
# Compression
rdbcompression yes # Use LZF compression (CPU vs size trade-off)
# Checksum
rdbchecksum yes # CRC64 integrity check
What's Inside an AOF File?
AOF (Append-Only File) is a text-based log of every write operation, stored in RESP protocol format (Redis Serialization Protocol).
graph TD
AOF[AOF File] --> Cmd1[Command 1: SET key1 value1]
AOF --> Cmd2[Command 2: INCR counter]
AOF --> Cmd3[Command 3: LPUSH list item]
AOF --> CmdN[Command N: ...]
Cmd1 --> RESP1[RESP Format]
Cmd2 --> RESP2[RESP Format]
AOF File Structure (Sequential Log):
┌─────────────────────────────────────────────────────────┐
│ *2 ← Array with 2 elements │
│ $6 ← Bulk string, 6 bytes │
│ SELECT ← Command name │
│ $1 ← Bulk string, 1 byte │
│ 0 ← Database number │
├─────────────────────────────────────────────────────────┤
│ *3 ← Array with 3 elements │
│ $3 ← Bulk string, 3 bytes │
│ SET ← Command name │
│ $4 ← Bulk string, 4 bytes │
│ key1 ← Key │
│ $6 ← Bulk string, 6 bytes │
│ value1 ← Value │
├─────────────────────────────────────────────────────────┤
│ *2 │
│ $4 │
│ INCR │
│ $7 │
│ counter │
├─────────────────────────────────────────────────────────┤
│ ... (commands continue chronologically) │
└─────────────────────────────────────────────────────────┘
RESP Protocol Format:
RESP Types:
*N = Array with N elements
$N = Bulk string with N bytes
+ = Simple string
- = Error
: = Integer
Example: Storing a Hash
HSET user:1 name "Alice" age 30
Becomes in AOF:
*5
$4
HSET
$6
user:1
$4
name
$5
Alice
$3
age
$2
30
Does AOF Have Indexes?
❌ No — AOF also has NO indexes.
AOF is:
How AOF Restore Works:
sequenceDiagram
participant Redis
participant AOF File
participant Memory
Redis->>AOF File: Open file
loop For each command
AOF File->>Redis: Read RESP command
Redis->>Redis: Parse command
Redis->>Memory: Execute command
Redis->>Memory: Update data structures
end
AOF File->>Redis: EOF reached
Redis->>Memory: Ready to serve
AOF Rewrite Process:
Over time, AOF grows as it logs every write. Redis periodically rewrites AOF to compact it.
flowchart TD
Start[AOF grows too large] --> Trigger[Auto-rewrite triggered]
Trigger --> Fork[fork() child process]
Fork --> Parent[Parent: Continue logging to old AOF]
Fork --> Child[Child: Iterate current memory]
Child --> Snapshot[Read current state from memory]
Snapshot --> Generate[Generate minimal commands]
Generate --> NewAOF[Write to temp AOF file]
Parent --> Buffer[Buffer new writes in memory]
NewAOF --> Append[Append buffered writes]
Append --> Rename[Atomic rename to appendonly.aof]
Rename --> Replace[Old AOF replaced]
Before Rewrite:
SET counter 1
INCR counter
INCR counter
INCR counter
DEL temp_key
SET temp_key value
DEL temp_key
SET user:1 Alice
SET user:1 Bob
... (thousands of operations)
After Rewrite:
SET counter 4
SET user:1 Bob
... (only final state)
AOF Rewrite = Converting history into current state
AOF Fsync Strategies:
AOF offers three durability levels via appendfsync:
| Strategy | Behavior | Durability | Performance | Data Loss Risk |
|---|---|---|---|---|
always | fsync() after every write | Maximum | Slowest | ~0 (only crash during write) |
everysec | fsync() once per second | Good | Fast | ~1 second of data |
no | Let OS decide when to flush | Minimal | Fastest | Potentially 30+ seconds |
Technical Detail:
// Simplified AOF write flow
write_command_to_aof_buffer(command);
if (appendfsync == ALWAYS) {
fsync(aof_fd); // Force write to disk NOW
} else if (appendfsync == EVERYSEC) {
// Background thread calls fsync() every second
schedule_fsync_background();
} else {
// OS will flush when it wants
}
Storage Organization:
Redis Data Directory:
/var/lib/redis/
├── appendonly.aof ← Main AOF file
├── appendonly.aof.manifest ← Manifest for multi-part AOF (Redis 7+)
├── appendonly.aof.1.base.rdb ← Base RDB (Redis 7+)
├── appendonly.aof.1.incr.aof ← Incremental AOF (Redis 7+)
└── temp-rewriteaof-<pid>.aof ← Temporary during rewrite
Redis 7+ Multi-Part AOF:
Redis 7 introduced AOF 2.0 with multiple files:
graph LR
Base[Base RDB Snapshot] --> Incr1[Incremental AOF 1]
Incr1 --> Incr2[Incremental AOF 2]
Incr2 --> IncrN[Incremental AOF N]
Benefits:
AOF Trade-offs:
| Advantage ✅ | Disadvantage ❌ |
|---|---|
| Much better durability (per-write) | Larger file size than RDB |
| Human-readable (RESP text format) | Slower recovery (must replay all commands) |
| Automatic rewrite compaction | Rewrite can spike CPU/memory |
| Granular fsync control | Not portable across Redis versions (sometimes) |
| Append-only = crash-safe writes | Can grow very large without rewrite |
| Preserves command history | No random access to specific keys |
AOF Configuration:
# Enable AOF
appendonly yes
# Fsync strategy
appendfsync everysec # Recommended balance
# Auto-rewrite trigger
auto-aof-rewrite-percentage 100 # Rewrite when 100% larger
auto-aof-rewrite-min-size 64mb # Minimum size to trigger
# AOF file name
appendfilename "appendonly.aof"
# Use RDB preamble (hybrid mode)
aof-use-rdb-preamble yes # Store base snapshot as RDB + incremental AOF
How They Work Together:
graph TD
Write[Write Command] --> Memory[Update Memory]
Memory --> AOF[Append to AOF]
Timer[Periodic Timer] --> RDB[Create RDB Snapshot]
RDB --> Disk1[dump.rdb]
AOF --> Disk2[appendonly.aof]
Memory -.Source of Truth.-> Both[Both derive from memory]
Restart Priority:
flowchart TD
Start[Redis Starts] --> Check{AOF enabled?}
Check -->|Yes| AOFExists{AOF file exists?}
Check -->|No| UseRDB[Load dump.rdb]
AOFExists -->|Yes| LoadAOF[Load AOF<br/>Higher priority]
AOFExists -->|No| LoadRDB[Load dump.rdb]
LoadAOF --> Validate[Validate data]
LoadRDB --> Validate
UseRDB --> Validate
Why AOF Has Priority?
AOF is more up-to-date (potentially has writes after last RDB snapshot)
Hybrid Persistence Strategy (Best Practice):
# Enable both
save 900 1
appendonly yes
appendfsync everysec
# Use RDB-format base in AOF rewrites
aof-use-rdb-preamble yes
Result:
RDB Versions:
| Version | Redis Version | Key Changes |
|---|---|---|
| 1 | 1.0 | Initial format |
| 5 | 2.6 | Millisecond expiry |
| 6 | 2.8 | LZF compression improvements |
| 7 | 3.2 | Quick list encoding |
| 9 | 5.0 | Stream data type, 64-bit lengths |
Compatibility:
AOF Format:
RDB Performance:
graph LR
Fork[fork] -->|Instant| Child[Child Process]
Child -->|Sequential Write| Disk[Disk I/O]
Disk -->|~1-10 seconds| Complete[Complete]
Fork -.Copy-on-Write.-> Memory[Memory Overhead]
Typical RDB Save Time:
| Dataset Size | Approximate Time | Memory Overhead |
|---|---|---|
| 1 GB | 1-2 seconds | < 50 MB (COW) |
| 10 GB | 10-20 seconds | < 500 MB (COW) |
| 100 GB | 100-200 seconds | < 5 GB (COW) |
AOF Performance:
Comparison:
Recovery Time:
RDB: 10 GB → ~30 seconds (sequential read + hash table rebuild)
AOF: 10 GB → ~5 minutes (parse + execute millions of commands)
Durability:
RDB: Last snapshot (potentially minutes/hours old)
AOF: Last fsync (potentially 0-1 seconds old)
Linux Kernel Optimizations Redis Uses:
fork() + Copy-on-Write
mmap() for file I/O
fallocate() for AOF
fdatasync() vs fsync()
fdatasync() = faster, skips metadata updatesWrite Amplification:
AOF Write Amplification = How many disk writes per logical write?
appendfsync always: 1 fsync per write = High durability, high cost
appendfsync everysec: 1 fsync per second = Good balance
appendfsync no: OS buffering = Low cost, low durability
RDB:
✅ Fast restart
✅ Compact file
✅ No runtime overhead
✅ Perfect for backups
❌ Data loss risk (between snapshots)
❌ Memory spike during fork
❌ Long save time for huge datasets
❌ No incremental updates
AOF:
✅ High durability (per-write)
✅ Human-readable
✅ Continuous updates
✅ Automatic compaction (rewrite)
❌ Larger files
❌ Slower recovery
❌ Slight write latency overhead
❌ Rewrite can be expensive
Hybrid (Recommended):
✅ Best of both worlds
✅ Fast recovery (RDB base)
✅ Minimal data loss (AOF incremental)
⚠️ Slightly more complex configuration
⚠️ Uses more disk space
Mental Model:
┌─────────────────────────────────────────────┐
│ In-Memory Data Structures │
│ (Source of Truth) │
└─────────┬─────────────────────┬─────────────┘
│ │
▼ ▼
┌──────────┐ ┌──────────┐
│ RDB │ │ AOF │
│ Snapshot │ │ Command │
│ File │ │ Log │
└──────────┘ └──────────┘
No Indexes No Indexes
Sequential I/O Append-Only
Fast Recovery High Durability
Replication means copying data from one Redis to others.
graph TD
Master --> Replica1
Master --> Replica2
Why Replicas Matter:
Key correction:
❗ Master is NOT always the only interface to the application
That's a design choice, not a Redis rule.
Logical model:
graph TD
App --> Master
Master --> Replica1
Master --> Replica2
How data flows:
There is no separate sync daemon.
Replication is done by:
Replication phases:
Phase 1: Full sync (initial)
sequenceDiagram
Replica->>Master: SYNC
Master->>Replica: RDB snapshot
Master->>Replica: buffered writes
Phase 2: Continuous replication
sequenceDiagram
Client->>Master: Write
Master->>Replica: Propagate command
Can applications talk to replicas?
Yes — but carefully.
| Operation | Safe on Replica? |
|---|---|
| Reads | ✅ |
| Writes | ❌ |
| Strong consistency | ❌ |
Master and Replica Relationship:
Replacement happens via failover, not magically.
graph LR
Master --> Replica
Replica -->|Failover| NewMaster
Redis Sentinel watches Redis instances.
What Sentinel Does:
Sentinel Architecture:
graph TD
Sentinel1 --> Master
Sentinel2 --> Master
Sentinel3 --> Master
Master --> Replica
Sentinels vote to avoid false failover.
Why 3 Sentinels?
Quorum and fault tolerance
Sentinel is NOT Redis:
Why multiple Sentinels?
If only 1 Sentinel:
If 2 Sentinels:
If 3 Sentinels:
Odd number rule:
graph TD
Sentinel1 --> Vote
Sentinel2 --> Vote
Sentinel3 --> Vote
Majority = 2 out of 3 Failover only happens if quorum is reached
Sentinel failure tolerance:
| Sentinels | Can lose |
|---|---|
| 3 | 1 |
| 5 | 2 |
Automatic Failover Flow:
sequenceDiagram
Sentinel->>Master: Ping
Master--x Sentinel: No response
Sentinel->>Replica: Promote
Replica->>Sentinel: I am master
Terminology:
| Term | Meaning |
|---|---|
| Node | A machine / VM / container |
| Redis instance | One Redis process |
| Master | A Redis instance that owns slots |
| Replica | A Redis instance replicating a master |
Important:
🔴 One Redis instance = one role (master OR replica) 🔴 One node can host multiple Redis instances (rare in prod)
Correct architecture:
graph TD
Node1 --> MasterA
Node2 --> ReplicaA1
Node3 --> ReplicaA2
Best practice:
One Redis instance per node for:
When a single Redis node is not enough, you use Redis Cluster.
What Is Sharding?
Splitting data across multiple Redis nodes.
graph TD
App --> Slot1[Redis Node A]
App --> Slot2[Redis Node B]
App --> Slot3[Redis Node C]
Correct Redis Cluster Layout:
A Redis Cluster has multiple masters, each responsible for a subset of slots
graph TD
Master1 --> Replica1
Master2 --> Replica2
Master3 --> Replica3
Each master:
Why multiple masters are required:
| Reason | Explanation |
|---|---|
| Scalability | One master cannot hold entire keyspace |
| Throughput | Writes distributed across masters |
| Availability | Failure isolates only part of data |
Correct definition:
Sharding partitions the KEYSPACE, not snapshots
Sharding:
Correct sharding model:
graph TD
Keyspace --> Slot0-5000 --> MasterA
Keyspace --> Slot5001-10000 --> MasterB
Keyspace --> Slot10001-16383 --> MasterC
Replica relationship:
graph TD
MasterA --> ReplicaA1
MasterB --> ReplicaB1
❗ Replicas do NOT participate in sharding ❗ Replicas mirror exactly one master's shard
Snapshot relationship:
graph TD
MasterA --> RDB_A
MasterB --> RDB_B
ReplicaA1 --> RDB_A_replica
📌 Each instance snapshots only its own shard
Important Clarifications:
Shard Definition:
Shard = Master + its assigned slots (+ replicas)
One master owns one or more slot ranges, which together form a shard.
A master can own multiple non-contiguous slot ranges, but conceptually they are treated as one shard.
graph TD
MasterA --> Slot1_100
MasterA --> Slot300_500
Critical Understanding:
Redis has NO "cluster main controller" Redis Cluster is fully decentralized
Every Redis node:
What Redis Cluster Solves:
| Problem | Solved? |
|---|---|
| Horizontal scaling | ✅ |
| Node failure | ✅ |
| Cross-slot transactions | ❌ (limited) |
Big Picture:
graph TD
App --> RedisCluster
RedisCluster --> Master1
RedisCluster --> Master2
Master1 --> Replica1
Master2 --> Replica2
Sentinel --> RedisCluster
RedisCluster --> Disk
Before Redis Cluster (pre-3.0), people did:
node = hash(key) % number_of_nodes
Problems with that approach:
Redis Cluster Solution: Hash Slots
Redis introduces an indirection layer:
Key → Hash → Slot → Master Node
flowchart LR
Key --> Hash
Hash --> Slot[0..16383]
Slot --> Master
Fixed number of slots:
This is the foundation of Redis Cluster scalability.
Step 1: Extract hash tag (if any)
key = user:{42}:profile
hash_input = 42
If {} exists → only content inside is hashed
If not → entire key is hashed
Step 2: CRC16
slot = CRC16(hash_input) % 16384
Why CRC16?
Step 3: Slot → Master lookup
flowchart LR
Slot -->|Slot table| MasterA
Important:
🔹 Only the key string matters for slot calculation 🔹 The value is irrelevant for routing
Is the Redis slot just an integer calculated by CRC16?
✅ Yes — in modern Redis Cluster:
slot = CRC16(key) % 16384
0 – 16383Important:
A key always hashes to the same slot forever. Slot migration only changes which node owns that slot, not how the slot is computed.
Example: 3-Master Cluster
Total slots = 16384
| Master | Slots |
|---|---|
| Master A | 0 – 5460 |
| Master B | 5461 – 10922 |
| Master C | 10923 – 16383 |
This is:
📌 These ranges are not required to be contiguous.
Can slots in the same shard be non-contiguous?
✅ Yes — absolutely allowed
Redis Cluster rule:
A master can own any set of slots, contiguous or not
Example:
graph TD
Slot1_100 --> MasterA
Slot201_300 --> MasterA
Slot101_200 --> MasterB
✔ Redis does not require continuity ✔ Continuity is usually chosen for operational simplicity
Why Redis allows non-contiguous slots:
Why operators prefer contiguous slots:
Who assigns and manages slot ranges?
❌ Not applications ❌ Not hashing logic ❌ Not modulo arithmetic
✅ Redis Cluster itself (with admin tools)
Slot assignment happens via:
redis-cli --cluster createredis-cli --cluster reshardHow slot assignment works internally:
Each master maintains:
slots[0..16383] → owned / not owned
Cluster-wide metadata looks like:
flowchart TB
SlotMap["Cluster Slot Map"]
SlotMap -->|0-5460| MasterA
SlotMap -->|5461-10922| MasterB
SlotMap -->|10923-16383| MasterC
Clients cache this map.
Where Is Slot Ownership Stored?
Each Redis node keeps:
Stored in:
nodes.conf
This file is:
Are Slots Physical Things?
❌ No — slots are NOT physical objects
A slot is:
There is:
What is physical?
| Physical | Description |
|---|---|
| Redis instance | OS process |
| Memory | Actual key-value storage |
| Keys | Stored in RAM |
| RDB / AOF | Persistence files |
| TCP connections | Client & cluster bus |
Slots are just numbers used to decide which keys belong where.
What Does "Slot Migration" Actually Mean?
Slot migration is a two-part operation:
1. Move ownership of a slot
2. Move keys belonging to that slot
Step-by-Step Slot Migration:
Let's migrate slot 8000 from Master A → Master B
Step 1: Mark slot as "migrating" and "importing"
sequenceDiagram
participant A as Master A
participant B as Master B
participant C as Clients
A->>A: slot 8000 = MIGRATING to B
B->>B: slot 8000 = IMPORTING from A
Step 2: Move keys belonging to the slot
flowchart LR
Key1 -->|DUMP| A
A -->|RESTORE| B
Redis internally:
This is key-level movement, not memory blocks.
What actually moves:
KEY + VALUE + TTL + TYPE
Internally via:
MIGRATE host port key db timeout
Step 3: Finalize slot ownership
sequenceDiagram
A->>A: remove slot 8000
B->>B: own slot 8000
A->>Cluster: update slot map
B->>Cluster: update slot map
Now:
slot 8000 → Master B
Important Corrections:
There is no structure like:
{
"slot": 5793,
"data": {...}
}
Who Coordinates Slot Migration?
❌ Not Sentinel ❌ Not a central controller ❌ Not ZooKeeper ❌ Not etcd
✅ Redis nodes coordinate themselves
Redis Cluster is fully decentralized.
Coordination happens via:
client_port + 10000)Who initiates migration?
Usually:
redis-cli --cluster reshardBut once initiated: ➡ Redis nodes do everything themselves
How Do Clients Survive During Migration?
This is where ASK / MOVED comes in.
During migration:
sequenceDiagram
Client->>MasterA: GET key_in_8000
MasterA-->>Client: ASK MasterB
Client->>MasterB: ASKING + GET key
After migration:
sequenceDiagram
Client->>AnyNode: GET key
Node-->>Client: MOVED slot 8000 MasterB
Problem:
Multi-key operations must be on same slot
MGET user:1 user:2 ❌ (likely different slots)
Solution: Hash tags
user:{42}:name
user:{42}:email
Both hash on 42
flowchart LR
user42name --> SlotX
user42email --> SlotX
SlotX --> MasterA
✔ Multi-key ops work ✔ Transactions work ✔ Lua scripts work
Hash Tag Rule:
Only substring inside { } is hashed
Best practice:
{US})What you CANNOT customize:
| Item | Customizable |
|---|---|
| Number of slots | ❌ Fixed at 16384 |
| Hash algorithm | ❌ CRC16 is fixed |
| Slot computation | ❌ Fixed |
| Slot redirection behavior | ❌ Fixed |
Redis must behave identically across all clients.
What you CAN influence:
1. Key design
This is your only real control
user:{id}:*
order:{id}:*
session:{id}
2. Slot distribution
Via:
CLUSTER ADDSLOTSCLUSTER DELSLOTSredis-cli --cluster reshard3. Read routing
Example: Writing and Reading Data
Data: {user: "whereq"}
Write Flow:
Application stores:
SET user:whereq '{"user":"whereq"}'
Redis sees:
KEY = "user:whereq"
VALUE = '{"user":"whereq"}'
Slot calculation:
slot = CRC16("user:whereq") % 16384 = 5793
Cluster metadata contains:
MasterA owns slots: 4096–8191 (example)
→ slot 5793 ∈ MasterA
Data stored in MasterA's RAM (not directly in RDB)
Read Flow:
Application sends:
GET user:whereq
Client hashes:
CRC16("user:whereq") → slot 5793
Client (or Redis redirect) routes to MasterA
MasterA returns:
{"user":"whereq"}
After Migration:
If slot 5793 migrates to MasterB:
Key Principle:
Slots route data; nodes store data
What happens when you add a new master node?
Step 1: New master joins the cluster
redis-cli --cluster add-node NewMaster ExistingMaster
Result:
flowchart LR
A[Master A<br/>Slots 0–5460]
B[Master B<br/>Slots 5461–10922]
C[Master C<br/>Slots 10923–16383]
D[New Master D<br/>Slots: none]
A --- B --- C --- D
📌 Redis will never auto-assign slots. This is intentional.
Why Redis does NOT auto-rebalance slots:
Auto-rebalance would be dangerous:
| Risk | Explanation |
|---|---|
| Massive data movement | Millions/billions of keys |
| Latency spikes | MIGRATE blocks CPU |
| Unpredictable IO | Disk + network storms |
| App impact | Cascading failures |
Redis follows the principle:
Data movement must be explicit, observable, and reversible
Step 2: Resharding (manual or scripted)
You explicitly run:
redis-cli --cluster reshard
Or specify:
redis-cli --cluster reshard \
--cluster-from <old-master-id> \
--cluster-to <new-master-id> \
--cluster-slots 4096
This:
Example: 3 Masters → 4 Masters
Before:
| Master | Slots |
|---|---|
| A | 0–5460 |
| B | 5461–10922 |
| C | 10923–16383 |
Target distribution (example):
| Master | Slots |
|---|---|
| A | 0–4095 |
| B | 4096–8191 |
| C | 8192–12287 |
| D | 12288–16383 |
During migration:
sequenceDiagram
participant Client
participant A as MasterA
participant D as MasterD
Client->>A: GET key(slot=13000)
A-->>Client: ASK 13000 D
Client->>D: GET key
D-->>Client: value
Redis uses ASK redirects during migration.
What happens to RDB/AOF after migration?
Persistence follows memory ownership:
No syncing between RDBs ever happens.
Adding replicas:
You normally add replicas AFTER slot resharding:
redis-cli --cluster add-node ReplicaOfD NewMasterD --cluster-slave
Replicas automatically sync data.
Does application need to change anything?
❌ No changes required
Clients:
MOVED / ASKThis is why slots exist.
Redis masters must keep data in RAM:
✅ Yes — Redis is fundamentally an in-memory database
Redis guarantees:
What if data exceeds RAM?
Redis gives you explicit strategies, not silent paging.
Option 1: Eviction (most common)
maxmemory 32gb
maxmemory-policy allkeys-lru
Policies include:
allkeys-lruvolatile-lruallkeys-randomnoeviction (write fails)📌 Redis will never swap to disk automatically.
Option 2: Redis Cluster (scale horizontally)
Instead of bigger RAM:
Add masters
Reshard slots
Distribute memory
Option 3: Redis on Flash (specialized)
Enterprise / Redis Stack only:
Slot layout:
Key naming:
Bad:
abc123:user:1
xyz999:user:2
Good:
user:1
user:2
Hash tags:
Use only when required:
✔ Sessions ✔ User-scoped objects ❌ Global counters ❌ Time series
Avoid hotspots:
Bad:
counter:{global}
Better:
counter:{region}:{id}
When Adding Masters:
✔ Add master first (empty) ✔ Reshard gradually ✔ Move slots from multiple masters ✔ Monitor latency during migration ❌ Don't auto-rebalance ❌ Don't migrate too many slots at once
KEY --(CRC16 % 16384)--> SLOT --(cluster map)--> MASTER --> REPLICAS
KEY
↓
CRC16 hash
↓
SLOT (0–16383)
↓
Cluster Metadata
↓
Owning Master
↓
RAM (source of truth)
↓
RDB / AOF (recovery only)
| Topic | Core Truth |
|---|---|
| RDB | Disk snapshot of memory |
| AOF | Write command log |
| Both together | Memory is source of truth |
| Replication | Master streams commands |
| Sentinel | External HA coordinator |
| Sharding | Data split by slots |
| Slots | Logical routing layer |
Redis Cluster shards the keyspace using hash slots, assigns slots to masters, replicates data to replicas, and persists each instance independently using RDB/AOF — all without a central controller.
When designing with Redis, always ask:
Redis is powerful because it is opinionated.
If you:
Redis becomes one of the most reliable and elegant tools in distributed systems.
1. Cache Pattern:
Application → Check Redis → Hit? Return : Fetch from DB → Store in Redis → Return
2. Counter Pattern:
INCR page_views
INCRBY user:123:credits 50
3. Distributed Lock:
SET lock:resource "token" NX PX 30000 # Atomic lock with 30s TTL
4. Rate Limiting:
INCR rate:user:123:20260108 # Count requests per day
EXPIRE rate:user:123:20260108 86400
5. Session Store:
SETEX session:abc123 3600 '{"user_id":123,"role":"admin"}'
| Operation | Command | Example |
|---|---|---|
| Set value | SET key value [EX seconds] | SET user:1 "Alice" EX 3600 |
| Get value | GET key | GET user:1 |
| Increment | INCR key | INCR counter |
| Hash set | HSET key field value | HSET user:1 name "Alice" |
| Hash get | HGET key field | HGET user:1 name |
| List push | LPUSH key value | LPUSH queue:jobs task1 |
| List pop | RPOP key | RPOP queue:jobs |
| Set add | SADD key member | SADD tags:post:1 "redis" |
| Sorted set add | ZADD key score member | ZADD leaderboard 100 "player1" |
| Check existence | EXISTS key | EXISTS user:1 |
| Delete | DEL key | DEL user:1 |
| Set expiry | EXPIRE key seconds | EXPIRE session:abc 3600 |
| Get TTL | TTL key | TTL session:abc |
Persistence:
# RDB Snapshots
save 900 1 # Save if 1+ keys changed in 900s
save 300 10 # Save if 10+ keys changed in 300s
save 60 10000 # Save if 10000+ keys changed in 60s
rdbcompression yes
rdbchecksum yes
# AOF
appendonly yes
appendfsync everysec # Options: always | everysec | no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-use-rdb-preamble yes # Hybrid mode
# Hybrid (Recommended)
save 900 1
appendonly yes
appendfsync everysec
aof-use-rdb-preamble yes
Memory Management:
maxmemory 2gb
maxmemory-policy allkeys-lru # Options:
# allkeys-lru - Evict any key, LRU
# volatile-lru - Evict keys with TTL, LRU
# allkeys-random - Evict any key, random
# volatile-random - Evict keys with TTL, random
# volatile-ttl - Evict keys with shortest TTL
# noeviction - Return errors when memory limit reached
Replication:
# On replica
replicaof <master-ip> <master-port>
replica-read-only yes
Cluster:
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 15000
| Problem | Possible Causes | Solutions |
|---|---|---|
| Slow queries | Large keys, blocking commands | Use SCAN instead of KEYS, avoid SMEMBERS on large sets |
| Memory spikes | RDB fork during save | Increase memory, schedule saves during low traffic |
| Data loss | No persistence enabled | Enable AOF with appendfsync everysec |
| Connection timeouts | Network issues, maxclients limit | Check maxclients, network latency |
| Replication lag | Master overloaded, slow network | Optimize write load, check network bandwidth |
| Cluster MOVED errors | Slot migration in progress | Normal during resharding, wait or retry |
| OOM errors | Memory limit reached | Check maxmemory-policy, add more nodes |
DO:
MULTI/EXEC) for atomicityINFO command and Redis monitoring toolsDON'T:
KEYS * in production (use SCAN instead)BLPOP with long timeout) without care| Version | Key Features |
|---|---|
| 3.0 | Redis Cluster introduced |
| 3.2 | GEO commands, Quick List |
| 4.0 | Modules system, lazy freeing |
| 5.0 | Streams data type, new sorted set commands |
| 6.0 | SSL/TLS, ACLs, RESP3, threaded I/O |
| 6.2 | Tracking, client-side caching |
| 7.0 | Functions, sharded pub/sub, ACL improvements, Multi-part AOF |
Key Metrics to Watch:
# Memory usage
INFO memory
- used_memory
- used_memory_rss
- mem_fragmentation_ratio
# Performance
INFO stats
- instantaneous_ops_per_sec
- total_commands_processed
- keyspace_hits / keyspace_misses (hit ratio)
# Persistence
INFO persistence
- rdb_last_save_time
- rdb_changes_since_last_save
- aof_current_size
# Replication
INFO replication
- role (master/replica)
- connected_slaves
- master_repl_offset
# Cluster
CLUSTER INFO
- cluster_state
- cluster_slots_assigned
- cluster_slots_ok
Sizing Guidelines:
| Use Case | Recommended Setup | Example |
|---|---|---|
| Small cache | Single master | 1-4 GB RAM |
| Production cache | Master + 1-2 replicas + Sentinel | 8-32 GB RAM per node |
| Large dataset | 3+ master cluster, 1 replica each | 32-256 GB RAM per node |
| High availability | Sentinel (3+ instances) or Cluster | Odd number of sentinels |
Network Requirements:
Important:
Correct Understanding:
Important:
RDB = raw key/value memory dump
Cluster metadata = routing truth
Summary Answers:
| Question | Correct Answer |
|---|---|
| Snapshot after migration? | ❌ No |
| Temporary RDB inconsistency? | ✅ Yes, expected |
| Crash before snapshot safe? | ✅ Yes |
| Slot logic during RDB load? | Metadata-driven |
| Hash during load? | Only for validation |
| Data always in RAM? | ✅ Yes |
| Exceed RAM? | Evict or scale |
Does the application care which master or replica it talks to?
✅ Conceptually correct
Writes:
MOVED redirect✔ Application logic remains unaware
Reads:
⚠️ Replica reads may be eventually consistent
When a new master is added later, Redis does NOT automatically rebalance slots. Slot redistribution (resharding) is an explicit, controlled operation.
Key Principle:
Adding a master does NOT change hashing. It only changes slot ownership — explicitly.
KEY → SLOT (fixed) → MASTER (mutable)