SlideShare a Scribd company logo
1 of 37
Download to read offline
深入了解Redis
               宋传胜
   Chuansheng.song@langtaojin.com
简单介绍
•   文本协议 memcached类似
•   KEY 可打印字符
•   VALUE支持的类型
    –   STRINGS
    –   LIST
    –   SET
    –   SORTED SET
    –   HASH
•   高性能 (100k+ SET/80k+ GET)/s
•   序列化支持
•   主从同步支持
•   客户端自己实现sharding
基本数据结构
• RedisObject (Redis.h)
  – key/value对象的基础
KEY
• 基本命令
 –   get/set/exists O(1)
 –   setnx/randomkey/rename/renamenx/dbsize/type
 –   keys pattern
 –   del 一次删除多个key
 –   expire key seconds 指定过期时间
 –   ttl key 剩余过期时间
 –   select db
 –   move key newdb
 –   flush db/flushall
EXPIRE AND LRU CACHE
• Volatile Key (Db.c)
   – When the key is set to a new value using the SET command, or when a
     key is destroyed via DEL, the timeout is removed from the key.
• Enhanced Lazy Expiration algorithm
   – Redis does not constantly monitor keys that are going to be expired.
     Keys are expired simply when some client tries to access a key, and the
     key is found to be timed out.
• Work as LRU cache (memcached)
   – Maxmemory/maxmemory-policy
   – When memory limit was already reached, server will remove some old
     data deleting a volatile key, even if the key is still far from expiring
     automatically.
   – Random get keys, delete by lru rules.
基本数据结构
• Hashtable(Dict.c)
基本Value数据结构
• STRING(sds.c)



  – set/setnx/get/getset/mget/mset/msetnx
    •nx – not exists
    •m -- multiple
  – incr/decr/incrby/decrby
  – getrange/append
基本Value数据结构
• LIST (T_list.c)
  – REDIS_LIST 类型, 如果其 entry 小于配置值: list-
    max-ziplist-entries 或 value字符串的长度小于 list-
    max-ziplist-value,使用ziplist数据结构,否则使用
    标准的Doubly linked list
  – (l/r)push/(l/r)pop/llen O(1)
  – b(l/r)pop支持阻塞等待,避免了轮循
  – lrange/ltrim/lrem/lset/rpoplpush
LIST
• Ziplist (Ziplist.c)
   – O(mem_size) add/delete
   – list-max-ziplist-entries (default: 1024)
   – list-max-ziplist-value (default: 32)
LIST
• Ziplist (continue)
  – zlentry (unencode structure)
LIST
• Doubly linked list (Adlist.c)
基本Value数据结构
• SET (T_set.c) – hashtable + Intset
  – String的无序集合
  – sadd/srem/sismember/scard O(1)
  – spop/srandmember
  – smove
  – smembers
  – sinter(store) O(C)
  – sunion(store)/sdiff(store) O(N)
INTSET
• INTSET (Intset.c)
  – 都是整型时数据结构退化成排序的intset
  – Good fit for size up to 20-50K
  – set-max-intset-entries (default: 4096)
  – O(n) search
  – O(log(n) + mem_size) add/del
基本Value数据结构
• SORTED SET(T_zset.c) hashtable + skip list
  – 按照key的score排序
  – zadd/zrem/zrank/zrevrank O(log(n))
  – zcard O(1)
  – zincrby 修改score
  – zrange/zrevrange/zrangebyscore/zscore
    O(log(N)+M)
  – zremrangebyrank/zremrangebyscore O(log(N)+M)
SKIP LIST
• Skip List (T_zset.c)
  – skip list  redblack tree  AVL tree (more
    balanced)
  – lockfree/concurrency
基本Value数据结构
• HASH (T_hash.c) zipmap + hashtable
  – 如果其 entry 小于配置值: hash-max-zipmap-
    entries 或 value字符串的长度小于 hash-max-
    zipmap-value ,使用zipmap数据结构
  – Value只能是string类型
  – hset/hget/hexists O(1)
  – hmset/hexists/hincrby/hdel/hlen
  – hmget O(N)
  – hkeys/hvals/hgetall
ZIPMAP
• zipmap
  – O(n) search O(mem_size) add/delete
内存管理
• zmalloc.c
  – 简单内存管理
  – 支持tcmalloc USE_TCMALLOC
  – zmalloc(size_t size)  malloc(size+PREFIX_SIZE)
  – fragmentation_ratio =
    zmalloc_get_rss/zmalloc_used_memory
PUBSUB
• SUBSCRIBE/UNSUBSCRIBE/PSUBSCRIBE/PU
  NSUBSCRIBE
• PUBLISH
• 连接断开后信息丢失
TRANSACTION
• MULTI (不支持CAS)
  –   MULTI
  –   SET foo 0
  –   INCR foo
  –   EXEC
• WATCH (Check and Set )
  –   WATCH theKey
  –   v = GET theKey
  –   MULTI
  –   SET theKey v+1
  –   EXEC
• UNWATCH/DISCARD
提升网络效率、减少内存占用
• MGET/MSET
• PIPELINING
  – $ (echo -en "PINGrnPINGrnPINGrn"; sleep 1) | nc localhost 6379
    +PONG
    +PONG
    +PONG
• 压缩数据,触发ziplist/zipmap/intset
• 使用 GETBIT/SETBIT/GETRANGE/SETRANGE 压缩多
  个属性
• 使用HASH
• 使用32位redis + sharding
Client Side Sharding
• Consistent Hashing
  – Redis Sharding at Craigslist
网络层
• Anet.c
• 事件驱动 ae.c 
  ae_epoll.c/ae_kqueue.c/ae_select.c
• 非阻塞单线程
• 支持timer
• 在VM场景下有可能多线程
网络层对比
• memcached/varnish/scribed的网络IO模型
 – 多线程,非阻塞IO复用
网络层对比
• lighttpd/nginx的网络IO模型
  – 多进程,单线程非阻塞IO
• apache的MPM模型
  – 多进程prefork,多线程
PERSISTENCE
• SNAPSHOT (Rdb.c)
• 可以关闭
• BGSAVE or save xseconds ychanges
• fork
   – parent - disable rehash, periodic check child on serverCron
     (100ms)
   – child - copy-on-write, save whole db, exit
• Need to turn on the overcommit_memory setting if you want to
  deal with a dataset more than 1/2 of RAM, which contradicts
  our habit to battle against OOM killer as a database
  administrator.
Append Only File
• Append to log file on every change (Aof.c)
• fsync() policy
   – always/os decide/every second
   – Always with battery backed raid controllers
   – Every second by default (innodb_flush_log_at_trx_commit=2 in
     mysql)
• Compact aof file
   – BGREWRITEAOF/REWRITEAOF
• Fork
   – child – write new aof in temporary file
   – parent – write change in both old aof file and memory buffer,
      append memory buffer to temporary file when child done, rename
      temporary to aof file.
• redis-check-aof --fix <filename>
Replication
• 支持多级同步 (Replication.c)
Replication
• 支持AUTH
• 起动过程
 – slave
   •SYNC
   •Wait
 – master
   •Issue a BGSAVE, or wait if BGSAVE in progress
Virtual Memory
• Redis VM is now deprecated. Redis 2.4 will be the
  latest Redis version featuring Virtual Memory!
• Why not OS VM
  – Varnish What’s wrong with 1975 programming ?
  – Redis What’s wrong with 2006 programming?
     • Blocking when page fault
     • 4k pages granularity
     • Optimal disk persistent format, 10:1
• Keys can’t be swapped out
• vm is read only when BGSAVE/BGREWRITEAOF in
  progress
• vm-max-memory/vm-pages/vm-page-size
Virtual Memory
• Blocking vm
• Threaded vm
  – Before command is executed, check the value
  – Block client and load, then signal by unix pipe
  – Continue
• Recommended filesystem – ext3 or any other file
  system with good support for sparse files
• redis-stat vmstat
• InnoDB innodb_buffer_pool_size – swap and
  durability
Future
• Diskstore (diskstore.c dscache.c)
  – All data on disk, B+tree in future
  – Swaps data into memory when needed
• Leveldb/InnoDB Change Buffer
  – LSM tree
其他相关的代码
• quicksort (pqsort.c)
• lzf compression (lzf_c.c lzf_d.c)
• Sha1 (Sha1.c)
• Syncio.c
• SORT (Sort.c)
线上Redis集群情况
• 16台服务器,兼做后端服务器
• 45个Redis Instance进程,三组服务
• 总内存 77G VIRT,71G RSS
• 使用了主从复制、snapshot持久化,VM支持
  ,客户端sharding、pipelining、hash优化
Redis应用
•   Resque – message queue of github (celery)
•   Nginx HttpRedis module
•   Redweb - Web administrative and query UI
•   A fast, fuzzy, full-text index using Redis
•   Redis-backed BloomFilter(s) in Ruby
•   Soulmate : Auto Complete with Redis
•   Rate limiting with Redis
•   Redis Sharding
•   Locking with setnx
•   A Collection Of Redis Use Cases
•   Radishapp Redis monitoring
•   openredis.com hosting service
Contribute to Redis
• Fork redis on github
  – https://github.com/antirez/redis
• Choose branch and commit your changes
• Create a pull request
• Wait for approval
  – https://github.com/antirez/redis/pulls
• My contribute for read only slave
  – https://github.com/antirez/redis/pull/47
Resources
• DOCUMENATION
 – http://redis.io/documentation
 – http://antirez.com/
 – http://groups.google.com/group/redis-db
• 本文参考的Redis资料集合
 – http://www.delicious.com/fakechris/redis
 – http://www.delicious.com/tag/redis

More Related Content

What's hot

A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redisZhichao Liang
 
Your 1st Ceph cluster
Your 1st Ceph clusterYour 1st Ceph cluster
Your 1st Ceph clusterMirantis
 
High Performance Weibo QCon Beijing 2011
High Performance Weibo QCon Beijing 2011High Performance Weibo QCon Beijing 2011
High Performance Weibo QCon Beijing 2011Tim Y
 
Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture ForumChristopher Spring
 
Ceph Performance and Sizing Guide
Ceph Performance and Sizing GuideCeph Performance and Sizing Guide
Ceph Performance and Sizing GuideJose De La Rosa
 
Development to Production with Sharded MongoDB Clusters
Development to Production with Sharded MongoDB ClustersDevelopment to Production with Sharded MongoDB Clusters
Development to Production with Sharded MongoDB ClustersSeveralnines
 
Caching solutions with Redis
Caching solutions   with RedisCaching solutions   with Redis
Caching solutions with RedisGeorge Platon
 
微博cache设计谈
微博cache设计谈微博cache设计谈
微博cache设计谈Tim Y
 
Red Hat Storage Server Administration Deep Dive
Red Hat Storage Server Administration Deep DiveRed Hat Storage Server Administration Deep Dive
Red Hat Storage Server Administration Deep DiveRed_Hat_Storage
 
Redis trouble shooting_eng
Redis trouble shooting_engRedis trouble shooting_eng
Redis trouble shooting_engDaeMyung Kang
 
Алексей Лесовский "Тюнинг Linux для баз данных. "
Алексей Лесовский "Тюнинг Linux для баз данных. "Алексей Лесовский "Тюнинг Linux для баз данных. "
Алексей Лесовский "Тюнинг Linux для баз данных. "Tanya Denisyuk
 

What's hot (20)

A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redis
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
redis basics
redis basicsredis basics
redis basics
 
Redis acc 2015_eng
Redis acc 2015_engRedis acc 2015_eng
Redis acc 2015_eng
 
Redis 101
Redis 101Redis 101
Redis 101
 
Redis ndc2013
Redis ndc2013Redis ndc2013
Redis ndc2013
 
Your 1st Ceph cluster
Your 1st Ceph clusterYour 1st Ceph cluster
Your 1st Ceph cluster
 
High Performance Weibo QCon Beijing 2011
High Performance Weibo QCon Beijing 2011High Performance Weibo QCon Beijing 2011
High Performance Weibo QCon Beijing 2011
 
Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture Forum
 
MyRocks Deep Dive
MyRocks Deep DiveMyRocks Deep Dive
MyRocks Deep Dive
 
Redis database
Redis databaseRedis database
Redis database
 
Ceph Performance and Sizing Guide
Ceph Performance and Sizing GuideCeph Performance and Sizing Guide
Ceph Performance and Sizing Guide
 
Development to Production with Sharded MongoDB Clusters
Development to Production with Sharded MongoDB ClustersDevelopment to Production with Sharded MongoDB Clusters
Development to Production with Sharded MongoDB Clusters
 
Caching solutions with Redis
Caching solutions   with RedisCaching solutions   with Redis
Caching solutions with Redis
 
微博cache设计谈
微博cache设计谈微博cache设计谈
微博cache设计谈
 
Redis introduction
Redis introductionRedis introduction
Redis introduction
 
Red Hat Storage Server Administration Deep Dive
Red Hat Storage Server Administration Deep DiveRed Hat Storage Server Administration Deep Dive
Red Hat Storage Server Administration Deep Dive
 
Bluestore
BluestoreBluestore
Bluestore
 
Redis trouble shooting_eng
Redis trouble shooting_engRedis trouble shooting_eng
Redis trouble shooting_eng
 
Алексей Лесовский "Тюнинг Linux для баз данных. "
Алексей Лесовский "Тюнинг Linux для баз данных. "Алексей Лесовский "Тюнинг Linux для баз данных. "
Алексей Лесовский "Тюнинг Linux для баз данных. "
 

Viewers also liked

新浪微博开放平台Redis实战
新浪微博开放平台Redis实战新浪微博开放平台Redis实战
新浪微博开放平台Redis实战mysqlops
 
高性能No sql数据库redis
高性能No sql数据库redis高性能No sql数据库redis
高性能No sql数据库redispaitoubing
 
redis 适用场景与实现
redis 适用场景与实现redis 适用场景与实现
redis 适用场景与实现iammutex
 
美团点评沙龙 飞行中换引擎--美团配送业务系统的架构演进之路
美团点评沙龙 飞行中换引擎--美团配送业务系统的架构演进之路美团点评沙龙 飞行中换引擎--美团配送业务系统的架构演进之路
美团点评沙龙 飞行中换引擎--美团配送业务系统的架构演进之路美团点评技术团队
 
Redis 常见使用模式分析
Redis 常见使用模式分析Redis 常见使用模式分析
Redis 常见使用模式分析vincent253
 
美团点评技术沙龙010-Redis Cluster运维实践
美团点评技术沙龙010-Redis Cluster运维实践美团点评技术沙龙010-Redis Cluster运维实践
美团点评技术沙龙010-Redis Cluster运维实践美团点评技术团队
 
Redis cluster
Redis clusterRedis cluster
Redis clusteriammutex
 
Everything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to askEverything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to askCarlos Abalde
 
Redis and Groovy and Grails - gr8conf 2011
Redis and Groovy and Grails - gr8conf 2011Redis and Groovy and Grails - gr8conf 2011
Redis and Groovy and Grails - gr8conf 2011Ted Naleid
 
美团点评沙龙012-初创电商的物流摸索
美团点评沙龙012-初创电商的物流摸索美团点评沙龙012-初创电商的物流摸索
美团点评沙龙012-初创电商的物流摸索美团点评技术团队
 
美团点评技术沙龙010-点评RDS系统介绍
美团点评技术沙龙010-点评RDS系统介绍美团点评技术沙龙010-点评RDS系统介绍
美团点评技术沙龙010-点评RDS系统介绍美团点评技术团队
 
美团点评技术沙龙09 - 美团配送智能调度实践
美团点评技术沙龙09 - 美团配送智能调度实践美团点评技术沙龙09 - 美团配送智能调度实践
美团点评技术沙龙09 - 美团配送智能调度实践美团点评技术团队
 
美团点评技术沙龙010-美团Atlas实践
美团点评技术沙龙010-美团Atlas实践美团点评技术沙龙010-美团Atlas实践
美团点评技术沙龙010-美团Atlas实践美团点评技术团队
 
美团点评技术沙龙011 - 移动app兼容性测试工具Spider
美团点评技术沙龙011 - 移动app兼容性测试工具Spider 美团点评技术沙龙011 - 移动app兼容性测试工具Spider
美团点评技术沙龙011 - 移动app兼容性测试工具Spider 美团点评技术团队
 
美团点评沙龙012-从零到千万量级的实时物流平台架构实践
美团点评沙龙012-从零到千万量级的实时物流平台架构实践美团点评沙龙012-从零到千万量级的实时物流平台架构实践
美团点评沙龙012-从零到千万量级的实时物流平台架构实践美团点评技术团队
 
美团点评技术沙龙011 - 团购系统流量和容量评估实践
美团点评技术沙龙011 - 团购系统流量和容量评估实践美团点评技术沙龙011 - 团购系统流量和容量评估实践
美团点评技术沙龙011 - 团购系统流量和容量评估实践美团点评技术团队
 
美团点评技术沙龙011 - 客户端用户体验数据量化
美团点评技术沙龙011 - 客户端用户体验数据量化美团点评技术沙龙011 - 客户端用户体验数据量化
美团点评技术沙龙011 - 客户端用户体验数据量化美团点评技术团队
 
Taste Rabbitmq
Taste RabbitmqTaste Rabbitmq
Taste Rabbitmqjeff kit
 

Viewers also liked (20)

Redis介绍
Redis介绍Redis介绍
Redis介绍
 
新浪微博开放平台Redis实战
新浪微博开放平台Redis实战新浪微博开放平台Redis实战
新浪微博开放平台Redis实战
 
高性能No sql数据库redis
高性能No sql数据库redis高性能No sql数据库redis
高性能No sql数据库redis
 
redis 适用场景与实现
redis 适用场景与实现redis 适用场景与实现
redis 适用场景与实现
 
美团点评沙龙 飞行中换引擎--美团配送业务系统的架构演进之路
美团点评沙龙 飞行中换引擎--美团配送业务系统的架构演进之路美团点评沙龙 飞行中换引擎--美团配送业务系统的架构演进之路
美团点评沙龙 飞行中换引擎--美团配送业务系统的架构演进之路
 
Redis 常见使用模式分析
Redis 常见使用模式分析Redis 常见使用模式分析
Redis 常见使用模式分析
 
美团点评技术沙龙010-Redis Cluster运维实践
美团点评技术沙龙010-Redis Cluster运维实践美团点评技术沙龙010-Redis Cluster运维实践
美团点评技术沙龙010-Redis Cluster运维实践
 
Redis cluster
Redis clusterRedis cluster
Redis cluster
 
Everything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to askEverything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to ask
 
Redis and Groovy and Grails - gr8conf 2011
Redis and Groovy and Grails - gr8conf 2011Redis and Groovy and Grails - gr8conf 2011
Redis and Groovy and Grails - gr8conf 2011
 
美团点评沙龙012-初创电商的物流摸索
美团点评沙龙012-初创电商的物流摸索美团点评沙龙012-初创电商的物流摸索
美团点评沙龙012-初创电商的物流摸索
 
美团点评技术沙龙010-点评RDS系统介绍
美团点评技术沙龙010-点评RDS系统介绍美团点评技术沙龙010-点评RDS系统介绍
美团点评技术沙龙010-点评RDS系统介绍
 
美团点评技术沙龙09 - 美团配送智能调度实践
美团点评技术沙龙09 - 美团配送智能调度实践美团点评技术沙龙09 - 美团配送智能调度实践
美团点评技术沙龙09 - 美团配送智能调度实践
 
美团点评技术沙龙010-美团Atlas实践
美团点评技术沙龙010-美团Atlas实践美团点评技术沙龙010-美团Atlas实践
美团点评技术沙龙010-美团Atlas实践
 
美团点评技术沙龙011 - 移动app兼容性测试工具Spider
美团点评技术沙龙011 - 移动app兼容性测试工具Spider 美团点评技术沙龙011 - 移动app兼容性测试工具Spider
美团点评技术沙龙011 - 移动app兼容性测试工具Spider
 
Redis Replication
Redis ReplicationRedis Replication
Redis Replication
 
美团点评沙龙012-从零到千万量级的实时物流平台架构实践
美团点评沙龙012-从零到千万量级的实时物流平台架构实践美团点评沙龙012-从零到千万量级的实时物流平台架构实践
美团点评沙龙012-从零到千万量级的实时物流平台架构实践
 
美团点评技术沙龙011 - 团购系统流量和容量评估实践
美团点评技术沙龙011 - 团购系统流量和容量评估实践美团点评技术沙龙011 - 团购系统流量和容量评估实践
美团点评技术沙龙011 - 团购系统流量和容量评估实践
 
美团点评技术沙龙011 - 客户端用户体验数据量化
美团点评技术沙龙011 - 客户端用户体验数据量化美团点评技术沙龙011 - 客户端用户体验数据量化
美团点评技术沙龙011 - 客户端用户体验数据量化
 
Taste Rabbitmq
Taste RabbitmqTaste Rabbitmq
Taste Rabbitmq
 

Similar to 深入了解Redis

Build an High-Performance and High-Durable Block Storage Service Based on Ceph
Build an High-Performance and High-Durable Block Storage Service Based on CephBuild an High-Performance and High-Durable Block Storage Service Based on Ceph
Build an High-Performance and High-Durable Block Storage Service Based on CephRongze Zhu
 
Bottom to Top Stack Optimization with LAMP
Bottom to Top Stack Optimization with LAMPBottom to Top Stack Optimization with LAMP
Bottom to Top Stack Optimization with LAMPkatzgrau
 
Bottom to Top Stack Optimization - CICON2011
Bottom to Top Stack Optimization - CICON2011Bottom to Top Stack Optimization - CICON2011
Bottom to Top Stack Optimization - CICON2011CodeIgniter Conference
 
Cassandra
CassandraCassandra
Cassandraexsuns
 
ZFS for Databases
ZFS for DatabasesZFS for Databases
ZFS for Databasesahl0003
 
Comparison of foss distributed storage
Comparison of foss distributed storageComparison of foss distributed storage
Comparison of foss distributed storageMarian Marinov
 
美团技术沙龙04 - 高性能服务器架构设计和调优
美团技术沙龙04 - 高性能服务器架构设计和调优美团技术沙龙04 - 高性能服务器架构设计和调优
美团技术沙龙04 - 高性能服务器架构设计和调优美团点评技术团队
 
Comparison of-foss-distributed-storage
Comparison of-foss-distributed-storageComparison of-foss-distributed-storage
Comparison of-foss-distributed-storageMarian Marinov
 
DEF CON 27- ITZIK KOTLER and AMIT KLEIN - gotta catch them all
DEF CON 27- ITZIK KOTLER and AMIT KLEIN - gotta catch them allDEF CON 27- ITZIK KOTLER and AMIT KLEIN - gotta catch them all
DEF CON 27- ITZIK KOTLER and AMIT KLEIN - gotta catch them allFelipe Prado
 
(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep DiveAmazon Web Services
 
What's new in Jewel and Beyond
What's new in Jewel and BeyondWhat's new in Jewel and Beyond
What's new in Jewel and BeyondSage Weil
 
Automatic tool for static analysis
Automatic tool for static analysisAutomatic tool for static analysis
Automatic tool for static analysisChong-Kuan Chen
 
Scaling web applications with cassandra presentation
Scaling web applications with cassandra presentationScaling web applications with cassandra presentation
Scaling web applications with cassandra presentationMurat Çakal
 
Community Update at OpenStack Summit Boston
Community Update at OpenStack Summit BostonCommunity Update at OpenStack Summit Boston
Community Update at OpenStack Summit BostonSage Weil
 
Talk about apache cassandra, TWJUG 2011
Talk about apache cassandra, TWJUG 2011Talk about apache cassandra, TWJUG 2011
Talk about apache cassandra, TWJUG 2011Boris Yen
 
Talk About Apache Cassandra
Talk About Apache CassandraTalk About Apache Cassandra
Talk About Apache CassandraJacky Chu
 
High Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouseHigh Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouseAltinity Ltd
 

Similar to 深入了解Redis (20)

Build an High-Performance and High-Durable Block Storage Service Based on Ceph
Build an High-Performance and High-Durable Block Storage Service Based on CephBuild an High-Performance and High-Durable Block Storage Service Based on Ceph
Build an High-Performance and High-Durable Block Storage Service Based on Ceph
 
Bottom to Top Stack Optimization with LAMP
Bottom to Top Stack Optimization with LAMPBottom to Top Stack Optimization with LAMP
Bottom to Top Stack Optimization with LAMP
 
Bottom to Top Stack Optimization - CICON2011
Bottom to Top Stack Optimization - CICON2011Bottom to Top Stack Optimization - CICON2011
Bottom to Top Stack Optimization - CICON2011
 
Strata - 03/31/2012
Strata - 03/31/2012Strata - 03/31/2012
Strata - 03/31/2012
 
Cassandra
CassandraCassandra
Cassandra
 
ZFS for Databases
ZFS for DatabasesZFS for Databases
ZFS for Databases
 
Comparison of foss distributed storage
Comparison of foss distributed storageComparison of foss distributed storage
Comparison of foss distributed storage
 
Scale 10x 01:22:12
Scale 10x 01:22:12Scale 10x 01:22:12
Scale 10x 01:22:12
 
美团技术沙龙04 - 高性能服务器架构设计和调优
美团技术沙龙04 - 高性能服务器架构设计和调优美团技术沙龙04 - 高性能服务器架构设计和调优
美团技术沙龙04 - 高性能服务器架构设计和调优
 
Comparison of-foss-distributed-storage
Comparison of-foss-distributed-storageComparison of-foss-distributed-storage
Comparison of-foss-distributed-storage
 
DEF CON 27- ITZIK KOTLER and AMIT KLEIN - gotta catch them all
DEF CON 27- ITZIK KOTLER and AMIT KLEIN - gotta catch them allDEF CON 27- ITZIK KOTLER and AMIT KLEIN - gotta catch them all
DEF CON 27- ITZIK KOTLER and AMIT KLEIN - gotta catch them all
 
(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive
 
What's new in Jewel and Beyond
What's new in Jewel and BeyondWhat's new in Jewel and Beyond
What's new in Jewel and Beyond
 
Automatic tool for static analysis
Automatic tool for static analysisAutomatic tool for static analysis
Automatic tool for static analysis
 
Scaling web applications with cassandra presentation
Scaling web applications with cassandra presentationScaling web applications with cassandra presentation
Scaling web applications with cassandra presentation
 
Community Update at OpenStack Summit Boston
Community Update at OpenStack Summit BostonCommunity Update at OpenStack Summit Boston
Community Update at OpenStack Summit Boston
 
Talk about apache cassandra, TWJUG 2011
Talk about apache cassandra, TWJUG 2011Talk about apache cassandra, TWJUG 2011
Talk about apache cassandra, TWJUG 2011
 
Talk About Apache Cassandra
Talk About Apache CassandraTalk About Apache Cassandra
Talk About Apache Cassandra
 
Hbase: an introduction
Hbase: an introductionHbase: an introduction
Hbase: an introduction
 
High Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouseHigh Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouse
 

More from iammutex

Scaling Instagram
Scaling InstagramScaling Instagram
Scaling Instagramiammutex
 
NoSQL误用和常见陷阱分析
NoSQL误用和常见陷阱分析NoSQL误用和常见陷阱分析
NoSQL误用和常见陷阱分析iammutex
 
MongoDB 在盛大大数据量下的应用
MongoDB 在盛大大数据量下的应用MongoDB 在盛大大数据量下的应用
MongoDB 在盛大大数据量下的应用iammutex
 
8 minute MongoDB tutorial slide
8 minute MongoDB tutorial slide8 minute MongoDB tutorial slide
8 minute MongoDB tutorial slideiammutex
 
Thoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency ModelsThoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency Modelsiammutex
 
Rethink db&tokudb调研测试报告
Rethink db&tokudb调研测试报告Rethink db&tokudb调研测试报告
Rethink db&tokudb调研测试报告iammutex
 
Introduction to couchdb
Introduction to couchdbIntroduction to couchdb
Introduction to couchdbiammutex
 
What every data programmer needs to know about disks
What every data programmer needs to know about disksWhat every data programmer needs to know about disks
What every data programmer needs to know about disksiammutex
 
redis运维之道
redis运维之道redis运维之道
redis运维之道iammutex
 
Realtime hadoopsigmod2011
Realtime hadoopsigmod2011Realtime hadoopsigmod2011
Realtime hadoopsigmod2011iammutex
 
[译]No sql生态系统
[译]No sql生态系统[译]No sql生态系统
[译]No sql生态系统iammutex
 
Couchdb + Membase = Couchbase
Couchdb + Membase = CouchbaseCouchdb + Membase = Couchbase
Couchdb + Membase = Couchbaseiammutex
 
Redis cluster
Redis clusterRedis cluster
Redis clusteriammutex
 
Hadoop introduction berlin buzzwords 2011
Hadoop introduction   berlin buzzwords 2011Hadoop introduction   berlin buzzwords 2011
Hadoop introduction berlin buzzwords 2011iammutex
 
Couchdb and me
Couchdb and meCouchdb and me
Couchdb and meiammutex
 
10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators  10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators iammutex
 
MongoDB开发应用实践
MongoDB开发应用实践MongoDB开发应用实践
MongoDB开发应用实践iammutex
 

More from iammutex (20)

Scaling Instagram
Scaling InstagramScaling Instagram
Scaling Instagram
 
NoSQL误用和常见陷阱分析
NoSQL误用和常见陷阱分析NoSQL误用和常见陷阱分析
NoSQL误用和常见陷阱分析
 
MongoDB 在盛大大数据量下的应用
MongoDB 在盛大大数据量下的应用MongoDB 在盛大大数据量下的应用
MongoDB 在盛大大数据量下的应用
 
8 minute MongoDB tutorial slide
8 minute MongoDB tutorial slide8 minute MongoDB tutorial slide
8 minute MongoDB tutorial slide
 
skip list
skip listskip list
skip list
 
Thoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency ModelsThoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency Models
 
Rethink db&tokudb调研测试报告
Rethink db&tokudb调研测试报告Rethink db&tokudb调研测试报告
Rethink db&tokudb调研测试报告
 
Introduction to couchdb
Introduction to couchdbIntroduction to couchdb
Introduction to couchdb
 
What every data programmer needs to know about disks
What every data programmer needs to know about disksWhat every data programmer needs to know about disks
What every data programmer needs to know about disks
 
Ooredis
OoredisOoredis
Ooredis
 
Ooredis
OoredisOoredis
Ooredis
 
redis运维之道
redis运维之道redis运维之道
redis运维之道
 
Realtime hadoopsigmod2011
Realtime hadoopsigmod2011Realtime hadoopsigmod2011
Realtime hadoopsigmod2011
 
[译]No sql生态系统
[译]No sql生态系统[译]No sql生态系统
[译]No sql生态系统
 
Couchdb + Membase = Couchbase
Couchdb + Membase = CouchbaseCouchdb + Membase = Couchbase
Couchdb + Membase = Couchbase
 
Redis cluster
Redis clusterRedis cluster
Redis cluster
 
Hadoop introduction berlin buzzwords 2011
Hadoop introduction   berlin buzzwords 2011Hadoop introduction   berlin buzzwords 2011
Hadoop introduction berlin buzzwords 2011
 
Couchdb and me
Couchdb and meCouchdb and me
Couchdb and me
 
10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators  10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators
 
MongoDB开发应用实践
MongoDB开发应用实践MongoDB开发应用实践
MongoDB开发应用实践
 

Recently uploaded

Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesZilliz
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfSeasiaInfotech2
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 

Recently uploaded (20)

Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector Databases
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdf
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 

深入了解Redis

  • 1. 深入了解Redis 宋传胜 Chuansheng.song@langtaojin.com
  • 2. 简单介绍 • 文本协议 memcached类似 • KEY 可打印字符 • VALUE支持的类型 – STRINGS – LIST – SET – SORTED SET – HASH • 高性能 (100k+ SET/80k+ GET)/s • 序列化支持 • 主从同步支持 • 客户端自己实现sharding
  • 3. 基本数据结构 • RedisObject (Redis.h) – key/value对象的基础
  • 4. KEY • 基本命令 – get/set/exists O(1) – setnx/randomkey/rename/renamenx/dbsize/type – keys pattern – del 一次删除多个key – expire key seconds 指定过期时间 – ttl key 剩余过期时间 – select db – move key newdb – flush db/flushall
  • 5. EXPIRE AND LRU CACHE • Volatile Key (Db.c) – When the key is set to a new value using the SET command, or when a key is destroyed via DEL, the timeout is removed from the key. • Enhanced Lazy Expiration algorithm – Redis does not constantly monitor keys that are going to be expired. Keys are expired simply when some client tries to access a key, and the key is found to be timed out. • Work as LRU cache (memcached) – Maxmemory/maxmemory-policy – When memory limit was already reached, server will remove some old data deleting a volatile key, even if the key is still far from expiring automatically. – Random get keys, delete by lru rules.
  • 7. 基本Value数据结构 • STRING(sds.c) – set/setnx/get/getset/mget/mset/msetnx •nx – not exists •m -- multiple – incr/decr/incrby/decrby – getrange/append
  • 8. 基本Value数据结构 • LIST (T_list.c) – REDIS_LIST 类型, 如果其 entry 小于配置值: list- max-ziplist-entries 或 value字符串的长度小于 list- max-ziplist-value,使用ziplist数据结构,否则使用 标准的Doubly linked list – (l/r)push/(l/r)pop/llen O(1) – b(l/r)pop支持阻塞等待,避免了轮循 – lrange/ltrim/lrem/lset/rpoplpush
  • 9. LIST • Ziplist (Ziplist.c) – O(mem_size) add/delete – list-max-ziplist-entries (default: 1024) – list-max-ziplist-value (default: 32)
  • 10. LIST • Ziplist (continue) – zlentry (unencode structure)
  • 11. LIST • Doubly linked list (Adlist.c)
  • 12. 基本Value数据结构 • SET (T_set.c) – hashtable + Intset – String的无序集合 – sadd/srem/sismember/scard O(1) – spop/srandmember – smove – smembers – sinter(store) O(C) – sunion(store)/sdiff(store) O(N)
  • 13. INTSET • INTSET (Intset.c) – 都是整型时数据结构退化成排序的intset – Good fit for size up to 20-50K – set-max-intset-entries (default: 4096) – O(n) search – O(log(n) + mem_size) add/del
  • 14. 基本Value数据结构 • SORTED SET(T_zset.c) hashtable + skip list – 按照key的score排序 – zadd/zrem/zrank/zrevrank O(log(n)) – zcard O(1) – zincrby 修改score – zrange/zrevrange/zrangebyscore/zscore O(log(N)+M) – zremrangebyrank/zremrangebyscore O(log(N)+M)
  • 15. SKIP LIST • Skip List (T_zset.c) – skip list  redblack tree  AVL tree (more balanced) – lockfree/concurrency
  • 16. 基本Value数据结构 • HASH (T_hash.c) zipmap + hashtable – 如果其 entry 小于配置值: hash-max-zipmap- entries 或 value字符串的长度小于 hash-max- zipmap-value ,使用zipmap数据结构 – Value只能是string类型 – hset/hget/hexists O(1) – hmset/hexists/hincrby/hdel/hlen – hmget O(N) – hkeys/hvals/hgetall
  • 17. ZIPMAP • zipmap – O(n) search O(mem_size) add/delete
  • 18. 内存管理 • zmalloc.c – 简单内存管理 – 支持tcmalloc USE_TCMALLOC – zmalloc(size_t size)  malloc(size+PREFIX_SIZE) – fragmentation_ratio = zmalloc_get_rss/zmalloc_used_memory
  • 19. PUBSUB • SUBSCRIBE/UNSUBSCRIBE/PSUBSCRIBE/PU NSUBSCRIBE • PUBLISH • 连接断开后信息丢失
  • 20. TRANSACTION • MULTI (不支持CAS) – MULTI – SET foo 0 – INCR foo – EXEC • WATCH (Check and Set ) – WATCH theKey – v = GET theKey – MULTI – SET theKey v+1 – EXEC • UNWATCH/DISCARD
  • 21. 提升网络效率、减少内存占用 • MGET/MSET • PIPELINING – $ (echo -en "PINGrnPINGrnPINGrn"; sleep 1) | nc localhost 6379 +PONG +PONG +PONG • 压缩数据,触发ziplist/zipmap/intset • 使用 GETBIT/SETBIT/GETRANGE/SETRANGE 压缩多 个属性 • 使用HASH • 使用32位redis + sharding
  • 22. Client Side Sharding • Consistent Hashing – Redis Sharding at Craigslist
  • 23. 网络层 • Anet.c • 事件驱动 ae.c  ae_epoll.c/ae_kqueue.c/ae_select.c • 非阻塞单线程 • 支持timer • 在VM场景下有可能多线程
  • 25. 网络层对比 • lighttpd/nginx的网络IO模型 – 多进程,单线程非阻塞IO • apache的MPM模型 – 多进程prefork,多线程
  • 26. PERSISTENCE • SNAPSHOT (Rdb.c) • 可以关闭 • BGSAVE or save xseconds ychanges • fork – parent - disable rehash, periodic check child on serverCron (100ms) – child - copy-on-write, save whole db, exit • Need to turn on the overcommit_memory setting if you want to deal with a dataset more than 1/2 of RAM, which contradicts our habit to battle against OOM killer as a database administrator.
  • 27. Append Only File • Append to log file on every change (Aof.c) • fsync() policy – always/os decide/every second – Always with battery backed raid controllers – Every second by default (innodb_flush_log_at_trx_commit=2 in mysql) • Compact aof file – BGREWRITEAOF/REWRITEAOF • Fork – child – write new aof in temporary file – parent – write change in both old aof file and memory buffer, append memory buffer to temporary file when child done, rename temporary to aof file. • redis-check-aof --fix <filename>
  • 29. Replication • 支持AUTH • 起动过程 – slave •SYNC •Wait – master •Issue a BGSAVE, or wait if BGSAVE in progress
  • 30. Virtual Memory • Redis VM is now deprecated. Redis 2.4 will be the latest Redis version featuring Virtual Memory! • Why not OS VM – Varnish What’s wrong with 1975 programming ? – Redis What’s wrong with 2006 programming? • Blocking when page fault • 4k pages granularity • Optimal disk persistent format, 10:1 • Keys can’t be swapped out • vm is read only when BGSAVE/BGREWRITEAOF in progress • vm-max-memory/vm-pages/vm-page-size
  • 31. Virtual Memory • Blocking vm • Threaded vm – Before command is executed, check the value – Block client and load, then signal by unix pipe – Continue • Recommended filesystem – ext3 or any other file system with good support for sparse files • redis-stat vmstat • InnoDB innodb_buffer_pool_size – swap and durability
  • 32. Future • Diskstore (diskstore.c dscache.c) – All data on disk, B+tree in future – Swaps data into memory when needed • Leveldb/InnoDB Change Buffer – LSM tree
  • 33. 其他相关的代码 • quicksort (pqsort.c) • lzf compression (lzf_c.c lzf_d.c) • Sha1 (Sha1.c) • Syncio.c • SORT (Sort.c)
  • 34. 线上Redis集群情况 • 16台服务器,兼做后端服务器 • 45个Redis Instance进程,三组服务 • 总内存 77G VIRT,71G RSS • 使用了主从复制、snapshot持久化,VM支持 ,客户端sharding、pipelining、hash优化
  • 35. Redis应用 • Resque – message queue of github (celery) • Nginx HttpRedis module • Redweb - Web administrative and query UI • A fast, fuzzy, full-text index using Redis • Redis-backed BloomFilter(s) in Ruby • Soulmate : Auto Complete with Redis • Rate limiting with Redis • Redis Sharding • Locking with setnx • A Collection Of Redis Use Cases • Radishapp Redis monitoring • openredis.com hosting service
  • 36. Contribute to Redis • Fork redis on github – https://github.com/antirez/redis • Choose branch and commit your changes • Create a pull request • Wait for approval – https://github.com/antirez/redis/pulls • My contribute for read only slave – https://github.com/antirez/redis/pull/47
  • 37. Resources • DOCUMENATION – http://redis.io/documentation – http://antirez.com/ – http://groups.google.com/group/redis-db • 本文参考的Redis资料集合 – http://www.delicious.com/fakechris/redis – http://www.delicious.com/tag/redis