简介

Redis-Cluster采用无中心结构，每个节点保存数据和整个集群状态,每个节点都和其他所有节点连接。

其结构特点：

  • 所有的redis节点彼此互联(PING-PONG机制),内部使用二进制协议优化传输速度和带宽。
  • 节点的fail是通过集群中超过半数的节点检测失效时才生效。
  • 客户端与redis节点直连,不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可。
  • redis-cluster把所有的物理节点映射到[0-16383]slot上（不一定是平均分配）,cluster 负责维护node<->slot<->value。
  • Redis集群预分好16384个桶，当需要在 Redis 集群中放置一个 key-value 时，根据 CRC16(key) mod 16384的值，决定将一个key放到哪个桶中。

环境准备

Redis-cluster最少需要3个master节点，推荐6个节点

主机 IP 角色
node1 172.16.10.11 redis
node2 172.16.10.12 redis
node3 172.16.10.13 redis

安装redis

1yum -y install epel-release
2yum -y install redis

配置

 1# 修改监听地址
 2sed -i '/^bind/s/127.0.0.1/0.0.0.0/' /etc/redis.conf
 3# 启用aof
 4sed -i '/appendonly/s/no/yes/' /etc/redis.conf
 5# 启用cluster
 6sed -i '/cluster-enabled/s/^# //' /etc/redis.conf
 7# cluster配置文件
 8sed -i '/cluster-config-file/s/^# //' /etc/redis.conf
 9# cluster超时时间
10sed -i '/cluster-node-timeout/s/^# //' /etc/redis.conf

运行

1systemctl enable --now redis

可以看到redis的3个节点已经启动成功 **注意：**这里并没有创建集群

1redis-cli info replication
2# Replication
3role:master
4connected_slaves:0
5master_repl_offset:0
6repl_backlog_active:0
7repl_backlog_size:1048576
8repl_backlog_first_byte_offset:0
9repl_backlog_histlen:0

使用redis-trib.rb创建集群

yum源中自带的ruby版本2.0.0，版本过低，可以从官网下载源码编译安装或者通过rvm进行安装

 1yum -y install gcc zlib-devel openssl-devel
 2wget https://cache.ruby-lang.org/pub/ruby/2.6/ruby-2.6.6.tar.gz
 3tar xf ruby-2.6.6.tar.gz
 4cd ruby-2.6.6
 5./configure
 6make
 7make install
 8
 9gem install redis
10Fetching redis-4.2.1.gem
11Successfully installed redis-4.2.1
12Parsing documentation for redis-4.2.1
13Installing ri documentation for redis-4.2.1
14Done installing documentation for redis after 1 seconds
151 gem installed

下载redis-trib.rb

1wget https://github.com/antirez/redis/raw/3.2/src/redis-trib.rb
2chmod +x redis-trib.rb

创建集群

 1./redis-trib.rb create --replicas 0 172.16.10.11:6379 172.16.10.12:6379 172.16.10.13:6379
 2>>> Creating cluster
 3>>> Performing hash slots allocation on 3 nodes...
 4Using 3 masters:
 5172.16.10.11:6379
 6172.16.10.12:6379
 7172.16.10.13:6379
 8M: 0dcfb481c80ec21a411e73e93af344e50aaa2713 172.16.10.11:6379
 9   slots:0-5460 (5461 slots) master
10M: 30804166e60b5ed55cb5be06185c6931942fcc2b 172.16.10.12:6379
11   slots:5461-10922 (5462 slots) master
12M: 41a8af9dafc1be00602cfeb9dd66fcf8bcef2f9a 172.16.10.13:6379
13   slots:10923-16383 (5461 slots) master
14Can I set the above configuration? (type 'yes' to accept): yes
15>>> Nodes configuration updated
16>>> Assign a different config epoch to each node
17>>> Sending CLUSTER MEET messages to join the cluster
18Waiting for the cluster to join.
19>>> Performing Cluster Check (using node 172.16.10.11:6379)
20M: 0dcfb481c80ec21a411e73e93af344e50aaa2713 172.16.10.11:6379
21   slots:0-5460 (5461 slots) master
22   0 additional replica(s)
23M: 41a8af9dafc1be00602cfeb9dd66fcf8bcef2f9a 172.16.10.13:6379
24   slots:10923-16383 (5461 slots) master
25   0 additional replica(s)
26M: 30804166e60b5ed55cb5be06185c6931942fcc2b 172.16.10.12:6379
27   slots:5461-10922 (5462 slots) master
28   0 additional replica(s)
29[OK] All nodes agree about slots configuration.
30>>> Check for open slots...
31>>> Check slots coverage...
32[OK] All 16384 slots covered.

测试

 1redis-cli -c
 2127.0.0.1:6379> set age 30
 3OK
 4127.0.0.1:6379> set name jerry
 5-> Redirected to slot [5798] located at 172.16.10.12:6379
 6OK
 7172.16.10.12:6379> get name
 8"jerry"
 9172.16.10.12:6379> get age
10-> Redirected to slot [741] located at 172.16.10.11:6379
11"30"

数据平均分布到3个节点上面，但是总共只有3个节点，3个都是主节点，当某个节点挂了之后，存储在该节点的数据便无法获取。因此redis-cluster建议6个节点以上，3主3从，某个节点挂了之后，该节点的从节点成为主节点接替其工作，达到高可用。

错误处理

1./redis-trib.rb create --replicas 172.16.10.11:6379 172.16.10.12:6379 172.16.10.13:6379
2/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- redis (LoadError)
3	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
4	from ./redis-trib.rb:25:in `<main>'

解决办法

1gem install redis

ruby版本过低

1gem install redis
2Fetching: redis-4.2.1.gem (100%)
3ERROR:  Error installing redis:
4	redis requires Ruby version >= 2.3.0.
5
6ruby -v
7ruby 2.0.0p648 (2015-12-16) [x86_64-linux]

解决办法

上官网下载2.3.0以后的版本

 1# 卸载旧版本
 2yum -y remove ruby rubygems
 3
 4yum -y install gcc zlib-devel
 5wget https://cache.ruby-lang.org/pub/ruby/2.6/ruby-2.6.6.tar.gz
 6tar xf ruby-2.6.6.tar.gz
 7cd ruby-2.6.6
 8./configure
 9make
10make install
11
12ruby -v
13ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]

zlib

1gem install redis
2ERROR:  Loading command: install (LoadError)
3	cannot load such file -- zlib
4ERROR:  While executing gem ... (NoMethodError)
5    undefined method `invoke_with_build_args' for nil:NilClass

解决版本

安装zlib-devel包，然后重新编译安装ruby

1yum -y install zlib-devel

openssl

 1gem install redis
 2...
 3/usr/local/lib/ruby/2.6.0/rubygems/request.rb:84:in `rescue in configure_connection_for_https': Unable to require openssl, install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources (Gem::Exception)
 4	22: from /usr/local/bin/gem:21:in `<main>'
 5	21: from /usr/local/lib/ruby/2.6.0/rubygems/gem_runner.rb:59:in `run'
 6	20: from /usr/local/lib/ruby/2.6.0/rubygems/command_manager.rb:148:in `run'
 7	19: from /usr/local/lib/ruby/2.6.0/rubygems/command_manager.rb:178:in `process_args'
 8	18: from /usr/local/lib/ruby/2.6.0/rubygems/command.rb:321:in `invoke_with_build_args'
 9	17: from /usr/local/lib/ruby/2.6.0/rubygems/commands/install_command.rb:165:in `execute'
10	16: from /usr/local/lib/ruby/2.6.0/rubygems/commands/install_command.rb:258:in `install_gems'
11	15: from /usr/local/lib/ruby/2.6.0/rubygems/commands/install_command.rb:258:in `each'
12	14: from /usr/local/lib/ruby/2.6.0/rubygems/commands/install_command.rb:264:in `block in install_gems'
13	13: from /usr/local/lib/ruby/2.6.0/rubygems/commands/install_command.rb:201:in `install_gem'
14	12: from /usr/local/lib/ruby/2.6.0/rubygems/dependency_installer.rb:478:in `resolve_dependencies'
15	11: from /usr/local/lib/ruby/2.6.0/rubygems/resolver/installer_set.rb:56:in `add_always_install'
16	10: from /usr/local/lib/ruby/2.6.0/rubygems/resolver/installer_set.rb:155:in `find_all'
17	 9: from /usr/local/lib/ruby/2.6.0/rubygems/resolver/best_set.rb:29:in `find_all'
18	 8: from /usr/local/lib/ruby/2.6.0/rubygems/resolver/best_set.rb:23:in `pick_sets'
19	 7: from /usr/local/lib/ruby/2.6.0/rubygems/source_list.rb:98:in `each_source'
20	 6: from /usr/local/lib/ruby/2.6.0/rubygems/source_list.rb:98:in `each'
21	 5: from /usr/local/lib/ruby/2.6.0/rubygems/resolver/best_set.rb:24:in `block in pick_sets'
22	 4: from /usr/local/lib/ruby/2.6.0/rubygems/source.rb:87:in `dependency_resolver_set'
23	 3: from /usr/local/lib/ruby/2.6.0/rubygems/remote_fetcher.rb:253:in `fetch_path'
24	 2: from /usr/local/lib/ruby/2.6.0/rubygems/remote_fetcher.rb:278:in `rescue in fetch_path'
25	 1: from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
26/usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- openssl (LoadError)

解决办法

安装openssl-devel，然后重新编译安装

1yum -y install openssl-devel

常用第三方库

python

  • redis-py-cluster

1pip install redis-py-cluster

示例

 1>>> from rediscluster import RedisCluster
 2
 3>>> # Requires at least one node for cluster discovery. Multiple nodes is recommended.
 4>>> startup_nodes = [{"host": "172.16.10.11", "port": "6379"}]
 5
 6>>> rc = RedisCluster(startup_nodes=startup_nodes, decode_responses=True)
 7
 8>>> rc.get("name")
 9'jerry'
10
11>>> rc.get("age")
12'30'

golang

  • redis-go-cluster

安装

1go get github.com/chasex/redis-go-cluster

示例

 1package main
 2
 3import (
 4	"fmt"
 5	"github.com/chasex/redis-go-cluster"
 6	"log"
 7	"time"
 8)
 9
10func main() {
11
12	cluster, err := redis.NewCluster(
13		&redis.Options{
14			StartNodes:   []string{"172.16.10.11:6379", "172.16.10.12:6379", "172.16.10.13:6379"},
15			ConnTimeout:  50 * time.Millisecond,
16			ReadTimeout:  50 * time.Millisecond,
17			WriteTimeout: 50 * time.Millisecond,
18			KeepAlive:    16,
19			AliveTime:    60 * time.Second,
20		})
21	if err != nil {
22		log.Fatal(err.Error())
23	}
24
25	reply, err := redis.String(cluster.Do("GET", "name"))
26	if err != nil {
27		log.Println(err.Error())
28	}
29	fmt.Println(reply)
30}

运行结果

1go run main.go
2jerry