龚哥哥 - 山里男儿 爱生活、做自己!
redis分布式缓存搭建与使用
发表于 2017-6-11 | 浏览(18350) | 服务器

redis官网地址 https://redis.io/

准备6台redis服务

127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
127.0.0.1:7003
127.0.0.1:7004
127.0.0.1:7005

下载安装redis

mkdir /data
cd /data
mkdir src soft
cd /data/src/
wget http://download.redis.io/releases/redis-3.2.9.tar.gz
tar -zxvf redis-3.2.9.tar.gz
cd redis-3.2.9
make
make install

创建集群位置

cd /data/soft
mkdir redis-cluster
cd redis-cluster
mkdir 7000 7001 7002 7003 7004 7005

拷贝配置文件

cp /data/src/redis-3.2.9/redis.conf /data/soft/redis-cluster/7000
cp /data/src/redis-3.2.9/src/redis-trib.rb /data/soft/redis-cluster/

修改配置文件

cd /data/soft/redis-cluster/7000
vim redis.conf 修改以下选项内容

port 7000
daemonize yes  # 后台启动
cluster-enabled yes
cluster-config-file nodes.conf  # 指定集群节点配置文件
cluster-node-timeout 5000  # 指定集群节点超时时间
appendonly yes  # 指定持久化方式

完成后分别把 redis.conf 文件拷贝到7000 ~ 7005目录下,修改 port 对应目录名称的端口号

完成后分别启动6个实例

重启需要先删除数据文件和节点配置文件

for((i=0;i<=5;i++)); do cd /data/soft/redis-cluster/700$i/;rm -rf appendonly.aof  dump.rdb nodes.conf; done

循环启动redis

for((i=0;i<=5;i++)); do cd /data/soft/redis-cluster/700$i/; redis-server redis.conf; done

创建集群

/data/soft/redis-cluster/redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

循环停止redis服务

for((i=0;i<=5;i++)); do redis-cli -c -p 700$i shutdown; done

启动redis服务,启动完成后查看实例列表 [ ps -ef | grep redis ],如下显示则说明启动成功

ps -ef | grep redis

root      32631      1  0 15:39 ?        00:00:00 redis-server 127.0.0.1:7000 [cluster]
root      32668      1  0 15:40 ?        00:00:00 redis-server 127.0.0.1:7001 [cluster]
root      32675      1  0 15:40 ?        00:00:00 redis-server 127.0.0.1:7002 [cluster]
root      32682      1  0 15:41 ?        00:00:00 redis-server 127.0.0.1:7003 [cluster]
root      32687      1  0 15:41 ?        00:00:00 redis-server 127.0.0.1:7004 [cluster]
root      32696      1  1 15:41 ?        00:00:00 redis-server 127.0.0.1:7005 [cluster]

安装ruby,由于redis集群客户端实现很少,redis集群的启动需要用到ruby实现的redis-trib.rb,所以我们需要先安装ruby。

yum -y install ruby

安装gem

yum -y install rubygems

gem 安装 redis ruby 接口

gem install redis

创建redis集群

--replicas 则指定了为 Redis Cluster 中的每个 Master 节点配备几个 Slave 节 点,节点角色由顺序决定,先 master 之后是 slave

/data/soft/redis-cluster/redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
Adding replica 127.0.0.1:7003 to 127.0.0.1:7000
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
M: 443adf2c1d13b41866dc0dc108ffe858d931ac77 127.0.0.1:7000
   slots:0-5460 (5461 slots) master
M: d74ee0110d7bcf0858ca76eb32caeca5094fa5ff 127.0.0.1:7001
   slots:5461-10922 (5462 slots) master
M: 13fd8b25f070382657c700fc9d8184418a82f8b6 127.0.0.1:7002
   slots:10923-16383 (5461 slots) master
S: 229a83d4997539771bf6792455c992cd215b7273 127.0.0.1:7003
   replicates 443adf2c1d13b41866dc0dc108ffe858d931ac77
S: 179a946b3ab66fac33d19316076e8c93d607a53a 127.0.0.1:7004
   replicates d74ee0110d7bcf0858ca76eb32caeca5094fa5ff
S: f6894bfcca6a22f426722a9b9843ba95cd20bb53 127.0.0.1:7005
   replicates 13fd8b25f070382657c700fc9d8184418a82f8b6
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.
....
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 443adf2c1d13b41866dc0dc108ffe858d931ac77 127.0.0.1:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: d74ee0110d7bcf0858ca76eb32caeca5094fa5ff 127.0.0.1:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 13fd8b25f070382657c700fc9d8184418a82f8b6 127.0.0.1:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 229a83d4997539771bf6792455c992cd215b7273 127.0.0.1:7003
   slots: (0 slots) slave
   replicates 443adf2c1d13b41866dc0dc108ffe858d931ac77
S: f6894bfcca6a22f426722a9b9843ba95cd20bb53 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 13fd8b25f070382657c700fc9d8184418a82f8b6
S: 179a946b3ab66fac33d19316076e8c93d607a53a 127.0.0.1:7004
   slots: (0 slots) slave
   replicates d74ee0110d7bcf0858ca76eb32caeca5094fa5ff
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

这里输入 [ yes ] 如下
Can I set the above configuration? (type 'yes' to accept): yes

检查集群 状态

/data/soft/redis-cluster/redis-trib.rb check 127.0.0.1:7000

>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 443adf2c1d13b41866dc0dc108ffe858d931ac77 127.0.0.1:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: d74ee0110d7bcf0858ca76eb32caeca5094fa5ff 127.0.0.1:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 13fd8b25f070382657c700fc9d8184418a82f8b6 127.0.0.1:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 229a83d4997539771bf6792455c992cd215b7273 127.0.0.1:7003
   slots: (0 slots) slave
   replicates 443adf2c1d13b41866dc0dc108ffe858d931ac77
S: f6894bfcca6a22f426722a9b9843ba95cd20bb53 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 13fd8b25f070382657c700fc9d8184418a82f8b6
S: 179a946b3ab66fac33d19316076e8c93d607a53a 127.0.0.1:7004
   slots: (0 slots) slave
   replicates d74ee0110d7bcf0858ca76eb32caeca5094fa5ff
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

连接redis服务测试一下

[root@My src]# redis-cli -c -p 7000
127.0.0.1:7000> set name 200
-> Redirected to slot [5798] located at 127.0.0.1:7001
OK
127.0.0.1:7001> set devil mogui
-> Redirected to slot [893] located at 127.0.0.1:7000
OK
127.0.0.1:7000> get devil
"mogui"
127.0.0.1:7000> quit
[root@My src]# redis-cli -c -p 7003
127.0.0.1:7003> get devil
-> Redirected to slot [893] located at 127.0.0.1:7000
"mogui"
127.0.0.1:7000> 
127.0.0.1:7000> quit
[root@My src]# redis-cli -c -p 7002
127.0.0.1:7002> get name
-> Redirected to slot [5798] located at 127.0.0.1:7001
"200"
127.0.0.1:7001> get devil
-> Redirected to slot [893] located at 127.0.0.1:7000
"mogui"
127.0.0.1:7000> set age 20
OK
127.0.0.1:7000> quit
[root@My src]# redis-cli -c -p 7005
127.0.0.1:7005> get age
-> Redirected to slot [741] located at 127.0.0.1:7000
"20"
127.0.0.1:7000> set sex 30
OK
127.0.0.1:7000> quit
[root@My src]# redis-cli -c -p 7001
127.0.0.1:7001> get sex
-> Redirected to slot [2584] located at 127.0.0.1:7000
"30"
127.0.0.1:7000>

使用add-node命令来添加节点,第一个参数是新节点的地址,第二个参数是任意一个已经存在的节点的IP和端口

添加一个从节点,使用[ --slave ] 参数

/data/soft/redis-cluster/redis-trib.rb add-node --slave 127.0.0.1:7007 127.0.0.1:7000

添加从节点还可以指定这个从节点的主节点id,使用[ --master-id ] 参数指定masterid

/data/soft/redis-cluster/redis-trib.rb add-node --slave --master-id 3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e 127.0.0.1:7007 127.0.0.1:7000

重新分配哈希槽

/data/soft/redis-cluster/redis-trib.rb reshard 127.0.0.1:7006

移除一个节点,第一个参数是集群中的任意一个节点地址,第二个是需要移除的节点id(移除前请确保节点哈希槽数量是空的)当移除一个master节点的时候,如果节点有哈希槽则失败

/data/soft/redis-cluster/redis-trib.rb del-node 127.0.0.1:7000 fea8868aeb4ce79d447b602e1cf07b4318adb852

改变从节点的主节点id[ CLUSTER REPLICATE <master-node-id> ](如把7007的主节点id改成7001的id)

[root@My redis-cluster]# redis-cli -c -p 7007
127.0.0.1:7007> cluster replicate cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf
OK


添加一个新节点演示

cd /data/soft/redis-cluster/
cp -r 7005 7006
cd 7006
rm -rf dump.rdb appendonly.aof nodes.conf
vim redis.conf # 修改7005为7006保存
redis-server redis.conf

使用add-node命令来添加节点,第一个参数是新节点的地址,第二个参数是任意一个已经存在的节点的IP和端口. 我们可以看到新的节点已经添加到集群中

/data/soft/redis-cluster/redis-trib.rb add-node 127.0.0.1:7006 127.0.0.1:7000

查看7006节点是否正常加入了集群(可以看出 7006 成为了master节点,但是还未分配哈希槽)

[root@My redis-cluster]# redis-cli -p 7000 cluster nodes

e3cb1f4805d31dc0dc11b1e5539653b2c4144216 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460
c7a0afa1c64045bad02de7478cc23f1d65a3b035 127.0.0.1:7006 master - 0 1497061063704 0 connected
64d1a9e01ca3820288c7e5c4adc3a161b7816895 127.0.0.1:7005 slave 20c27ce2235485f2d5aa5d941674d7f81ee22682 0 1497061064812 6 connected
dffe09f5b1739dc5b53f8299d20b06508eccb15d 127.0.0.1:7004 slave cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 0 1497061065820 5 connected
20c27ce2235485f2d5aa5d941674d7f81ee22682 127.0.0.1:7002 master - 0 1497061064309 3 connected 10923-16383
cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 127.0.0.1:7001 master - 0 1497061065316 2 connected 5461-10922
3f363ffcacfa225ee098819b67ab4bd5d0e5def8 127.0.0.1:7003 slave e3cb1f4805d31dc0dc11b1e5539653b2c4144216 0 1497061063805 4 connected

给7006分配哈希槽

[root@My redis-cluster]# /data/soft/redis-cluster/redis-trib.rb reshard 127.0.0.1:7006

How many slots do you want to move (from 1 to 16384)? 需要分配的哈希槽数量 如:100
What is the receiving node ID? 7006节点的ID 如:c7a0afa1c64045bad02de7478cc23f1d65a3b035
Source node #1:需要从哪个节点分配哈希槽,all则从所有master节点上随机凑100个哈希槽。或者输入节点ID换行以done结束。如输入7001的节点id:cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf
Source node #2:done
Do you want to proceed with the proposed reshard plan (yes/no)? 打印被移动的 slot 后,输入 yes 开始移动 slot 以及对应的数据

哈希槽分配完成,可以看出7006刚好100个哈希槽数量,7001也刚好少了100个哈希槽

[root@My redis-cluster]# redis-cli -p 7000 cluster nodes

e3cb1f4805d31dc0dc11b1e5539653b2c4144216 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460
c7a0afa1c64045bad02de7478cc23f1d65a3b035 127.0.0.1:7006 master - 0 1497061327609 7 connected 5461-5560
64d1a9e01ca3820288c7e5c4adc3a161b7816895 127.0.0.1:7005 slave 20c27ce2235485f2d5aa5d941674d7f81ee22682 0 1497061325594 6 connected
dffe09f5b1739dc5b53f8299d20b06508eccb15d 127.0.0.1:7004 slave cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 0 1497061327609 5 connected
20c27ce2235485f2d5aa5d941674d7f81ee22682 127.0.0.1:7002 master - 0 1497061326097 3 connected 10923-16383
cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 127.0.0.1:7001 master - 0 1497061327106 2 connected 5561-10922
3f363ffcacfa225ee098819b67ab4bd5d0e5def8 127.0.0.1:7003 slave e3cb1f4805d31dc0dc11b1e5539653b2c4144216 0 1497061326602 4 connected

当移除一个master节点的时候,不能有哈希槽数量(把当7002前master节点的哈希槽转移到7006节点上去)

[root@My redis-cluster]# ./redis-trib.rb reshard 127.0.0.1:7002
How many slots do you want to move (from 1 to 16384)? 输入7002的哈希槽数量,16383-10923如:5461
What is the receiving node ID? 输入7006的id,如:c7a0afa1c64045bad02de7478cc23f1d65a3b035
Source node #1: 从哪个节点分配哈希槽,这里输入7002的id,ID换行以done结束如:20c27ce2235485f2d5aa5d941674d7f81ee22682
Source node #2:done
Do you want to proceed with the proposed reshard plan (yes/no)? 打印被移动的 slot 后,输入 yes 开始移动 slot 以及对应的数据

查看7002集群状态,由此可以看到7002节点已经没有哈希槽数量了,而7006已经多了5461个哈希槽数[ 10923-16383 ]

[root@My redis-cluster]# redis-cli -p 7000 cluster nodes
e3cb1f4805d31dc0dc11b1e5539653b2c4144216 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460
c7a0afa1c64045bad02de7478cc23f1d65a3b035 127.0.0.1:7006 master - 0 1497070326503 7 connected 5461-5560 10923-16383
64d1a9e01ca3820288c7e5c4adc3a161b7816895 127.0.0.1:7005 slave c7a0afa1c64045bad02de7478cc23f1d65a3b035 0 1497070326000 7 connected
dffe09f5b1739dc5b53f8299d20b06508eccb15d 127.0.0.1:7004 slave cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 0 1497070327004 5 connected
20c27ce2235485f2d5aa5d941674d7f81ee22682 127.0.0.1:7002 master - 0 1497070326000 3 connected
cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 127.0.0.1:7001 master - 0 1497070327507 2 connected 5561-10922
3f363ffcacfa225ee098819b67ab4bd5d0e5def8 127.0.0.1:7003 slave e3cb1f4805d31dc0dc11b1e5539653b2c4144216 0 1497070327004 4 connected

从集群中移除7002节点

[root@My redis-cluster]# ./redis-trib.rb del-node 127.0.0.1:7000 20c27ce2235485f2d5aa5d941674d7f81ee22682
>>> Removing node 20c27ce2235485f2d5aa5d941674d7f81ee22682 from cluster 127.0.0.1:7000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

查看7002已经不在集群中了,原来的哈希槽已由7006接手,当前集群的数据也是不会受到任何影响

[root@My redis-cluster]# redis-cli -p 7000 cluster nodes
e3cb1f4805d31dc0dc11b1e5539653b2c4144216 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460
c7a0afa1c64045bad02de7478cc23f1d65a3b035 127.0.0.1:7006 master - 0 1497070633845 7 connected 5461-5560 10923-16383
64d1a9e01ca3820288c7e5c4adc3a161b7816895 127.0.0.1:7005 slave c7a0afa1c64045bad02de7478cc23f1d65a3b035 0 1497070634348 7 connected
dffe09f5b1739dc5b53f8299d20b06508eccb15d 127.0.0.1:7004 slave cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 0 1497070633343 5 connected
cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 127.0.0.1:7001 master - 0 1497070632843 2 connected 5561-10922
3f363ffcacfa225ee098819b67ab4bd5d0e5def8 127.0.0.1:7003 slave e3cb1f4805d31dc0dc11b1e5539653b2c4144216 0 1497070632340 4 connected

添加7002为master,7007为slave两个节点(可以看出7002为master还未分配哈希槽,7007为slave)

[root@My redis-cluster]# ./redis-trib.rb add-node 127.0.0.1:7002 127.0.0.1:7000
[root@My redis-cluster]# ./redis-trib.rb add-node --slave 127.0.0.1:7007 127.0.0.1:7000

[root@My redis-cluster]# redis-cli -p 7000 cluster nodes
e3cb1f4805d31dc0dc11b1e5539653b2c4144216 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460
c7a0afa1c64045bad02de7478cc23f1d65a3b035 127.0.0.1:7006 master - 0 1497078346719 7 connected 5461-5560 10923-16383
64d1a9e01ca3820288c7e5c4adc3a161b7816895 127.0.0.1:7005 slave c7a0afa1c64045bad02de7478cc23f1d65a3b035 0 1497078346216 7 connected
dffe09f5b1739dc5b53f8299d20b06508eccb15d 127.0.0.1:7004 slave cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 0 1497078347221 5 connected
bde1b4378b10c6c7fbb8ba62189fc4d8fbbc6da7 127.0.0.1:7007 slave a7aeb2c3bef177ff671c5304d3baf0a825025d3c 0 1497078347723 0 connected
cba8ca301ef0fc8039af8313d580b0a5bc8ea8cf 127.0.0.1:7001 master - 0 1497078347221 2 connected 5561-10922
a7aeb2c3bef177ff671c5304d3baf0a825025d3c 127.0.0.1:7002 master - 0 1497078346719 0 connected
3f363ffcacfa225ee098819b67ab4bd5d0e5def8 127.0.0.1:7003 slave e3cb1f4805d31dc0dc11b1e5539653b2c4144216 0 1497078348226 4 connected

把7002节点从集群中移除(7002还未分配哈希槽数量,所以可以直接移除成功)

[root@My redis-cluster]# ./redis-trib.rb del-node 127.0.0.1:7000 a7aeb2c3bef177ff671c5304d3baf0a825025d3c
>>> Removing node a7aeb2c3bef177ff671c5304d3baf0a825025d3c from cluster 127.0.0.1:7000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> 127.0.0.1:7007 as replica of 127.0.0.1:7000
>>> SHUTDOWN the node.

把7007节点中添加点数据

[root@My redis-cluster]# ./redis-trib.rb del-node 127.0.0.1:7000 bde1b4378b10c6c7fbb8ba62189fc4d8fbbc6da7
>>> Removing node bde1b4378b10c6c7fbb8ba62189fc4d8fbbc6da7 from cluster 127.0.0.1:7000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

PHP中使用方式

<?php

$redis = new RedisCluster(null, ['127.0.0.1:7000', '127.0.0.1:7001', '127.0.0.1:7002', '127.0.0.1:7003', '127.0.0.1:7004', '127.0.0.1:7005']);
var_dump($redis->set('gong', 'gongfuxiang'));
var_dump($redis->set('xxx', 'XIANG'));
var_dump($redis->get('devil'));
var_dump($redis->get('sex'));
var_dump($redis->get('age'));
var_dump($redis->get('ggg'));
var_dump($redis->get('ppp'));
?>

docker镜像下载

docker pull gongfuxiang/centos7.3-redis-cluster

阅读全文

TOP