Redis-full-check for the Redis Instance Comparison Tool

This article comes from the public number: Java Geek Technology Author: Duck Blood Fans

Hello hello everyone, I am a fan of duck blood, the previous article introduced how SpringBoot project from a stand-alone switch access cluster, have not seen the small partner can go to see the SpringBoot project access Redis cluster. This article introduces you to a Redis tool, redis-full-check, which is primarily used to verify the accuracy of migration data, so let's take a look.

Installation

Redis-full-check is an open source tool for Ali, GitHub address https://github.com/alibaba/RedisFullCheck, we need to find a Linux machine before installation, and the GLIBC version needs to be higher than 2.14, otherwise it will be prompted /lib64/libc.so.6: version GLIBC_2.14 not found T here are two ways to download, the first is to download directly locally and then upload it to the server, and the other is to perform a wget https://github.com/alibaba/RedisFullCheck/releases/download/release-v1.4.8-20200212/redis-full-check-1.4.8.tar.gz directly on the server to download. U nzip tar xzvf redis-full-check-1.4.8.tar.gz after download is complete. The specific process we carry out as follows:

1. Check the GLIBC version of the current server, execute the command strings /lib64/libc.so.6 |grep GLIBC_ as shown below, if there is more than 2.14 can be, if there is no consideration to change a server or their own update, but the update is risky please be careful, the specific update method self Baidu;

1. Download the package and perform: wget https://github.com/alibaba/RedisFullCheck/releases/download/release-v1.4.8-20200212/redis-full-check-1.4.8.tar.gz Unzip after downloading. A powder here has been down, do not repeat the download, unzip into the directory, enter ./redis-full-check -v if you can see the version number normally to indicate that the download installation was successful.

use

Before you can use this tool, you need two different Redis instances, which you already have because you switched from stand-alone to cluster. H ere's a stand-alone and cluster demonstration. W e execute the following command: ./redis-full-check -s "172.20.xxx.xxx:6379" -p "sourcePassword" --sourcedbfilterlist=0 -t "172.20.xxx.xxx:6379;172.20.yyy.yyy:6379" -a "targetPassword" --targetdbtype=1

illustrate:

1. -s: Represents the source Redis instance
2. p: Source Redis password
3. --sourcedbfilterlist: match the specified db library, single set Redis can set up a specific db library, the cluster environment can not, according to their own circumstances to decide whether to adopt;
4. -t: Target Redis, A powder this side is a cluster so there will be multiple nodes, each node separated by a semicolon, in addition to the document says that here must fill in all master nodes or all slave nodes, can not be mixed to fill in. A powder here is filled with master node is successful, but all slave seems not to succeed, we can try it ourselves.
5. -a: Represents the password for the target Redis
6. --targetdbtype=1:Type of target Redis environment, 0:db (standalone single node, master slave), 1: cluster (cluster version), 2: Alibaba Cloud

The detailed parameters are as follows:

 -s, --source=SOURCE               源redis库地址（ip:port），如果是集群版，那么需要以分号（;）分割不同的db，只需要配置主或者从的其中之一。例如：10.1.1.1:1000;10.2.2.2:2000;10.3.3.3:3000。
  -p, --sourcepassword=Password     源redis库密码
      --sourceauthtype=AUTH-TYPE    源库管理权限，开源reids下此参数无用。
      --sourcedbtype=               源库的类别，0：db(standalone单节点、主从)，1: cluster（集群版），2: 阿里云
      --sourcedbfilterlist=         源库需要抓取的逻辑db白名单，以分号（;）分割，例如：0;5;15表示db0,db5和db15都会被抓取
  -t, --target=TARGET               目的redis库地址（ip:port）
  -a, --targetpassword=Password     目的redis库密码
      --targetauthtype=AUTH-TYPE    目的库管理权限，开源reids下此参数无用。
      --targetdbtype=               参考sourcedbtype
      --targetdbfilterlist=         参考sourcedbfilterlist
  -d, --db=Sqlite3-DB-FILE          对于差异的key存储的sqlite3 db的位置，默认result.db
      --comparetimes=COUNT          比较轮数
  -m, --comparemode=                比较模式，1表示全量比较，2表示只对比value的长度，3只对比key是否存在，4全量比较的情况下，忽略大key的比较
      --id=                         用于打metric
      --jobid=                      用于打metric
      --taskid=                     用于打metric
  -q, --qps=                        qps限速阈值
      --interval=Second             每轮之间的时间间隔
      --batchcount=COUNT            批量聚合的数量
      --parallel=COUNT              比较的并发协程数，默认5
      --log=FILE                    log文件
      --result=FILE                 不一致结果记录到result文件中，格式：'db    diff-type    key    field'
      --metric=FILE                 metric文件
      --bigkeythreshold=COUNT       大key拆分的阈值，用于comparemode=4
  -f, --filterlist=FILTER           需要比较的key列表，以分号（;）分割。例如："abc*|efg|m*"表示对比'abc', 'abc1', 'efg', 'm', 'mxyz'，不对比'efgh', 'p'。
  -v, --version

View the results

After executing the above command, three files are generated in the current directory: result .db.1, result .db.2, and result .db.3. We can query through the sqlite3 tool, as follows:

Enter the terminal via sqlite3 result.db.3 command and query the data we need from the key table. The sqlite3 tool is a MySQL-like database that you can study for yourself how to use it and then have the opportunity to share it with you.

From the picture above can be found that this result looks very uncomfortable, A powder to teach everyone a few more tricks, let look a little cool! After entering the terminal, we enter the commands in the figure below in turn

1. .header on opens the header, id is just a serial number, key represents key in source Redis, type represents type, db represents the db library of the source Redis where key is located, source_len, and target_len represents the length of value in the source Redis and target Redis, respectively. We can quickly see different data by length.
2. .mode column sets the output mode
3. .widht int int... Set the length of each column display to make it more beautiful
4. .quit exit terminal

From this output we can clearly see which data is inconsistent, so compare the data of the two Redis instances, it is important to note that Redis-full-check compares whether the source instance is a subset of the target instance!

summary

Today A powder introduced you to a Redis instance data comparison tool, can really use in production of an Ali open source very good tool, I hope to help you! Specific more details of use, we can study their own research, a good tool is worth a good in-depth study.

That's what W3Cschool编程狮 has to say about Redis-full-check, the Redis-full-check for Redis instance comparison tools, and I hope it's helpful.