Visualize the basic capabilities of Redis by telling Redis' data structure and key commands
This article will start with the basic features of Redis, and introduce the basic capabilities of Redis through the data structure and main commands of Redis. Afterwards, we will provide more in-depth introduction and guidance on performance tuning.
Overview
Redis is an open source, memory-based structured data storage medium that can be used as a database, cache service, or messaging service.
Redis supports a variety of data structures, including strings, hash tables, linked lists, collections, ordered collections, bitmaps, Hyperloglogs, and more.
Redis has LRU retirement, transactional implementation, and different levels of hard disk persistence, and supports replica sets and high availability scenarios through Redis Sentinel, as well as automatic data fragmentation with Redis Cluster.
Redis' main features are based on a single-threaded model, which means that Redis uses one thread to serve all client requests, while Redis uses non-blocking IO and fine-tunes the algorithmic time complexity of various commands. mean:
Redis is thread-safe (because there is only one thread), all operations are atomic, and no data exceptions are generated concurrently.
Redis is very fast (because of the use of non-blocking IO, and the algorithmic time complexity of most commands is O(1))
Using a time-consuming Redis command is dangerous and takes up a lot of processing time for a single thread, causing all requests to be slowed down. (For example, the KEYS command with time complexity of O(N) is strictly prohibited in production environments)
Redis data structure and related commands
This section describes the main data structures supported by Redis and the related common Redis commands. This section only provides a brief introduction to the Redis command and lists only the more commonly used commands. If you want to know the full Redis command set, or learn how to use a command in detail,
Commonly used commands, Key
Redis uses the basic data structure of the Key-Value type. Any binary sequence can be used as the Key of Redis (such as a normal string or a JPEG picture). Some notes about Key:
Do not use a Key that is too long. For example, using a 1024-byte key is not a good idea, it will not only consume more memory, but will also reduce the efficiency of the search.
It's not good for Key to be short enough to be readable. For example, "u1000flw" saves awkward storage space compared to "user:1000:followers", but it causes troublesome readability and maintainability.
It is best to use a uniform specification to design the Key, such as "object-type: id:attr". The Key designed with this specification might be "user:1000" or "comment:1234:reply-to"
The maximum key length allowed by Redis is 512MB (the length limit for Value is also 512MB)
Commonly used commands II, String
String is the underlying data type of Redis. Redis has no concept of data types such as Int, Float, and Boolean. All basic types are represented by String in Redis.
Common commands related to String:
SET: Set the value for a key. You can use the EX/PX parameter to specify the validity period of the key. Use the NX/XX parameter to distinguish whether the key exists or not. The time complexity O(1)
GET: Get the value corresponding to a key, time complexity O(1)
GETSET: set the value for a key, and return the original value of the key, time complexity O (1)
MSET: set value for multiple keys, time complexity O(N)
MSETNX: Same as MSET, if any one of the specified keys already exists, no operation is performed, time complexity O(N)
MGET: Get the value corresponding to multiple keys, time complexity O(N)
As mentioned above, the basic data type of Redis is only String, but Redis can use String as an integer or floating-point number, mainly in the commands of INCR and DECR:
INCR: Increases the value of the key by 1 and returns the incremented value. Only works with String data that can be converted to an integer. Time complexity O(1)
INCRBY: increments the value corresponding to the key by the specified integer value and returns the incremented value. Only works with String data that can be converted to an integer. Time complexity O(1)
DECR/DECRBY: Same as INCR/INCRBY, self-increase to self-reduction.
The INCR/DECR family of commands requires that the value type of the operation be a String and can be converted to a 64-bit signed integer number, otherwise an error is returned.
In other words, the value of the INCR/DECR series command must be in the range [-2^63 ~ 2^63 - 1].
As mentioned earlier, Redis uses a single-threaded model that is naturally thread-safe, which makes the INCR/DECR command very convenient for precise control in high-concurrency scenarios.
Example 1: Inventory Control
Accurate verification of inventory margins in high concurrency scenarios ensures that no oversold conditions occur.
Set the total inventory:
SET inv:remain "100"
Inventory deduction + margin verification:
DECR inv:remain
When the return value of the DECR command is greater than or equal to 0, the stock balance check is passed. If the value less than 0 is returned, the stock is exhausted.
Assuming that there are 300 concurrent requests for inventory deduction, Redis can ensure that the 300 requests get a return value of 99 to -200 respectively. The return value obtained for each request is unique, and absolutely no two requests are found. The same return value case.
Example 2: Auto-incremental sequence generation
Implement a Sequence function similar to RDBMS to generate a series of unique serial numbers
Set the sequence start value:
SET sequence "10000"
Get a sequence value:
INCR sequence
Simply use the return value as a sequence.
Get a batch (such as 100) sequence value:
INCRBY sequence 100
Assuming the return value is N, then the values ​​of [N - 99 ~ N] are all available sequence values.
When multiple clients request an auto-increment sequence from Redis at the same time, Redis can ensure that the sequence value or sequence range obtained by each client is globally unique, and there is absolutely no case where different clients get duplicate sequence values.
Commonly used commands three, List
Redis's List is a linked-list data structure that can be used to insert elements and pop-up elements at both ends of the List using commands such as LPUSH/RPUSH/LPOP/RPOP. Although List also supports the ability to insert and read elements on a specific index, its time complexity is high (O(N)) and should be used with care.
Common commands related to List:
LPUSH: Inserts one or more elements to the left side of the specified List (that is, the header), and returns the length of the inserted List. Time complexity O(N), N is the number of inserted elements
RPUSH: Same as LPUSH, inserting 1 or more elements to the right side (ie the tail) of the specified List
LPOP: removes an element from the left side of the specified List (ie the header) and returns, time complexity O(1)
RPOP: Same as LPOP, remove 1 element from the right side of the specified List (ie tail) and return
LPUSHX/RPUSHX: Similar to LPUSH/RPUSH, except that the key of the LPUSHX/RPUSHX operation does not operate if it does not exist.
LLEN: Returns the length of the specified List, time complexity O(1)
LRANGE: Returns the specified range of elements in the specified List (double-ended inclusion, ie LRANGE key 0 10 will return 11 elements), time complexity O(N). You should control the number of elements you get at once, and getting a large list of List elements at once will cause delays, while avoiding the full traversal operation of LRANGE key 0 -1 for Lists whose length is unpredictable.
List related commands that should be used with caution:
LINDEX: Returns the specified index on the specified index, if the index is out of bounds, returns nil. The index value is loopback, that is, -1 represents the last position of the List, and -2 represents the second-to-last position of the List. Time complexity O(N)
LSET: Sets the element on the specified List specified index to value. If the index is out of bounds, it returns an error, the time complexity is O(N). If the head/tail element is operated, the time complexity is O(1).
LINSERT: Inserts a new element before/after the specified element in the specified List, and returns the length of the List after the operation. Returns -1 if the specified element does not exist. If the specified key does not exist, no operation will be performed, time complexity O(N)
Since the List of Redis is a linked list structure, the above three commands are less efficient, and the List needs to be traversed. The time consumption of the commands cannot be estimated. When the length of the List is large, the time consumption will increase significantly. It should be used with caution. .
In other words, Redis's List is actually designed to implement queues, not to implement lists like ArrayList. If you don't want to implement a double-ended queue, try not to use Redis's List data structure.
In order to better support the characteristics of the queue, Redis also provides a series of blocking operation commands, such as BLPOP/BRPOP, which can implement the ability similar to BlockingQueue, that is, when the List is empty, the connection is blocked until there are objects in the List. Can return when you leave the team. Commands for blocking classes are not discussed in detail here. Please refer to the section "Blocking operations on lists" in the official documentation (https://redis.io/topics/data-types-intro).
Commonly used commands four, Hash
Hash is a hash table. Redis Hash is a field-value type data structure, just like a traditional hash table. It can be understood as moving a HashMap into Redis.
Hash is very suitable for representing the data of the object type, and the field in the hash corresponds to the field of the object.
The advantages of Hash include:
Binary lookups can be implemented, such as "find the age of a user with an ID of 1000"
Hash can effectively reduce the consumption of network transmissions compared to serializing the entire object as a String storage method.
Provides a much more efficient random access command than List when using Hash to maintain a collection
Common commands related to Hash:
HSET: Set the field in the hash corresponding to the key to value. If the hash does not exist, one will be created automatically. Time complexity O(1)
HGET: Returns the value of the field field in the specified hash, time complexity O(1)
HMSET/HMGET: With HSET and HGET, you can batch operate multiple fields under the same key. Time complexity: O(N), N is the number of fields in one operation.
HSETNX: Same as HSET, but if the field already exists, HSETNX will not perform any operation, time complexity O(1)
HEXISTS: Determines whether the field exists in the specified hash, there is a return of 1, there is no return 0, time complexity O (1)
HDEL: Delete the field (1 or more) in the specified hash, time complexity: O(N), N is the number of fields in the operation
HINCRBY: With the INCRBY command, INCRBY is performed on a field in the specified hash, time complexity O(1)
Hash related commands that should be used with caution:
HGETALL: Returns all field-value pairs in the specified hash. The result is an array with fields and values ​​alternating in the array. Time complexity O(N)
HKEYS/HVALS: Returns all field/value in the specified hash, time complexity O(N)
The above three commands will complete the traversal of the hash. The number of fields in the hash is linearly related to the time-consuming of the command. For unpredictable hashes, the above three commands should be strictly avoided, and the HSCAN command should be used instead of the cursor. Traversing,
Commonly used commands five, Set
Redis Set is an unordered, non-repeatable String collection.
Common commands related to Set:
SADD: Add one or more members to the specified Set. If the specified Set does not exist, one will be created automatically. Time complexity O(N), N is the number of members added
SREM: Remove 1 or more members from the specified Set, time complexity O(N), N is the number of members removed
SRANDMEMBER: Randomly returns one or more members from the specified Set, time complexity O(N), N is the number of members returned
SPOP: randomly removes and returns count members from the specified Set, time complexity O(N), N is the number of members removed
SCARD: Returns the number of members in the specified Set, time complexity O(1)
SISMEMBER: Determines whether the specified value exists in the specified Set, time complexity O(1)
SMOVE: Move the specified member from one Set to another
Set related commands with caution:
SMEMBERS: Returns all members in the specified hash, time complexity O(N)
SUNION/SUNIONSTORE: Calculates the union of multiple Sets and returns/stores them to another Set, time complexity O(N), where N is the total number of members of all collections participating in the calculation
SINTER/SINTERSTORE: Calculate the intersection of multiple Sets and return / store to another Set, time complexity O(N), N is the total number of members of all the collections participating in the calculation
SDIFF/SDIFFSTORE: Calculates the difference between 1 Set and 1 or more Sets and returns/stores them to another Set, time complexity O(N), where N is the total number of members of all the sets participating in the calculation.
The above commands involve a large amount of calculations and should be used with caution, especially if the size of the Set participating in the calculation is unknown. You can consider traversing the SSCAN command to get all the members of the relevant Set (see https://redis.io/commands/scan for details). If you need to do the union/intersection/differential calculation, you can do it on the client, or not. The service is performed on the Slave on the real-time query request.
Commonly used commands six, Sorted Set
Redis Sorted Set is an ordered, non-repeatable String collection. Each element in the Sorted Set needs to be assigned a score, and the Sorted Set sorts the elements in ascending order according to the score. If multiple members have the same score, they are sorted in ascending order in lexicographical order.
Sorted Set is great for ranking.
The main commands of the Sorted Set:
ZADD: Add 1 or more members to the specified Sorted Set, time complexity O(Mlog(N)), M is the number of members added, and N is the number of members in the Sorted Set.
ZREM: Delete one or more members from the specified Sorted Set, time complexity O(Mlog(N)), M is the number of deleted members, and N is the number of members in the Sorted Set.
ZCOUNT: Returns the number of members in the specified score range in the specified Sorted Set, time complexity: O(log(N))
ZCARD: Returns the number of members in the specified Sorted Set, time complexity O(1)
ZSCORE: Returns the score of the specified member in the specified Sorted Set, time complexity O(1)
ZRANK/ZREVRANK: Returns the rank of the specified member in the Sorted Set, ZRANK returns the rank in ascending order, and ZREVRANK returns the rank in descending order. Time complexity O(log(N))
ZINCRBY: Same as INCRBY, increments the score of the specified member in the specified Sorted Set, time complexity O(log(N))
Use the Sorted Set command with caution:
ZRANGE/ZREVRANGE: Returns all members in the specified ranking range in the specified Sorted Set. ZRANGE is sorted in ascending order by score, ZREVRANGE is sorted in descending order of score, time complexity O(log(N)+M), M is returned this time. Number of members
ZRANGEBYSCORE/ZREVRANGEBYSCORE: Returns all members in the specified score range in the specified Sorted Set. The returned results are sorted in ascending/descending order. min and max can be specified as - inf and + inf, which means that all members are returned. Time complexity O(log(N)+M)
ZREMRANGEBYRANK/ZREMRANGEBYSCORE: Removes all members in the specified rank range/specified score range in the Sorted Set. Time complexity O(log(N)+M)
The above commands should try to avoid passing parameters such as [0 -1] or [-inf +inf] to make a one-time complete traversal of the Sorted Set, especially if the size of the Sorted Set is unpredictable. Cursor traversal can be done with ZSCAN commands (see https://redis.io/commands/scan for details), or the number of returned members (for ZRANGEBYSCORE and ZREVRANGEBYSCORE commands) can be restricted by the LIMIT parameter to implement cursors. Traversal.
Commonly used commands 7, Bitmap and HyperLogLog
Redis's two data structures are not commonly used compared to the previous ones. They are only briefly introduced in this article. For details on the two data structures and their related commands, please refer to the official documentation https://redis.io/ Related chapters in topics/data-types-intro
Bitmap is not an actual data type in Redis, but a way to use String as a Bitmap. Can be understood as converting a String to a bit array. Using Bitmap to store simple data of type true/false is extremely space efficient.
HyperLogLogs is a data structure mainly used for quantitative statistics. It is similar to Set and maintains a non-repeatable String collection. However, HyperLogLogs does not maintain specific member content, only the number of members. In other words, HyperLogLogs can only be used to calculate the number of elements that are not repeated in a collection, so it saves a lot of memory space than Set.
Other commonly used commands
EXISTS: Determines whether the specified key exists, returns 1 for existence, 0 for non-existence, time complexity O(1)
DEL: Delete the specified key and its corresponding value, time complexity O(N), N is the number of deleted keys
EXPIRE/PEXPIRE: Set the validity period for a key in seconds or milliseconds, time complexity O(1)
TTL/PTTL: Returns the remaining valid time of a key in seconds or milliseconds, time complexity O(1)
RENAME/RENAMENX: Rename the key to newkey. When using RENAME, if newkey already exists, its value will be overwritten; when using RENAMENX, if newkey already exists, no operation will be performed, time complexity O(1)
TYPE: Returns the type of the specified key, string, list, set, zset, hash. Time complexity O(1)
CONFIG GET: Get the current value of a Redis configuration item, you can use * wildcard, time complexity O (1)
CONFIG SET: Set a new value for a Redis configuration item, time complexity O(1)
CONFIG REWRITE: Let Redis reload the configuration in redis.conf 
Redis performance tuning
Although Redis is a very fast memory data storage medium, it does not mean that Redis will not cause performance problems. As mentioned earlier, Redis uses a single-threaded model. All commands are executed serially by a thread, so when a command takes a long time to execute, it will slow down all subsequent commands, which makes Redis The efficiency of each task is more sensitive.
For the performance optimization of Redis, we mainly start from the following aspects:
First and foremost, make sure that Redis does not take long time-consuming commands
Use pipelining to combine consecutively executed commands
The operating system's Transparent huge pages feature must be turned off:
Echo never > /sys/kernel/mm/transparent_hugepage/enabled
If you run Redis in a virtual machine, there may be inherent delays in the virtual machine environment. The inherent delay can be viewed with the ./redis-cli —intrinsic-latency 100 command. At the same time, if there is a high requirement for the performance of Redis, Redis should be deployed directly on the physical machine.
Check data persistence strategy
Consider introducing a read-write separation mechanism
Long time-consuming command
The time complexity of most Redis read and write commands is between O(1) and O(N), and the time complexity of each command is explained in both text and official documents.
In general, the O(1) command is safe, and the O(N) command should be used when it is used. If the order of magnitude of N is unpredictable, it should be avoided. For example, if you execute the HGETALL/HKEYS/HVALS command on a hash data whose field number is unknown, these commands are usually executed very quickly, but if the number of fields in the hash is extremely large, the time will multiply.
If you use SUNION to perform a Union operation on two Sets, or use SORT to perform a sort operation on a List/Set, you should pay close attention to it.
There are several ways to avoid problems when using these O(N) commands:
Don't use List as a list, just use it as a queue
Strictly control the size of Hash, Set, and Sorted Set by mechanism
If possible, put sorting, union, intersection, etc. on the client side.
It is absolutely forbidden to use the KEYS command
Avoid traversing all members of the collection type at once, but use the SCAN class's commands for batch, cursor-style traversal
Redis provides SCAN commands that allow cursor-like traversal of all keys stored in Redis, avoiding performance issues with KEYS commands. There are also commands such as SSCAN/HSCAN/ZSCAN for cursor traversal of elements in the Set/Hash/Sorted Set. Please refer to the official documentation for the use of the SCAN class command: https://redis.io/commands/scan
Redis provides the Slow Log feature to automatically log long-running commands. There are two related configuration parameters:
Slowlog-log-slower-than xxxms #Execution time is slower than xxx milliseconds. The command counts into Slow Logslowlog-max-len xxx #Slow Log, which is the maximum number of Slow Log records.
Use the SLOWLOG GET [number] command to output the number of commands that have recently entered the Slow Log. Use the SLOWLOG RESET command to reset the Slow Log
Network-induced delay
Use long connections or connection pools whenever possible to avoid frequent creation of destroyed connections
Bulk data operations performed by the client should be done in one interaction using the Pipeline feature. Please refer to the Pipelining chapter of this article for details.
Delay caused by data persistence
Redis' data persistence work itself introduces delays that require a reasonable persistence strategy based on the security level and performance requirements of the data:
Although the setting of AOF + fsync always can ensure data security absolutely, each operation will trigger fsync once, which will have a significant impact on the performance of Redis.
AOF + fsync every second is a better compromise, fsync once per second
AOF + fsync never provides optimal performance under AOF persistence schemes. Using RDB persistence usually provides higher performance than using AOF, but you need to pay attention to the policy configuration of RDB.
Every time an RDB snapshot and AOF Rewrite require a Redis main process for a fork operation. The fork operation itself can be time consuming, depending on the amount of memory used by the CPU and Redis. Properly configure RDB snapshots and AOF Rewrite timings based on specific conditions to avoid delays caused by too frequent forks
Redis needs to copy the memory paging table to the child process in the fork child process. Take the Redis instance that takes up 24GB of memory as an example. A total of 24GB / 4kB * 8 = 48MB of data needs to be copied. On a physical machine using a single Xeon 2.27Ghz, this fork operation took 216ms.
You can view the time (microseconds) of the last fork operation from the latest_fork_usec field returned by the INFO command.
Swap caused delay
When Linux moves the memory used by Redis to the swap space, it will block the Redis process, causing an abnormal delay in Redis. Swap usually occurs when there is insufficient physical memory or some processes are performing a large number of I/O operations, and the above two situations should be avoided as much as possible.
The swap record of the process is saved in the /proc//smaps file. By looking at this file, you can determine whether the delay of Redis is generated by Swap. If a large Swap size is recorded in this file, the delay is most likely caused by Swap.
Delay caused by data elimination
Redis delays are also raised when a large number of keys expire in the same second. Try to stagger the failure time of the key when using it.
Introduce read and write separation mechanism
Redis' master-slave replication capability enables a multi-node architecture with multiple masters and slaves. Under this architecture, the master node receives all write requests and synchronizes data to multiple slave nodes.
On this basis, we can let the slave node provide a read request service with low real-time requirements to reduce the pressure on the master node.
Especially for some statistical tasks that use long time-consuming commands, you can specify to execute on one or more slave nodes to avoid these long time-consuming commands affecting the response of other requests.
For specific instructions on read and write separation, please refer to the subsequent chapters.
Master-slave replication and cluster sharding
Master-slave replication
Redis supports a master-slave replication architecture with one master and multiple slaves. A Master instance handles all write requests, and the Master synchronizes writes to all Slaves.
Read and write separation and high availability with Redis' master-slave replication:
Real-time requirements are not particularly high read requests and can be done on the Slave to improve efficiency. In particular, some statistical tasks that are performed periodically. These tasks may require some long-time Redis commands. You can specifically plan one or several slaves to serve these statistical tasks.
High availability is achieved with Redis Sentinel, and when Master crashes, Redis Sentinel automatically promotes a slave to Master and continues to provide services.
Enabling master-slave replication is as simple as configuring multiple Redis instances and configuring them in a Redis instance that is a slave:
Slaveof 192.168.1.1 6379 #Specify the IP and port of the Master
When the Slave starts, it will perform a cold start data synchronization from the Master. The Master triggers the BGSAVE to generate the RDB file and pushes it to the Slave for import. After the import is completed, the Master then synchronizes the incremental data to the Slave through the Redis Protocol. After that, the data between the master and the slave is always synchronized with the Redis Protocol.
Use Sentinel to do automatic failover
Redis's master-slave replication function itself is just data synchronization, does not provide monitoring and automatic failover capabilities, to achieve high availability of Redis through master-slave replication, and also needs to introduce a component: Redis Sentinel
Redis Sentinel is an officially developed monitoring component of Redis that monitors the status of Redis instances, automatically discovers Slave nodes through the Master node, and elects a new Master when it detects that the Master node fails, and pushes new Masters to all Redis instances. Configuration.
Redis Sentinel needs to deploy at least 3 instances to form an election relationship.
Key configuration:
Sentinel monitor mymaster 127.0.0.1 6379 2 #IP instance of the IP, port, and the number of votes required for the election sentinel down-after-milliseconds mymaster 60000 #How long does not respond is considered Master failure sentinel failover-timeout mymaster 180000 #two failover Sentinel parallel-syncs mymaster 1 # If there are multiple slaves, you can use this configuration to specify the number of slaves to synchronize data from the new master at the same time, avoiding simultaneous synchronization of all slaves and making the query service unavailable.
It should also be noted that the automatic failover implemented by Redis Sentinel is not completed on the same IP and port. That is to say, the IP and port of the new Master service provided by automatic failover are different from the previous Master, so it is implemented. HA also requires the client to support Sentinel and be able to interact with Sentinel to get information about the new Master.
Cluster fragmentation
Why do clustering:
The amount of data stored in Redis is large, and the physical memory of one host cannot be accommodated.
Redis's write request is concurrency, a Redis instance cannot be hosted
When the above two problems arise, it is necessary to slice Redis.
Redis has a variety of sharding schemes. For example, many Redis clients implement fragmentation by themselves, and there is a proxy-based Redis sharding scheme like Twemproxy. However, the preferred solution should also be the Redis Cluster sharding scheme that Redis officially introduced in version 3.0.
This article does not cover the specific installation and deployment details of Redis Cluster, focusing on the benefits and drawbacks of Redis Cluster.
Redis Cluster capabilities
Ability to automatically spread data across multiple nodes
Automatically forward requests to the correct shard when the accessed key is not on the current shard
Can still provide services when some nodes in the cluster fail
The third point is based on master-slave replication. Each data fragment of Redis Cluster adopts the structure of master-slave replication. The principle is exactly the same as the master-slave replication described above. The only difference is that Redis is omitted. Sentinel is an additional component that is responsible for node monitoring and automatic failover within a shard.
Redis Cluster fragmentation principle
There are 16384 hash slots in the Redis Cluster. Redis calculates the CRC16 of each key and modulates the result with 16384 to determine which hash slot the key is stored in. It also needs to specify that each data fragment in the Redis Cluster is responsible. Slot number. The allocation of Slots can be redistributed at any point in time.
When the client reads and writes the key, it can connect to any fragment in the cluster. If the key of the operation is not within the Slot scope of the fragment, Redis Cluster will automatically redirect the request to the correct fragment.
Swarm tags
In terms of the basic sharding principle, Redis also supports the hash tags function. The key specified in the format required by hash tags will ensure that it enters the same Slot. For example: {uiv}user:1000 and {uiv}user:1001 have the same hash tag {uiv}, which will be saved in the same Slot.
When using Redis Cluster, the keys involved in the pipelining, transaction, and LUA Script functions must be on the same data slice, otherwise an error will be returned. To use the above features in a Redis Cluster, you must use hash tags to ensure that all keys in a pipeline or a transaction are in the same slot.
Some clients (such as Redisson) implement clustered pipelining operations, which can automatically group the commands in a pipeline according to the shards where the keys are located, and send them to different shards for execution. However, Redis does not support cross-sharding transactions, and transactions and LUA Scripts must still follow the rules of all keys on a shard.
Master-slave replication vs cluster fragmentation
When designing a software architecture, how do you choose between a master-slave replication and a cluster sharding deployment?
In all respects, Redis Cluster is a better solution than master-slave replication.
Redis Cluster can solve the problem of excessive data on a single node
Redis Cluster can solve the problem of excessive pressure on single-node access
Redis Cluster includes master-slave replication capabilities
Does that mean that Redis Cluster is always better than master-slave replication?
No.
Software architecture is never the more complex, the complex architecture will bring significant benefits, but it will also bring corresponding drawbacks. The drawbacks of using Redis Cluster include:
Maintenance difficulty increases. When using Redis Cluster, the number of Redis instances that need to be maintained doubles, the number of hosts that need to be monitored increases, and the complexity of data backup/persistence increases. At the same time, when performing the addition and subtraction operation of the slice, the reshard operation is required, which is far more complicated than adding a Slave in the master-slave mode.
Client resource consumption increases. When the client uses the connection pool, it needs to maintain a connection pool for each data fragment. The number of connections that the client needs to maintain at the same time is multiplied, which increases the consumption of the client itself and operating system resources.
Performance optimization is more difficult. You may need to view the Slow Log and Swap logs on multiple shards to locate performance issues.
The cost of using transactions and LUA Script is increasing. There are strict restrictions on using transaction and LUA script features in Redis Cluster. The keys for operations in transactions and scripts must be on the same shard. This makes it necessary to plan and standardize the keys involved in the corresponding scenarios during development. Claim. If the application scenario involves a large amount of transactions and the use of scripts, how to average the data into multiple data slices under the premise of ensuring the normal operation of these two functions becomes a difficulty.
Therefore, when making choices between master-slave replication and cluster sharding, it should be considered from the aspects of application software features, data and access levels, future development planning, etc., only when it is necessary to introduce data points. Use Redis Cluster when you are in the movie. Here are some suggestions:
How big is the data that needs to be stored in Redis? How big is it possible to develop in the next 2 years? Do these data need to be stored for a long time? Can I use the LRU algorithm to eliminate non-hotspot data? Consider the previous factors and evaluate the physical memory that Redis needs to use.
What is the physical memory of the host used to deploy Redis? How many can be assigned to Redis? Is the memory demand assessment in (1) sufficient?
How big is the pressure of concurrent writing by Redis? Redis write performance can exceed 100,000 times per second without pipelining (more benchmarks can be found at https://redis.io/topics/benchmarks)
Will Pipelining and transaction functions be used when using Redis? How many scenes are you using? Based on the above considerations, if the available physical memory of a single host is sufficient to support the capacity requirements of Redis, and the concurrent write pressure faced by Redis is still far from the Benchmark value, it is recommended to adopt a master-slave replication architecture, which can save a lot of unnecessary trouble. At the same time, if pipelining and transactions are heavily used in applications, it is also recommended to choose a master-slave replication architecture as much as possible to reduce the complexity of design and development.
Connecting Terminals,Micro Connecting Terminal,Aluminum Connecting Terminals,Connecting Copper Terminal
Taixing Longyi Terminals Co.,Ltd. , https://www.longyicopperterminals.com