Notes are based of 7DB - Redis - Day 1
Redis was first released around 2009 and was intended to be a fast "data structures" server. For example, it is great to use within a system as a cache storage or as a queue.
Follow the setup instructions
Let's say that you wanted to build a URL shortening service. Let's explore how a DB like Redis can help us in the process.
Like other DBs, we will start off with CRUD operations. Setting is very easy,
just SET <key> <value>
set msu http://montana.edu
And getting is easy as well
get msu
We can also set multiple
mset um http://www.umt.edu goog http://www.google.com
And get multiple
mget msu um goog
Next we explore some of the data types in Redis. They are the building blocks for many systems. First, our system will need some users
Let's assume that we wanted to store users with a name a password (NOTE Never store passwords in clear text, this is only for an illustration). So far we could get fancy with our keys
mset user:dave:name "David Millman" user:dave:password "12345"
mget user:dave:name user:dave:password
But, this is a "data structure" DB, we can do better. In fact, we have a hash type that allows us to store this more naturally.
hmset user:dave name "David Millman" password "12345"
hvals user:dave
Or get the keys
hkeys user:dave
Or one value
hget user:dave password
However, you can only go one level deep on a hash. Redis is very fast to use in a system, but the speed does come at the cost of a simpler data model.
What if we wanted to build a feature that allowed uses to queue up pages that they want to look at. Since a queue is just a list with FIFO semantics it is easy:
rpush dave:wishlist msu goog um
We can get the length of the list
llen dave:wishlist
Or a sublist over a range of indices
lrange dave:wishlist 1 2
Note that the last element is inclusive and we can all to the end with -1
lrange dave:wishlist 1 -1
(We can remove intermediate elements with LREM, see docs for more info.) If
we are going to build a queue, since we pushed from the right, we need to be
able to pop from the left:
lpop dave:wishlist
Question How would you update so that we process the wishlist as a stack?
What about moving content from one list to another? We can use the command
RPOPLPUSH for example:
rpoplpush dave:wishlist dave:visited
Note that RPOPLPUSH is the only command that moves from one list to anther
(since this has to be done atomically), so if you need to move data between
lists, you will need to plan your data structures accordingly.
I think that I have used Redis in some form in almost every big project that I have created over the past 7 years. In part because it is really easy to create a message queue. Let's see an example. In a second terminal start a second client
docker run --net=host -v ${PWD}/env/redis/mnt:/mnt/ -it adv-db/clients redis-cli
In the first client block on a list that waits for comments to post for 300 seconds
BRPOP comments 300
Now, hop back to the first client and post a comment
LPUSH comments "This is super cool!"
Notice that the second client stopped waiting and outputted the comments and if
we check out the contents of comments, our comment was popped. There are
a few other various, see other BR* commands for options.
No collection of data structures would be complete without sets. To illustrate, let's add tags to our sites:
sadd edu cs.montana.edu umt.edu nyu.edu unc.edu
sadd cs cs.montana.edu google.com acm.org
But sets are only really useful if we can perform set operations like intersection
sinter edu cs
Or difference
sdiff edu cs
Or Union
sunion edu cs
Sorted sets are a little slower to work with over hashes (since operations take
time proportional to the log of the size of the set instead of const). Sorted
operations are prefixed with Z. Let's created a sorted list of visits per
shorted url:
zadd visits 10 um 100 msu 50 goog
We can retrieve by range
zrange 1 2
And change scores with ZINCRBY
zincrby visits 10 msu
One of my favorite uses for Redis is a cache, but as with all good caching
systems, you must have someway to determine if data is no longer valid. Time is
a great option, which can be accomplished with EXPIRY for example
set ice "I'm melting...."
expire ice 10
And now lets check
> exists ice
(integer) 1
> exists ice
(integer) 0
In fact, we can set and expire as a 1 liner
setex ice 10 "I'm melting...."
exists ice
...
And we can check the TTL
setex ice 10 "I'm melting...."
ttl ice
...
ttl ice
A common operation in a system is incrementing a counter. Often, one is tempted to do the following:
set my-counter 0
get my-counter
... proccessing ...
... update counter
set my-counter 1
Question If there are multiple users, what is the problem?
So, if all you are doing is incrementing, redis has an increment function to make this easier:
incr my-counter
In fact we can control the amount of the increment
incrby my-counter 10
But what if you want to do a bunch of commands together? Transactions would be helpful!
127.0.0.1:6379> multi
OK
127.0.0.1:6379> set dave http://millman.us
QUEUED
127.0.0.1:6379> incr my-counter
QUEUED
127.0.0.1:6379> exec
1) OK
2) (integer) 13
Question What happens if you miss type a command and have an error while in a transaction?
Note that transactions in Redis are different than we have seen in other DBs. Each command is queued and then run as a batch. So if I run
127.0.0.1:6379> multi
127.0.0.1:6379> set a 1
127.0.0.1:6379> set b 2
127.0.0.1:6379> set c 3
127.0.0.1:6379> discard
None of the previous commands will have run
You can use the commands FLUSHDB and FLUSHALL to remove all keys from the
current db and all dbs. (Note that we didn't cover here, but you can change to
different DBs with SELECT).
Question How would you implement symmetric difference? (e.g. Union \ Intersection)
We will need to make some temp sets (look at SUNIONSTORE and SINTERSTORE)