Introduction
Redis is an open-source, in-memory data structure store. In addition to its core data types, Redis provides the ability to execute server-side programs similar to stored procedures in relational databases. However, in Redis, scripts are executed by an embedded engine, which only supports the Lua interpreter. Lua is a scripting language that supports multiple programming paradigms, including procedural, functional, and object orientation. As part of a Lua script, you can write logic that uses Redis commands to read and write data.
Benefits of Lua scripting with Redis include:
- Efficiency: Lua scripting allows you to execute logic closer to the data. By executing the logic in a Redis server, you can reduce latency and save network bandwidth in return.
- Atomicity: A Lua script is executed atomically in Redis because the entire server is blocked during the duration of script execution. Although this is a powerful feature, it requires you to be careful about the correctness and execution time of Lua scripts.
- Flexibility: Lua is a full-fledged programming language, with it, you can implement server-side features by combining Redis commands. This allows you to build capabilities specific to a set of requirements that may not be natively supported by Redis.
- This article explains how to implement Lua scripting using a Vultr Managed Database for Redis.
Prerequisites
Before you begin:
- Deploy a Vultr Managed Database for Redis.
- When deployed, and ready. Copy the database connection string to connect to the Redis database. For example:
- rediss://default:[DATABASE_PASSWORD]@[DATABASE_HOST]:[DATABASE_PORT]
- Deploy a Ubuntu 22.04 management server.
- Using SSH, access the server as a non-root sudo user and install:
The latest stable GO programming language.
$ sudo snap install go --classic
The Redis CLI tool.
$ sudo apt-get install redis
Lua scripting commands
Redis supports Lua scripting usage via the EVAL, EVALSHA, SCRIPT LOAD, SCRIPT EXISTS, SCRIPT FLUSH, and SCRIPT KILL commands. In this section, try the commands on your database using the Redis CLI tool.
To get started, use redis-cli to access your Vultr Managed Database for Redis using the connection string as below.
$ redis-cli -u rediss://default:[DATABASE_PASSWORD]@[DATABASE_HOST]:[DATABASE_PORT]
Replace DATABASE_PASSWORD, DATABASE_HOST, and DATABASE_PORT with your actual Redis database values.
EVAL
EVAL runs a Lua script on the server. For example, to increment the count of orders for a user, write a script in Lua and run it using EVAL as below.
> EVAL "return redis.call('incr', KEYS[1])" 1 user:123:orders
Output:
(integer) 1
The command increments the value stored in user:123:orders where 1 is the number of keys that the Lua script takes.
SCRIPT LOAD
SCRIPT LOAD adds a script into Redis memory and returns its SHA1 hash for use with EVALSHA. For example:
> SCRIPT LOAD "return redis.call('incr', KEYS[1])"
Your output should look like the one below:
2bab3b661081db58bd2341920e0ba7cf5dc77b25
As displayed in the output, the command loads the script and returns a SHA1 hash. Later, you will use the hash with EVALSHA to run the script.
EVALSHA
[size=5][/size]To run a cached script on the server using its SHA1 hash generated by the SCRIPT LOAD command, use the syntax below.
> EVALSHA 2bab3b661081db58bd2341920e0ba7cf5dc77b25 1 user:123:orders
Output:
(integer) 2
The command runs the previously loaded script using its SHA1 hash.
SCRIPT EXISTS
SCRIPT EXISTS checks if a script exists in the Redis cache as below:
> SCRIPT EXISTS 2bab3b661081db58bd2341920e0ba7cf5dc77b25
Output:
1) (integer) 1
The command returns 1 if the script exists in the cache and 0 if otherwise.
SCRIPT FLUSH
SCRIPT FLUSH removes all scripts from the Redis script cache.
> SCRIPT FLUSH
Output:
OK
SCRIPT KILL
SCRIPT KILL kills the Lua script in execution, assuming no write operation is performed by the script.
> SCRIPT KILL
Depending on the script state, your output may look like the one below:
(error) NOTBUSY No scripts in execution right now.
The command stops the Lua script from execution if no write operations are performed yet.
Initialize the project
In this section, execute Lua scripts in a Go application with the help of the go-redis client as described in the following steps.
1.Exit the Redis shell.
> EXIT
2.Create a directory to store your Lua scripts. For example redis-lua-scripting.
$ mkdir redis-lua-scripting
3.Switch to the directory.
$ cd redis-lua-scripting
4.Create a new Go module.
$ go mod init redis-lua-scripting
The above command creates a new go.mod file
5.Create a new file main.go.
$ touch main.go
Use Eval and EvalRO functionsThe Eval function makes it possible to invoke a server-side Lua script. It has a variant, EvalRO that prohibits commands which mutate data from executing in Redis. When a script is executed, it's cached in Redis, and can be uniquely identified using its SHA1 hash. The EvalSha function loads a script from the Redis cache its SHA1 hash and executes it as described in this section.
1.Using a text editor such as Nano, edit the main.go file.
$ nano main.go
2.Add the following code to the file.
package main
import (
"context"
"crypto/tls"
"fmt"
"log"
"os"
"github.com/go-redis/redis/v9"
)
var client *redis.Client
const luaScript = `
local key = KEYS[1]
local value = ARGV[1] + ARGV[2]
redis.call("SET", key, value)
return value
`
func init() {
redisURL := os.Getenv("REDIS_URL")
if redisURL == "" {
log.Fatal("missing environment variable REDIS_URL")
}
opt, err := redis.ParseURL(redisURL)
if err != nil {
log.Fatal("invalid redis url", err)
}
opt.TLSConfig = &tls.Config{}
client = redis.NewClient(opt)
_, err = client.Ping(context.Background()).Result()
if err != nil {
log.Fatal("ping failed. could not connect", err)
}
}
func main() {
script := redis.NewScript(luaScript)
result, err := script.Eval(context.Background(), client, []string{"sum"}, "10", "20").Int()
if err != nil {
fmt.Println("lua scrip eval failed", err)
}
fmt.Println("lua script eval sum result", result)
result, err = script.EvalSha(context.Background(), client, []string{"sum"}, "10", "20").Int()
if err != nil {
fmt.Println("lua scrip eval failed", err)
}
fmt.Println("lua script eval sha sum result", result)
_, err = script.EvalRO(context.Background(), client, []string{"sum"}, "40", "21").Int()
if err != nil {
fmt.Println("lua scrip eval read_only failed", err)
}
}
Save and close the file.
3.To run the program, fetch the Go module dependencies.
$ go get
4.To use your Redis connection in the file, add your Vultr Managed Database for Redis connection string to the REDIS_URL variable.
$ export REDIS_URL=rediss://default:[DATABASE_PASSWORD]@[DATABASE_HOST]:[DATABASE_PORT]
Replace DATABASE_PASSWORD, DATABASE_HOST, and DATABASE_PORT with your actual database values.
5.Run the program.
$ go run main.go
Your output should look like the one below:
lua script eval sum result 30
lua script eval sha sum result 30
lua scrip eval read_only failed ERR Write commands are not allowed from read-only scripts. script: abf78cadfdce42a3df3df54bf8545340bccd5289, on @user_script:4.In this section, you imported the required packages and defined the Lua script. Then, the init function reads the Vultr Managed Database for Redis URL from the REDIS_URL environment variable and fails if it's missing. The program creates a new redis.Client instance and verifies connectivity to Vultr Managed Database for Redis by using the Ping utility. If connectivity fails, the program exists with an error message.
In the Go file, a Lua script is used to calculate the sum of two numbers and store it in another key. The Eval function executes the script, when successful, the result is printed. The EvalSha function also works as expected, and the EvalRO function fails with an error.
How to use Run and RunRO functions
The Run function provides a convenient way of executing Lua scripts because it optimistically uses EvalSha and retries the execution using Eval if the script does not exist. Similar to the EvalRO function, RunRO uses EvalSha_RO to run the script and retries using Eval_RO when required. In this section, use the Run or RunRO functions to run Lua scripts as described below.
1.Back up the original main.go file.
$ mv main.go main.ORIG
2.Create a new main.go file.
$ nano main.go
3.Add the following contents to the file.
package main
import (
"context"
"crypto/tls"
"fmt"
"log"
"os"
"github.com/go-redis/redis/v9"
)
var client *redis.Client
const luaScript = `
local key = KEYS[1]
local value = ARGV[1] + ARGV[2]
redis.call("SET", key, value)
return value
`
func init() {
redisURL := os.Getenv("REDIS_URL")
if redisURL == "" {
log.Fatal("missing environment variable REDIS_URL")
}
opt, err := redis.ParseURL(redisURL)
if err != nil {
log.Fatal("invalid redis url", err)
}
opt.TLSConfig = &tls.Config{}
client = redis.NewClient(opt)
_, err = client.Ping(context.Background()).Result()
if err != nil {
log.Fatal("ping failed. could not connect", err)
}
}
func main() {
script := redis.NewScript(luaScript)
result, err := script.Run(context.Background(), client, []string{"sum_run"}, "10", "20").Int()
if err != nil {
fmt.Println("lua script run failed", err)
}
fmt.Println("lua script run sum result", result)
_, err = script.RunRO(context.Background(), client, []string{"sum_run"}, "40", "21").Int()
if err != nil {
fmt.Println("lua script run read_only failed", err)
}
}Save and close the file.
4.Run the program.
$ go run main.go
Your output should look like the one below:
lua script run sum result 30
lua scrip run read_only failed ERR Write commands are not allowed from read-only scripts. script: abf78cadfdce42a3df3df54bf8545340bccd5289, on @user_script:4.
In this section, you imported required packages to the file and used the Run function to execute the Lua script, when successful, the result is printed, and as expected, the RunRO function fails with an error.
Implement increment functionality with a Lua script
Redis has an in-built INCRBY command that increments a counter atomically. In this section, implement the functionality using a Lua script and examine its atomicity characteristics.
1.Back up the main.go file.
$ mv main.go main.ORIG.1
2.Create a new main.go file.
$ nano main.go
3.Add the following contents to the file.
package main
import (
"context"
"crypto/tls"
"fmt"
"log"
"os"
"sync"
"github.com/go-redis/redis/v9"
)
var client *redis.Client
const counterName = "test_counter"
func init() {
redisURL := os.Getenv("REDIS_URL")
if redisURL == "" {
log.Fatal("missing environment variable REDIS_URL")
}
opt, err := redis.ParseURL(redisURL)
if err != nil {
log.Fatal("invalid redis url", err)
}
opt.TLSConfig = &tls.Config{}
client = redis.NewClient(opt)
_, err = client.Ping(context.Background()).Result()
if err != nil {
log.Fatal("ping failed. could not connect", err)
}
}
func main() {
var wg sync.WaitGroup
wg.Add(5)
for i := 1; i <= 5; i++ {
go func() {
LuaIncrBy(client, counterName, 2)
wg.Done()
}()
}
fmt.Println("waiting for operations to finish")
wg.Wait()
fmt.Println("all operations finished")
result := client.Get(context.Background(), counterName).Val()
fmt.Println("final result", result)
}
func LuaIncrBy(c *redis.Client, key string, counter int) int {
incrByScript := redis.NewScript(`
local key = KEYS[1]
local counter = ARGV[1]
local value = redis.call("GET", key)
if not value then
value = 0
end
value = value + counter
redis.call("SET", key, value)
return value
`)
k := []string{key}
val, err := incrByScript.Run(context.Background(), c, k, counter).Int()
if err != nil {
log.Fatal("lua script execution failed", err)
}
return val
}
Save and close the file.
Elements of the Lua script include:
key and counter are local variables retrieved from the script key and argument, respectively.
If the key is not found, its value is set to 0.
The value is incremented by the provided counter and the updated value is SET.
The main function:
A sync.WaitGroup object is defined and initialized to 5.
goroutine spawns five instances, each of which invokes the LuaIncrBy operation concurrently. This verifies that the Lua script execution is indeed atomic.
Waits for all the instances of goroutine to finish.
When all five operations are complete, the program prints the final value (result of increment) and exits.
4.Run the program.
$ go run main.go
Your output should look like the one below:
waiting for operations to finish
all operations finished
final result 10
In this section, you imported necessary packages, and used the LuaIncrBy function which provides the ability to increment a given key with the specified counter. Within the file, a Lua script is defined, and it's executed using the Run function by passing in the key and counter value. When successfully executed, the result is returned, or the program exits with an error.
Conclusion
In this article, you have implemented Lua scripts to execute server-side logic using a Vultr Managed Database for Redis database. You ran Lua scripting commands with the Redis CLI and several programs that use Lua scripts in Go.