Speed Up MongoDB Queries with Redis on Golang

[Administrator]

Introduction

A cache utilizes the RAM of your computer to store and serve frequently accessed data from backend systems like databases. As a result, a caching layer improves performance, minimizes costs, and reduces the load on a database.

The Redis in-memory database server is a suitable caching application because it requires minimal setup and scales well in a production environment. When you implement a cache, your application first queries Redis to get data. If the cache is empty, the application fetches data from the database. Then, it creates a cache for future queries. This in-memory caching solution is several times faster when compared to rotating or modern Solid State Disks (SSD).

This guide walks you through the process of implementing a caching layer with Redis to speed up MongoDB queries using the Go programming language.
Prerequisites
Deploy an Ubuntu 20.04 server.
Create a non-root sudo user.
Install and configure a MongoDB server.
Set up a Redis server.
Download and install the Go programming language.
1. Set Up a Sample Database

Code Select Expand

This guide uses a sample MongoDB database to store data permanently on a disk. Follow the steps below to create the database.

Log in to your MongoDB server. Replace mongo_db_admin with the correct user account.

$ mongosh -u mongo_db_admin -p --authenticationDatabase admin
Enter your password to proceed.

Run the statement below to create an e-commerce database.

test> use e-commerce
Output.

switched to db e-commerce
Insert five documents into a new products collection.

e-commerce> db.products.insertMany([
            {"product_id" : 1,
             "product_name" : "INSTANT WATER HEATER",
             "retail_price" : 45.55   
            },
            {"product_id" : 2,
             "product_name" : "DOUBLE SOCKET WITH PATTRESS",
             "retail_price" : 6.65   
            },
            {"product_id" : 3,
             "product_name" : "80MM USB PRINTER",
             "retail_price" : 125.95
            },
            {"product_id" : 4,
             "product_name" : "FITNESS SMARTWATCH",
             "retail_price" : 39.85
            },
            {"product_id" : 5,
             "product_name" : "3.1A FAST CHARGER",
             "retail_price" : 23.90
            }     
           ]);
Output.

{
  acknowledged: true,
  insertedIds: {
    '0': ObjectId("625d2694dd1d8bd34f469875"),
    '1': ObjectId("625d2694dd1d8bd34f469876"),
    '2': ObjectId("625d2694dd1d8bd34f469877"),
    '3': ObjectId("625d2694dd1d8bd34f469878"),
    '4': ObjectId("625d2694dd1d8bd34f469879")
 }
}
Querying the products collection to verify the documents.

e-commerce> db.products.find()
Output.

[
  {
    _id: ObjectId("625d2694dd1d8bd34f469875"),
    product_id: 1,
    product_name: 'INSTANT WATER HEATER',
    retail_price: 45.55
  },
  {
    _id: ObjectId("625d2694dd1d8bd34f469876"),
    product_id: 2,
    product_name: 'DOUBLE SOCKET WITH PATTRESS',
    retail_price: 6.65
  },
  {
    _id: ObjectId("625d2694dd1d8bd34f469877"),
    product_id: 3,
    product_name: '80MM USB PRINTER',
    retail_price: 125.95
  },
  {
    _id: ObjectId("625d2694dd1d8bd34f469878"),
    product_id: 4,
    product_name: 'FITNESS SMARTWATCH',
    retail_price: 39.85
  },
  {
    _id: ObjectId("625d2694dd1d8bd34f469879"),
    product_id: 5,
    product_name: '3.1A FAST CHARGER',
    retail_price: 23.9
  }
]
Log out from the MongoDB database.

e-commerce> quitCreate the main.go File
The main.go file is the entry point in this sample application. This file contains the main(...){...} function that fires when your application starts. Execute the steps below to set up the file.

Create a new directory for your project. This approach keeps your project organized and avoids conflicting your source code with the rest of the Linux files.

Code Select Expand

$ mkdir project
Switch to the new project directory.

$ cd project
Open a new main.go file in an editor.

$ nano main.go
Enter the following information into the file. Replace mongo_db_admin and EXAMPLE_PASSWORD with the correct account details for the MongoDB server.

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "net/http"
)

const ( 
    MONGO_DB_USER = "mongo_db_admin"
    MONGO_DB_PASS = "EXAMPLE_PASSWORD"
    MONGO_DB_NAME = "e-commerce"   
)

func main() {
     http.HandleFunc("/products", requestHandler)
     http.ListenAndServe(":8080", nil)
}

func requestHandler(w http.ResponseWriter, req *http.Request) {   

    w.Header().Set("Content-Type", "application/json")

    var respMessage map[string]interface{}
    var respErr error

    ctx := context.Background()

    isCached, productsCache, err := getFromCache(ctx)

    if err != nil { 

        respErr = err

    } else {

        if isCached == true {

            respMessage = productsCache
            respMessage["_source"] = "Redis Cache"

        } else {

            respMessage, err = getFromDb(ctx);

            if err != nil {       
                respErr = err
            }   

            err = addToCache(ctx, respMessage)           

            if err != nil {       
                respErr = err
            }

            respMessage["_source"] = "MongoDB database"
        }
    }

    if respErr != nil  {

        fmt.Fprintf(w, respErr.Error())

    } else {

        enc := json.NewEncoder(w)
        enc.SetIndent("", "  ")

        if err := enc.Encode(respMessage); err != nil {       
            fmt.Fprintf(w, err.Error())
        }

    }   
}Save and close the file.

In the above file, you're creating an HTTP server that listens for incoming connections on port 8080.

In the main(...){...} function, you're redirecting incoming requests to the requestHandler(...){...} function. Under this function, you're using the isCached, productsCache, err := getFromCache(ctx) statement to retrieve products from a Redis Cache. In case the cache is empty, you're fetching the data from the MongoDB database using the statement respMessage, err = getFromDb(ctx);.

Then, you're outputting the products' data in a JSON format. You're also appending a _source attribute to mark the data source. That is, either from the cache or the database.