How to Build a Serverless Order Processing System on Google Cloud

Introduction

In this guide, we’ll design and build a serverless order processing system on Google Cloud. This system will allow users to place orders via a REST API, process these orders in the background, send email notifications upon completion, and log all relevant events. The key components involved are:

• Google Cloud API Gateway: To expose a RESTful API for order placement.
• Cloud Functions: To handle order creation and processing.
• Firestore: To store order details.
• Cloud Tasks: To queue and process orders asynchronously.
• Cloud Logging: To log all operations for monitoring and debugging.
• Firebase Cloud Messaging API: To deliver notifications and messages to iOS, Android, and web applications.

Setting Up Our Google Cloud Project

Create a New Project

First, we need to create a new project within the Google Cloud Console. Here’s how we can do it:

1. Navigate to Google Cloud Console: Open the Google Cloud Console in our browser.
2. Access the Project Dropdown: In the top navigation bar, we’ll find a dropdown menu that lists our current projects. Click on this dropdown and select “New Project”.
3. Name Our Project: A dialog box will appear prompting us to name our project. Let’s name it something relevant, like “OrderProcessingSystem”, to keep our projects organized. After naming, click “Create”.

By following these steps, we create a dedicated workspace for our order processing application, ensuring that all configurations and resources are neatly contained.

Enable Required APIs

With our project created, the next crucial step is to enable the necessary APIs that our serverless order processing system will use. Enabling these APIs allows our project to access Google Cloud’s powerful features. Here’s what we need to do:

1. Navigate to APIs & Services: On the left-hand side of the Google Cloud Console, we’ll find the navigation menu. From here, select “APIs & Services” and then click on “Library”.
2. Enable Cloud Functions API: In the API Library, search for “Cloud Functions API”. Click on it and then hit the “Enable” button. Cloud Functions allow us to run our code in the cloud without managing servers, making it perfect for event-driven logic.
3. Enable Firestore API: Next, search for “Firestore API”. Enable it to use Google’s NoSQL document database for our application’s data storage, offering flexibility and scalability.
4. Enable Cloud Tasks API: Search for “Cloud Tasks API” and enable it. This API is essential for managing the execution of large numbers of distributed tasks, helping us handle asynchronous processing efficiently.
5. Enable Firebase Cloud Messaging API: Next, search for “Firebase Cloud Messaging API” and enable it. This API allows us to send notifications and messages to users, enhancing our application’s communication capabilities.

Now, with our project set up and all required APIs enabled, we are ready to start developing our Order Processing System with the powerful features of Google Cloud.

Setting Up Firestore

In this phase, we’ll set up Firestore as our NoSQL database and configure Cloud Functions to handle our CRUD operations and background processing. Let’s initialize Firestore and create the necessary collection:

1. Access Firestore in Google Cloud Console: Go to the Google Cloud Console and navigate to the Firestore section.
2. Create a Database: Click “Create Database”. When prompted, choose the “Native” mode for Firestore, as it’s typically suited for most applications.
3. Select Database Location: Choose a database location that is geographically close to your user base for optimal performance.
4. Select Secure Rules: For this tutorial, we can choose “Test rules”. After selecting the location, click “Create Database”.

By following these steps, we have set up our Firestore database, ready to store and manage our application data.

Create Orders Collection

1. Navigate to Firestore Database: In the Firestore section, go to your newly created database.
2. Create a Collection: Click on “Start Collection”. Name the collection “orders” and proceed to create it.

With the “orders” collection in place, we now have a dedicated space in our database to store order information. This setup will help us manage order data efficiently and facilitate easy retrieval.

Set Up Cloud Functions

Cloud Functions allow us to run backend code in response to events without managing servers. We’ll create functions to handle CRUD operations for our orders.

Create a Cloud Function for Order Creation:

1. Navigate to Cloud Functions: In the Google Cloud Console, go to the “Cloud Functions” section.
2. Create a New Function: Click “Create Function”.
3. Configure the Function:

• Name: Set the function name to “createOrder”.
• Trigger: Choose “HTTP” as the trigger type, allowing the function to be invoked via an HTTP request.
• Authentication: For this tutorial, we choose “Allow unauthenticated invocations”. Click “Save” then “Next”.
• Runtime: Select the runtime environment, such as “Python 3.9”.

Next, we need to write the function code to handle order creation. We’ll use the Firestore SDK to write orders to our database. Here’s an example in Python:

import firebase_admin
from firebase_admin import credentials, firestore
from flask import jsonify, request
from google.cloud import firestore

# Initialize Firestore and Flask app
cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred, {
  'projectId': 'orderprocessingsystem'
})
db = firestore.Client()

def create_order(request):
    try:
        # Parse request body for order data
        order_data = request.get_json()
        
        # Add order to Firestore collection
        db.collection('orders').add(order_data)
        
        # Return success response
        return jsonify({"message": "Order created successfully"}), 200
    except Exception as e:
        # Return error response
        return jsonify({"error": str(e)}), 500

In this code, we initialize Firebase Admin SDK and set up an HTTP-triggered Cloud Function named createOrder. The function reads the order data from the request body and writes it to the “orders” collection in Firestore.

Deploy the Function

• Make sure that all necessary dependencies, including firebase_admin, flask, and any other libraries used in our function, are included in the requirements.txt file.
• Specify the entry point to create_order.
• Deploy the function via the Cloud Console or using the gcloud CLI.
• gcloud functions deploy createOrder --runtime python39 --trigger-http --entry-point create_order --requirements requirements.txt

By following these steps, we have successfully set up Firestore as our database and created a Cloud Function to handle order creation.

After deploying our Cloud Function on Google Cloud, navigate to the Testing tab. In the “Test the Function” section, we can configure the payload for your HTTP request. This is where we specify the JSON data that our function expects to receive.

Example JSON Payload:

{
    "order_id": "1000001",
    "product": "Laptop",
    "quantity": 1,
    "customer_name": "John Doe",
    "customer_email": "john_doe@hotmail.com"
}

After configuring the JSON payload, click on the “Test the function” button.

Navigate to the Firestore to verify the order data has been inserted successfully.

When we deploy a Cloud Function with an HTTP trigger, Google Cloud automatically assigns a URL that we can use to invoke the function. This URL follows the format:

https://REGION-PROJECT_ID.cloudfunctions.net/FUNCTION_NAME

Cloud Tasks for Background Processing

In this step, we’ll integrate Cloud Tasks with Google Cloud Functions to handle background processing of orders. Cloud Tasks allows us to queue tasks asynchronously and process them independently from client-facing operations, ensuring efficient background processing without impacting immediate API responses.

Create a Cloud Task Queue

1. Navigate to Cloud Tasks: Go to the Google Cloud Console and select “Cloud Tasks”
2. Create a Task Queue: Click on “Create Queue” to define a new task queue.
3. Enter Queue Name: Enter “orderqueue” as the queue name.
4. Select The Region: Select the region that is close to our country. Then click “Create”

Cloud Tasks is ideal for scenarios where we need to offload time-consuming tasks (like order processing, and sending email) from our main API thread. It allows us to manage and prioritize background tasks independently, ensuring reliable execution even under varying load conditions.

Develop a Cloud Function for Processing Orders

Now let’s, create another Cloud Function named processOrder that handles order processing tasks asynchronously. This function updates the status of orders in Firestore after a specified delay and sends an email to the customer.

import firebase_admin
from firebase_admin import credentials, firestore
from flask import jsonify, request
import time

# Initialize Firestore and Flask app
cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred, {
  'projectId': 'orderprocessingsystem',
})
db = firestore.Client()

def process_order(request):
  try:
    # Parse request body for order data
    order_data = request.get_json()

    # Simulate order processing
    order_id = order_data.get('orderId')

    doc = db.collection('orders').where('order_id', '==', order_id).limit(1).get()
    doc = doc[0]
  
    # Perform processing tasks (e.g., update order status)
    db.collection('orders').document(doc.id).update({'status': 'processed'})

    # Simulate processing delay
    time.sleep(5)  # Simulate processing taking 5 seconds

    # Send email confirmation (simulated)
    send_email(doc.to_dict().get('customer_email'), 'Order Processed', 'Your order has been processed successfully.')

    # Return success response
    return jsonify({"message": "Order processed successfully"}), 200
  except Exception as e:
    print(f"Error updating status: {e}")
    # Return error response
    return jsonify({"error": str(e)}), 500

def send_email(recipient, subject, message):
  # Simulate sending email (replace with your email service integration)
  print(f"Sending email to {recipient}: {subject} - {message}")

This Python function process_order updates the order status in Firestore and simulates sending an email confirmation to the customer. Using Cloud Tasks ensures that processing-intensive tasks like sending emails are handled asynchronously, preventing delays in API responses to clients.

Verification

Let’s verify our function with the below payload:

{
    "orderId": "1000001"
}

Now, go back to modify our create_order function to enqueue a task for background processing and trigger the process_order function, we can integrate Cloud Tasks to create a task queue and dispatch tasks to our Cloud Function for processing. Here’s how we can achieve this:

import firebase_admin
from firebase_admin import credentials, firestore
from flask import jsonify, request
from google.cloud import firestore
from google.cloud import tasks_v2
import json

# Initialize Firestore and Flask app
cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred, {
    'projectId': 'orderprocessingsystem'
})
db = firestore.Client()

# Initialize Cloud Tasks client
project = 'orderprocessingsystem'
location = 'asia-southeast1'
queue = 'orderqueue'

def create_order(request):
    try:
        # Parse request body for order data
        order_data = request.get_json()
        
        # Add order to Firestore collection
        order_ref = db.collection('orders').add(order_data)
        
        # Enqueue task for background processing
        client = tasks_v2.CloudTasksClient()

        task = tasks_v2.Task(
            http_request=tasks_v2.HttpRequest(
                http_method=tasks_v2.HttpMethod.POST,
                url="https://asia-southeast1-orderprocessingsystem.cloudfunctions.net/processOrder",
                headers={"Content-type": "application/json"},
                body=json.dumps({'orderId': order_data["order_id"]}).encode(),
            )
        )
            
        parent = cloud_tasks_client.queue_path(project, location, queue)
        response = client.create_task(
            tasks_v2.CreateTaskRequest(
                # The queue to add the task to
                parent=client.queue_path(project, location, queue),
                # The task itself
                task=task,
            )
        )
        print(response)
        
        # Return success response
        return jsonify({"message": "Order created successfully", "taskId": response.name}), 200
    
    except Exception as e:
        # Return error response
        return jsonify({"error": str(e)}), 500

Next, go to the requirements.txt file and add in the google-cloud-tasks dependency. After deployment, let’s verify our function with the below payload:

{
    "order_id": "1000002",
    "product": "Laptop",
    "quantity": 1,
    "customer_name": "John Doe",
    "customer_email": "john_doe@hotmail.com",
    "status": "Pending"
}

Now, navigate to the Cloud Task, we should able to see a new task created.

After a while, the task should disappear from the queue. This indicates the task has been processed. Let’s navigate to the Firestore to verify our order status has been updated to processed.

Conclusion

In the next tutorial, we will explore how to integrate a client user interface (UI) with this serverless order processing system. We will focus on building a web or mobile interface that interacts with the API Gateway endpoints (createOrder) and listens for notifications via Firebase Cloud Messaging (FCM). This integration will enable users to place orders and receive real-time notifications on their devices when their orders are processed.