Progressive web app essentials: Service worker background sync

Progressive web apps (PWAs) are an important concept in web development because they combine the universality of browser deployment with the features of natively installed software. In delivering native-like functionality, being able to run background and offline operations is key.

An especially critical feature of progressive web apps is their ability to deal with situations where a user has submitted something like an email but the network is not available to process it. In this article, you’ll learn how service workers and the Sync API support offline processing in PWAs. You’ll also be introduced to IndexedDB, the browser’s built-in database.

Introduction to service workers

Considerable thinking goes into making a browser-based application behave more like a native one. The service worker is a key part of that process. A service worker is a more constrained variant of a worker thread, one that communicates with the main browser thread only via event messages, and which does not have access to the DOM. A service worker is a kind of environment unto itself, and we’ll get a taste of that here.

Despite their limitations, service workers are quite powerful in that they have their own lifecycle, independent of the main thread, in which a variety of background operations can be performed. In our case, we are interested in the Sync API, which lets a service worker observe the state of the network connection and attempt to retry a network call until it is successful.

The Sync API and sync events

Imagine if you wanted to configure a retry mechanism that said something like this:

• If the network is up, send the request right now.
• If the network is down, wait until the network is available and try again.
• If any retries fail, be smart about trying again with an exponential back-off and give-up settings.

This would require a lot of finagling. Fortunately, service workers include a specialized “sync” event for this very purpose.

Service workers are registered using the navigator.serviceWorker. This object is only available in a secure context, meaning the website must be loaded in HTTPS, not HTTP.

Our application will deal with the creation of a to-do in the tradition of the canonical TODO sample app. We’ll specifically see how we can take the creation of a new to-do and handle it using a service worker’s sync event, giving us the resilient retry mechanism we need. We’ll ignore everything else to focus on service worker syncing for this example.

Setting up a service worker sync

Let’s dive right into the code. We’ll see how to manage the whole lifecycle of a PWA application that has an operation that requires syncing. For that, we need a full-stack setup. In this setup, a request to create a new TODO is made, and if the network is good, it is sent to the server and saved, but if the network is down, the request is retried in the background whenever the network is available. 

We’ll use Node and Express to serve the front end and handle the API request. 

To begin we need Node/npm installed. Assuming that’s set, we can start a new app with: the following call:

$ npm init -y

That call creates the scaffolding for a new application. Next, we add the express dependency:

$ npm install express

Now we can create a simple Express server that will serve some static files. Next, we’ll create a new index.js file in the root directory and put the following in there:

const express = require('express');
const path = require('path');  // Required for serving static files

const app = express();
const port = process.env.PORT || 3000;  // default to 3000

app.use(express.static(path.join(__dirname, 'public')));

// Serve the main HTML file for all unmatched routes (catch-all)
app.get('*', (req, res) => {
  res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

app.listen(port, () => {
  console.log(`Server listening on port ${port}`);
});

If you run the app, it will serve whatever is in /public. To make it simple to run, open the package.json file the NPM created for us and add a start script like so:

"scripts": {
    "start": "node index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  }

From the command line, you can run the app with:

$ npm run start

This application won’t do anything yet. Add a new index.html file to the /public directory:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>PWA InfoWorld</title>
</head>
<body>
  <h1>My To-Do List</h1>
  <input type="text" id="new-task" placeholder="Add a task">
  <button onclick="addTask(document.getElementById('new-task').value)">Add</button>
  <ul id="task-list"></ul>
  <script src="script.js"></script>
</body>
</html>

Now, if you visit localhost:3000, you’ll get the basic HTML layout with a title, input box, and button. Notice that the button, when clicked, will grab the value of the new-task input and pass it to the addTask() function. 

The main script

addTask() is provided by script.js. You can paste-in the contents of that file if you are following along:

if ('serviceWorker' in navigator) { // 1
  window.addEventListener('load', () => { // 2
    navigator.serviceWorker.register('sw.js') // 3
      .then(registration => { // 4
        console.log('Service Worker registered', registration); // 5
      })
      .catch(err => console.error('Service Worker registration failed', err)); // 6
  });
}

const taskChannel = new BroadcastChannel('task-channel'); // 7

function addTask(task) { // 8
  taskChannel.postMessage({ type: 'add-task', data: task }); // 9
}

Notice that we are not actually concerned with displaying the to-do’s; we are exclusively following the process of submitting it to the back end.

I’ve put comments with numbers in the code. Let’s walk through them here:

1. Checks for the existence of serviceWorker on navigator. It may not be present if it’s not a secure context.
2. If the serviceWorker is present, add a callback to the load observer so it will react when the page is loaded.
3. Use the register method to request that the sw.js file be loaded as a service worker. (We’ll walk through sw.js next.)
4. After the sw.js file is loaded, we get a callback with the registration object. 
5. The registration object can be used to perform various tasks, but for our needs, we just log the success.
6. We also log any errors using the catch() promise callback.
7. We create a BroadcastChannel named “task-channel”. This is a simple way to communicate events to the service worker that will be running based on the code in sw.js.
8. The addTask() function itself, called by the HTML file.
9. We send a message on the task-channel broadcast, and set the type to “add-task” and the data field to the task itself.

In this example, we are ignoring how the UI itself would handle the task creation. We could use a few different approaches including an optimistic one, wherein we put the task into the UI list and then attempt syncing with the back end. Alternatively, we could attempt the back-end sync and then, once successful, send a message to the UI to add the task. (The BroadcastChannel makes it easy to send messages in either direction, from the main thread to the service worker or vice versa.)

Running on HTTPS

We’ve noted that you have to be running on HTTPS (not HTTP) for serviceWorker to exist on navigator. This is a security constraint. To get an HTTPS connection with the minimum of effort, I used ngrok. This handy command-line tool lets you expose your local environment to the world with no configuration and it includes HTTPS. For example, if you run the sample app ($ npm run start) and then type $ ngrok http 3000, it will spawn a tunnel and display the HTTP and HTTPS endpoints you can put into the browser address bar. For example:

Forwarding                    https://8041-35-223-70-178.ngrok-free.app -> http://localhost:3000

Now you can visit https://8041-35-223-70-178.ngrok-free.app, and the app will be served over HTTPS. 

Interacting with the service worker

The sw.js is used to interact with the actual service worker. Remember, we told the browser to load this file with serviceWorker. 

You’ll notice we are using IndexedDB in this code. The reason is that the browser is free to destroy and create service worker contexts as needed to handle events, so we have no guarantee that the same context will be used to handle the broadcast event and the sync event. So, that eliminates local variables as a reliable medium. LocalStorage is not available in service workers. It might be possible to use CacheStorage (which is available in both main and service threads), but that is really designed for caching request responses.

That leaves IndexedDB, which lives across instances of the service worker. We just use it to push the new task in when the add-task broadcast happens, and then pull it out when the sync event occurs.

Here are the contents of sw.js. We’ll walk through this code after you’ve had a look.

const URL = "https://8014-35-223-70-178.ngrok-free.app/"; // 1
const taskChannel = new BroadcastChannel('task-channel'); // 2
taskChannel.onmessage = event => { // 3
  persistTask(event.data.data); // 4
  registration.sync.register('task-sync'); // 5
};

let db = null; // 6
let request = indexedDB.open("TaskDB", 1); // 7
request.onupgradeneeded = function(event) { // 8
  db = event.target.result; // 9
  if (!db.objectStoreNames.contains("tasks")) { // 10
    let tasksObjectStore = db.createObjectStore("tasks", { autoIncrement: true }); // 11
  }
};
request.onsuccess = function(event) { db = event.target.result; }; // 12
request.onerror = function(event) { console.log("Error in db: " + event); }; // 13

persistTask = function(task){ // 14
  let transaction = db.transaction("tasks", "readwrite");
  let tasksObjectStore = transaction.objectStore("tasks");
  let addRequest = tasksObjectStore.add(task);
  addRequest.onsuccess = function(event){ console.log("Task added to DB"); };
  addRequest.onerror = function(event) { console.log(“Error: “ + event); };
}
self.addEventListener('sync', async function(event) { // 15
  if (event.tag == 'task-sync') {
    event.waitUntil(new Promise((res, rej) => { // 16
      let transaction = db.transaction("tasks", "readwrite");
      let tasksObjectStore = transaction.objectStore("tasks");
      let cursorRequest = tasksObjectStore.openCursor();
      cursorRequest.onsuccess = function(event) { // 17
        let cursor = event.target.result;
        if (cursor) {
          let task = cursor.value; // 18
          fetch(URL + 'todos/add', // a
            { method: 'POST', 
              headers: { 'Content-Type': 'application/json' },
              body: JSON.stringify({ "task" : task }) 
            }).then((serverResponse) => {
              console.log("Task saved to backend.");
              deleteTasks(); // b
              res(); // b
            }).catch((err) => {
              console.log("ERROR: " + err);
              rej(); //c
            })
          }
        } 
    }))
  }
})
async function deleteTasks() { // 19
  const transaction = db.transaction("tasks", "readwrite");
  const tasksObjectStore = transaction.objectStore("tasks");
  tasksObjectStore.clear();
  await transaction.complete;
}

Now let’s talk about what is happening in this code.

1. We need to route our requests through the same secure tunnel we created with ngrok, so we save the URL here.
2. Create the broadcast channel with the same name so we can listen for messages.
3. Here, we are watching for task-channel message events. In responding to these events, we do two things:
4. Call persistTask() to save the new task to IndexedDB.
5. Register a new sync event. This is what invokes the special capability for retrying requests intelligently. The sync handler allows us to specify a promise that it will retry when the network is available, and implements a back off strategy and give-up conditions.
6. With that done, we create a reference for our database object.
7. Obtain a “request” for the handle on our database. Everything on IndexedDB is handled asynchronously. (For an excellent overview of IndexedDB, I recommend this series.)
8. The onupgradeneeded event fires if we are accessing a new or up-versioned database. 
9. Inside onupgradeneeded, we get a handle on the database itself, with our global db object.
10. If the tasks collection is not present, we create the tasks collection.
11. If the database was successfully created, we save it to our db object.
12. Log the error if the database creation failed.
13. The persistTask() function called by the add-task broadcast event (4). This simply puts the new task value in the tasks collection.
14. Our sync event. This is called by the broadcast event (5). We check for the event.tag field being task-sync so we know it’s our task-syncing event.
15. event.waitUntil() allows us to tell the serviceWorker that we are not done until the Promise inside it completes. Because we are in a sync event, this has special meaning. In particular, if our Promise fails, the syncing algorithm will keep trying. Also, remember that if the network is unavailable, it will wait until it becomes available.
    1. We define a new Promise, and within it we begin by opening a connection to the database.
16. Within the database onsuccess callback, we obtain a cursor and use it to grab the task we saved. (We are leveraging our wrapping Promise to deal with nested asynchronous calls.)
17. Now we have a variable with the value of our broadcast task in it. With that in hand:
    1. We issue a new fetch request to our expressJS /todos/add endpoint.
    2. Notice that if the request succeeds, we delete the task from the database and call res() to resolve our outer promise.
    3. If the request fails, we call rej(). This will reject the containing promise, letting the Sync API know the request must be retried.
18. The deleteTasks() helper method deletes all the tasks in the database. (This is a simplified example that assumes one tasks creation at a time.)

Clearly, there is a lot to this, but the reward is being able to effortlessly retry requests in the background whenever our network is spotty. Remember, we are getting this in the browser, across all kinds of devices, mobile and otherwise.

Testing the PWA example

If you run the PWA now and create a to-do, it’ll be sent to the back end and saved. The interesting test is to open devtools (F12) and disable the network. You can find the “Offline” option in the “throttling” menu of the network tab like so:

Created by Matthew Tyson.

Now, in offline mode, if you add a task and click Add, nothing will happen. The Sync API is watching the network status.

If you then re-enable the network in the network tab, you will see the request being issued to the back end.

Conclusion

Progressive web applications are not simple, but they deliver something extremely valuable: cross-device, native-like functionality within the browser. In this tutorial, you got hands-on, code-first familiarity with several important components of a PWA: service workers, events, syncing, and IndexedDB.

The code for this example can be found here.

More by Matthew Tyson:

• HTMX for Java with Spring Boot and Thymeleaf
• Stream gatherers: A new way to manipulate Java streams
• Intro to multithreaded JavaScript