Puppeteer on Ubuntu 16.04 and Digital Ocean for Integration Tests

Demo code here

Last week I posted about doing integration tests with Puppeteer. This week I want to go more into how to automate these tests. The idea behind these tests is to be able to set and forget it.

Testing in real life

If I was hired by a client to set up these kind of integration tests up, what I would do is exactly what I am going to show in real life. This assumes that I am not doing the development for their application but have simply been hired to do the testing. I’d have my tests recur at an automated time so I could constantly be testing any changes that the client makes during development.

If I was doing the development of the application, I’m not as sure if it would be as important to automate the testing on a recurring, daily basis. In theory, I should be in control of when any changes happen and would test at that point and so I’m not sure if recurring tests at set times make sense.

I am hosting my tests on a droplet on Digital Ocean. I am a big fan of Digital Ocean. It’s been extremely simple

Adding notifications

I want to be notified when the tests start and then if there are any failures. I originally had it to just notify me when there are failures. I think I might have trust issues because without knowing that my tests are running regularly I think I might always wonder if something broke or not. So, I have the simple notification for when the tests start.

If I was to start running tests more often, such as every hour, the start tests notifications probably starts to be noise and is less helpful. In that case, I think it might be more helpful to do something like a reporter that happens once a day that the tests were running and working correctly.

// Main file
import puppeteer, { Browser, Page } from 'puppeteer';
import { getPropertyBySelector, getPropertyByHandle } from 'puppeteer-helpers';
import { expect } from 'chai';
import { fail } from 'assert';
import * as dotenv from 'dotenv';
import Webhook from 'webhook-discord';


// .env file looks something like this

I’m using the dotenv package here to manage my webhook urls. It’s a really simple package to manage environment variables. If you want to use it, just rename the included .sample.env file and update the credentials for whatever you need.

async function notify(message: string) {

    return new Promise(async (resolve) => {

        if (process.env.discordWebhookURL) {
            const hook = new Webhook(process.env.discordWebhookURL);
            await hook.info('Citadel Packaging Testing', message);
        else {


One thing I wanted to do differently with this (and future open source project examples) is make set up as minimal as possible. I want to be notified when the tests start and fail and so I’m going to use Discord webhooks. If someone else picks up this project, I didn’t want them to have to worry about setting up Discord webhooks. Right now, the script will function out of the box without setting up any third party services. You can also easily update your notification third party service and swap to slack or even just email. So, this checks to see if the environment file has a discordWebhookURL and if it does, it’s going to send the notifications to Discord. If it doesn’t, it just logs it to the console.

Installing puppeteer on Ubuntu 16.04

This part is going to assume that you have already Node installed on your Ubuntu box. If you do not or don’t know how to do that, I went into more detail in this post.

Now if we were on a windows (and probably mac?) machine, we’d be good to go. Running even our npm run test:ubuntu which should run Puppeteer ready for ubuntu will still throw the following error:

An error has occurred:  Something went wrong in the initial set up: Error: Failed to launch chrome!
/home/<path>/node_modules/puppeteer/.local-chromium/linux-609904/chrome-linux/chrome: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory

You will need to install some additional depedencies:

sudo apt-get install libx11-xcb1 libxcomposite1 libxi6 libxext6 libxtst6 libnss3 libcups2 libxss1 libxrandr2 libasound2 libpangocairo-1.0-0 libatk1.0-0 libatk-bridge2.0-0 libgtk-3-0

After this, running npm run test:ubuntu should work!

Scheduling the tests

To the cron! We’re simply going to open up our cronjob with crontab -e and add the cronjob. This below code will run the task twice a day every 12 hours.

# This happens every 12 hours, at 1am and 1pm UTC
0 1,13 * * * cd /home/jordan-does-integration-tests-on-citadel-packaging && /root/.nvm/versions/node/v6.9.5/bin/mocha --timeout 30000 -r /home/jordan-does-integration-tests-on-citadel-packaging/node_modules/ts-node/register /home/jordan-does-integration-tests-on-citadel-packaging/src/test/**/*.spec.ts --exit ubuntu

Scheduling Mocha tests with cron wasn’t the easiest thing for me to get working. There may be a better way to do this but in order to get Mocha running from the path, I had to add the following to my package.json: "preinstall": "npm i -g mocha", . This will ensure that Mocha is installed globally and allows you to run it as above. If you find a better way, I’d love to hear it!

If you want a different frequency, there is a bunch of information on how to do this. I think https://crontab.guru/ is a helpful tool in figuring out what you need for the frequency you want.

And that is the end! Get puppeteer installed, schedule the tests, and get notified.

Demo code here

Looking for business leads?

Using the techniques talked about here at javascriptwebscrapingguy.com, we’ve been able to launch a way to access awesome web data. Learn more at Cobalt Intelligence!

Leave a Reply