How to do regression testing for your WordPress site using BackStop JS

WordPress updates breaking your site?

If you’ve been updating your WordPress plugins and themes regularly, you’ve probably accidentally broken your site—or a client’s site—at least once.

The most common issues you’ll encounter after doing updates are:

• Broken styling – pages or posts on your site look all messed up
• Broken functionality – things that used to work stop working, e.g. your optin, your sidebar, etc.

Or, gahd fuhbid, you can even break your “mission critical” checkout or lead generation functionality.

That means you’re missing out on leads and sales until you fix the issue.

Why should I do regression testing?

The biggest reason you should do regression testing on your WordPress sites it because it’s not always obvious that there’s an issue.

You simply can’t keep track of everything.

That goes double for you if you’re managing multiple websites or running an agency.

Often you only find out about a problem because somebody contacts you and lets you know about it.

Maybe even a pissed off customer.

Embarrassing!

We all know WordPress is an amazing and highly flexible CMS that lets you do just about anything you want with your website.

The price? Your SOUL…

The other price?

You have to perpetually stay on top of things like hosting, security, performance, and most relevant to the topic of the day: plugin and theme updates.

So how do you prevent this stressful and potentially expensive situation?

Well, you can sign up to one of our Care Plans and let us worry about it for you.

Or, you can read this tutorial on how to perform regression testing yourself! 🙂

What is regression testing for WordPress?

Regression testing is a process where you compare ‘before’ and ‘after’ snapshots of your WordPress website to see if the updates you’ve performed have caused any changes to the visual layout or functionality of your site.

Sometimes the before and after difference can be so small that you might miss it with the naked eye.

For example, in the screenshot below there’s a subtle difference in font rendering.

There are many tools and scripts that help automate the regression testing process.

We tried a bunch of them and in the end settled on a great free tool called Backstop.

And though we try to automate as much as possible, we also run manual checks on key functionality like checkouts, login pages, etc.

Tools you need to perform WordPress Regression Testing with Backstop

For this regression testing method, you will need the following:

• Node.JS – you can download it for free here 
• NPM – this will be installed alongside Node.JS
• Backstop JS – you can download it for free here or, as I recommend, install it by using the terminal
• Code editor – I’m using Visual Studio Code, but you can use your code editor of choice

Install BackStop JS globally

Note: I strongly recommend you perform the regression testing on a staging website. I use CloudWays for this because they make it so easy to launch and work with staging applications.

After you have installed Node.JS and NPM, now you need to install BackStop JS globally using NPM.

To do that, follow these steps:

1 – Open your terminal and type $ npm install -g backstopjs (for Windows versions) and npm install -g backstopjs (for Mac versions)

Note: for Mac users, in case npm install -g backstopjs doesn’t work, try with sudo npm install -g backstopjs

2 – Create a folder on your desktop (preferably) and use your terminal to create the initial files for testing

For this example, I’ve named my folder ‘MemberFix’.

3 – Open your terminal and use the following commands to go into your folder:

• cd desktop then press the Enter key (this command will select your desktop)
• cd MemberFix then press the Enter key (this command will select your folder)
• backstop init then press the Enter key (this command will create the two initial files required for testing)

4 – Open the newly created folder in your preferred code editor and click on the backstop.json file.

5 – Go to viewports and create a new desktop viewport by using the following code snippet. Make sure that you put a comma (“,”) after the last bracket (“}“).

{

     “label”: “desktop”,

     “width”: 1920,

     “height”: 1080

     }

6 – Go to Scenarios and rename the default label “BackstopJS Homepage” with the name of the page you want to test.

In this case, it will be Memberfix Homepage.

7 – Provide the URL of the page and add a delay of 20.

8 – Copy the last 2 steps for each page you want to test.

Usually, if the Homepage has no issues, the other pages will be ok.

9 – Don’t forget to save the file.

10 – Go back to the browser, open the terminal, and write backstop test

Note: As this is an initial test, you will get some errors because the system doesn’t have any information to compare with yet.

10 – Go back to the terminal and write backstop approve to approve the initial test.

11 – Open the terminal and write backstop test to see if the test has been approved.

You should now see differences between the initial test and the regression test.

If there are any major errors like images or entire sections disappearing, you should fix them before approving the test.

As you can see in the screenshot, the errors are there because the images or fonts are loading too slowly. 

To solve this error, you can increase the delay.

Is regression testing for WordPress worth it?

Yes, by implementing this regression testing process you’ll save a ton of time and catch many issues that you would otherwise miss.

But while this process automates a lot of your post-updates checking, it’s still a good idea to look at key pages and functionality manually.

For example, you want to ensure that your core pages are up, look correct, and are actually working correctly, e.g.

• Your Checkout page
• Login page
• Members home page
• My account page

…etc.

We’re actually in the process of writing scenario templates to automate these checks as well, so stay tuned.

In the meantime, leave a comment below and share your thoughts. 🙂