How to Build a QA Test Plan for Your WordPress Plugin Business

Bugs are a part and parcel of all software. If someone says they ship bug free software they are delusional. 

Quality assurance is hard. Quality assurance in the WordPress space is nothing short of a nightmare, but it doesn’t need to be that way. 

I’ve personally worked in quality assurance roles for multiple WordPress plugin businesses including names like WPMU DEV, MailPoet, LearnDash, and more. 

Manual Or Automated Testing 

The harsh reality in the WordPress space is that most plugin companies don’t have the resources available to build out fully automated test suites and code coverage. 

Usually QA testing in the WordPress space comes down to manual QA. 

If you can, add end to end testing (E2E) and try to add tests for everything you can, realistically you won’t be able to. 

So how do we go about building a manual testing plan that actually makes sense?

Building a Manual QA Testing Plan

You might think, note down a few functions, a few features and check they work and away you go. You’d be wrong. And that’s ok, remember QA is hard.

Before you even start with your manual QA testing plan you need to know the product inside and out. You can’t test a product well enough if you don’t understand it. 

So as a first port of call, spend as long as you need building familiarity with the product. Speak to the developers, ask questions, and be curious. 

What You Need in Your Manual QA Testing Plan

Primarily you need your show stopping functionality. For a eCommerce based WordPress plugin like an LMS, a Membership plugin etc that could look like:

• Registration and checkout. 
• Upgrades/downgrades.
• Building a membership in the admin.
• Content protection.
• Integrations and automations. 
• Initial Setup.
• Compatibility with major WordPress plugins (think WooCommerce, Yoast, Elementor, Jetpack). When I say compatibility I mean does your plugin break when those plugins are active? Or do you break them? Not do you integrate with them. 

That’s a really simple high-level list. Each of those bullet points branches into multiple sub points, of sub points. 

Then you need to actually put your expectations and what happened, along with what went wrong. 

That can be as simple as three rows in a spreadsheet. 

Function/Feature

What are you testing? Be as specific as possible here. For example “Testing new user registration with a valid email address”. 

That’s much different from “Testing new user registration”. But why? Let’s break it down. 

Ok. So “Testing new user registration with a valid email address” means that we aren’t trying to break anything here in this test, rather we’re providing a valid data input and making sure that the registration action succeeds. 

We’re also limiting the scope of the test. Notice how we aren’t checking if an email sends to let the user know about the account? In this test we don’t care about that. 

Instead we just want to make sure that when we hit that registration button that WordPress correctly registers a new user. 

However “Testing new user registration” can mean that we’re providing a valid, invalid input. Testing the post processing and all aspects in between. Rather more vague and prone to failure right? 

Expected Outcome

What’s your expectation when you run the test? Let’s use our example from before “Testing new user registration with a valid email address”. 

For that we could say our expected outcome is: “The user is successfully added to the system and is visible in the wp-admin > users screen, and can log in with their registered details”. 

Again notice we’re being specific. All we want to do is ensure the user registration was 100% successful, we don’t care about emails, payments, access to content or anything else at this stage. We just want to know:

1. Is the user added to WordPress? Yes. 
2. Can the user log into the website with their details? Yes. 

Those are our test conditions. If either of those conditions fail we’d send the ticket back to the development team. If those conditions pass we approve the ticket and merge, or however you handle your development process. 

Outcome Result

Now you’ve completed your test, what was the outcome? Did the test pass or fail? Here you want to detail what happened, for example:

“After performing the test, the user failed to be registered which upon inspection appeared to be caused by an internal server error on the wp-admin-ajax.php call to process the registration which links back to the function added in 2.4.5”. 

Remember this is your sheet to run through every time you test a ticket and the why. You want to be able to articulate in shorthand here to pass all the details to your development team. 

As QA you want to be able to provide the following to assist with finding out what is wrong. 

• What did you do? — The details from the Function/Feature section. 
• What did you expect to happen? — The details from the expected outcome section of your sheet. 
• What happened? — the details from the outcome result section of your sheet. 

In addition developers appreciate details like the following, and you can add columns for these in your sheet. 

• WordPress version
• Theme used — try and test on default themes when testing plugins. E.g, the Twenty X series of themes, like Twenty Twenty One etc. 
• Replicable in the previous version — can you replicate the bug in the previous version of the software? E.g, is it a new bug or something that has been in the software a while and just found. 

Iterations of Your Testing Plan 

As QA your plan is a living breathing animal. As you come across new use cases, new test cases, add them in! Even if you don’t flesh them out right away, always add them in as quick as possible so you don’t forget. 

Not to mention as the software you are testing matures, new features and functionality will be added that needs testing. 

Wrapping Up

This is a pretty high-level overview that has been simplified so you don’t fall to sleep reading, I’ll continue with a series of QA posts though that’ll go more in depth on the technical side and the why.