Skip to main content

Tackling Test Automation Part I: What problem are you trying to solve?

For the past seven months, since I came back from maternity leave, I've spent the majority of my time focussing on test automation. I have both set up a test automation framework as well as overhauled two existing test automation frameworks for mobile (both Android and iOS).

But one of the struggles I have faced is not getting lost as to fail to see the forest for the trees. During the day to day, it can be easy to be constantly knee deep in code and forget the problem I am actually trying to solve. Why am I actually doing test automation in the first place?

I have found having some focussing questions to be great for both starting a project where I have to write Test Automation but also to come back to these questions throughout a project to make sure I am still on track.


1. What problem am I trying to solve?

This is the main one. For me, at my current project, I want to cover regression so that as the team adds features; makes changes etc. they can do so without worrying about inadvertently breaking something else. 

In general, I like to have basic flows covered as well as targeting areas that I have noticed have been a bit problematic in the past (I do this by talking to people in the team and going through the bug tracking tool to see if I can see patterns).


2. Why do I have this test?

Taking a step further and looking at the tests in more detail, I start thinking why do I have this test? What value does this specific test add to my test suite? 


3. Does this test still have value?

I also then think, does this test still have value? As changes to our apps are made and I may be looking through our existing suite to see if/how I need to update tests, I may then find that some tests no longer have value.

This is most likely because:

  • Other tests I have added since overlap that area or cover that area in a better way

To be honest, I'm pretty brutal, I don't like have an excessively high number of tests and I don't think you need to have heaps and heaps of tests to achieve good coverage. If I can achieve a good amount of coverage, with a reasonable number of tests (where each test still has its specific focus/goal) then I'm all for it.


4. What type of regression bugs can my test automation suite miss?

I then take a step back and ask myself, what type of regression bugs will my test automation suite miss?

Asking myself this question, helps me change my perspective to see what else I need to address. Often when I try to solve problems (not just test automation related), I ask myself questions in different ways to try and figure out what else I could be missing. This question is an example of that.




Comments

Popular posts from this blog

Getting started on a testing project

I started on a new project a few weeks ago and thought it would be a good idea to share a checklist for what new testers on a project need and some good starting questions when you, as a tester, are new on a project Checklist for what new testers on a project need (Note, your project may not include all of the below) Note to check if user credentials are needed for any of the below

My Most Used Test Heuristics (with examples)

First, what is a heuristic? A heuristic is a guideline ,  it is fallible.  Therefore, it will give you a good idea of what behaviour you should see BUT it isn't definitely what should happen - it's up to you to confirm that the behaviour you are seeing is correct.  In a previous blog post I shared a step by step guide on  how to test without requirements/little requirements.    But I figured it's good to share my most used test heuristics that I use for testing without requirements. They are: Consistency with History Consistency with User Expectations Consistency within Product Let's take a look at each of them along with some examples to illustrate the concept. 1. Consistency with History  The feature's or function's current behaviour should be consistent with its past behaviour, assuming there is no good reason for it to change. This heuristic is especially useful when testing a new version of an existing program. (Source: developsense) Example: Whitcoulls, a

Bloggers Club: The Importance of Exploratory Testing

 Before we dive into the importance of Exploratory Testing, I would like to clear three things up. Firstly, I align with this definition of Exploratory Testing, by Cem Kamer, it is an approach to software testing that consists of simultaneous learning, test design and test execution. Secondly, I don't think Exploratory Testing has to be a substitute for test cases, it can complement test cases. (It's up to you, how or if you choose to combine both Exploratory Testing and Test cases when you test a feature) Lastly, exploratory testing is not adhoc testing - adhoc testing is random, unstructured testing, exploratory testing forced you to think critically about the application under test. (For more about the difference go here .) Job Interview Analogy To illustrate the importance of Exploratory Testing, I'd like to first use the analogy of a job interview. (I wrote about this in a previous blog post but will expand on this further) The Test Cases in this analogy In a job inte