Nicky Tests Software: 2017

Monday, November 27, 2017

Reminding myself about how one's experience shapes one's point of view

As I am helping introduce Exploratory Testing to our current project, there is one thing I've had to remind myself over and over and over again.

One's experience shapes one's point of view.

When having a discussion, or trying to convince someone of my point of view, I try to consciously remember this.

If the people I am having a discussion with, have a different point of view to me, that doesn't necessarily mean I should jump to the conclusion that they are wrong and I am right (or vice versa). Based on our own experiences, chances are, we are both right in our own minds. Which means it's not up to me to try and figure out how to convince them that they are wrong and I am right


I need to figure out how to close the information gap.

I love analogies so let me use an analogy to further explain what I mean:

Working Remotely Analogy
Let's say you want to have the option to work from home and are going to propose remote working in your team.
You have had great experiences working from home. You've been able to get more done (less disturbance), you get to enjoy having no commute and you've had access to the right tools etc. so you can still get your job done and communicate with your team.
But then one of your teammates raises their concern about this because they have also worked remotely and it didn't work out so well for them. Your teammate says that they struggled to contact people who were working remotely and that people who worked remotely often had problems around logging into the VPN and around the communication tool.

We're not going to get anywhere by just having one person be right and another person be wrong. Each person's experience resulted in that person's opinion. Therefore each opinion is valid.

The goal here is to first get a shared understanding of what working remotely is (should be easy enough) but more importantly what working remotely requires by both the project and each individual.

Some questions that may run through the team's mind when discussing working remotely may include:

  • What are their experiences of working remotely?
  • How have these experiences affected their understanding and opinion of what working remotely is?
  • Since I can't just share my own experiences (I can't just tell them), is there any way I can get them to experience what I experienced when it comes to working remotely?

Ideas on the thought process
When it comes to introducing Exploratory Testing to our current project and helping dispel people's misconceptions about ET, I'm keeping the following in mind:

  • What do they think Exploratory Testing is?
  • How can I check to see our understanding of Exploratory Testing is the same thing? (Before trying to advocate for the use of Exploratory Testing, it might be worthwhile seeing if we are discussing the same concept or only the same term)
  • What are their experiences with Exploratory Testing?
  • How have these experiences affected their understanding and opinion of what Exploratory Testing is?

  • Self reflection
  • Am I happy with my use of words, to describe and explain Exploratory Testing?
  • Am I listening to understand, not to answer? (this is a very difficult one for me, working on this)
  • With my use of words and how I say things, am I showing I am open to discussion about the topic and that I am open to questions?
  • Since I can't just share my own experiences (I can't just tell them), is there any way I can get them to experience what I experienced when it comes to Exploratory Testing?
Note: This is an effort to document my thought process when it comes to certain discussions at work, not all of these questions run through my mind with each and every conversation. But I do try to be aware of these questions and again remind myself that:

One's experience shapes one's point of view.

Thursday, October 12, 2017

Introducing people to Exploratory Testing Part I

 A bit of context

For the past 2 months(ish) I've been working on introducing Exploratory Testing to people in my project, starting with people in my immediate team of 3 testers, which is distributed between 3 countries. The project, as a whole, has a lot more than that, but the plan is to introduce  this as a pilot, see what the testers think of it, and then (hopefully) introduce this approach to other teams and other features.

I'm still a relatively new person on this project as I've been on it for 6 months, but the other testers in my team have been on this project for 3-5 years. So I've made sure to ask their thoughts, listen to their ideas and address their concerns about this - they know things about our context (which I don't) because of their experience here.

Currently, on the project, we write test scripts, link them to test cases, then execute those test cases. I'm under the impression that a lot of people on the project have only ever used test cases to formally do testing (when they're not doing "Exploratory Testing").

Another thing to keep in mind, is that this process is still ongoing (hence 'Part I'), but while these thoughts are still fresh on my mind, I wanted to get them down.

Sites/resources I shared

Spotify Offline: Exploratory Testing by Rikard Edgren
I asked my Test Manager (who also thinks Exploratory Testing is effective) about resources I can share and she recommended two Youtube videos by Rikard Edgren.

James Bach has a lot of useful posts on his blog helping explain what Exploratory Testing is.
Few examples:
Exploratory Testing 3.0
What is Exploratory Testing

I also shared a post from Michael Bolton's series - what Exploratory Testing is not

Lastly, I decided that Session Based Test Management (SBTM) would be a great way to help us structure our Exploratory Testing, so I shared some resources around that including this powerpoint by Anders Claesson

Managing others' expectations

I've noticed that a lot of people on this project have a very different understanding (to me) of what Exploratory Testing is. Based on what people on this project say, it seems that they think Exploratory Testing and Ad hoc testing are the same thing.

Since initially introducing the idea to the other two testers I work with, I'd say I've been met with cautious reception. Test cases is seen, by one, as proper testing and Exploratory Testing is not-  I'm still working on breaking that misconception. Aside from that, it does seem to be a welcome idea - you get to see results faster and are able to react to what you find as you test.

In terms of time estimates and how this affects our team meeting it's goals - I've been sure to communicate with our team that this is a new way of working which we need to learn - so any time savings may not be seen straight away.

Lastly, there is the idea of coverage - to cover this, I've decided to specifically mention, at the start of each charter, which Acceptance Criteria is covered in the charter. People on this project like reports and seeing the number of passed test cases etc. - I'm still learning how to deal with that mindset and any obstacles which arise there.

Managing my own expectations

This has been tough. It's been a while since I've been in a work environment with people avid fans of test cases. I don't think that test cases offer no value at all, but I think people overestimate the value it provides.  Test cases can give people a false sense of security of the state of the product when they see that 95% of their test cases have passed, but they can't properly attach meaning to how a 95% pass rate affects the customer. It's just a number.

I've also been trying to manage my expectations around other people's understanding of good testing and my own understanding of good testing. I constantly remind myself that their understanding is based on their own experiences. So neither of us are necessarily wrong - we are both right in our own mind.

My goal is to effectively show people on my project another way of doing testing. Then they can make more informed decisions in the future and choose which approach is best for their context.

Moving Forward

I'm hoping to sit with the tester who'll soon be on-site and pair test with them as we do Exploratory Testing. I should also organise a pair testing session with the tester who'll still be offsite. Once we've done this, I'll seek more feedback on this approach and see what they like and what they are concerned about (in the context of this project). We're also working on a Low Tech Dashboard to help us communicate the testing status for our features and help others attach meaning to what we present. 

In time, I'm hoping to help introduce this approach to other teams in our project - but for now we need to continue the pilot first. 

Tuesday, May 30, 2017

The limitations of Acceptance Criteria

According to Software Testing Class, Acceptance Criteria are conditions which a software application should satisfy to be accepted by a user or customer.

Often these can also be used to guide the testing for a testing team. If the acceptance criteria are met, then the story has passed. You can choose to test strictly against the acceptance criteria by using test cases or exploratory testing etc. and then once each acceptance criteria has been "ticked off", you can mark testing as done.

The thing is - acceptance criteria has its limitations.

You are expecting someone to know in advance, before seeing the software, exactly how the software application should be. So if you are testing strictly against the Acceptance Criteria - you are in essence trusting that, that person (or group of people) who wrote the acceptance criteria knows everything about what is needed before the software is built.

People don't know what they want until they see it (same goes for knowing what they don't want)
I think it's possible to build a product that the customer did say they want then still find them to be unhappy because they realised once they saw it - that it wasn't quite what they wanted. After seeing something they are then better able to articulate what they want or need from the software application. They may not, however, be able to articulate what they want clearly - until they have something in front of them.

Here's a fun analogy to help me explain this further
Finding a romantic partner
Now of course finding a romantic partner and acceptance criteria are vastly different things, but let me explain. If you've ever, as a single person, talked to your friends etc. about what you want in a potential romantic partner, you may rattle off things such as:

  • Funny
  • Kind
  • Attractive
  • Likes sports
  • Same religion

Among other things, and these are things you may deem to be important. So let's assume the above criteria are all must-haves, they are your "acceptance criteria".

But as I said before - acceptance criteria has its limitations.

If you think you know exactly what you want when it comes to a romantic partner, then technically a friend can introduce you to someone who is Funny, Kind etc. and you'll be happy.

But it's not that simple, while there are some things that you may not want to compromise on, there are also some things you may not realise are important to you, until you have met someone who has those special qualities, which you didn't think to define. (or they may be missing some qualities that you didn't think to define)

 Image courtesy of

Wednesday, May 24, 2017

My Testbash Belfast 2017 Experience Report Part I

This is a two-part Experience Report, the first part will cover preparing my talk, the pre-Testbash Shindig and the first half of the conference. The second part will cover the second half of the conference and the Post-Testbash shindig.

Preparing my talk

I started preparing my talk around 2-2.5 months before the conference. But I didn't properly gain momentum until about 1.5 months before my talk. Initially I tried to write the whole talk in Google Docs - but I found that didn't work for me. Instead, I ended up creating the slides and writing speaker notes below.

I aimed to have a completed presentation ASAP and then just edit it continuously up until I gave my talk. I find it a lot easier to edit a presentation that's complete than to add more to one that is incomplete.

Tuesday, May 16, 2017

My Experience at Romanian Testing Conference 2017 - told through photos

I thought I'd share my experience at Romanian Testing Conference 2017 with the use of photos :)

Here are some photos of our talented speakers on the evening of Thursday 11 May, before the main conference day.

Here's Rob Lambert , the conference chair, welcoming all of us to the Romanian Testing Conference 2017

One of my favourite slides from Santhosh Tuppad's opening keynote

Some photos from Adam Knight's talk on communicating risk

Marcel Gehlen sharing his expertise on creating a test friendly environment

One of the slides from Elizabeth Zagroba's talk on how to succeed as an introvert

At lunch we had quite the dessert offering, I ate less than half of what's on this plate. It was very rich. But I saw lots of other people eat twice the amount on this plate for dessert. 

My certificate :)

Harry Girlea giving his closing keynote

Sightseeing - I'm doing my classic thumbs up pose here

The Opera House in Cluj

View over Cluj

Going for a walk in central park

Monday, May 15, 2017

Don't call it exploratory testing (if it's not exploratory testing)

Well, this is a bit of a rant - but seriously.

I don't like people calling an activity exploratory testing when it's actually ad hoc testing.

Get it right.

I think using the term "exploratory testing" loosely - takes away from the value that actual exploratory testing can add to a project.

Below are a few questions to ask yourself, to see if you are actually doing exploratory testing (or if it's in fact ad hoc testing)

  • Are you doing concurrent test design and test execution at the same time?
  • Are past findings influencing what you do next? 
  • Have you written some test ideas or goals in mind - to help you explore the application?
  • Are you documenting your test session?
  • Are you focussing on both positive and negative scenarios? (if you're just looking for bugs - this is likely to be ad hoc testing)
  • Can you explain the process/what you did during the test session to someone else?
  • Are you a skilled tester? (ad hoc can be done by anyone, doing proper exploratory testing is a skill in itself)
  • Are you doing SBTM (Session Based Test Management)? This is a way to structure your exploratory testing. If you are doing SBTM, this is a good way to indicate you are doing Exploratory Testing. BUT, if you're not doing SBTM, that doesn't necessarily mean you're not doing Exploratory Testing.

Updated: I  have updated this blog post to help explain the difference between ad hoc testing and exploratory testing by posing a few questions

Monday, May 8, 2017

Interview with Mark Winteringham

In his own words, here's a bit about Mark Winteringham:

I am a tester, coach, mentor, teacher and international speaker, presenting workshops and talks on technical testing techniques. I’ve worked on award winning projects across a wide variety of technology sectors ranging from broadcast, digital, financial and public sector working with various Web, mobile and desktop technologies.

I’m an expert in technical testing and test automation and a passionate advocate of risk-based automation and automation in testing practices which I regularly blog about at and the co-founder of the Software Testing Clinic. in London, a regular workshop for new and junior testers to receive free mentoring and lessons in software testing. I also have a keen interest in various technologies, developing new apps and Internet of thing devices regularly. You can get in touch with me on Twitter: @2bittester

I noticed that the next event for Software Testing Clinic is sold out for both mentors and students - any plans to expand in the near future?

Wednesday, May 3, 2017

Getting started on a testing project

I started on a new project a few weeks ago and thought it would be a good idea to share a checklist for what new testers on a project need and some good starting questions when you, as a tester, are new on a project

Checklist for what new testers on a project need (Note, your project may not include all of the below)

Note to check if user credentials are needed for any of the below

Thursday, April 6, 2017

Developing your thoughts: Talking out loud

Now that I've started actively preparing for my "Testing so you can move on" talk at Testbash Belfast and Romanian Testing Conference, I've started to notice how my thought process is developing as a result of talking out loud.

Bit of background first so you have an idea how I'm preparing:

  • I'm focussing on getting my first draft done (almost there, just need to expand a bit more on one of my key points first)
  • Then I'll start on the slides 
  • I'll look into how the slides and talk "fit" each other
  • Then I'll go back and forth between slides and talk aiming to reach a deliverable presentation ASAP
  • After that, just refine until I deliver the presentation

I'd like to share a little bit about my thought process and what I learn/gain from talking out loud when preparing to give talks at conferences or give speeches at Toastmasters .

Monday, April 3, 2017

Living in Sweden: Laundry rooms and apartments

Since I moved to Sweden in September 2015, I've had to learn to get used to a lot of different aspects of life. I'm in another country on the other side of the world from my native country after all.

Aside from the obvious like the language difference, there are some aspects of Swedish life that I'm still getting used to. Here are two of them:

Laundry rooms
It seems to be very common to have to book laundry times. If you live in an apartment, chances are, you need to book laundry times each week. You share a laundry room with other people and book the room. This can be done in various ways, these include:

  • Digitally (have heard of this, never done it)
  • Writing your apartment number in the spreadsheet in the bulletin board
  • Using your apartment peg and moving it to the time slot you want 

Thursday, March 30, 2017

Do you see what I see?

Yesterday I had an interview with the Test Manager for a potential project and the subject of providing valuable information, as testers, came up.

We had a bit of back and forth regarding what exactly is valuable information - I asked him what the term "valuable information" meant to him. He then proceeded to tell me it's information that helps stakeholders make informed decisions.

We were on the same page.

Wednesday, March 8, 2017

Find (and surround yourself by) inspiring people at work

I've been really lucky to have found and been inspired by some amazing people at work. First off, let me give a few brief reasons as to why I think it's important to find (and surround yourself by) inspiring people at work.

  • If you're impressionable, then it's great to be surrounded by these people - and hope some of them awesomeness rubs off on you (at least that's what I've been doing).  
  • You will learn a lot from them.
  • Lastly, they can help you discover what you can offer the world, that you didn't know you could offer. There may be strengths you have that you don't recognise, but with their experience and knowledge - they may be able to spot it and let you know that it sets you apart.

Below I'd like to mention a few (not all, blog post would be too long) of these people and share what they taught me.

Aaron Hodder
  • Question things
  • Opened up world of CDT

I met Aaron while I was on the Graduate programme at Assurity. He was the person who opened up the Context Driven Testing world to me - thanks to this so many more doors opened as a result. I also distinctly remember him teaching us (graduates) the dice game which taught me to not prove that your theory is correct, but find a way to disprove your theory. He also taught us to question the ISTQB exam and that it has its limitations.

Katrina Clokie
  • Make stuff happen

The main thing I have learned from Katrina is that you can make stuff happen (she even has spoken about the topic at conferences). WeTest Auckland is initially created because of all the resources and learnings Katrina shared with us when we were first starting up the meet-up. Then when I moved to Sweden, I started up Stockholm Software Testing Talks using what I learned from helping run the WeTest Auckland meet-up.

Shirley Tricker
  • Self awareness
Shirley was my mentor both when she was at Assurity and after she left. I asked her for career advice and from her, learned more about what I have to offer and what's unique about me in the workforce. 

  • Don't focus only your weaknesses, focus also on strengthening your strengths
I reported to Pete while I was at Vend and one piece of advice I distinctly remember is to not forget your strengths. I was so focussed on trying to be a better tester my working on areas in which I thought I was weak, that I forgot to work on my strengths - what helps me stand out.

  • How to write proposals for conferences
Carsten helped me write the first proposal that got accepted at a conference. He provided a lot of useful feedback and helped me gain some perspective of where the submission reviewers are coming from.

Tuesday, February 21, 2017

A Recent Encounter with Dark Patterns

I first came across the term of "dark patterns" when I saw Emma Keaveny's talk about it on The Dojo. While watching it, then later looking more into it, I realised how many companies are out there purposely trying to get the user to do something, the user doesn't actually want to do.

Time passed.

Then I signed up to The Economist.

I signed up online and found it pretty easy to do so. But then I soon struggled to keep up with new issues - so I decided to cancel. Unfortunately, it wasn't simply a case of finding some "Manage Your Account" link and then clicking the "Cancel Subscription" button. Instead I had to "Contact my local print service centre"

Wednesday, February 8, 2017

How to incorporate humour into speeches

About 6 months ago, I completed my last project in the Humorously Speaking Manual. To be honest, I was really happy to get it out of the way. One of the objectives for each speech in this manual is "make people laugh". And I found that stressful. I mean, I sometimes make people laugh spontaneously when I talk to them but having to make people laugh in purpose? Well, that's another story.

I remember my 2nd speech from the Humorously Speaking manual was close to a disaster. I got a few awkward smiles at best and I thought "f*** why did I pick this manual?"

Then I reminded myself - it's because I love to listen to funny speeches or speeches that have incorporated elements of  humour. I want to do that. 

So I learned how.

Tuesday, February 7, 2017

Handling nerves when public speaking

Yesterday I competed at my Toastmasters club's annual Evaluation Contest. During this contest, you have a test speaker give a 5-7min speech and then we compete to give the best Evaluation to the Test Speaker.

While the judges votes were being counted, us 5 contestants were asked to come up to the stage and be interviewed. The question I was asked, was something along the lines of "How did you find the Evaluation Contest?" or "What was your experience in participating in the Evaluation Contest?"

And I answered truthfully.

I said I was fine, up until 30 seconds before I had to deliver my evaluation. Then my heart started racing. It's like a switch flipped as soon as I had to enter the room to give my evaluation (we had to stand outside and wait our turn to give evaluations so we can't cheat and copy the other evaluators).

It amazed me how one minute I was joking with the other contestants saying "Don't forget us when you make it to the next level of the competition" and then next minute I can feel my heart racing and thinking "hmm, wonder what my Fitbit says my heart rate is right now".

I told the audience how I felt. And they also knew I've been a Toastmaster for 4 years.

Monday, January 30, 2017


Earlier today, Dorothy Graham presented a webinar on her thoughts about test coverage. For me, listening to this webinar was a trip down memory lane - I immediately thought of the ISTQB Foundation exam I sat in 2012 and the multi-choice questions I had to prepare for when it came to the different types of coverage (But I really enjoyed hearing Dorothy’s analogies on coverage (I find analogies are a great way to explain things). This post is not a summary of webinar, but just some of my highlights.

The travel analogy

Dorothy started off with an analogy using Scratchie maps - those ones where you (typically) scratch off the countries you have visited. She questioned “But what do I scratch off? cities? states? countries?” Initially she scratched off 74 cities in the map, but we couldn’t really see them.

She then proceeded to scratch off states then countries. But she commented that it wasn’t really representative of how much of these places she’s actually experienced - it was “rather shallow coverage”. (For example: scratching off all of Brazil when you have only been to Rio de Janeiro)

Wednesday, January 4, 2017

The State of Testing Survey for 2017 is now open!

The 4th State of Testing Survey is now open:

Have your say in how you do testing and your company, how many people are in your test team etc. to contribute to this survey, then later reap the benefits and get to see how everyone's jobs look like in the testing world.