Software Testing-Requirements Traceability Matrix

What is the need for Requirements Traceability Matrix in Software Testing?

Automation requirement in an organization initiates it to go for a custom built Software. The client who had ordered for the product specifies his requirements to the development Team and the process of Software Development gets started.

In addition to the requirements specified by the client, the development team may also propose various value added suggestions that could be added on to the software. But maintaining a track of all the requirements specified in the requirement document and checking whether all the requirements have been met by the end product is a cumbersome and a laborious process.

The remedy for this problem is the Requirements Traceability Matrix.

What is Requirements Traceability Matrix in Software Testing?
Requirements tracing is the process of documenting the links between the user requirements for the system you're building and the work products developed to implement and verify those requirements. These work products include Software requirements, design specifications, Software code, test plans and other artifacts of the systems development process. Requirements tracing helps the project team to understand which parts of the design and code implement the user's requirements, and which tests are necessary to verify that the user's requirements have been implemented correctly.

Requirements Traceability Matrix Document is the output of Requirements Management phase of SDLC.

The Requirements Traceability Matrix (RTM) captures the complete user and system requirements for the system, or a portion of the system. The RTM captures all requirements and their traceability in a single document, and is a mandatory deliverable at the conclusion of the lifecycle.

The RTM is used to record the relationship of the requirements to the design, development, testing and release of the software as the requirements are allocated to a specific release of the software. Changes to the requirements are also recorded and tracked in the RTM. The RTM is maintained throughout the lifecycle of the release, and is reviewed and baselined at the end of the release.
It is very useful document to track Time, Change Management and Risk Management in the Software Development.
Here I am providing the sample template of Requirement Traceability Matrix, which gives detailed idea of the importance of RTM in SDLC.

The RTM Template shows the Mapping between the actual Requirement and User Requirement/System Requirement.
Any changes that happens after the system has been built we can trace the impact of the change on the Application through RTM Matrix. This is also the mapping between actual Requirement and Design Specification. This helps us in tracing the changes that may happen with respect to the Design Document during the Development process of the application. Here we will give specific Document unique ID, which is associated with that particular requirement to easily trace that particular document.

In any case, if you want to change the Requirement in future then you can use the RTM to make the respective changes and you can easily judge how many associated test scripts will be changing.

Requirements Traceability Matrix Template Instructions:

Introduction:
This document presents the requirements traceability matrix (RTM) for the Project Name [workspace/workgroup] and provides traceability between the [workspace/workgroup] approved requirements, design specifications, and test scripts.
The table below displays the RTM for the requirements that were approved for inclusion in [Application Name/Version]. The following information is provided for each requirement:
1. Requirement ID
2. Risks
3. Requirement Type (User or System)
4. Requirement Description
5. Trace to User Requirement/Trace From System Requirement
6. Trace to Design Specification
7. UT * Unit Test Cases
8. IT * Integration Test Cases
9. ST * System Test Cases
10. UAT * User Acceptance Test Cases
11. Trace to Test Script

The following is the sample Template of Requirements Traceability Matrix.

Read more...

Requirements Traceability Matrix (RTM)

1. Requirements Traceability Matrix (RTM)

A document showing the relationship/mapping between Test Requirements and Test Cases.

Elements of RTM:

a. Requirements ID

b. Requirements Description

c. Test Case ID

d. Status [Open, Closed, Defer (Later), On hold]

2. Verification and Validation

Verification is the process confirming that something software meets its specification. Validation is the process confirming that it meets the user's requirements.

Difference between Verification and Validation:

Suppose, you are going to buy a pair of shoes having number 9 for you. You have chosen a pair and seen the tag with 9 written on it. This is verification, because your requirement was to buy a pair of shoes with 9 number.

But when you tried to wear it and found that shoe is not fitted into your feet. After inquery, you have found that company has tagged it 9 number by mistake. Actually it was 7 number shoe. This process is called Validation.

Example of Verification: Creating Traceability Matrix

Example of Validation: Executing Test Cases

3. Static and Dynamic Testing

Static black-box testing: Testing the specification is static black-box testing.

Two Types of Static black-box testing:

1. High-level review techniques

a. Research Existing Standards and Guidelines

b. Review and Test Similar Software

2. Low-level techniques

a. Specification Attributes Checklist (e.g. Spec must be complete, accurate, precise, consistent etc)

b. Specification Terminology Checklist (e.g. focus on the terms in Spec like "If…Then…(but missing Else)." or "Etc., And So Forth, And So On" etc)

Dynamic Black-Box Testing: Testing software without knowledge of code is dynamic black-box testing.

Static White-Box Testing: Static white-box testing is the process of carefully reviewing the software design, architecture, or code for bugs without executing it.

Three Types of Static White-Box Testing:

a. Peer Reviews: Peer Reviews are the least formal method. Peer reviews are often held with just the programmer who wrote the code and one or two other programmers or testers acting as reviewers.

b. Walkthroughs: In a Walkthrough, the programmer who wrote the code formally presents it to a small group of five or so other programmers and testers. The presenter reads through the code line by line, or function by function, explaining what the code does and why. The reviewers listen and question anything that looks suspicious.

c. Inspections: Inspections are the most formal type of reviews and more formalized than a 'walkthrough', typically with 3-8 people including a moderator, reader, and a recorder to take notes. The other participants are called inspectors.

Walkthrough:

1. It's a type of Semi Formal Review.

2. 2 to 7 People ate attaining it.

3. Author is Presenter.

4. Lead by Author only.

5. Reviewers are not aware of the subject/topic.

Inspection:

1. It's totally a Formal Review.

2. 2 to 10 or more People attaining it.

3. Author is not presenter. Some one else is giving presentation.

4. Lead by Moderator.

5. Reviewers are aware & well prepared for the subject/topic.

6. Recorder is noting down everything. Like defects, changes, improvements etc.

Dynamic White-Box Testing: is a method of testing software that tests internal structures or workings of an application.

Difference between Dynamic White-Box Testing and Debugging:

The goal of dynamic white-box testing is to find bugs. The goal of debugging is to fix them.

Read more...

How to write software Testing Weekly Status Report

How to write software Testing Weekly Status Report

Writing effective status report is as important as the actual work you did! How to write a effective status report of your weekly work at the end of each week?

Here I am going to give some tips. Weekly report is important to track the important project issues, accomplishments of the projects, pending work and milestone analysis. Even using these reports you can track the team performance to some extent. From this report prepare future actionables items according to the priorities and make the list of next weeks actionable.

So how to write weekly status report?

Follow the below template:
Prepared By:
Project:
Date of preparation:
Status:

A) Issues:
Issues holding the QA team from delivering on schedule:
Project:
Issue description:
Possible solution:
Issue resolution date:

You can mark these issues in red colour. These are the issues that requires managements help in resolving them.

Issues that management should be aware:

These are the issues that not hold the QA team from delivering on time but management should be aware of them. Mark these issues in Yellow colour. You can use above same template to report them.

Project accomplishments:
Mark them in Green colour. Use below template.
Project:
Accomplishment:
Accomplishment date:

B) Next week Priorities:
Actionable items next week list them in two categories:

1) Pending deliverables: Mark them in blue colour: These are previous weeks deliverables which should get released as soon as possible in this week.
Project:
Work update:
Scheduled date:
Reason for extending:

2) New tasks:
List all next weeks new task here. You can use black colour for this.
Project:
Scheduled Task:
Date of release:

C) Defect status:

Active defects:
List all active defects here with Reporter, Module, Severity, priority, assigned to.

Closed Defects:
List all closed defects with Reporter, Module, Severity, priority, assigned to.

Test cases:
List total number of test cases wrote, test cases passed, test cases failed, test cases to be executed.

Read more...

Things Every New Software Tester Should Learn

This is a guide to learning more about software testing. As you start on your journey you will have tasks you can work through. Software testers are always learning but we cannot always quantify it. We learn about the product we are testing. We learn and develop relationships with developers, managers and testers. This makes us great at what we do. We are also chameleons. We change based on the environment we are in or the product we have to test. We have to continue to educate ourselves about the tools we need to test each product.

1. Are you interested in software testing?

Let's start with a little self-reflection.
An important part of being a tester is your mindset and being aware of it.

• Why is software testing valuable to you?
• What do you believe software testing is? We'll talk about this more in the next Task but it's good to think about what you currently believe it is.

Take a moment to think about the questions above and other questions you might have. By the end of this guide the aim is to have the majority of those answered. You're going to have more questions as you go too. Questions prompt a search for answers which results in learning.
We are surrounded by tech daily. There may be things you use in your everyday life that bother you. Perhaps a software package at work that takes longer than a minute to load. The app on your phone that freezes just after an update. Or perhaps when you get locked out of your emails and discover you can't recover or reset your password. You might have started to think more about that software package by maybe looking at your task manager as it loads.
Can you be a person who strives to make software better? Can you be an advocate for the users? The business? Your work colleagues?
Do you know what it takes to get there? Do you know what you need to learn? Think about all the good and horribly bad software you have come across in your life. You are the chameleon in both. In the good situation you want the product to stay good so you keep learning to stay on top of it. In the bad situation you want to learn more so you can feel like you did everything you could before the software was released. Learning is the key. No matter the situation you will always be learning even if you aren't aware of it.
This sounds daunting, don't let it put you off. Start with the first question and think about it. By the end of this guide you will be more confident in your answers to all of these questions.
What to do next:

• Commit to print why you are here (a post-it or whatever medium you prefer)
• Where do you plan to be at the end of this?
• What skills do you have now that you think are useful for testing?
• What do you currently perceive software testing to be?

2. What is software testing?

Start by writing down what you think software testing is and what a software tester would do.
There are many aspects to software testing. It does not always involve using the product. It is not just about finding bugs. Testing can start around the requirements stage. Thinking about what the product should do, where risks could be, and how the user/customer navigates the product is all part of testing.
One discussion I recently had, about whether it is fair to assume that customers are your users, gave me perspective on terms we use to represent stakeholders. Stakeholders, to me, covers everyone who has an interest in the software product. In the end we agreed that the person who pays for your software may not be the one who actually uses it. This is a great example of how software testers should look at things from different perspectives. Below are examples gathered from discussions with other testers about what they think testing is or is not.

What Testing Is	What Testing Is Not
Verifying whether software meets the expected value to its users. Investigating the product to find any information that is of value to our stakeholders.	Simple, quick, predictable. Only verifying that the product matches the description.
Mitigating surprises - Trying to find problems before software is released to stakeholders.	Increasing quality - this is a whole team exercise. Prove that the software has worked once.
Exploring products. A valuable activity in software development but often misunderstood due to its unpredictable and creative nature.	Well understood or valued by everyone. A process that is fixed, unimaginative and best kept under strict rules.
Communicating - with stakeholders (customers, users, developers, etc.) and work together to find out if the product is improving.	A phase that a project needs to go through in order to be successful.
Infinite. All testing is sampling. For every non trivial product, there are an unimaginable number of parameters with a great number of possible values. How do you know you are testing the important ones?	Ever finished - we can make decisions about stopping criteria but there are an infinite amount of combinations that could be checked.

What to do next:

• Read this list of things of what software testers do. 
• Try asking people around you about the title of this task. Ask people you work with testers and non-testers, ask people on Slack and maybe even ask your family. Has this changed much from what you originally thought? How different were the responses you got?
• Research other industries such as pharmaceuticals, appliances etc. What is their approach to testing? How different or similar is it to the tech industry? Do any of the differences change your thoughts on how you would approach testing?

3. Testing for non-testers

When I started in software testing I had no idea what testing was. I also had no clue of where to start. One of the most useful resources I came across was the pathway Testing for non testers by Katrina Clokie. Using Katrina's pathway I was able to understand testing and the value it provided. I was also able to investigate her references further to start expanding my knowledge and list of people I should look to for advice.
This is one of the things that will not be done overnight! I recommend reading it and referring to this guide each time. Are there any things you want to practice? Are there any things you don't and if so, why? Has this changed the thoughts you wrote down?

What to do next:

• Share the link with your team
• Read an article a day from Testing for non-testers
• Create your own list of things to learn and practice. You will most likely be updating this constantly

4. Get social

This will be a slightly easier task to get started with but you will need to put in effort to keep it up. Start as simply as you like or feel comfortable with. I do recommend achieving all of the steps below when you can.
Engaging in social activities has enabled me to meet local testers who are willing to help me when I come across a tough problem. Through these social gatherings, I have made some friends in a city I am new to. It's nice to have a sympathetic ear with friends who understand what you are talking about.

What to do next:

• Go to Meetup.com and look for a local software testing meetup
• Join the testers.io or Ministry of Testing Slack
• Join Twitter, start finding testers to follow and get involved in some conversations 
• Join The Software Testing Club
• Read Andrew Morton's blog posts on attending the Bristol STC meetup and the Cardiff STC meetup
• Start looking up testing conferences such as TestBash, Agile Testing Days, EuroStar, STPCon, Global Testing Retreat, TestCon, STAREAST, Let's Test, Nordic Testing Days. They are a big part of getting social.

5. Get organised

It can be difficult to stay organised when there is so much information being thrown at you. I struggled with this a lot, I still do occasionally. Tiny Habits was recommended to me by another tester when I was getting started. I go back to it frequently and set myself new tiny habits to achieve, particularly when I feel like things are going awry. This does not take a lot of time but it will save you in the long run.
You might also want to think about writing mind maps. There are a lot of free tools out there to help you with this. I find a mind map can help me streamline a random thought process a lot easier. Ever had a million ideas running through your mind that are loosely related but you can't get them into a sensible order to make something of? Mind maps can help with this, so can the good ole reliable post-its on a wall approach.
What to do next:

• Sign up to Tiny Habits and do it for a week
• Start a mind map
• Create a personal Kanbanflow or Trello board. Start with the column headers "To Do", "In Progress" and "Done". Add a card for each thing you would like to achieve and move them across the columns as you progress with them.

6. What is a bug?

I know a bug in software when I see one, or at least I like to think I do. I thought it would be easy to explain what a bug is. When I tried to find a definition I found many that just didn't cover the scope enough. James Bach defines a bug as "Anything that threatens the value of the product. Something that bugs someone whose opinion matters". That is a nice high level definition. On Testing Computer Software you will find "most bugs cause a program to change its behavior when the programmer didn't want or expect it to, or cause the program not to change its behavior when the programmer did expect it to".
In most of the definitions I found them to only mention the code or the developer. None of these are wrong. This would threaten the value of the product and it would not then be a bug with the code.

Think about the technology you use:

• Have you encountered bugs in any of it?
• Why did you feel like they were bugs?
• Did you do anything about it (e.g. report a bug in an app through the app store)?
• Can you think of three different things that help you decide whether a bug is actually a bug or not? These are called Oracles and we will look at them in more detail in Task 13.

7. Write a user story

A user story is like a set of instructions you get with flat pack furniture. By writing a user story it will help you to understand their purpose.
It should provide:

• A description of what needs to be done.
• Acceptance criteria outlining what the story needs to accomplish.
• Any criteria which are out-of-scope for the story to be considered accomplished.
• Note any dependencies required to start the story.

Mountain Goat Software has examples of agile user stories to give you an idea of what to expect when you're out in the wild. EPICs are essentially a large user story. They are broken down into smaller digestible or deliverable pieces called user stories.
Take the user story below as an example, the overall EPIC would be "Flat Pack TV Stand". The user story is then a smaller piece of that "Left Leg and Top Shelf Construction". The dependencies are things that we need but not things we would do, for example we would need a screwdriver to be able to assemble the shelf. The acceptance criteria are things that we actually have to do.

Writing a user story will help you to think about writing requirements for users more. It will also help you to think about the approach to take for testing these user stories. As a tester you will have a unique set of skills when it comes to analysing user stories. You will be able to highlight missed details, misuse cases, abuse cases and how it can be misunderstood. Not everyone gets the chance to provide early input with user story writing. It is an excellent process, you are testing before there is ever a piece of code written!
Your tasks for today:

• Write a user story for making toast, a peanut butter sandwich or even Ikea furniture using the templates above.
• How do you know if you missed something?
• Have you made any assumptions?
• Think about the order things need to be done in. Are there any optional inputs?
• Are there any requirements or prerequisites (assume the user has bread for example)?
• Try to make toast or a sandwich from scratch following your user story exactly. 

8. Testing personas

When you are testing you shouldn't only test for you. You need to think about impatient users, users who take their time, users who have a bit of a tester in them and like to "break" things and many more. As you get familiar with an application, things like this may slip from your mind and you may go into autopilot somewhat. I find trying to get into the user persona more helps me to avoid this pitfall.
Before thinking that personas may not already exist within a project, it may also be worth looking into whether personas research and documentation already exists. You can then use it to build things from a testing perspective.

For today:

• Pick an app on your phone or a program on your computer to test
• Navigate through it using the different user personas 
• What did you learn from this?
• How did the application perform under each persona?
• Think about and research how others approach personas

9. Test a user story

There are many ways to document testing but we will not focus on those here. If you've never written testing documentation, you could look at the test case templates on Tutorialspoint.
While writing the user story, you were testing it just as well, weren't you? But the thing that has changed is that you now have an actual product at your disposal and that makes a world of difference.
You now have something to get hands on with, you can leave the theory behind and put it to practice. This is where you start using your ability to learn. The speed and precision with which you can learn new and interesting information will define your skill and value as a tester.
Concentrate on your mindset as you test the story. This is the most important part when testing a user story.

Questions to ask yourself as you test this story:

• Who is this story important to?
• What is the one problem this User Story tries to solve?
• How could this be misunderstood?
• How could accidental bugs find their way into this?
• What behaviour should it absolutely NOT show?
• When are you done?
• At what point did you feel like quitting your analysis and want to start testing?
• Does this feel reasonable?

10. White & Black Box Testing

White and Black box testing are two approaches to software testing. Think of them as two umbrellas for types of testing to fall under.
White or Glass box testing is a testing method where the tester knows the internal workings of the program.
The tester writes a strategy to test the program based on knowledge of the application's internal code. They understand what the function is doing and what inputs will produce what outputs. This type of testing typically focuses on coverage of statements, lines of code, branches of code, etc. White box testing most often applies to unit tests.
One of the biggest risks with this type of testing is the inability to find missing features. You cannot cover a line of code with a test if the line of code never existed. The tester is required to have knowledge of the language that the program was written in. With this type of testing you are also naturally biased with your thinking patterns given the time you have spent within the product. As a result, you might miss obvious or non-obvious problems due to being too close too it. Think about signing in to your emails. It's something you do every day and you follow the same pattern every day. When was the last time you noticed a change on the login page or process? You become so used to what you expect to see that you may not notice the unexpected.
Black box testing is a method where the internal workings of the program are unknown to the tester. Tests are based on the requirements and functionality. One of the advantages of this over white box testing is the possibility of identifying missing code statements. This approach does not require the tester to have any knowledge of the language that the program was written in. A risk with this type of testing is the unknown level of coverage that testing gives.

What to do next:

• Try out doing some of the Black Box Puzzles
• Try testing any mobile app you have on your phone, you would essentially be doing black box testing

11. Accessibility Testing

Do you know anyone who is colourblind or maybe uses a screen reader? I know people who are colourblind, visually impaired, deaf, dyslexic or have photosensitive epilepsy. As a result I am always thinking about these users along with the 'standard' user set for our application. Luckily there are tools that can test an application for its accessibility level. A fun and easy trick to try is navigate through an application you are testing with no mouse or trackpad. Sounds easy right? Give it a go and see what you think. There are a few more quick things to try in this 6 Simplest Web Accessibility Tests Anyone Can Do. There are various plugins you can add to your browser to view your application the way someone with colourblindness would.

What to do next:

• Pick an application and try some simple accessibility testing on it. Did it perform as you expected? Were there any issues you encountered?
• Read A Software Tester's Guide to Web Accessibility.
• An Alphabet of Accessibility Issues is a great article to give you insight into the variety of needs people have.

12. User Experience (UX) as a tester 

The user experience is pretty much what it says on the tin. It is how the user experiences your application and how they respond when they use the application. Don Norman describes it as even more than that. Testers are often tasked with understanding user experiences and helping ensure the user has a good experience by reporting back any possible issues to the rest of the team. Even before the product is out on the market, testing can help to highlight UX issues. An example would be a page takes a long time to load. A user wouldn't wait that long because the competitor product hasn't got the same issue. We should investigate the delay and try to fix it.
The Nielsen Norman Group have an article about heuristics for user interface design which can help with the user experience.
At the end of the day it is users who will give you feedback on their experience with the product. As a tester you can help to ensure that their initial experience is a good one.

What to do next:

• Start to really think about user experience.
• Can you recall a stand out positive user experience you had?
• Can you explain why you felt it was positive?
• Do the same for the negative case.

13. Regression testing

A big part of software development is getting things to work together and keeping them working together. Even if loads of time and effort go into harnessing these implementations you'll find cracks within the surface.
Regression testing is a technique that tries to find problems in software that has worked before, but is now broken because something was changed.
Many Testing concepts are introduced because of regression and the uncertainty this gives developers and managers. Automated checks, rerunning Test Cases, elaborate smoke tests, are all in place because of regression issues.
Regression testing involves re-testing features that previously worked to ensure that they are not affected by the addition of new features or refactoring of existing ones.

What to do next:

• Think about a time when one feature in one of your favourite apps didn't work anymore. This might be in a game such as Pokémon Go, Facebook, Netflix application, or any other product that does frequent updates.We've grown quite used to these frequent updates and bugfixes. Additionally, a malfunctioning feature might be fixed tomorrow. Sometimes, this backfires and users get fed up and discard the product entirely.
• Consider the strategies these companies use to tackle their regression issues and compare them with strategies used in your company. Are they very different? Does that make sense to you? Can you come up with other strategies?
• Do you have something that you can practice regression testing on? Let's think back to that app on your phone. Did it recently update? Have a look around at the new functionality that the app creators say they have added. Has it broken functionality that previously worked? If so report it to them. They might already be aware of it but if not you've helped improve someone's experience.

14. Agile testing

If you're quite new to the software scene you will not have heard of development practices such as Waterfall, Agile or DevOps. These are methods or approaches to software development. A key thing to notice in the diagram below is where testing fits into each of the approaches. DevOps is relatively new to the scene. We'll start with Agile to ease you in and you can research DevOps more later.

15. Pairing (Pair Testing)

Pair testing involves two people working together at the same machine with a single keyboard to test a piece of software. One person has the keyboard while the other suggests ideas or tests, pays attention and takes notes, listens, asks questions, grabs reference material, etc.. There should be a shared goal with this approach and the two people will constantly communicate to ensure that the goal is achieved. Communication is key with this testing approach. Both people involved should have a shared understanding of what they are doing and why they are doing it.
This is easy and fun to do. You only need you and one other person. I try to do this as much as I can in work. It is a great way to get a different perspective on your work and maybe even learn a new skill.

What to do next:

• If you're not in a software house ask someone from the meetups you attended or off Twitter or Slack, you could even try remote pairing.
• Pair with this person and reflect on your experience.
• What did you learn from it?
• What could be better next time?
• Would you do it again?
• What did you teach the other person?

16. Mob Testing

This is not something you can do by yourself but you should read up about it in the Mob Programming Guidebook. If you can, try to do it in work or get a few people together to try this out. A good chance to do this is at the end of a sprint demo session. Start with a 15-30 minute slot with the whole team. Get them to start asking questions and testing the product as a team. This is a great way to get non testers involved in testing and possibly help generate some new ideas or approaches.
If you're lucky the meetup you attend might try this. If they don't try it, ask them! Any meetup organisers I know are crying out for feedback from the community of what they would like to see. It may not be something you can do at the moment but it is definitely something you need to be aware of.

17. Exploratory testing

Exploratory testing is one of my personal favourites. You get to explore areas of the application that you most likely didn't cover with other testing methods or covered, but you felt like you could investigate more. There are some really interesting articles I will advise you to read for this. Most of them are quite recent!
James Bach has written his thoughts about What is Exploratory Testing. Simon Tomes recently wrote an excellent article where he gave three digestible diagrams to describe exploratory testing. I particularly like this article as someone who learns more from diagrams than text. I would also like to highlight the book Explore It! that I mentioned earlier. This book is definitely a go to for exploratory testing.
What to do next:

• When you are done reading them pick something you would like to explore, maybe an app on your phone as you take the train to work.
• Explore with intent for the day!
• Did you find anything unusual?
• Did you feel like you explored it enough?
• Would you approach it differently the next time? 

18. Mobile testing

I would call this a specialisation. I've put it in here so that you have resources to be aware of the specialisation. Maybe you'd like to specialize in mobile testing or your job requires knowledge of it. Check out this quick introduction to mobile application testing. There are channels on both Ministry of Testing and testers.io Slack for mobile testing where you can ask for guidance if you wish.
Think about the apps on your phone. If you go into the about section for them does it say anything about the operating systems they support? Given what you now know about mobile testing does this surprise you?
When you're finished getting to grips with mobile testing you might want to try the 30 days of mobile testing challenge. 

19. Security testing

We've all seen the big hacks over the past year or more: TalkTalk, Ashley Madison, 3pay, Tesco. Security testing is becoming more important than ever before.
As the world around us changes and we start to (sometimes unknowingly) put more information out there about ourselves.
This task is just a starter to get a flavour of security testing. Again, it is probably one of those tasks that won't be just a day.

Your mission for today is:

• Watch or read about an introduction to security testing and see if it is something you would like to explore further 
• Check out OWASP and the incredible resources they have

20. Automation

Automation is a huge topic of debate and discussion within the software testing community. When you delve into this topic be prepared to hear such things like "automation fixes everything". These perceptions are the possible cons of automation. You may also read things about "testing vs checking", please do not be put off by the language used, I know I found it overwhelming at the beginning. One of the pros of automation is that it is a way to enhance the coverage you have of your application. You are only one person and you can miss things in re-checking of features, you are human. Computers can be very good at doing some things that people often fail at. Possible tools to use for automation are varied and can depend on the type of application you are working with. One of the main things I struggle with for automation is how do you decide what to automate.
If you want to give automation a try I recommend checking out the course from Richard Bradshaw or Alan Richardson to get you started. Of course Selenium is not the only automation tool out there, but I have personally taken these two courses and can highly recommend them. Have you found any others? Did you find them useful? Share them with others looking for these resources.

21. Performance testing

Have you ever been in an online queue for either of these? Load and performance testing can help the whole team to get an estimate of possible weakness in the product before it is released to customers.
Performance testing aims to determine how responsive an application is under normal load conditions. Examples of questions that performance testing tries to answer are how long a transaction takes to complete or a page takes to load. It can help teams see if a site can handle the projected number of users. A performance test in a tool like JMeter would keep the number of users at a normal level, where normal would be based on the requirements for the system.
Load testing is done to determine how the system responds under heavy load conditions. It would measure response times and resource utilization under this heavy load. Load testing also helps to determine when the system breaks and if it is graceful when it does so. A question we may like answered when we do load testing would be how many transactions are we actually capable of dealing with when the system is under heavy load. When running a load test the level of users is kept high to get an idea of how the system will respond under the level of usage.

What to do next:

• Read this Load Test Article by Scott Stirling which gives a nice breakdown of load, background, stress and performance testing
• Investigate the JMeter User Manual

Read more...

Difference between Bug, Defect and Error

Bug:
An Error found in the development environment before the product is shipped to the customer.

Bug: Simply Bug is an error found BEFORE the application goes into production. A programming error that causes a program to work poorly, produce incorrect results, or crash.  An error in software or hardware that causes a program to malfunction.

Defect:
Defect is the difference between expected and actual result in the context of testing. Defect is the deviation of the customer requirement. An Error found in the product itself after it is shipped to the customer. Defect is an error found AFTER the application goes into production. Simply defect can be defined as a variance between expected and actual. Defect is an error found AFTER the application goes into production.

Categories of defects – There are three main category of defects;

Wrong: The requirements have been implement the wrong way. This defect is a variance from the given specification.

Missing: There was a requirement given by the client and it was not done. This is a variance from the specifications, an indication that a specification was not implemented, or a requirement of the customer was not noted correctly.

Extra: A requirement incorporated into the product that was not given by the end   customer. This is always a variance from the specification, but may be an attribute  desired by the user of the product. However, it is considered a defect because it’s  a variance from the existing requirements.


Error: It the one which is generated because of wrong login, loop or due to syntax. Error means normally arises in software Error means to change the functionality of the program.

Fault: A wrong or mistaken step, process or Data definition in a computed program which causes the program to perform in an unintended or unanticipated manner.

Difference between a defect and a failure –
When a defect reaches the end customer it is called a failure and if the defect is detected internally and resolved it’s called a defect.

Read more...

What is a Defect Life Cycle or a Bug Life Cycle in Software Testing?

Defect life cycle is a cycle which a defect goes through during its lifetime. It starts when defect is found and ends when a defect is closed, after ensuring it’s not reproduced. Defect life cycle is related to the bug found during testing.
The bug has different states in the Life Cycle. The Life cycle of the bug can be shown diagrammatically as follows:

Bug or defect life cycle includes following steps or status:

1. New:  When a defect is logged and posted for the first time. It’s state is given as new.
2. Assigned:  After the tester has posted the bug, the lead of the tester approves that the bug is genuine and he assigns the bug to corresponding developer and the developer team. It’s state given as assigned.
3. Open: At  this state the developer has started analyzing and working on the defect fix.
4. Fixed: When developer makes necessary code changes and verifies the changes then he/she can make bug status as ‘Fixed’ and the bug is passed to testing team.
5. Pending retest:  After fixing the defect the developer has given that particular code for retesting to the tester. Here the testing is pending on the testers end. Hence its status is pending retest.
6. Retest:  At this stage the tester do the retesting of the changed code which developer has given to him to check whether the defect got fixed or not.
7. Verified: The tester tests the bug again after it got fixed by the developer. If the bug is not present in the software, he approves that the bug is fixed and changes the status to “verified”.
8. Reopen: If the bug still exists even after the bug is fixed by the developer, the tester changes the status to “reopened”. The bug goes through the life cycle once again.
9. Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to “closed”. This state means that the bug is fixed, tested and approved.
10. Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “duplicate“.
11. Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state of the bug is changed to “rejected”.
12. Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next releases. The reasons for changing the bug to this state have many factors. Some of them are priority of the bug may be low, lack of time for the release or the bug may not have major effect on the software.
13. Not a bug: The state given as “Not a bug” if there is no change in the functionality of the application. For an example: If customer asks for some change in the look and field of the application like change of colour of some text then it is not a bug but just some change in the looks of the  application.

Defect Life Cycle (Bug Life cycle) is the journey of a defect from its identification to its closure. The Life Cycle varies from organization to organization and is governed by the software testing process the organization or project follows and/or the Defect tracking tool being used.
Nevertheless, the life cycle in general resembles the following:

Status	Alternative Status
NEW
ASSIGNED	OPEN
DEFERRED
DROPPED	REJECTED
COMPLETED	FIXED, RESOLVED, TEST
REASSIGNED	REOPENED
CLOSED	VERIFIED

Defect Status Explanation

• NEW: Tester finds a defect and posts it with the status NEW. This defect is yet to be studied/approved. The fate of a NEW defect is one of ASSIGNED, DROPPED and DEFERRED.
• ASSIGNED / OPEN: Test / Development / Project lead studies the NEW defect and if it is found to be valid it is assigned to a member of the Development Team. The assigned Developer’s responsibility is now to fix the defect and have it COMPLETED. Sometimes, ASSIGNED and OPEN can be different statuses. In that case, a defect can be open yet unassigned.
• DEFERRED: If a valid NEW or ASSIGNED defect is decided to be fixed in upcoming releases instead of the current release it is DEFERRED. This defect is ASSIGNED when the time comes.
• DROPPED / REJECTED: Test / Development/ Project lead studies the NEW defect and if it is found to be invalid, it is DROPPED / REJECTED. Note that the specific reason for this action needs to be given.
• COMPLETED / FIXED / RESOLVED / TEST: Developer ‘fixes’ the defect that is ASSIGNED to him or her. Now, the ‘fixed’ defect needs to be verified by the Test Team and the Development Team ‘assigns’ the defect back to the Test Team. A COMPLETED defect is either CLOSED, if fine, or REASSIGNED, if still not fine.
• If a Developer cannot fix a defect, some organizations may offer the following statuses:
  • Won’t Fix / Can’t Fix: The Developer will not or cannot fix the defect due to some reason.
  • Can’t Reproduce: The Developer is unable to reproduce the defect.
  • Need More Information: The Developer needs more information on the defect from the Tester.
• REASSIGNED / REOPENED: If the Tester finds that the ‘fixed’ defect is in fact not fixed or only partially fixed, it is reassigned to the Developer who ‘fixed’ it. A REASSIGNED defect needs to be COMPLETED again.
• CLOSED / VERIFIED: If the Tester / Test Lead finds that the defect is indeed fixed and is no more of any concern, it is CLOSED / VERIFIED. This is the happy ending.

Defect Life Cycle Implementation Guidelines

• Make sure the entire team understands what each defect status exactly means. Also, make sure the defect life cycle is documented.
• Ensure that each individual clearly understands his/her responsibility as regards each defect.
• Ensure that enough detail is entered in each status change. For example, do not simply DROP a defect but provide a reason for doing so.
• If a defect tracking tool is being used, avoid entertaining any ‘defect related requests’ without an appropriate change in the status of the defect in the tool. Do not let anybody take shortcuts. Or else, you will never be able to get up-to-date defect metrics for analysis.

Read more...