Blog

Software testing is one of the most interesting fields in modern computer science. In theory, every technical person knows what tests are. He often even knows how to make them. However, if you ask about the basic principles of testing, few people can mention even one. This article deals with these principles.

In many areas of our lives, we try to see certain patterns, patterns and truths. This is not surprising. Our brain works largely like an associative (associative) machine. So the occurrence of repetitive phenomena (like the rotation of the earth around its axis) is simply called (e.g. day). In the case of universal patterns, schemes or truths about software testing, we are talking about 7 principles. It sounds smart and it is. However, this does not mean that these principles are secret and difficult to understand, especially when we outline them well.

To understand the actual state of science, which is computer science, let me quote one of the fathers of today’s computer science Donald Knuth:

People think that computer science is the art of geniuses but the actual reality is the opposite, just many people doing things that build on each other, like a wall of mini stones.

This analogy, however, with greater pathos, is presented to us by the song of Juliusz Słowacki „My Testament ”, from which the title of beautiful and difficult novel „Stones for the rampart ” was taken. At this point, it’s worth asking yourself why I write about it at all?

The topics of this article are likely to go to many beginner testers, and the topic itself is one of the most important and relatively simple to test software. That is why I would like to encourage and wish readers that they can refer to a fragment of the above-mentioned poem with their attitude:

But I swear: let the living not lose hope
And before the nation they carry an education muzzle

Without extending, let me discuss the 7 principles of software testing according to the latest syllabus ISTQB^®.

Rule 1. Testing reveals the presence of defects, not their absence

This means that in the software testing process, the errors found indicate the existence of defects (often colloquially, not quite correctly, called bugs from bug. The term bug is due to one of Grace Hooper’s IT pioneers. Together with her colleague, she found a moth that caused one of the systems to be overvolted on Mark II’s computer), and not completely missing. Testing only reduces the likelihood of them occurring. Software testing does not prove its correctness, as there may be errors that were not detected in the testing process.

 

Rule 2. Comprehensive testing exhaustive testing ) is impossible

Apart from trivial cases, testing the program with all possible inputs / conditions is simply impossible. Imagine such a function as calculating the distance between two points. This function takes the parameters: x1, x2, y1, y2. It subtracts the appropriate parameters from each other, then squared them and adds to finally draw the square root. Suppose:

• x1, x2, y1, y2 are 64-bit numbers (we do not enter in the way of writing and interpreting values)
• we have an external machine that will always indicate the correct result of the operation
• today’s computers have an average processor clock speed of 5 GHz
• this function will literally take 1 processor clock, and the machine externally comparing whether the result is correct immediately returns the result.

So we have (2^64)*4 or (2 ^ 66) operations to be tested. You can reduce their amount by noticing that some data is trivial (e.g. x1 = x2). In addition, part of the operation (y1 = n1 y2 = n2 and y1 = n2 y2 = n1) is redundant. However, this will not be exhaustive testing then.

By dividing the number of operations by the number of tests performed per second (5*10^9 – as much as processor clock speed), then for 60 (seconds), 60 (minutes) and 24 (hours), we get:

>>> (2 ** 66) / (5 * 10 ** 9) / (60 * 60 * 24)
170803L
>>> (2 ** 66) / (5 * 10 ** 9) / (60 * 60 * 24) / 365
467L

To test such a trivial function we need in a very optimistic variant (high clock speed, the operation takes literally one processor cycle) 467 years. I don’t need to explain why more advanced features and programs by their very nature cannot be thoroughly (another name for exhaustive testing) tested.

Rule 3. Early testing saves time and money

The principle itself is quite obvious – finding an error in the early stages is simply cheap. It is therefore important that the testing process is started as soon as possible. Early testing avoids making cost changes to the project at later stages of his life.

One of my favorite articles on this topic is the material prepared by NASA, taking into account many real software development projects and specialized equipment – Error Cost Escalation Through the Project Life Cycle. According to it, in the worst case, repairing the error at the stage of implementing the solution and its maintenance can be up to 1500 times more expensive than detecting it when collecting requirements.

Rule 4. Defects accumulate (cluster) together

Most design errors are usually found in a small number of modules. Finding an error in the module gives you a high probability of finding many other errors in its vicinity. It is also important when analyzing risk and estimating which modules should be devoted to more broadly understood work. Some researchers indicate the existence of the Pareto principle, where 20% of modules are responsible for 80% of errors.

Rule 5. Pesticide paradox

Pesticides are measures designed to eliminate a certain group of harmful organisms. Along with the use of pesticides, the population of organisms is naturally resistant to their effects. Antibiotics – work similarly, as evidenced by the presence of antibiotic-resistant bacteria, e.g. tuberculosis.

The same tests are designed to detect and then eliminate defects. When testing, errors / code defects become resistant to unchanged tests. Tests lose their effectiveness in detecting defects over time, so you should update them on a regular basis and create new ones. This can be compared to changing (proportion and composition) of the mixture, which is sprayed with an arable field to eliminate „bugs ”.

Rule 6. Testing depends on the context

Individual system components are tested differently at the stage of their integration, and in other words, e.g. a single function. Testing also depends on the software development methodology and from what we produce. The client portal, or space rocket software, is tested in a different way. There is a lot of specialization in software testing for a reason, such as built-in software tester, web application tester or automation tester.

Rule 7. Convincing about the lack of errors is a misconception

This is one of the errors in software perception that usually occurs in non-technical people, such as project manager or CEO. They often expect the tester to be able to carry out all test cases (this is not feasible – see rule 2) and that the program will not contain any errors (this is not feasible – see rule 1). Accounting for people testing only from the number of errors detected is doomed to failure – the lack or small number of errors detected (again in accordance with the first rule) does not mean that the system is ready for implementation. This also applies the other way around, detecting a large number of errors may / need not mean ensuring the quality of the code.

In addition, even the best-tested software may still not meet customer expectations, be difficult to use or simply act improperly (but as intended!).

What’s next?

Knowing the 7 rules about software testing, I recommend that all those interested in studying the ISTQB – syllabus have knowledge accumulated in a nutshell.

In the near future, more testing articles will appear on our blog along with more technical examples. If you liked this article, I encourage you to subscribe to our newsletter and to share your opinions in the comments.