Software testing: the story behind flawless code
Software testing can feel a lot like being a demolition man. Or like being a curious kid with an appetite for dismantling expensive toys. It comes with the daily practice of getting into the mindset of end-users of all kinds.
At Tremend we’ve been testing software since we started coding for our clients. Over the past 11 years it has become clear that functional and usability testing is essential for high quality software. Here are some of the things we’ve learned about the process and the people involved.
The software tester: a profile
It all starts with some illegible piece of code that developers hand over to QA testers one hot Friday afternoon. The code works, says the developer. It meets the specs, he says.
Not so fast, replies the QA engineer with a grin. Over the following days, the Bugminator will subject the code to every possible scenario that end-users might think of. Or run into. Just like in the story about the fellow tester who walks into a bar and asks for a beer. Then he orders 0 beers. Then he orders 999999999999 beers. Then he orders a lizard. Then he orders -1 beers. Then he orders NULL beers. Then he orders Asnwikfjsdf. Then he tries to leave the bar without paying. All these scenarios are valid scenarios for a software that has a large number of users. Testing only the “golden path” scenario, with positive “non-destructive” use-cases, does not give us an actual overview of the software’s quality.
For example, one QA engineer might be required to test the login screen of a website. Using only “golden path” use scenarios, the testing would finish quite fast. But an expert QA engineer can look further into use case scenarios and even end up seizing control of the website’s database via a classical hacker method – an SQL injection.
But there is more to testing than a passion for creative-destructive trials.
QA engineers need to understand the client’s business and workflows. That’s because testing is about both making sure the code works flawlessly and about making sure it is easy to use. So testers are curious people. They are eager to know what is under the hood, behind the product and the business.
Moreover, QA specialists should know how users behave. And they talk with clients in order to align the testing strategy with the usage scenarios.
Software bugs can be deceitful
Bugs come in different flavors and behaviors. For sure, they can be much more than a basic malfunction.
We have seen cases when users became accustomed with operational problems of their software and no longer saw them as bugs. As a result, using the software implied losing precious time because of software defects or usability quirks. Eventually they embraced the problems, instead of dealing with them.
Our trained QA specialist spotted such issues and suggested usability improvements and pointed out that the current behaviour is not the most efficient one. They have also noticed bugs that lurked within faulty workflows. There can be invisible issues that simply drive new customers away – issues such as a clumsy web interface or insufficient interface communication, lack of user options and more.
Testing software vs. building a new solution
It is cost-effective to audit an existing solution before deciding to develop a new one. There are cases when a legacy software platform can be patched, secured and stabilized with lower expenses than the costs of building a new one.
But when the existing solution becomes obsolete, developing a new platform can make better sense. The same happens when game-changing technologies emerge that are difficult to integrate with the current one. Or when the cost of patching the current tool becomes comparable to the alternative.
In order to properly assess a current solution, a QA specialist should be involved, a test suite should be defined and run against the current software solution. Thus a baseline for the current product quality can be established and a properly informed decision can be taken.
Software testing: in-house or outsourced?
Having an outsourced testing team in a different location has its advantages. In case of a large non-software company that has a coding department, we have often noticed that developers tend to argue that the software behaves in a particular way, due to specific technologies they use. There is no extra technical trained eye, that can offer a second opinion to the one of the development team. If testers are in the same location and interact on a daily basis, they may become indulgent and eventually, they may forget to support the point of view of an average user.
Therefore, a separation of testers from the development team can prevent biases. It can foster stronger, business-oriented opinions. Needless to say, sometimes the internal testing team is not so well trained an experienced as compared to an outsourced one. It’s hard to maintain quality assurance with non-specialized personnel. That is why QA outsourcing can be a better option for companies that develop their own software, but have a different core business (banking, telecom, etc.).
In the case where in-house testing is preferred but there is no available personnel, it is recommended to start the team by employing a trained QA consultant that can kick-off the quality assurance department or implicate an entire specialized team together with it’s team leader. It can be difficult to start a new department without an experienced opinion to guide you through.
How to choose a software testing company
Here is a useful checklist for choosing a software testing supplier:
- Analyze the technical diversity of the team. Look for automation test framework diversity. One automation framework may not be suitable for all types of software products. For example for websites developed in Angular JS, Protractor is a much more suitable test automation framework compared to Selenium. Also, having experience in testing with both mobile and desktop applications allows a better understanding and consequently better test results.
- Verify their expertise with your own kind of software
- Check the team’s certifications, ISTQB® being the best known and also included among Tremend QA team’s qualifications.
- Ask about the team’s ability to conduct various types of QA processes – including functional, usability, performance, compatibility and scalability testing.
- Make sure the team can develop automated tests and a strategy to conduct them.
- Look for a positive can-do attitude, which will ease the daily development-testing communication
- Ask about the tools they are using. At Tremend, for instance, we rely on some of the most advanced testing solutions in the industry, including the Appium automation framework , Django tests, Selenium Grid, Selenium WebDriver, Jenkins, TestNG, Junit, Cucumber, Mink/Gherkin, Robot Framework and LDRA. These tests are usually run from Jenkins in a continuous integration environment. After each build is done, an automatic smoke test suite is run to ensure proper stability of the product, ensuring a go/no-go status for the product. Then the full regression test suite is run by Jenkins, to test all the rest of product features. It’s important to separate test from the build server, by creating a Jenkins master slave configuration. Tests for website products should run in parallel on multiple browsers by using Selenium Grid.
Our advice is that you should never ignore a QA team, nor consider it as being optional. If you need support in deciding how to approach software testing for your organization, feel free to say firstname.lastname@example.org.
Tremend software solutions include the full range from online stores to complex software for the banking industry. For over 11 years we have developed for the Internet of Things, e-commerce platforms, enterprise solutions, embedded software, CRM, CMS, ERP integration and custom software. Over two million users benefit from solutions developed by our team of software engineers.