A functional requirement specifies a function that a system or system component must be able to perform. Probably the easiest way to explain ‘non-functional’ requirements is that they specify all the remaining requirements not covered by the functional requirements. Non-functional requirements specify the system’s ‘quality characteristics’ or ‘quality attributes’. Two products could have the same functions, but their attributes can make them entirely different products. A Rolls Royce has more or less the same functionality as a Lada but many different attributes!
What if Entry Criteria for Load and Performance Testing are not met?
A little while ago I wrote a blog called ‘What are the entry criteria for Load Testing?’.
Lack of Appetite to Test
The most debilitating and possibly biggest challenge I face when it comes to load testing is the lack of appetite to load and performance test from the senior project stakeholders.
Businesses, Development, Operations and IT teams aim for faster delivery, greater stability and higher quality IT systems to gain a competitive edge.
That’s where DevOps comes in. Having Development, Test and Operations perfectly combine with one another whilst reducing restrictions, barriers between the teams and transitions can lead to high quality products being produced quickly.
DevOps is not about cutting corners, forgetting good practice or reducing quality assurance activities. It’s rather doing things with minimal touch points between separate teams and individuals, automating where both possible and beneficial to do so. It’s leveraging the value of continuous testing and creating build, deploy test scenarios that work.
To ensure high quality products, automated unit testing and Test Driven Development (TDD) (or similar) can be used to ensure that the code works as designed and can be checked in to the latest release. Continuous integration and continuous testing is used to ensure there is no breaking or regression of code. This means that code can be integrated on demand and tested dozens of times a day.
Automated build, deploy test scenarios means the automated regression packs are vital and need to have a good depth and breadth of tests. The selection of what to test and what not to test with what data is where many teams go wrong. A lack of experience with DevOps and trying to apply traditional automation approaches can leave Use Cases and objects untested, opening up quality risks for the product.
It must be recognised that the automated tests must be written in a robust and effective way to ensure quality. This is also where I see many teams failing as the effort required to maintain the tests and the test framework becomes a large overhead because they have not been architected correctly.
Automation is critical in DevOps, but it is important to realise that not everything can or should be automated. This is where experienced DevOps testers play a vital role. Often it is more efficient or more effective to test manually – either using a test script or via exploratory testing. This therefore minimalises cost and maximises efficiency.
The challenge of continuous testing with the right test scope, right test data and optimised amount of both automated and manual tests is a tough one. Getting this right by engaging with a professional testing consultancy, well versed in DevOps can generate a huge return on investment by practical yet helpful advice and coaching.
nFocus are a Microsoft DevOps Gold Partner. If you would like more information or to discuss how nFocus can help you – call us on 0370 242 6235 or visit our contact us page here.
Topics: Testing in DevOps
Hiring a team that consists of qualified people that are well versed in Agile principles is extremely important for any software development team. But the task of interviewing and hiring good Agile testers often proves to be difficult. With that in mind, we've created a new white paper with 10 great questions you can ask when conducting interviews, and a guide to the kind of responses we would expect. You can download the full white paper here, with a sneak peak below.
For years, good testers were urging their Project Managers to build plans that allowed them to test early and test often. Agile delivery forces that model and so problems can be found and remedied quicker, moving the project to faster completion. Involving testers in the entire project leads to a new level of synergy that can deliver software products with a high level of quality.
Topics: Benefits of Agile