In Scrum, quality is defined as “The ability of the completed product or deliverables to meet the Acceptance Criteria and achieve the business value expected by the customer”.
I hope you find this Performance Test Plan template helpful. Please use it wisely. You see, I have mixed feelings about template test plans – performance or otherwise.
If the Pentagon and the Whitehouse can both be hacked, then so can you. Now, please don’t think for a second that I think the members of the Pentagon and Whitehouse are in any way more intelligent or better than you (I’m convinced of quite the opposite – without even meeting you), but I am assuming they have pretty good cyber security.
High user expectations and the risk of putting poorly performing code live makes performance engineering more important than it has even been. Fortunately, incremental change makes it easier to engineer system performance and build a workable process. For those doing DevOps, having administrators and technical subject matter experts as part of the delivery team have a much better ecosystem within which to deliver effective performance engineering.
A functional requirement specifies a function that a system or system component must be able to perform. Probably the easiest way to explain ‘non-functional’ requirements is that they specify all the remaining requirements not covered by the functional requirements. Non-functional requirements specify the system’s ‘quality characteristics’ or ‘quality attributes’. Two products could have the same functions, but their attributes can make them entirely different products. A Rolls Royce has more or less the same functionality as a Lada but many different attributes!
What if Entry Criteria for Load and Performance Testing are not met?
A little while ago I wrote a blog called ‘What are the entry criteria for Load Testing?’.
Lack of Appetite to Test
The most debilitating and possibly biggest challenge I face when it comes to load testing is the lack of appetite to load and performance test from the senior project stakeholders.
Businesses, Development, Operations and IT teams aim for faster delivery, greater stability and higher quality IT systems to gain a competitive edge.
That’s where DevOps comes in. Having Development, Test and Operations perfectly combine with one another whilst reducing restrictions, barriers between the teams and transitions can lead to high quality products being produced quickly.
DevOps is not about cutting corners, forgetting good practice or reducing quality assurance activities. It’s rather doing things with minimal touch points between separate teams and individuals, automating where both possible and beneficial to do so. It’s leveraging the value of continuous testing and creating build, deploy test scenarios that work.
To ensure high quality products, automated unit testing and Test Driven Development (TDD) (or similar) can be used to ensure that the code works as designed and can be checked in to the latest release. Continuous integration and continuous testing is used to ensure there is no breaking or regression of code. This means that code can be integrated on demand and tested dozens of times a day.
Automated build, deploy test scenarios means the automated regression packs are vital and need to have a good depth and breadth of tests. The selection of what to test and what not to test with what data is where many teams go wrong. A lack of experience with DevOps and trying to apply traditional automation approaches can leave Use Cases and objects untested, opening up quality risks for the product.
It must be recognised that the automated tests must be written in a robust and effective way to ensure quality. This is also where I see many teams failing as the effort required to maintain the tests and the test framework becomes a large overhead because they have not been architected correctly.
Automation is critical in DevOps, but it is important to realise that not everything can or should be automated. This is where experienced DevOps testers play a vital role. Often it is more efficient or more effective to test manually – either using a test script or via exploratory testing. This therefore minimalises cost and maximises efficiency.
The challenge of continuous testing with the right test scope, right test data and optimised amount of both automated and manual tests is a tough one. Getting this right by engaging with a professional testing consultancy, well versed in DevOps can generate a huge return on investment by practical yet helpful advice and coaching.
nFocus are a Microsoft DevOps Gold Partner. If you would like more information or to discuss how nFocus can help you – call us on 0370 242 6235 or visit our contact us page here.
Topics: Testing in DevOps
Hiring a team that consists of qualified people that are well versed in Agile principles is extremely important for any software development team. But the task of interviewing and hiring good Agile testers often proves to be difficult. With that in mind, we've created a new white paper with 10 great questions you can ask when conducting interviews, and a guide to the kind of responses we would expect. You can download the full white paper here, with a sneak peak below.