Infrastructure testing

For most of my testing career I never needed to think about Infrastructure. It was simply “there” to make sure the actual test object could be functionally tested. But what if the infrastructure is actually the test object?

When I was a member of the test line dedicated for a fraud management application the team was suddenly forced to think about this question. After years of testing new queries, data streams, reports and such we were asked to perform functional acceptance testing for an upgrade of the Oracle database and the application. But what would be the scope? And how would we approach this?

And that was only the start of it. The next step was to move this application to an entirely new infrastructure on a different location including a cold standby environment in a different location, which was inactive but ready for use in case of major production incidents. I was asked to write a master test plan for this project and manage the entire test project. That required a major shift in focus from the functional projects I was used to do.

With the help of the technical and application experts I created a master test plan approved by all and helped them create detailed test plans for all infrastructure components and aspects delivered during the first phase of this project. However even with so much input and cooperation from all these experts we did find during testing that there had been some gaps.

The backup team delivered their solution but when we were about to test this we found out they never actually scheduled any back-ups, because they were waiting for the application team to provide details on what to back up and when. During failover testing we found out the Windows team had misunderstood the design of our cold standby environment and delivered a virtual machine there with only an OS installed but no application in it. During functional testing we found out the database team had delivered the databases in a location where they were inaccessible for the data analysts, which required a last minute solution for secured access.

By now I’ve been involved in over 25 of such infrastructure projects and in hindsight I would’ve certainly done some things differently for this first project. Without high-level knowledge of infrastructure concepts I was entirely dependent on the technical experts and unable to keep the complete overview necessary to define and realize my test strategy.

It is to this end that I decided to document some this knowledge that I’ve gained through these projects in a new TMap Suite building block “infrastructure testing”. It is my goal to share an understanding of the complexities and relationships involved in infrastructure testing and some of the challenges that a test manager will face in such projects. This building block is aligned with Sogeti’s standards InFraMe® and TMap | Infrastructuur. Please bear in mind that in your organization these concepts and their relationships may be interpreted differently.

I would like to thank Thomas Veltman, Robert Cornelisse and Conrad Horsten for their reviews of and input for this building block.