Where are my requirements please?

My experience of modern software dev projects, has been that business requirements generally start as discussion, which are quickly translated into development requirements. What I believe modern QA professionals bring to modern software development projects, is ability to reverse engineer requirements based on sound software testing principles, and ongoing discussions with project management.Â

The best way to start a discussion is a question. “Why does the customer want this software?” If no-one can answer this question, the team must be developing to their own requirements, based on initial “suggestions” from the client. QA is in a unique position, as QA is the conduit between management and technical, in the course of release cycles.

The customer is very likely to be unaware of all their requirements, as the initial sell can sometimes cloud/warp their real requirements. The advantage of QA is that the drive behind our test requirements is to have a complete picture of what an application is supposed to be doing, and that it is doing what everyone expects it to be doing. Raising issues doesnt just have to be coding issues, it can be requirements issues that require a business decision.

Where testers can fall down, is losing themselves in the development cycle, and effectively losing track of the bigger picture of the project. Not a failure of testers, but of test management, a role considered a luxury, rather than necessity, these days.

New Methodology

After seeing post I raised related to Agile and other related methodologies, and how QA can adapt and improve (see post), a friend of mine raised a very valid suggestion – write a new methodology. In the course of consulting QA management work, adaptation of traditional QA processes has been essential, as well as being the unpopular guy who points out faults in a company’s development methodology. Although I have the experience, creating a methodology is a challenge I am happy to take on.Â

I am not looking to reinvent the wheel, just for the sake of it.  But my approach will attempt to include positive elements from existing methodologies combined with with my experience in the field. Media/Gaming/Mobile projects are ideal to base a new methodology around.

Risk Based Test Strategy

Risk based testing is a common approach in the fast demanding modern methodologies in development. In this strategy we assume that it’s undoable to test everything. From a economic point of view it doesn’t even make sense; spending lots of time to parts of a system where the changes of having a bug are low, or even if a bug has been found, where the impact of it will be low. Risk and requirements based testing helps you to determine what to test first, in which sequence, so you spend the time you have to the parts that really matter.

The strategy starts with a risk analysis to determine the functions (requirements) with the highest risk, and plan your test activities guided by this analysis.

To help you identify the risks involved in all your requirements, consider the following aspects:

  • Functions often used by the users
  • Complex functions
  • Functions that have a lot of updates or bugfixes
  • Functions that require high availibility
  • Functions that require a consistent level of performance
  • Functions that are developed with new tools
  • Functions that require interfacing with external systems
  • Functions with requirements with a low level of quality
  • Functions developed by more programmers at the same time
  • New functions
  • Functions developed under extreme time pressure
  • Functions that are most important to the stakeholders
  • Functions that reflect a complex business process

Reverse engineering requirements

One of the great challenges of modern development projects is the phenomena that would have been unthinkable 10 years ago, which is very real possibility of little/no business requirements, or very out-of-date requirements.  Generally, once a project reaches development phase, most methodologies will assume requirements are in place. If they are not in place, then there will be some kind of development requirements. While these are sufficient to develop software, they are not sufficient to deliver software to a customer. In these circumstances, reverse enginnering requirements can help solve the issue.

No-one wants to go backwards, but sometimes it is a necessary evil. In my QA manager capacity, I have seen this situation many times. The way to reverse engineer requirements, by stealth, is a combination of:-

1. Get the customer more involved in the test cycle (so they can realise what their requirements are)
2. Generate some standard business requirements based on generic software standards.
3. Generate some standard business requirements based on conversations.
4. Use higher level development requirements.

Once you have a baseline set of business requirements, you can then put them forward for review.

Localization on a zero budget

This was an interesting exercise is battling the odds. I was providing QA management on a mature successful MMORPG that was approaching localization phase. The game was based on open source technology, and had been developed in Agile/SCRUM environment in the US office. The localization was managed from the UK. The first big hurdle was that the game had been totally hardcoded, so initial work was done to externalise all game data. Translations were done by external translations company, but up to the point when I arrived, the translation had been very ad-hoc, leading to basic errors. The localized versions were hosted by various ISP’s in different countries, and their involvement was limited to verifying the translations.

There was no real budget allowance for this phase of project, so the first move I made was to streamline the translation process, from the translation company, through development and through to the testing at the end of the cycle. As the data had been externalised into one comprehensive localizer file, I took this as the first point in the cycle. This file was provided (intact) to the translations company. To speed up process, and to ensure file integrity, I specified acceptable methods of viewing and editing this file (I chose Vi, as this had support for multiple language character sets, and there was free version available). In order to keep this file integrity, I used QA as the conduit between various people.

Once the translations company had translated the complete localizer file, it was uploaded to shared ftp drive, where QA validated the file integrity, before sending to US dev team for implementation on QA server. The ISP was then asked to validate the localizer file, and then perform some in-game testing to verify correct use of grammar, and identify issues, such as graphic overspill, caused by elongated text. I also used this process as a formal signoff stage, as the informality of the environment meant there was little or no formality. For the business this could prove disastrous as in the event of issues, it was difficult to trace where problem was first initiated.