My interest to performance requirement was started from a few practical questions about five years ago, and since that time I return to performance requirements again and again, trying to package the topic in a meaningful way. Now I am working on the second part of the Performance Requirements: An Attempt at a Systematic View article (the first part was published in the June 2011 issue of the Software Test & Quality Assurance magazine) and thinking about coming CMG presentation (see the last version of my performance requirements slides from the Connecticut CMG group).

One thing I have realized is that people have quite different notion of performance requirements. Some believe that it is a part of the thick volume that somebody (Business Analysts?) creates at the beginning of the project and the volume is kept on the shelf during the project. If you see articles like Ditch the Requirements – Focus on the Customer Instead, you may be sure that the author uses this notion. I am still looking for somebody to advocate creating systems completely without knowing what the system should do – so “Ditch the Requirements” usually means to describe the system in another way. Well, this notion is all but non-existing, at least nobody advocates creation of such static volume anymore (although, I guess, many still doing it to get a contract, to make a checkmark, etc. – but it is another story).

A little more elaborate “performance requirements” notion is that you define performance requirements at the beginning of the project, maybe adjust them a little during the project, and then verify that the system meets the requirements at the end of the project (usually by load testing). This notion probably matches today’s typical practice (where there is any – usually with the real-life correction of having performance requirements discussion just before load testing). Still this notion is pretty limited and assumes that you are only interested to make sure that the system meets criteria, so you may throw it over the wall (whatever the wall is). You don’t actually care what people will do with it on another side of the wall. Although it still may be the prevailing approach, it looks like the majority of thought-leaders agree that it doesn’t deliver the optimal results for the business.

You need to consider the whole life span of the system, “from conception to gravestone”. And performance view is a very important aspect of the system. You need to have the goals you always measure system performance against and make sure that your actions move you closer to these goals, addressing any negative trends or deviations. How can you manage performance if you don’t measure it or don’t have anything to compare it with? Actually it is the main idea of different management methodologies (see, for example, ITIL Continual Service Improvement or Six Sigma): you set the goals and meticulously track your progress toward them. When I am speaking about performance requirements, I keep this notion in the mind: we set some initial performance goals in the beginning of the project; elaborate them during the design and development, making sure that we are on our way to meet them; verify them with performance testing; and make sure that we continue to measure performance in the same consistent way in production through all stages of the system life cycle.

So when I am talking about performance requirements, I mean setting initial performance goals and tracking the progress towards these goals during the whole system lifecycle (with all needed adjustments along the way). Something we build the whole performance engineering process around. And here we face many more issues comparing with more limited approaches. For example, a performance goal which you can’t measure in production doesn’t make much sense. So if we set such goal at the beginning, we need to adjust at some point and replace it with similar measurable goal (better earlier than later).

I was referring to this as performance requirements because it is how most people would refer to this area, but looks like the term has too many different connotations and actually starting to hurt the point I am trying to convey. Do we need a better word for it?