“On the loftiest throne in the world we are still sitting on our own behind.” – Michel de Montaigne, Of Experience, cited in A History of Knowledge, Charles Van Doren.
How to measure response time (first appeared in InfoWorld)
Back when I studied electric fish I learned a valuable lesson from my research advisor: not all criticism in my professional life would be phrased with concern for my ego’s ongoing desire to inflate. The following episode, which also provided a second invaluable insight, led me to this conclusion.
My research involved a quantitative analysis of behavior. After some nasty FORTRAN programming and some manual charting and graphing, I showed it to my advisor.
“What’s it mean?” he asked. My answer involved the mathematics of information theory, capped by a precise measure of the communication between two interacting fish. Nobel Prize material.
“But what’s it mean?” he asked again, impatiently. When he failed to hear a satisfactory answer he told me to stop bothering him until I could explain it in English. I knew the number, but not what it meant.
Just for giggles, let’s take the same potshot at a basic number used in the specs for information systems: response time.
Oh, the concept is fine. Unfortunately, we usually measure it in units of time, which demonstrates our lack of insight into its meaning.
“The benchmark transaction completed in .782 seconds,” we might say. Or, in comparing brands and models of personal computer, “Model A completed our benchmark task in 1 minute 34 seconds, clearly outperforming Model B which completed it in 1 minute 51 seconds.”
Precise, measures, yes, but they mean very little.
The reason? Look from a customer-value perspective, remembering that “Customers define value,” is the first of the three rules of management. Response time isn’t a continuous numeric quantity. Customer-valued response time (for most of us) has only six values, and none of them is a number. Those values are Eye-blink, Quick, Pause, Delay, Break, and Overnight. To elaborate:
An eye-blink is, from a customer perspective, instantaneous. Never mind that sufficiently sensitive instruments can measure it. It takes place faster than a person’s ability to notice. Eye-blink response time is perfect. You can’t improve on it.
Quick processes happen fast enough that users don’t pay attention to them. The wait is noticeable, but not obtrusive. When we talk about “sub-second response time” we’re really saying we need it to be quick.
A pause breaks our rhythm, but doesn’t let us do anything else useful. Lots of computer processes cause us to pause. Printing a one-page memo results in a pause. So can saving a long document to disk or downloading a small, well-constructed Web page from an adequately powered server via modem. Likewise database updates when response time is bad. Pauses are awfully annoying. Every pause requires the exercise of patience, and each of us has only a fixed amount of patience available for our use each day.
Delays take longer than pauses – enough longer to do something useful. We can take a gulp of coffee during a delay. Sharpen a pencil. Dial a telephone number. File a document. In many respects a delay, while longer than a pause, tries our patience less. Loading nearly any application in Windows causes a delay. So do most Web pages.
Breaks are even better than delays. You can refill your coffee mug during a break. Ask your boss a quick question. File several documents. Booting your computer gives you a break. So does faxing a document through your fax/modem, or running a report off a medium-sized database.
Anything longer than a break may as well wait until you leave for the night. Start it up, go home, see the result the next morning. If you want to be daring, start it up before you leave for a meeting instead.
That’s it. Six possible response times. You haven’t improved response time until you cross a boundary from one to the other.
And that’s not always an improvement, because sometimes, you just need a break.