Peter Montague
The computer industry revealed an illuminating use of risk assessment recently. On November 24, the New York Times announced discovery of a bug in the Pentium computer chip, the flagship product of the giant Intel Corporation, the USA's premier chip maker.
A few days later, Intel confirmed a flaw in its Pentium microprocessor, the chip that serves as brain for millions of the spiffiest desktop "PC compatible" computers. An estimated 6 million new desktop computers rely on the Pentium chip. Now it turns out that the Pentium cannot do arithmetic correctly. For example, the Pentium gets a wrong answer if you tell it to divide 4195835 by 3145727. (It gives the answer 1.3337391, but the correct answer is 1.3338204. Likewise, the Pentium says 5505001 divided by 294911 = 18.66600093 but the correct answer is 18.66665197.
A Pentium processor (the chip itself) sells for roughly US$300. Since 6 million Pentium processors have been sold, that's about US$1.8 billion worth of flawed chips already built into high-end desktop computers. (Of these, 3 to 4 million have been sold; the remainder are still in stores and warehouses.)
Even for a large company like Intel, $1.8 billion is serious money. What should Intel do? Bite the bullet and offer to replace all those chips? You might think of that as the old-fashioned approach. A more modern approach — the one Intel adopted — is to maintain silence and do a risk assessment.
Intel said it had discovered the problem in June but revealed nothing publicly (and continued selling Pentiums) until the New York Times broke the story on November 24. After the story got out, Intel waited a week, then published results of an internal risk assessment showing that "average" computer users would get wrong answers only once every 27,000 years of normal computer use. A "heavy user", such as an accountant or financial analyst, might see an error once every 270 years, Intel's risk assessment concluded.
"The flaw is not meaningful for the vast majority of commercial PC applications", Intel said in a press statement on December 1.
On December 13, IBM announced it was halting all sales of IBM personal computers built around the Pentium chip. IBM said its own risk assessment had shown Pentium could cause an error once every 24 days for average users, not once every 27,000 years.
William R. Pulleybank, director of mathematical sciences at IBM's Watson Research Center in Yorktown, NY, said a large company running 500 Pentium-based computers might get 20 errors every day. Thus IBM's risk assessment concluded that the Pentium problem was roughly 400,000 times worse than Intel said it was.
In reporting this story, the New York Times acknowledged that both Intel's and IBM's risk assessments are possibly coloured by what economist Thorstein Veblen used to call "pecuniary exigencies" — the lure of loot. IBM will soon market its own microprocessor, called the Power PC, which will compete with the Pentium, so IBM has a business interest in making the Pentium look bad. On the other hand, Intel has business reasons for wanting the Pentium to look good; it doesn't want to replace $1.8 billion worth of brain-damaged chips, so it just seems natural to deny the problem — and what better vehicle for denying a problem than a risk assessment?
The Times quoted Professor Vaughan Pratt, a Stanford University computer scientist, who said he had investigated the Pentium independently and found its error rate "significantly higher than what Intel had reported".
Who are we to believe? If two of the USA's most resourceful corporations, analysing a well-defined arithmetic problem in a computer chip measuring about one inch square, get answers that differ by a factor of 400,000, what hope is there for reaching agreement on the hazards posed by a huge, complex machine like a solid waste incinerator, a nuclear power station or a petrochemical processing plant?
Clearly, risk assessment is not a tool for reaching true and reliable conclusions; it is just one more weapon that businesses can wield in their never-ending battle to increase market share and maintain profitability.
A similar conclusion must be drawn from a major study of risk assessment conducted by 11 European governments during the period 1988-1990, and published by the Commission of the European Communities as the Benchmark Exercise in Major Hazard Analysis in 1991.
Eleven European governments established teams of their best scientists and engineers and set them to work on a single problem: analysing the accident hazards of a small ammonia storage plant. Private companies like Rohm & Haas, Solvay, Battelle, and Fiat contributed experts as well.
The results were stunning: the 11 teams varied in their assessment of the hazards by a factor of 25,000. Analysing the hazards of a single small plant handling only one chemical, these world-class experts reached wildly different conclusions. For example, the individual risk at the "refrigerated storage site" was calculated by one group of experts to be one in 400, but by another group of experts to be one in 10 million.
The teams of experts disagreed on:
- which kinds of accidents were most important to study; some teams ignored as unimportant accidents that other teams modelled fully;
- behaviour of a plume of ammonia after release; the scientists reached different conclusions about where the chemical would go, and how fast;
- the possible consequences when ammonia left the plant site and entered the surrounding environment;
- how rapidly the ammonia plant operator would respond to the emergency: some assumed response after a few minutes; others thought the response might take half an hour;
- the probability of success of mitigation measures, such as the ability of an operator to isolate a major ammonia release.
The experts analysing this relatively simple problem reached vastly different conclusions and, for them, it was a sobering exercise.
Many of the differences between the teams' results could be summed up as differing assumptions. As their report said, "At any step of a risk analysis, many assumptions are introduced by the analyst, [and] it must be recognised that the numerical results are strongly dependent on these assumptions".
Meanwhile in Washington, the Republicans have announced an entirely new use for risk assessment. As part of their "Contract With America", they have unveiled a piece of legislation called "The Risk Communication Act of 1995". This legislation would require all government agencies to conduct a risk assessment and a cost/benefit analysis for every proposed regulation that would affect "more than 100 people" — in other words, just about every regulation.
The procedure for risk assessment is spelled out in the proposed legislation. As part of the procedure, government will be required to assemble a "peer review panel" made up of "scientific experts in the appropriate disciplines with recent professional experience with the substance for which risk assessment and cost/benefit analysis is conducted".
In other words, only scientists employed by the regulated industries are likely to meet the selection criteria. These "peer review panels" will analyse the risk assessments done by the government and, if the peer review experts disagree with the government's experts, the proposed regulation will be shelved until government can convince the independent peer reviewers that the government's analysis is right.
Let's see, now: what is the probability that industry-employed peer reviewers will find reasons to disagree with the government's risk assessors and thus bring the regulatory process to a standstill? Hint: to answer that, you probably will do not need to do a risk assessment. Common sense will suffice.
[From Rachel's Environment & Health Weekly.]