How expert is your expert system for performance management?

Diagnosing performance problems in computer systems is a complex task for which many expert systems have been developed. This paper studies the quality of the results produced by four techniques for diagnosing performance problems: threshold analysis, bottleneck analysis, what's-different analysis, and correlation analysis. All have strengths and weaknesses, and none of the techniques is preferred to the others in all cases. Further issues arise when diagnostic techniques are employed in practice. For example, the above technique were applied to a mainframe computer system that had a known performance problem; none of the techniques produced the correct diagnosis. These results suggest the need for a systematic study of diagnostic techniques. An approach to such a study is outlined in the conclusions of this paper.