< Prev Next >
NATO SOFTWARE ENGINEERING CONFERENCE 1968
67
6. Service
Kolence: If the users expect a supplier’s software to work exactly as and when predicted, they should in all fairness apply the same standards to the software that they develop themselves.
6.3.2. PERFORMANCE MONITORING
The major discussions on performance monitoring are reported in the sections of this report dealing with Design and with Production. The comments given below are of most immediate relevance to the question of monitoring the performance of an installed system.
Gillette: We have used performance monitoring principally as a maintenance tool to find bottlenecks and deficiencies.
Perlis: Delivered systems should have facilities for accumulating files of run time performance information, which could be shipped back to the manufacturer for analysis’ as well as being of value to the user installation. I do not know of anywhere this is being done as a standard operating procedure.
Kolence: We have experience of performance monitoring in a user environment. In such environments extra care must be taken to avoid interference with the running of the system, and in overloading the system. Information can be monitored in relation to the overall system performance or to particular user programs. Typical data that we produce concern core storage usage and I/O channel and device activity. Information on disk cylinder  usage can be very useful in learning how to reorganise data on the disk for improved transfer rates. We have found that almost all programs can be substantially improved (between 10 and 20 percent, say), without design changes, in one or two man-days.
Smith: Performance monitoring when first applied can certainly lead to spectacular improvements. However, this can lead one onto the treadmill of incremental improvements to a system which is basically unsound.
Gries: Optional performance monitors should be built into compilers, so as to be available to users.
Galler: Performance monitors should be part of the system, and not in individual compilers.
6.4. FEEDBACK TO MANUFACTURERS FROM USERS
Comments relevant to this topic occurred in entirely separate contexts, and included attempts to classify the various types of information that a user would want to feedback to a manufacturer and the possible means of so doing. See also Section 4.2.2.
Haller: There is feedback of different entities, on different paths, leading to three separate control loops with different time-lag:
1. On the correctness, or otherwise, of a system. This would go to the maintenance group; time to get a reply might be up to a week.
2. On system performance, to the production group, who might be expected to reply within, say, a month.
3. Requests for extra facilities would go to the design group and, if accepted, a year might be the expected delay before an appropriately modified system was released.
Hastings: A user needs a fast means of obtaining corrections to program bugs. We have used a terminal-oriented on-line system for our field engineers to obtain up-to-date information from the maintenance group. Some such system as this is the only substitute for the common means of unofficial feedback, possible when one knows the right person.
Babcock: (from Variations on software available to the user)
»Even with a company such as IBM — which incidentally, I am convinced, is the very best — the best was certainly none too good.
…

Time to me was money. Time-sharing is a revenue of the moment, once lost, it will not, nor cannot be captured. It took us approximately six months to learn the then current IBM service was archaic, and something must be