»Engineering advances and back-up faculties through duplex equipment have reduced the danger of hardware faults and increased user expectancy of reliability (especially in on-line real time installations) but programming has grown more complex and unreliable while software has not provided analogous back-up facilities.«
Harr: Time-shared systems need a level of reliability far beyond that which we have been getting from our batch-processing systems. This can be achieved only by a careful matching of hardware and software. The essence of the approach that we used to achieve high reliability in our Electronic Switching System was to plan on the basis that even when the system went live it would still have errors in it. Sure enough, it did.
d’Agapeyeff: I agree with this approach. You must design and implement your system so that it will continue to give service in the presence of both hardware and software errors.
Harr: It requires a very careful program design to ensure that your crucial system tables will not be mutilated except in the very rarest of types of error situation.
Bemer: One interesting question is how much the software people are prepared to pay for extra hardware to assist in maintaining data integrity. My guess would be perhaps 15-20 % of the normal hardware cost.
Harr: We definitely need better techniques for changing a program in the field while continuing to provide service.
d’Agapeyeff: In large on-line systems the cost of testing a change is almost 100 times as much as the cost of producing the change. We cannot afford to go through this process too often.
One final pair of quotations are given below, to indicate the need for quantifying our notions of reliability.
Opler: How many errors should we be prepared to accept in a system containing one million instructions?
Perlis: Seven [Laughter].
5.2. PRODUCTION — MANAGEMENT ASPECTS
5.2.1. PRODUCTION PLANNING
There was much discussion on the difficulties of planning the production of a system that involved a high research content.
Genuys: If the job has been done before, estimates are fairly easy. If the research content is high, estimates are difficult. The trouble is that it is not always possible to tell beforehand which jobs are which.
McClure: It’s research if it’s different. Even if it is just higher performance, it is still research.
An illustration of how the original estimate of the work involved in producing a particular piece of software gets more reliable as the group of people doing the work gains experience with that particular problem was provided by McClure (figure 7). This shows, for three Fortran compilers produced by the same group, both the original estimate of the work required and the work actually needed.
David: (from Some thoughts about production of large software systems (2))
»Computing has one property, unique I think, that seriously aggravates the uncertainties associated with software efforts. In computing, the research, development, and production phases are often telescoped into one process. In the competitive rush to make available the latest techniques, such as on-line consoles served by time-shared computers, we strive to take great forward leaps across gulfs of unknown width and depth. In the cold light of day, we know that a step-by-step approach separating research and development from production is less risky and more likely to be successful. Experience indeed indicates that for software tasks similar to previous ones, estimates are accurate to within 10–30% in many cases. This situation is familiar in all fields lacking a firm theoretical base.
Thus, there are good reasons why software tasks that include novel concepts involve not only uncalculated but also uncalculable risks.«