Opler: The latest release of OS/360 was intended to introduce 16 changes (many complex), and to correct 1074 errors. The current policy is to have releases at 90 day intervals. This indicates a rate of over 11 corrections per day between versions. It is obviously better to batch improvements to a system together. On the other hand, because customers need errors to be corrected the release frequency cannot be too low. Once a year would be much too infrequent.
Hastings: Many of these 1000 errors are of course quite trivial, perhaps being just documentation errors.
d’Agapeyeff: (from Reducing the cost of software)
»The growth of complexity in software has led, understandably enough, to many issues and versions of the same system. But no way has been found of making past issues sub-sets of new issues. This is causing great disturbance to users and is also causing incompatibilities (e.g. on machine A, operating system level X is unlikely to be compatible with level Y to either programmers or operators).«
Babcock: One way to have a successful system is to never make a change in it. However, this is obviously impractical. Systems such as our Rush system never remain static, and are never 100 percent checked out. So system managers should go easy on installing new versions of on-line systems, and check them out in a dynamic environment.
Pinkerton: With less frequent releases there would be increased stability, and more opportunity for users to generate responsible feedback on system performance to the manufacturers.
Galler: OS/360 has had 16 releases in two and a half years. We should have frequent updates for corrections, but decrease the frequency of traumatic upheavals.
Gillette: (from Comments on service group statements)
»[Babcock’s use of the phrase ‘works well’] bothers me as it is largely qualitative. For an engineering group I think a metric description would be more palatable and more meaningful.
To be specific, below I have written a copy of one of the paragraphs which has been put into a Product Objectives document. We struggled a great deal to define measurable objectives in the document and this is an example. The numbers used do have relevance, historically, and that is all that need be said about them. Finally our objectives may not have been high enough in this particular area — we tried to push our luck while at the same time being realistic.
The total number of unique bugs reported for all releases in one year on ECS SCOPE will not be greater than the number given by the following formula: Number of bugs ≤ 500 - 45/(I + 10) where I is the number of installations using ECS SCOPE. 85 percent of the reported PSRs (see Notes below) will be corrected within 30 days and 50 percent of these will be corrected within 15 days. All PSRs will be corrected within 60 days.
Notes:
1. A PSR is the reporting form which a customer uses to report a bug.
2. A bug is effectively equivalent to a PSR — it may be a mistake in the code or a misunderstanding on the part of the customer.
3. Correcting the bug may result in modified code or clarifying a definition with correction to the reference manual.
4. The limiting function is bounded and increases with the number of users.«
6.1.4. RESPONSIBILITY FOR MODIFIED SYSTEMS
This discussion led to the subject of assigning responsibility for user-modified systems.
Babcock: In those areas that have not been affected by user modifications, software manufacturers should be responsible for all maintenance and improvements.
Paul: One cannot easily define the borderline between affected and unaffected parts.