History of IBM mainframe operating systems
The history of operating systems running on IBM mainframes is a notable chapter of history of mainframe operating systems, because of IBM's long-standing position as the world's largest hardware supplier of mainframe computers.
Arguably the operating systems which IBM supplied to customers for use on its early mainframes have seldom been very innovative, except for the virtual machine systems beginning with CP-67. But the company's well-known reputation for preferring proven technology has generally given potential users the confidence to adopt new IBM systems fairly quickly. IBM's current mainframe operating systems, z/OS, z/VM, z/VSE, and z/TPF, are backward compatible successors to operating systems introduced in the 1960s, although of course they have been improved in many ways.
Both IBM-supplied operating systems and those supplied by others are discussed here if notably used on IBM mainframes.
Before System/360
IBM was slow to introduce operating systems: General Motors produced General Motors OS in 1955 and GM-NAA I/O in 1956 for use on its own IBM computers; and in 1962 Burroughs Corporation released MCP and General Electric introduced GECOS, in both cases for use by their customers.In fact the first operating systems for IBM computers were written by IBM customers who did not wish to have their very expensive machines sitting idle while operators set up jobs manually, and so they wanted a mechanism for maintaining a queue of jobs.
The operating systems described below ran only on a few processor models and were suitable only for scientific and engineering calculations. Users with other IBM computers or other applications had to manage without operating systems. But one of IBM's smaller computers, the IBM 650, introduced a feature which later became part of OS/360: if processing was interrupted by a "random processing error", the machine could automatically resume from the last checkpoint instead of requiring the operators to restart the job manually from the beginning.
From General Motors' GM-NAA I/O to IBSYS
' Research division produced GM-NAA I/O for its IBM 701 in 1956, and updated it for the 701's successor. In 1960 the IBM user association SHARE took it over and produced an updated version, SHARE Operating System.Finally IBM took over the project and supplied an enhanced version called IBSYS with the IBM 7090 and IBM 7094 computers. IBSYS required 8 tape drives. Its main components were: a card-based Job Control language, which was the main user interface; compilers for FORTRAN and COBOL; an assembler; and various utilities including a sort program.
In 1958, the University of Michigan Executive System adapted GM-NAA I/O to produce UMES, which was better suited to the large number of small jobs created by students. UMES was used until 1967 when it was replaced by the MTS timesharing system.
BESYS
Bell Labs produced BESYS and used it until the mid-1960s. Bell also made it available to others without charge or formal technical support.FORTRAN Monitor System
Before IBSYS, IBM produced for its IBM 709, 7090 and 7094 computers a tape-based operating system whose sole purpose was to compile FORTRAN programs—in fact FMS and the FORTRAN compiler were on the same tape.Early time-sharing and virtual machine systems
's Fernando Corbató produced the first experimental time-sharing systems, such as CTSS, from 1957 to the early 1960s, using slightly modified IBM 704 and IBM 7090 mainframes; these systems were based on a proposal by John McCarthy. In the 1960s IBM's own laboratories created experimental time-sharing systems, using standard mainframes with hardware and microcode modifications to support virtual memory: IBM M44/44X in the early 1960s; CP-40 from 1964 to 1967; CP-67 from 1967 to 1972. The company even released CP-67 without warranty or technical support to several large customers from 1968 to 1972. CP-40 and CP-67 used modified System/360 CPUs, but the M44/44X was based on the IBM 7044, an earlier generation of CPU which was very different internally.These experimental systems were too late to be incorporated into the System/360 series which IBM announced in 1964 but encouraged the company to add virtual memory and virtual machine capabilities to its System/370 mainframes and their operating systems in 1972:
- The M44/44X showed that a partial approach to virtual machines was not good enough and that thrashing could severely reduce the speed of virtual memory systems. Thrashing is a condition in which the system runs very slowly because it spends a lot of its time shuffling virtual memory pages between physical memory and disk files.
- IBM learned from CP-40 and CP-67: how to make the thrashing problem manageable; that its other virtual memory and virtual machine technologies were sufficiently fast and reliable to be used in the high-volume commercial systems which were its core business. In particular, IBM's David Sayre convinced the company that automated virtual memory management could consistently perform at least as well as the best programmer-designed overlay schemes.
Universities produced three other S/360 time-sharing operating systems in the late 1960s:
- The Michigan Terminal System was developed in 1967 by a consortium of universities led by the University of Michigan. All versions ran on IBM's mainframes which had virtual memory capability, starting with the S/360-67. MTS remained in use until 1999.
- McGill University in Montreal started developing MUSIC in 1969. MUSIC was enhanced several times and eventually supported text searches, web publishing and email as well as software development. MUSIC was marketed by IBM mainly to educational institutions as a cost-effective operating system for its hardware, and eventually became an IBM Systems Product in 1985. The last official version was released in 1999.
- ORVYL and WYLBUR were developed by Stanford University in 1967-68 for the IBM S/360-67. They provided some of the first time-sharing capabilities on IBM S/360 computers.
System/360 operating systems
IBM originally intended that System/360 should have only one batch-oriented operating system, OS/360. There are at least two accounts of why IBM later decided it should also produce a simpler batch-oriented operating system, DOS/360:
- because it found that OS/360 would not fit into the limited memory available on the smaller System/360 models;
- or because it realized that the development of OS/360 would take much longer than expected, and introduced DOS/360 as one of a series of stop-gaps to prevent System/360 hardware sales from collapsing the others were BOS/360 and TOS/360.
- They had to support multiprogramming switching to run another in-progress application when the current application was blocked waiting for I/O operations to complete. Without multiprogramming, the faster CPUs in the range would have spent most of their time idle, waiting for slow I/O operations. Hence, the operating systems had to be the real masters of the systems, to provide whatever services the applications validly requested, and to handle crashes or misbehavior in one application without stopping others that were running at the same time.
- They had to support a much wider range of machine sizes. Memory ranged from 16 KB to 1 MB and processor speeds from a few thousand instructions per second to 500,000.
- They had to support a wide range of application requirements. For example, some applications only needed to read through sequential files from start to finish; others needed fast, direct access to specific records in very large files; and a few applications spent nearly all their time doing calculations, with very little reading or writing of files.
IBM's competitors took advantage of the delays in OS/360 and the System/360 to announce systems aimed at what they thought were the most vulnerable parts of IBM's market. To prevent sales of System/360 from collapsing, IBM released four stop-gap operating systems:
- Basic Operating System/360, which loaded from a disk drive or tape drive and supported tape drives and a few disks. This system was supplied to beta test customers and may have been an early version of DOS/360.
- TOS/360, which was designed to provide an upgrade path for customers who had IBM 1401 computers with tape drives and no disks.
- DOS/360, which was built by the developers of BOS/360 and TOS/360 and went on to become a mainstream operating system whose descendant z/VSE is still widely used.
- Operating System/360 with only the Primary Control Program option, which didn't support multiprogramming.
The traumas of producing the System/360 operating systems gave a boost to the emerging discipline of software engineering, the attempt to apply scientific principles to the development of software and the management of software projects. Frederick P. Brooks, who was a senior project manager for the whole System/360 project and then was given specific responsibility for OS/360, wrote an acclaimed book, The Mythical Man-Month, based on the problems encountered and lessons learned during the project, two of which were:
- Throwing additional resources at a struggling project quickly becomes unproductive or even counter-productive because of communication difficulties. This is the "Mythical Man-Month" syndrome which gave the book its title.
- The successor to a successful system often runs into difficulties because it gets overloaded with all the features people wished had been in the earlier system. Brooks called this the "second-system effect", and cited OS/360 as a very comprehensive example.
DOS/360
- Sequential data sets were normally read one record at a time from beginning to end.
- In indexed files a specified section of each record was defined as a key which could be used to look up specific records.
- In direct access files, the application program had to specify the physical location on the disk of the data it wanted to access. BDAM programming was not easy and most customers never used it themselves; but it was the fastest way to access data on disks and many software companies used it in their products, especially database management systems such as ADABAS, IDMS and IBM's DL/I.
DOS/360 also offered BTAM, a data communications facility which was primitive and hard to use by today's standards. But BTAM could communicate with almost any type of terminal, which was a big advantage at a time when there was hardly any standardization of communications protocols.
But DOS/360 had significant limitations compared with OS/360, which was used to control most larger System/360 machines:
- The first version could run only one program at a time. A later enhancement allowed 3 at the same time, in one of 3 "partitions" whose size was set by each customer when DOS/360 was installed.
- The JCL it used for submitting jobs was designed to be easy for the low-end machines to process, and as a result programmers did not find it easy to read or write.
- There was no spooling sub-system to improve the efficiency of punched card and printer use. In the late 1960s an independent software company started selling a spooler called GRASP.
- DOS/360 had no relocating loader, so users had to link edit a separate executable version of each program for each partition in which the program was likely to be run.
- Executable programs were stored in the Core Image Library, which did not reclaim space when programs were deleted or replaced by newer versions. When the Core Image Library became full, it had to be compressed by one of the utility programs, and this could halt development work for as much as half a day.
- Its application programming interface was different from that of OS/360. DOS/360 programs written in high level languages such as COBOL needed small modifications before they could be used with OS/360 and assembly language programs needed larger changes.
- System/360 hardware sold very well
- Over 90% of the 360 systems sold were Models 20, 30 & 40
- Most of these cheaper models had far less core memory than required by OS/360.
OS/360
included multiple levels of support, a single API and much shared code. PCP was a stop-gap version which could run only one program at a time, but MFT and MVT were used until at least the late 1970s, a good five years after their successors had been launched. It is unclear whether the divisions among PCP, MFT and MVT arose because MVT required too much memory to be usable on mid-range machines or because IBM needed to release a multiprogramming version of OS as soon as possible.PCP, MFT and MVT had different approaches to managing memory, but provided very similar facilities:
- The same application programming interface, so application programs could be transferred among PCP, MFT and MVT without even needing re-compilation.
- The same JCL, which was more flexible and easier to use than that of DOS/360.
- The same facilities as DOS/360 for reading and writing files and for data communications.
- An additional file structure, partitioned, and access method, which was mainly used for managing program libraries. Although partitioned files needed to be compressed to reclaim free space, this seldom halted development work as it did with DOS/360's Core Image Library, because PCP, MFT and MVT allowed an indefinite number of partitioned files and each project generally had at least one.
- A file naming system which allowed files to be managed as hierarchies, e.g. PROJECT.USER.FILENAME.
- A spooling facility.
- Applications could create sub-tasks, which allowed multiprogramming within the one job.
MFT
When installing MFT, customers would specify up to four "partitions", areas of memory with fixed boundaries, in which application programs could be run simultaneously. MFT Version II raised the limit to 52.MVT
MVT was considerably larger and more complex than MFT and therefore was used on the most powerful System/360 CPUs. It treated all memory not used by the operating system as a single pool from which contiguous "regions" could be allocated as required by an indefinite number of simultaneous application programs. This scheme was more flexible than MFT's and in principle used memory more efficiently, but was liable to fragmentation after a while one could find that, although there was enough spare memory in total to run a program, it was divided into separate chunks none of which was large enough.In 1971 the Time Sharing Option for use with MVT was added. TSO became widely used for program development because it provided: an editor; the ability to submit batch jobs, be notified of their completion and view the results without waiting for printed reports; debuggers for some of the programming languages used on System/360. TSO communicated with terminals by using TCAM, which eventually replaced the earlier Queued Telecommunications Access Method. TCAM's name suggests that IBM hoped it would become the standard access method for data communications, but in fact TCAM was used almost entirely for TSO and was largely superseded by VTAM from the late 1970s onwards.
TP monitors
System/360's hardware and operating systems were designed for processing batch jobs which in extreme cases might run for hours. As a result, they were unsuitable for transaction processing, in which there are thousands of units of work per day and each takes between 30 seconds and a very few minutes. In 1968 IBM released IMS to handle transaction processing, and in 1969 it released CICS, a simpler transaction processing system which a group of IBM's staff had developed for a customer. IMS was only available for OS/360 and its successors, but CICS was also available for DOS/360 and its successors. For many years this type of product was known as a "TP monitor". Strictly speaking TP monitors were not operating system components but application programs which managed other application programs. In the 1970s and 1980s several third-party TP monitors competed with CICS, but IBM gradually improved CICS to the point where most customers abandoned the alternatives.Special systems for airlines
In the 1950s airlines were expanding rapidly but this growth was held back by the difficulty of handling thousands of bookings manually. In 1957 IBM signed a development contract with American Airlines for the development of a computerized reservations system, which became known as SABRE. The first experimental system went live in 1960 and the system took over all booking functions in 1964 in both cases using IBM 7090 mainframes. In the early 1960s IBM undertook similar projects for other airlines, and soon decided to produce a single standard booking system, PARS, to run on System/360 computers.In SABRE and early versions of PARS there was no separation between the application and operating system components of the software, but in 1968 IBM divided it into PARS and ACP. Later versions of ACP were named ACP / TPF and then TPF as non-airline businesses adopted this operating system for handling large volumes of online transactions. The latest version is z/TPF.
IBM developed ACP and its successors because: in the mid-1960s IBM's standard operating systems were batch-oriented and could not handle large numbers of short transactions quickly enough; even its transaction monitors IMS and CICS, which run under the control of standard general-purpose operating systems, are not fast enough for handling reservations on hundreds of flights from thousands of travel agents.
The last "public domain" version of ACP, hence its last "free" version, was ACP 9.2, which was distributed on a single mini-reel with an accompanying manual set and which could be restored to IBM 3340 disk drives and which would, thereby, provide a fully functional ACP system.
ACP 9.2 was intended, primarily, for bank card and other "financial" applications, but it could also be utilized for airlines reservation systems, too, as by this time ACP had become a more general-purpose OS.
Indeed, ACP had by then incorporated a "hypervisor" module which supported a virtual OS... usually VS1, but possibly also VS2... as a "guest", with which program development or file maintenance could be accomplished concurrently with the on-line functions.
In some instances, production work was run under VS2 under the hypervisor, including, possibly, IMS DB.
System/360 Model 20
The Model 20 was labeled as part of the System/360 range because it could be connected to some of the same peripherals, but it was a 16-bit machine and not entirely program-compatible with other members of the System/360 range. Three operating systems were developed by IBM's labs in Germany, for different 360/20 configurations; DPS—with disks ; TPS—no disk but with tapes ; and CPS—punched-card-based. These had no direct successors since IBM introduced the System/3 range of small business computers in 1969 and System/3 had a different internal design from the 360/20 and different peripherals from IBM's mainframes.System/360 Model 44
This was another processor which used the System/360 peripherals but had a different internal design. The 360/44 was designed for scientific computation using floating point numbers, such as geological or meteorological analyses. Because of the internal differences and the specialized type of work for which it was designed, the 360/44 had its own operating system, PS/44. An emulator for missing System/360 instructions allowed the Model 44 to run OS/360. The 360/44 and PS/44 had no direct successors.System/370 and virtual memory operating systems
When System/370 was announced in 1970 it offered essentially the same facilities as System/360 but with about 4 times the processor speeds of similarly-priced System/360 CPUs. Then in 1972 IBM announced "System/370 Advanced Functions", of which the main item was that future sales of System/370 would include virtual memory capability and this could also be retro-fitted to existing System/370 CPUs. Hence IBM also committed to delivering enhanced operating systems which could support the use of virtual memory.Most of the new operating systems were distinguished from their predecessors by the presence of "/VS" in their names. "VS" stands for "Virtual Storage" IBM avoided the term "virtual memory", allegedly because the word "memory" might be interpreted to imply that IBM computers could forget things.
All of today's IBM mainframe operating systems except z/TPF are descendants of those included in the "System/370 Advanced Functions" announcement z/TPF is a descendant of ACP, the system which IBM initially developed to support high-volume airline reservations applications.
DOS/VS
was the successor to DOS/360, and offered similar facilities, with the addition of virtual memory. In addition to virtual memory DOS/VS provided other enhancements:- Five memory partitions instead of three. Later releases increased this to seven.
- A relocating loader, so that it was no longer necessary to link-edit a separate copy of each program for each partition in which it was to run.
- An improved spooling component, POWER/VS.
OS/VS1
was the successor to MFT, and offered similar facilities, with the addition of virtual memory. IBM released fairly minor enhancements of OS/VS1 until 1983, and in 1984 announced that there would be no more. OS/VS1 and TSS/370 are the only IBM System/370 operating systems that do not have modern descendants.The Special Real Time Operating System, Programming RPQ Z06751, was a variant of OS/VS1 extended to support real-time computing. It was targeted at such industries as electric utility energy management and oil refinery applications.
OS/VS2 and MVS
OS/VS2 Release 1 was a replacement for MVT with virtual memory; while there were many changes it retained the overall structure. But in 1974 IBM released what it described as OS/VS2 Release 2 but which was a major rewrite that was upwards-compatible with the earlier OS/VS2 SVS. The new system's most noticeable feature was that it supported multiple virtual address spaces different applications thought they were using the same range of virtual addresses, but the new system's virtual memory facilities mapped these to different ranges of real memory addresses. As a result, the new system rapidly became known as "MVS", the original OS/VS2 became known as "SVS". IBM itself accepted this terminology and labelled MVS's successors "MVS/...".The other distinctive features of MVS were: its main catalog had to be a VSAM catalog; it supported "tightly-coupled multiprocessing" ; it included a System Resources Manager which allowed users to load additional work on to the system without reducing the performance of high-priority jobs.
IBM has released several MVS upgrades: MVS/SE, MVS/SP Version 1, MVS/XA, MVS/ESA, OS/390 and currently z/OS.
VM/370
combined a virtual machine facility with a single-user system called Conversational Monitor System ; this combination provided time-sharing by allowing each user to run a copy of CMS on his / her own virtual machine. This combination was a direct descendant of CP/CMS. The virtual machine facility was often used for testing new software while normal production work continued on another virtual machine, and the CMS timesharing system was widely used for program development.VM/370 was followed by a series of upgrades: VM/SEPP, VM/BSEPP, VM/SP, VM/SP HPO, VM/XA MA, VM/XA SF, VM/XA SP, VM/ESA, and z/VM. IBM also produced optional microcode assists for VM and successors, to speed up the hypervisor's emulation of privileged instructions on behalf of "guest" operating systems. As part of 370/Extended Architecture, IBM added the Start Interpretive Execution instruction to allow a further speedup of the CP hypervisor.
Technical notes
Time-sharing
Time-sharing is based on the idea that computers are much faster than humans, so while one human user is reading what a computer has just displayed on a screen the computer can do some useful work for another user. Large time-sharing systems can have hundreds or even thousands of simultaneous users, and the memory required by their programs and data generally adds up to much more than the physical memory attached to the computer. Time-sharing systems solve this problem by various combinations of:- virtual memory, described below.
- swapping: when the OS is waiting for a response from one user, a time slice has ended or the OS is trying to free up real storage, it may save a user's programs and data on a disk or drum and read them back into memory when the user sends a response, resources free up or another user is swapped out due to time slice end. Swapping does not require virtual memory; it was implemented on OS/360 without virtual memory. It transfers all of a user's programs and data between memory and disk/drum and is mainly driven by the user's responses to information displayed by the system.
Virtual memory
The main components of an IBM virtual memory system are:
- Virtual memory, consisting of all memory addresses accessible by the CPU hardware. Virtual memory is an abstraction, so systems can easily have more virtual than real memory.
- Pages, fixed-size blocks into which all virtual memory is divided. Most IBM operating systems use 4 KB pages, although some older systems ran quite well with 2 KB pages. Newer IBM System z systems also support 1 MB large pages in addition to the normal 4 KB pages.
- Real memory, random-access memory attached to the computing system.
- Page frames, realized by dividing all real memory into pieces equal to the system's page size. Virtual-memory pages must be placed into real-memory page frames before they can be used by the CPU and I/O channels.
- Page Tables track the location of every virtual-memory page, whether in a real-memory page frame or on disk, in a paging file. Critical to memory management, Page Table entries also record the last time each page was accessed.
- Dynamic Address Translation hardware is integrated into the CPU itself and participates in every memory reference. If the Page Table shows the page in a real-memory page frame, DAT translates the virtual address to a real one and allows the memory access to complete. If, on the other hand, the referenced page is not in real memory, the DAT hardware generates an interrupt which calls the Paging Supervisor into action.
- The Paging Supervisor manages all memory, both real and virtual, moving pages between real memory and disk as needed, keeping the Page Table updated, servicing memory allocation requests, and cleaning up after itself. As the load on the system increases, a page can be referenced when all page frames are in use. When this happens, the paging supervisor typically identifies the page that has not been read or written for the longest interval of time, copies the page to the paging file, updates the Page Table, and uses the newly available page frame to satisfy the memory request.
Virtual machine
Virtual machine techniques enable several operating systems or other software to run on the same computer so that each thinks it has a whole computer to itself, and each of these simulated whole computers is called a "virtual machine". The operating system which really controls the computer is usually called a hypervisor. Two of the major components of the hypervisor are:- Virtual memory management. Each virtual machine appears to have a complete range of addresses from 0 to some large number, and virtual memory techniques prevent different virtual machines from confusing each other.
- Simulating "privileged" functions on behalf of the "guest" operating systems. "Privileged" functions are those which enable programs to take over all or at least large parts of the computer, and usually operating systems immediately terminate any other program which tries to use them. But "guest" operating systems think they are entitled to use these functions, so the hypervisor detects their attempts to do so and runs the privileged functions on their behalf, using virtual memory techniques to prevent them from corrupting memory areas used by other "guest" operating systems.