Pick operating system
The Pick operating system is a demand-paged, multiuser, virtual memory, time-sharing computer operating system based around a unique MultiValue database. Pick is used primarily for business data processing. It is named after one of its developers, Richard A. Pick.
The term "Pick system" has also come to be used as the general name of all operating environments which employ this multivalued database and have some implementation of Pick/BASIC and ENGLISH/Access queries. Although Pick started on a variety of minicomputers, the system and its various implementations eventually spread to a large assortment of microcomputers, personal computers and mainframe computers.
Overview
The Pick operating system consists of a database, dictionary, query language, procedural language, peripheral management, multi-user management and a compiled BASIC Programming language.The database is a 'hash-file' data management system. A hash-file system is a collection of dynamic associative arrays which are organized altogether and linked and controlled using associative files as a database management system. Being hash-file oriented, Pick provides efficiency in data access time. Originally, all data structures in Pick were hash-files meaning records are stored as associated couplets of a primary key to a set of values. Today a Pick system can also natively access host files in Windows or Unix in any format.
A Pick database is divided into one or more accounts, master dictionaries, dictionaries, files and sub-files, each of which is a hash-table oriented file. These files contain records made up of fields, sub-fields and sub-sub-fields. In Pick, records are called items, fields are called attributes, and sub-fields are called values or sub-values. All elements are variable-length, with field and values marked off by special delimiters, so that any file, record, or field may contain any number of entries of the lower level of entity. As a result, a Pick item can be one complete entity, or is like a file on most conventional systems. Entities that are stored as 'files' in other common-place systems must be stored as records within files on Pick.
The file hierarchy is roughly equivalent to the common Unix-like hierarchy of directories, sub-directories, and files. The master dictionary is similar to a directory in that it stores pointers to other dictionaries, files and executable programs. The master dictionary also contains the command-line language.
All files are organized identically, as are all records. This uniformity is exploited throughout the system, both by system functions, and by the system administration commands. For example, the 'find' command will find and report the occurrence of a word or phrase in a file, and can operate on any account, dictionary, file or sub-file.
Each record must have a unique, primary key which determines where in a file that record is stored. To retrieve a record, its key is hashed and the resultant value specifies which of a set of discrete "buckets" to look in for the record. Therefore, most records can be read using one single disk-read operation. This same method is used to write the record back to its correct "bucket".
In its initial implementation, Pick records were limited to 32 KB in total, although this limit was removed in the 1980s. Files can contain an unlimited number of records, but retrieval efficiency is determined by the number of records relative to the number of buckets allocated to the file. Each file may be initially allocated as many buckets as required, although changing this extent later may require the file to be quiescent. All modern multi-value databases have a special file-type which changes extent dynamically as the file is used. These use a technique called linear hashing, whose cost is proportional to the change in file size, not the file size itself. All files start as a contiguous group of disk pages, and grow by linking additional "overflow" pages from unused disk space.
Initial Pick implementations had no index structures as they were not deemed necessary. Around 1990, a B-tree indexing feature was added. This feature makes secondary key look-ups operate much like keyed inquiries of any other database system: requiring at least two disk reads.
Pick data files are usually two levels. The first level is known as the "dictionary" level and is mandatory. It contains:
- Dictionary itemsthe optional items that serve as definitions for the names and structure of the items in the data fork, used in reporting
- The data-level identifiera pointer to the second or "data" level of the file
A Pick database has no data typing, since all data is stored as characters, including numbers. Data integrity, rather than being controlled by the system, is controlled by the applications and the discipline of the programmers. Because a logical document in Pick is not fragmented, intra-record integrity is automatic.
In contrast to many SQL database systems, Pick allows for multiple, pre-computed field aliases. For example, a date field may have an alias definition for the format "12 Oct 1999", and another alias formatting that same date field as "10/12/99". File cross-connects or joins are handled as a synonym definition of the foreign key. A customer's data, such as name and address, are "joined" from the customer file into the invoice file via a synonym definition of "customer number" in the "invoice" dictionary.
Pick record structure favors a non-first-normal-form composition, where all of the data for an entity is stored in a single record, obviating the need to perform joins. Managing large, sparse data sets in this way can result in efficient use of storage space. This is why these databases are sometimes called NF2 or NF-squared databases.
History
Pick was originally implemented as the Generalized Information Retrieval Language System on an IBM System/360 in 1965 by Don Nelson and Richard Pick at TRW, whose government contract for the Cheyenne Helicopter project required developing a database. It was supposed to be used by the U.S. Army to control the inventory of Cheyenne helicopter parts.Pick was subsequently commercially released in 1973 by Microdata Corporation as the Reality Operating System now supplied by Northgate Information Solutions. McDonnell Douglas bought Microdata in 1981.
Originally on the Microdata implementation, and subsequently implemented on all Pick systems, a BASIC language called Data/BASIC with numerous syntax extensions for smart terminal interface and database operations was the primary programming language for applications. A PROC procedure language was provided for executing scripts. A SQL-style language called ENGLISH allowed database retrieval and reporting, but not updates. ENGLISH did not fully allow manipulating the 3-dimensional multivalued structure of data records. Nor did it directly provide common relational capabilities such as joins. This was because powerful data dictionary redefinitions for a field allowed joins via the execution of a calculated lookup in another file. The system included a spooler. A simple text editor for file-system records was provided, but the editor was only suitable for system maintenance, and could not lock records, so most applications were written with the other tools such as Batch, RPL, or the BASIC language so as to ensure data validation and allow record locking.
By the early 1980s observers saw the Pick operating system as a strong competitor to Unix. BYTE in 1984 stated that "Pick is simple and powerful, and it seems to be efficient and reliable, too... because it works well as a multiuser system, it's probably the most cost-effective way to use an XT". Dick Pick founded Pick & Associates, later renamed Pick Systems, then Raining Data and TigerLogic and more recently Rocket Software. He licensed "Pick" to a large variety of manufacturers and vendors who have produced different "flavors" of Pick. The database flavors sold by TigerLogic were D3, mvBase, and mvEnterprise. Those previously sold by IBM under the "U2" umbrella are known as UniData and UniVerse. Rocket Software purchased IBM's U2 family of products in 2010 and TigerLogic's D3 and mvBase family of products in 2014.
Dick Pick died of stroke complications in October 1994.
Pick Systems often became tangled in licensing litigation, and devoted relatively little effort to marketing and improving its software. Subsequent ports of Pick to other platforms generally offered the same tools and capabilities for many years, usually with relatively minor improvements and simply renamed. Licensees often developed proprietary variations and enhancements.
Derivative and related products
The Pick database was licensed to roughly three dozen licensees between 1978 and 1984. Application-compatible implementations evolved into derivatives and also inspired similar systems.- Reality - The first implementation of the Pick database was on a Microdata platform using firmware and called Reality. The first commercial release was in 1973. Microdata acquired CMC Ltd. in the early 80s and were based in Hemel Hempstead, England. The Microdata implementations ran in firmware, so each upgrade had to be accompanied by a new configuration chip. Microdata itself was eventually bought by McDonnell-Douglas Information Systems. Pick and Microdata sued each other for the right to market the database, the final judgment being that they both had the right. In addition to the RealitySequoia and Pegasus] series of computers, Microdata and CMC Ltd. sold the Sequel series which was a much larger class able to handle over 1000 simultaneous users. The earlier Reality minicomputers were known to handle well over 200 simultaneous users, although things got pretty slow and it was above the official limit. Pegasus systems superseded Sequoia and could handle even more simultaneous users than its predecessors. The modern version of this original Pick implementation is owned and distributed by .
- Ultimate - The second implementation of the Pick database was developed in about 1978 by a New Jersey company called The Ultimate Corp, run by Ted Sabarese. Like the earlier Microdata port, this was a firmware implementation, with the Pick instruction set in firmware and the monitor in assembly code on a Honeywell Level 6 machine. The system had dual personalities in that the monitor/kernel functions were executed by the native Honeywell Level 6 instruction set. When the monitor "select next user" for activation control was passed to the Honeywell WCS to execute Pick assembler code for the selected process. When the user's time slice expired control was passed back to the kernel running the native Level 6 instruction set.
Over the years, many important and widely used applications have been written using Pick or one of the derivative implementations. In general, the end users of these applications are unaware of the underlying Pick implementation.
Criticisms and comparisons
Run-time environment
Native Pick did not require an underlying operating system to run. This later changed with later implementations when Pick required a host OS. While the host OS provided access to hardware resources, Pick had internal processes for memory management. Object-oriented Caché addressed some of these problems.Networking in mvBase was not possible without an accompanying application running in the host OS that could manage network connections via TCP ports and relay them to Pick internal networking.
Credentials and security
Individual user accounts must be created within the Pick OS, and cannot be tied to an external source.User passwords are stored within the Pick OS as an encrypted value. The encrypted password can be "cracked" via brute force methods, but requires system access and Pick programming skills as part of the attack vector.
The Rocket D3 implementation supports SSL file encryption.
Expertise and support
Companies looking to hire developers and support personnel for MultiValue-based systems recognize that although developers typically do not learn the environment in college and university courses, developers can be productive quickly with some mentoring and training. Due to the efficient design and nature of the programming language, the learning curve is generally considered low. Pick products such as D3, UniVerse, UniData, jBASE, Revelation, MVON, Caché, OpenQM, and Reality are still supported globally via well established distribution channels and resellers. The mvdbms Google Group is a useful place to start when looking for resources.MUMPS
The Pick OS invites comparison with MUMPS, which evolved into Caché. Similarities include:- Both systems are built on the efficient implementation of large, sparse, string-indexed arrays;
- Both historically commingled the language and the OS;
- Both have a similar domain of applicability.