High availability
High availability is a characteristic of a system which aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period.
Modernization has resulted in an increased reliance on these systems. For example, hospitals and data centers require high availability of their systems to perform routine daily activities. Availability refers to the ability of the user community to obtain a service or good, access the system, whether to submit new work, update or alter existing work, or collect the results of previous work. If a user cannot access the system, it is – from the users point of view – unavailable. Generally, the term downtime is used to refer to periods when a system is unavailable.
Principles
There are three principles of systems design in reliability engineering which can help achieve high availability.- Elimination of single points of failure. This means adding redundancy to the system so that failure of a component does not mean failure of the entire system.
- Reliable crossover. In redundant systems, the crossover point itself tends to become a single point of failure. Reliable systems must provide for reliable crossover.
- Detection of failures as they occur. If the two principles above are observed, then a user may never see a failure – but the maintenance activity must.
Scheduled and unscheduled downtime
If users can be warned away from scheduled downtimes, then the distinction is useful. But if the requirement is for true high availability, then downtime is downtime whether or not it is scheduled.
Many computing sites exclude scheduled downtime from availability calculations, assuming that it has little or no impact upon the computing user community. By doing this, they can claim to have phenomenally high availability, which might give the illusion of continuous availability. Systems that exhibit truly continuous availability are comparatively rare and higher priced, and most have carefully implemented specialty designs that eliminate any single point of failure and allow online hardware, network, operating system, middleware, and application upgrades, patches, and replacements. For certain systems, scheduled downtime does not matter, for example system downtime at an office building after everybody has gone home for the night.
Percentage calculation
Availability is usually expressed as a percentage of uptime in a given year. The following table shows the downtime that will be allowed for a particular percentage of availability, presuming that the system is required to operate continuously. Service level agreements often refer to monthly downtime or availability in order to calculate service credits to match monthly billing cycles. The following table shows the translation from a given availability percentage to the corresponding amount of time a system would be unavailable.| Availability % | Downtime per year | Downtime per month | Downtime per week | Downtime per day |
| 90% | 36.53 days | 73.05 hours | 16.80 hours | 2.40 hours |
| 95% | 18.26 days | 36.53 hours | 8.40 hours | 1.20 hours |
| 97% | 10.96 days | 21.92 hours | 5.04 hours | 43.20 minutes |
| 98% | 7.31 days | 14.61 hours | 3.36 hours | 28.80 minutes |
| 99% | 3.65 days | 7.31 hours | 1.68 hours | 14.40 minutes |
| 99.5% | 1.83 days | 3.65 hours | 50.40 minutes | 7.20 minutes |
| 99.8% | 17.53 hours | 87.66 minutes | 20.16 minutes | 2.88 minutes |
| 99.9% | 8.77 hours | 43.83 minutes | 10.08 minutes | 1.44 minutes |
| 99.95% | 4.38 hours | 21.92 minutes | 5.04 minutes | 43.20 seconds |
| 99.99% | 52.60 minutes | 4.38 minutes | 1.01 minutes | 8.64 seconds |
| 99.995% | 26.30 minutes | 2.19 minutes | 30.24 seconds | 4.32 seconds |
| 99.999% | 5.26 minutes | 26.30 seconds | 6.05 seconds | 864.00 milliseconds |
| 99.9999% | 31.56 seconds | 2.63 seconds | 604.80 milliseconds | 86.40 milliseconds |
| 99.99999% | 3.16 seconds | 262.98 milliseconds | 60.48 milliseconds | 8.64 milliseconds |
| 99.999999% | 315.58 milliseconds | 26.30 milliseconds | 6.05 milliseconds | 864.00 microseconds |
| 99.9999999% | 31.56 milliseconds | 2.63 milliseconds | 604.80 microseconds | 86.40 microseconds |
Uptime and availability can be used synonymously as long as the items being discussed are kept consistent. That is, a system can be up, but its services are not available, as in the case of a network outage. This can also be viewed as a system that is available to be worked on, but its services are not up from a functional perspective. The perspective is important here - whether the item being discussed is the server hardware, server OS, functional service, software service/process...etc. Keep the perspective consistent throughout a discussion, then uptime and availability can be used synonymously.
"Nines"
Percentages of a particular order of magnitude are sometimes referred to by the number of nines or "class of nines" in the digits. For example, electricity that is delivered without interruptions 99.999% of the time would have 5 nines reliability, or class five. In particular, the term is used in connection with mainframes or enterprise computing, often as part of a service-level agreement.Similarly, percentages ending in a 5 have conventional names, traditionally the number of nines, then "five", so 99.95% is "three nines five", abbreviated 3N5. This is casually referred to as "three and a half nines", but this is incorrect: a 5 is only a factor of 2, while a 9 is a factor of 10, so a 5 is 0.3 nines : 99.95% availability is 3.3 nines, not 3.5 nines. More simply, going from 99.9% availability to 99.95% availability is a factor of 2, but going from 99.95% to 99.99% availability is a factor of 5, over twice as much.
A formulation of the class of 9s based on a system's unavailability would be
.
A similar measurement is sometimes used to describe the purity of substances.
In general, the number of nines is not often used by a network engineer when modeling and measuring availability because it is hard to apply in formula. More often, the unavailability expressed as a probability, or a downtime per year is quoted. Availability specified as a number of nines is often seen in marketing documents. The use of the "nines" has been called into question, since it does not appropriately reflect that the impact of unavailability varies with its time of occurrence. For large amounts of 9s, the "unavailability" index is easier to handle. For example, this is why an "unavailability" rather than availability metric is used in hard disk or data link bit error rates.
Measurement and interpretation
Availability measurement is subject to some degree of interpretation. A system that has been up for 365 days in a non-leap year might have been eclipsed by a network failure that lasted for 9 hours during a peak usage period; the user community will see the system as unavailable, whereas the system administrator will claim 100% uptime. However, given the true definition of availability, the system will be approximately 99.9% available, or three nines. Also, systems experiencing performance problems are often deemed partially or entirely unavailable by users, even when the systems are continuing to function. Similarly, unavailability of select application functions might go unnoticed by administrators yet be devastating to users — a true availability measure is holistic.Availability must be measured to be determined, ideally with comprehensive monitoring tools that are themselves highly available. If there is a lack of instrumentation, systems supporting high volume transaction processing throughout the day and night, such as credit card processing systems or telephone switches, are often inherently better monitored, at least by the users themselves, than systems which experience periodic lulls in demand.
An alternative metric is mean time between failures.
Closely related concepts
Recovery time, also known as recovery time objective is closely related to availability, that is the total time required for a planned outage or the time required to fully recover from an unplanned outage. Another metric is mean time to recovery. Recovery time could be infinite with certain system designs and failures, i.e. full recovery is impossible. One such example is a fire or flood that destroys a data center and its systems when there is no secondary disaster recovery data center.Another related concept is data availability, that is the degree to which databases and other information storage systems faithfully record and report system transactions. Information management often focuses separately on data availability, or Recovery Point Objective, in order to determine acceptable data loss with various failure events. Some users can tolerate application service interruptions but cannot tolerate data loss.
A service level agreement formalizes an organization's availability objectives and requirements.
Military control systems
High availability is one of the primary requirements of the control systems in unmanned vehicles and autonomous maritime vessels. If the controlling system becomes unavailable, the Ground Combat Vehicle or ASW Continuous Trail Unmanned Vessel would be lost.System design
Adding more components to an overall system design can undermine efforts to achieve high availability because complex systems inherently have more potential failure points and are more difficult to implement correctly. While some analysts would put forth the theory that the most highly available systems adhere to a simple architecture, this architecture suffers from the requirement that the entire system must be brought down for patching and operating system upgrades. More advanced system designs allow for systems to be patched and upgraded without compromising service availability.High availability requires less human intervention to restore operation in complex systems; the reason for this being that the most common cause for outages is human error.
Redundancy is used to create systems with high levels of availability. In this case it is required to have high levels of failure detectability and avoidance of common cause failures. Two kinds of redundancy are passive redundancy and active redundancy.
Passive redundancy is used to achieve high availability by including enough excess capacity in the design to accommodate a performance decline. The simplest example is a boat with two separate engines driving two separate propellers. The boat continues toward its destination despite failure of a single engine or propeller. A more complex example is multiple redundant power generation facilities within a large system involving electric power transmission. Malfunction of single components is not considered to be a failure unless the resulting performance decline exceeds the specification limits for the entire system.
Active redundancy is used in complex systems to achieve high availability with no performance decline. Multiple items of the same kind are incorporated into a design that includes a method to detect failure and automatically reconfigure the system to bypass failed items using a voting scheme. This is used with complex computing systems that are linked. Internet routing is derived from early work by Birman and Joseph in this area. Active redundancy may introduce more complex failure modes into a system, such as continuous system reconfiguration due to faulty voting logic.
Zero downtime system design means that modeling and simulation indicates mean time between failures significantly exceeds the period of time between planned maintenance, upgrade events, or system lifetime. Zero downtime involves massive redundancy, which is needed for some types of aircraft and for most kinds of communications satellites. Global Positioning System is an example of a zero downtime system.
Fault instrumentation can be used in systems with limited redundancy to achieve high availability. Maintenance actions occur during brief periods of down-time only after a fault indicator activates. Failure is only significant if this occurs during a mission critical period.
Modeling and simulation is used to evaluate the theoretical reliability for large systems. The outcome of this kind of model is used to evaluate different design options. A model of the entire system is created, and the model is stressed by removing components. Redundancy simulation involves the N-x criteria. N represents the total number of components in the system. x is the number of components used to stress the system. N-1 means the model is stressed by evaluating performance with all possible combinations where one component is faulted. N-2 means the model is stressed by evaluating performance with all possible combinations where two component are faulted simultaneously.
Reasons for unavailability
A survey among academic availability experts in 2010 ranked reasons for unavailability of enterprise IT systems. All reasons refer to not following best practice in each of the following areas :- Monitoring of the relevant components
- Requirements and procurement
- Operations
- Avoidance of network failures
- Avoidance of internal application failures
- Avoidance of external services that fail
- Physical environment
- Network redundancy
- Technical solution of backup
- Process solution of backup
- Physical location
- Infrastructure redundancy
- Storage architecture redundancy