Cosmological constant problem
In cosmology, the cosmological constant problem or vacuum catastrophe is the disagreement between the observed values of vacuum energy density and theoretical large value of zero-point energy suggested by quantum field theory.
Depending on the Planck energy cutoff and other factors, the discrepancy is as high as 120 orders of magnitude, a state of affairs described by physicists as "the largest discrepancy between theory and experiment in all of science" and "the worst theoretical prediction in the history of physics."
History
The basic problem of a vacuum energy producing a gravitational effect was identified as early as 1916 by Walther Nernst. The value was predicted to be either zero or very small, so that the theoretical problem was already apparent, and began to be actively discussed in the 1970s.With the development of inflationary cosmology in the 1980s, the problem became much more important: as cosmic inflation is driven by vacuum energy, differences in modeling vacuum energy leads to huge differences in the resulting cosmologies.
Quantum description
After the development of quantum field theory in the 1940s, the first to address contributions of quantum fluctuations to the cosmological constant was Zel’dovich. In quantum mechanics, the vacuum itself should experience quantum fluctuations. In general relativity, those quantum fluctuations constitute energy that would add to the cosmological constant. However, this calculated vacuum energy density is many orders of magnitude bigger than the observed cosmological constant. Original estimates of the degree of mismatch were as high as 120 orders of magnitude; however, modern research suggests that, when Lorentz invariance is taken into account, the degree of mismatch is closer to 60 orders of magnitude.The calculated vacuum energy is a positive, rather than negative, contribution to the cosmological constant because the existing vacuum has negative quantum-mechanical pressure, and in general relativity, the gravitational effect of negative pressure is a kind of repulsion. Roughly, the vacuum energy is calculated by summing over all known quantum-mechanical fields, taking into account interactions and self-interactions between the ground states, and then removing all interactions below a minimum "cutoff" wavelength to reflect that existing theories break down and may fail to be applicable around the cutoff scale. Because the energy is dependent on how fields interact within the current vacuum state, the vacuum energy contribution would have been different in the early universe; for example, the vacuum energy would have been significantly different prior to electroweak symmetry breaking during the quark epoch.
Renormalization
The vacuum energy in quantum field theory can be set to any value by renormalization. This view treats the cosmological constant as simply another fundamental physical constant not predicted or explained by theory. Such a renormalization constant must be chosen very accurately because of the many-orders-of-magnitude discrepancy between theory and observation, and many theorists consider this ad-hoc constant as equivalent to ignoring the problem.Proposed solutions
Some physicists propose an anthropic solution, and argue that we live in one region of a vast multiverse that has different regions with different vacuum energies. These anthropic arguments posit that only regions of small vacuum energy such as the one we live in are reasonably capable of supporting intelligent life. Such arguments have existed in some form since at least 1981. Around 1987, Steven Weinberg estimated that the maximum allowable vacuum energy for gravitationally-bound structures to form is problematically large, even given the observational data available in 1987, and concluded the anthropic explanation appears to fail; however, more recent estimates by Weinberg and others, based on other considerations, find the bound to be closer to the actual observed level of dark energy. Anthropic arguments gradually gained credibility among many physicists after the discovery of dark energy and the development of the theoretical string theory landscape, but are still derided by a substantial skeptical portion of the scientific community as being problematic to verify. Proponents of anthropic solutions are themselves divided on multiple technical questions surrounding how to calculate the proportion of regions of the universe with various dark energy constants.Other proposals involve modifying gravity to diverge from general relativity. These proposals face the hurdle that the results of observations and experiments so far have tended to be extremely consistent with general relativity and the ΛCDM model, and inconsistent with thus-far proposed modifications. In addition, some of the proposals are arguably incomplete, because they solve the "new" cosmological constant problem by proposing that the actual cosmological constant is exactly zero rather than a tiny number, but fail to solve the "old" cosmological constant problem of why quantum fluctuations seem to fail to produce substantial vacuum energy in the first place. Nevertheless, many physicists argue that, due in part to a lack of better alternatives, proposals to modify gravity should be considered "one of the most promising routes to tackling" the cosmological constant problem.
Bill Unruh and collaborators have argued that when the energy density of the quantum vacuum is modeled more accurately as a fluctuating quantum field, the cosmological constant problem does not arise. Going in a different direction, George F. R. Ellis and others have suggested that in unimodular gravity, the troublesome contributions simply do not gravitate.
Another argument, due to Stanley Brodsky and Robert Shrock, is that in light front quantization, the quantum field theory vacuum becomes essentially trivial. In the absence of vacuum expectation values, there is no contribution from QED, Weak interactions and QCD to the cosmological constant. It is thus predicted to be zero in a flat space-time.
In 2018, a mechanism for cancelling Λ out has been proposed through the use of a symmetry breaking potential in a Lagrangian formalism in which matter shows a non-vanishing pressure. The model assumes that standard matter provides a pressure which counterbalances the action due to the cosmological constant. Luongo and Muccino have shown that this mechanism permits to take vacuum energy as quantum field theory predicts, but removing the huge magnitude through a counterbalance term due to baryons and cold dark matter only.