JPEG 2000
JPEG 2000 is an image compression standard and coding system. It was developed from 1997 to 2000 by a Joint Photographic Experts Group committee chaired by Touradj Ebrahimi, with the intention of superseding their original discrete cosine transform based JPEG standard with a newly designed, wavelet-based method. The standardized filename extension is .jp2 for ISO/IEC 15444-1 conforming files and .jpx for the extended part-2 specifications, published as ISO/IEC 15444-2. The registered MIME types are defined in RFC 3745. For ISO/IEC 15444-1 it is image/jp2.
JPEG 2000 code streams are regions of interest that offer several mechanisms to support spatial random access or region of interest access at varying degrees of granularity. It is possible to store different parts of the same picture using different quality.
JPEG 2000 is a discrete wavelet transform based compression standard that could be adapted for motion imaging video compression with the Motion JPEG 2000 extension. JPEG 2000 technology was selected as the video coding standard for digital cinema in 2004.
Aims of the standard
While there is a modest increase in compression performance of JPEG 2000 compared to JPEG, the main advantage offered by JPEG 2000 is the significant flexibility of the codestream. The codestream obtained after compression of an image with JPEG 2000 is scalable in nature, meaning that it can be decoded in a number of ways; for instance, by truncating the codestream at any point, one may obtain a representation of the image at a lower resolution, or signal-to-noise ratio – see scalable compression. By ordering the codestream in various ways, applications can achieve significant performance increases. However, as a consequence of this flexibility, JPEG 2000 requires codecs that are complex and computationally demanding. Another difference, in comparison with JPEG, is in terms of visual artifacts: JPEG 2000 only produces ringing artifacts, manifested as blur and rings near edges in the image, while JPEG produces both ringing artifacts and 'blocking' artifacts, due to its 8×8 blocks.JPEG 2000 has been published as an ISO standard, ISO/IEC 15444. The cost of obtaining all documents for the standard has been estimated to 2718 CHF., JPEG 2000 is not widely supported in web browsers, and hence is not generally used on the Internet.
Improvements over the 1992 JPEG standard
Multiple resolution representation
JPEG 2000 decomposes the image into a multiple resolution representation in the course of its compression process. This pyramid representation can be put to use for other image presentation purposes beyond compression.Progressive transmission by pixel and resolution accuracy
These features are more commonly known as progressive decoding and signal-to-noise ratio scalability. JPEG 2000 provides efficient code-stream organizations which are progressive by pixel accuracy and by image resolution. This way, after a smaller part of the whole file has been received, the viewer can see a lower quality version of the final picture. The quality then improves progressively through downloading more data bits from the source.Choice of lossless or lossy compression
Like the Lossless JPEG standard, the JPEG 2000 standard provides both lossless and lossy compression in a single compression architecture. Lossless compression is provided by the use of a reversible integer wavelet transform in JPEG 2000.Error resilience
Like JPEG 1992, JPEG 2000 is robust to bit errors introduced by noisy communication channels, due to the coding of data in relatively small independent blocks.Flexible file format
The JP2 and JPX file formats allow for handling of color-space information, metadata, and for interactivity in networked applications as developed in the JPEG Part 9 JPIP protocol.High dynamic range support
JPEG 2000 supports any bit depth, such as 16- and 32-bit floating point pixel images, and any color space.Side channel spatial information
Full support for transparency and alpha planes.JPEG 2000 image coding system – Parts
The JPEG 2000 image coding system consists of following parts:| Part | Number | First public release date | Latest public release date | Latest amendment | Identical ITU-T standard | Title | Description |
| Part 1 | 2000 | 2016 | Core coding system | the basic characteristics of JPEG 2000 compression | |||
| Part 2 | 2004 | 2004 | 2015 | Extensions | |||
| Part 3 | 2002 | 2007 | 2010 | Motion JPEG 2000 | |||
| Part 4 | 2002 | 2004 | Conformance testing | ||||
| Part 5 | 2003 | 2015 | Reference software | Java and C implementations | |||
| Part 6 | 2003 | 2016 | Compound image file format | e.g. document imaging, for pre-press and fax-like applications | |||
| Part 7 | abandoned | Guideline of minimum support function of ISO/IEC 15444-1 | |||||
| Part 8 | 2007 | 2007 | 2008 | Secure JPEG 2000 | JPSEC | ||
| Part 9 | 2005 | 2005 | 2014 | Interactivity tools, APIs and protocols | JPIP | ||
| Part 10 | 2008 | 2011 | Extensions for three-dimensional data | JP3D | |||
| Part 11 | 2007 | 2007 | 2013 | Wireless | JPWL | ||
| Part 12 | 2004 | 2015 | ISO base media file format | ||||
| Part 13 | 2008 | 2008 | An entry level JPEG 2000 encoder | ||||
| Part 14 | 2013 | XML structural representation and reference | JPXML |
Technical discussion
The aim of JPEG 2000 is not only improving compression performance over JPEG but also adding features such as scalability and editability. JPEG 2000's improvement in compression performance relative to the original JPEG standard is actually rather modest and should not ordinarily be the primary consideration for evaluating the design. Very low and very high compression rates are supported in JPEG 2000. The ability of the design to handle a very large range of effective bit rates is one of the strengths of JPEG 2000. For example, to reduce the number of bits for a picture below a certain amount, the advisable thing to do with the first JPEG standard is to reduce the resolution of the input image before encoding it. That is unnecessary when using JPEG 2000, because JPEG 2000 already does this automatically through its multiresolution decomposition structure. The following sections describe the algorithm of JPEG 2000.According to KB, «the current JP2 format specification leaves room for multiple interpretations when it comes to the support of ICC profiles, and the handling of grid resolution information».
Color components transformation
Initially images have to be transformed from the RGB color space to another color space, leading to three components that are handled separately. There are two possible choices:- Irreversible Color Transform uses the well known YCBCR color space. It is called "irreversible" because it has to be implemented in floating or fix-point and causes round-off errors.
- Reversible Color Transform uses a modified YUV color space that does not introduce quantization errors, so it is fully reversible. Proper implementation of the RCT requires that numbers are rounded as specified and cannot be expressed exactly in matrix form. The transformations are:
Tiling
After color transformation, the image is split into so-called tiles, rectangular regions of the image that are transformed and encoded separately. Tiles can be any size, and it is also possible to consider the whole image as one single tile. Once the size is chosen, all the tiles will have the same size. Dividing the image into tiles is advantageous in that the decoder will need less memory to decode the image and it can opt to decode only selected tiles to achieve a partial decoding of the image. The disadvantage of this approach is that the quality of the picture decreases due to a lower peak signal-to-noise ratio. Using many tiles can create a blocking effect similar to the older JPEG 1992 standard.Wavelet transform
These tiles are then wavelet transformed to an arbitrary depth, in contrast to JPEG 1992 which uses an 8×8 block-size discrete cosine transform. JPEG 2000 uses two different wavelet transforms:- irreversible: the CDF 9/7 wavelet transform. It is said to be "irreversible" because it introduces quantization noise that depends on the precision of the decoder.
- reversible: a rounded version of the biorthogonal LeGall-Tabatabai 5/3 wavelet transform. It uses only integer coefficients, so the output does not require rounding and so it does not introduce any quantization noise. It is used in lossless coding.
Quantization
After the wavelet transform, the coefficients are scalar-quantized to reduce the number of bits to represent them, at the expense of quality. The output is a set of integer numbers which have to be encoded bit-by-bit. The parameter that can be changed to set the final quality is the quantization step: the greater the step, the greater is the compression and the loss of quality. With a quantization step that equals 1, no quantization is performed.Coding
The result of the previous process is a collection of sub-bands which represent several approximation scales. A sub-band is a set of coefficients—real numbers which represent aspects of the image associated with a certain frequency range as well as a spatial area of the image.The quantized sub-bands are split further into precincts, rectangular regions in the wavelet domain. They are typically sized so that they provide an efficient way to access only part of the image, though this is not a requirement.
Precincts are split further into code blocks. Code blocks are in a single sub-band and have equal sizes—except those located at the edges of the image. The encoder has to encode the bits of all quantized coefficients of a code block, starting with the most significant bits and progressing to less significant bits by a process called the EBCOT scheme. EBCOT here stands for Embedded Block Coding with Optimal Truncation. In this encoding process, each bit plane of the code block gets encoded in three so-called coding passes, first encoding bits of insignificant coefficients with significant neighbors, then refinement bits of significant coefficients and finally coefficients without significant neighbors. The three passes are called Significance Propagation, Magnitude Refinement and Cleanup pass, respectively.
In lossless mode all bit planes have to be encoded by the EBCOT, and no bit planes can be dropped.
The bits selected by these coding passes then get encoded by a context-driven binary arithmetic coder, namely the binary MQ-coder. The context of a coefficient is formed by the state of its eight neighbors in the code block.
The result is a bit-stream that is split into packets where a packet groups selected passes of all code blocks from a precinct into one indivisible unit. Packets are the key to quality scalability.
Packets from all sub-bands are then collected in so-called layers.
The way the packets are built up from the code-block coding passes, and thus which packets a layer will contain, is not defined by the JPEG 2000 standard, but in general a codec will try to build layers in such a way that the image quality will increase monotonically with each layer, and the image distortion will shrink from layer to layer. Thus, layers define the progression by image quality within the code stream.
The problem is now to find the optimal packet length for all code blocks which minimizes the overall distortion in a way that the generated target bitrate equals the demanded bit rate.
While the standard does not define a procedure as to how to perform this form of rate–distortion optimization, the general outline is given in one of its many appendices: For each bit encoded by the EBCOT coder, the improvement in image quality, defined as mean square error, gets measured; this can be implemented by an easy table-lookup algorithm. Furthermore, the length of the resulting code stream gets measured. This forms for each code block a graph in the rate–distortion plane, giving image quality over bitstream length. The optimal selection for the truncation points, thus for the packet-build-up points is then given by defining critical slopes of these curves, and picking all those coding passes whose curve in the rate–distortion graph is steeper than the given critical slope. This method can be seen as a special application of the method of Lagrange multiplier which is used for optimization problems under constraints. The Lagrange multiplier, typically denoted by λ, turns out to be the critical slope, the constraint is the demanded target bitrate, and the value to optimize is the overall distortion.
Packets can be reordered almost arbitrarily in the JPEG 2000 bit-stream; this gives the encoder as well as image servers a high degree of freedom.
Already encoded images can be sent over networks with arbitrary bit rates by using a layer-progressive encoding order.
On the other hand, color components can be moved back in the bit-stream; lower resolutions could be sent first for image previewing.
Finally, spatial browsing of large images is possible through appropriate tile and/or partition selection.
All these operations do not require any re-encoding but only byte-wise copy operations.
Compression ratio
Compared to the previous JPEG standard, JPEG 2000 delivers a typical compression gain in the range of 20%, depending on the image characteristics. Higher-resolution images tend to benefit more, where JPEG-2000's spatial-redundancy prediction can contribute more to the compression process. In very low-bitrate applications, studies have shown JPEG 2000 to be outperformed by the intra-frame coding mode of H.264. Good applications for JPEG 2000 are large images, images with low-contrast edges – e.g., medical images.Computational complexity and performance
JPEG2000 is much more complicated in terms of computational complexity in comparison with JPEG standard. Tiling, color component transform, discrete wavelet transform, and quantization could be done pretty fast, though entropy codec is time consuming and quite complicated. EBCOT context modelling and arithmetic MQ-coder take most of the time of JPEG2000 codec.On CPU the main idea of getting fast JPEG2000 encoding and decoding is closely connected with AVX/SSE and multithreading to process each tile in separate thread. The fastest JPEG2000 solutions utilize both CPU and GPU power to get high performance benchmarks.
File format and code stream
Similar to JPEG-1, JPEG 2000 defines both a file format and a code stream. Whereas JPEG 2000 entirely describes the image samples, JPEG-1 includes additional meta-information such as the resolution of the image or the color space that has been used to encode the image. JPEG 2000 images should — if stored as files — be boxed in the JPEG 2000 file format, where they get the .jp2 extension. The part-2 extension to JPEG 2000, i.e., ISO/IEC 15444-2, also enriches this file format by including mechanisms for animation or composition of several code streams into one single image. Images in this extended file-format use the .jpx extension.There is no standardized extension for code-stream data because code-stream data is not to be considered to be stored in files in the first place, though when done for testing purposes, the extension .jpc or .j2k appear frequently.
Metadata
For traditional JPEG, additional metadata, e.g. lighting and exposure conditions, is kept in an application marker in the Exif format specified by the JEITA. JPEG 2000 chooses a different route, encoding the same metadata in XML form. The reference between the Exif tags and the XML elements is standardized by the ISO TC42 committee in the standard 12234-1.4.Extensible Metadata Platform can also be embedded in JPEG 2000.
Applications
Some markets and applications intended to be served by this standard are listed below:- Consumer applications such as multimedia devices
- Client/server communication
- Military/surveillance
- Medical imagery, esp. the DICOM specifications for medical data interchange.
- Biometrics.
- Remote sensing
- High-quality frame-based video recording, editing and storage.
- Live HDTV feed contribution, such as live HDTV feed of a sport event linked to the TV station studio
- Digital cinema
- JPEG 2000 has many design commonalities with the ICER image compression format that is used to send images back from the Mars rovers.
- Digitized Audio-visual contents and Images for Long term digital preservation
- World Meteorological Organization has built JPEG 2000 Compression into the new GRIB2 file format. The GRIB file structure is designed for global distribution of meteorological data. The implementation of JPEG 2000 compression in GRIB2 has reduced file sizes up to 80%.
Comparison with PNG format
The PNG format is still more space-efficient in the case of images with many pixels of the same color, such as diagrams, and supports special compression features that JPEG 2000 does not.
Legal status
ISO 15444 is covered by patents, but the contributing companies and organizations agreed that licenses for its first part—the core coding system—can be obtained free of charge from all contributors.The JPEG committee has stated:
However, the JPEG committee acknowledged in 2004 that undeclared submarine patents may present a hazard:
In the latest ISO/IEC 15444-1:2016, the JPEG committee stated in Annex L: Patent statement :
The International Organization for Standardization and the International Electrotechnical Commission draw attention to the fact that it is claimed that compliance with this Recommendation | International Standard may involve the use of patents.
The complete list of intellectual property rights statements can be obtained from the ITU-T and ISO patent declaration databases
ISO and IEC take no position concerning the evidence, validity and scope of these patent rights.
Attention is drawn to the possibility that some of the elements of this Recommendation | International Standard may be the subject of patent rights other than those identified in the above mentioned databases. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
The analysis of this ISO patent declaration database shows that 3 companies finalized their patent process, Telcordia Technologies Inc. US patent number 4,829,378, whose licensing declaration is not documented, Mitsubishi Electric Corporation, with 2 Japan patents 2128110 and 2128115, that have been expired since 20090131, 20100226 respectively, and IBM N.Y. with 11 patents under the option 1 declaration.
The Telcordia Technologies Inc. patent 4,829,378 may be checked on http://patft.uspto.gov/netahtml/PTO/srchnum.htm Its title is "Sub-band coding of images with low computational complexity", and it seems that its relation with JPEG 2000 is "distant", as the technique described and claimed is widely used.
Finally, search on the European patent and US patent databases on JPEG 2000 between 1978 and 15 March 2000 provides no patent registered on any of these 2 patent databases.
This provides an updated context of JPEG 2000 legal status in 2019, showing that since 2016, though ISO and IEC deny any responsibility in any hidden patent rights other than those identified in the above mentioned ISO databases, the risk of such a patent claim on ISO 15444-1 and its discrete wavelet transform algorithm appears to be low.
Related standards
Several additional parts of the JPEG 2000 standard exist;Amongst them are ISO/IEC 15444-2:2000, JPEG 2000 extensions defining the .jpx file format, featuring for example Trellis quantization, an extended file format and additional color spaces, ISO/IEC 15444-4:2000, the reference testing and ISO/IEC 15444-6:2000, the compound image file format, allowing compression of compound text/image graphics.
Extensions for secure image transfer, JPSEC, enhanced error-correction schemes for wireless applications, JPWL and extensions for encoding of volumetric images, JP3D are also already available from the ISO.
JPIP protocol for streaming JPEG 2000 images
In 2005, a JPEG 2000 based image browsing protocol, called JPIP has been published as ISO/IEC 15444-9. Within this framework, only selected regions of potentially huge images have to be transmitted from an image server on the request of a client, thus reducing the required bandwidth.JPEG 2000 data may also be streamed using the ECWP and ECWPS protocols found within the ERDAS ECW/JP2 SDK.
Motion JPEG 2000
,, originally defined in Part 3 of the ISO Standard for JPEG2000 as a standalone document, has now been expressed by ISO/IEC 15444-3:2002/Amd 2:2003 in terms of the ISO Base format, ISO/IEC 15444-12 and in ITU-T Recommendation T.802. It specifies the use of the JPEG 2000 format for timed sequences of images, possibly combined with audio, and composed into an overall presentation. It also defines a file format, based on ISO base media file format. Filename extensions for Motion JPEG 2000 video files are .mj2 and .mjp2 according to RFC 3745.It is an open ISO standard and an advanced update to MJPEG, which was based on the legacy JPEG format. Unlike common video formats, such as MPEG-4 Part 2, WMV, and H.264, MJ2 does not employ temporal or inter-frame compression. Instead, each frame is an independent entity encoded by either a lossy or lossless variant of JPEG 2000. Its physical structure does not depend on time ordering, but it does employ a separate profile to complement the data. For audio, it supports LPCM encoding, as well as various MPEG-4 variants, as "raw" or complement data.
Motion JPEG 2000 was considered as a digital archival format by the Library of Congress.
In June 2013, in an interview with Bertram Lyons from the Library of Congress for The New York Times Magazine, about "Tips on Archiving Family History", codecs such as FFV1, H264 or Apple ProRes are mentioned, but JPEG 2000 is not.
ISO base media file format
ISO/IEC 15444-12 is identical with ISO/IEC 14496-12 and it defines ISO base media file format. For example, Motion JPEG 2000 file format, MP4 file format or 3GP file format are also based on this ISO base media file format.GML JP2 georeferencing
The Open Geospatial Consortium has defined a metadata standard for georeferencing JPEG 2000 images with embedded XML using the Geography Markup Language format: GML in JPEG 2000 for Geographic Imagery Encoding , version 1.0.0, dated 2006-01-18. Version 2.0, entitled GML in JPEG 2000 Encoding Standard Part 1: Core was approved 2014-06-30.JP2 and JPX files containing GMLJP2 markup can be located and displayed in the correct position on the Earth's surface by a suitable Geographic Information System, in a similar way to GeoTIFF images.