Lidar
Lidar is a method for measuring distances by illuminating the target with laser light and measuring the reflection with a sensor. Differences in laser return times and wavelengths can then be used to make digital 3-D representations of the target. It has terrestrial, airborne, and mobile applications.
The term lidar was originally a portmanteau of light and radar.
It is now also used as an acronym of "light detection and ranging"
and "laser imaging, detection, and ranging".
Lidar sometimes is called 3-D laser scanning, a special combination of a 3-D scanning and laser scanning.
Lidar is commonly used to make high-resolution maps, with applications in surveying, geodesy, geomatics, archaeology, geography, geology, geomorphology, seismology, forestry, atmospheric physics, laser guidance, airborne laser swath mapping, and laser altimetry. The technology is also used in control and navigation for some autonomous cars.
History and etymology
Under the direction of Malcolm Stitch, the Hughes Aircraft Company introduced the first lidar-like system in 1961, shortly after the invention of the laser. Intended for satellite tracking, this system combining laser-focused imaging with the ability to calculate distances by measuring the time for a signal to return using appropriate sensors and data acquisition electronics. It was originally called "Colidar" an acronym for "COherent Light Detecting And Ranging," derived from the term "radar", itself an acronym for "Radio Detection And Ranging". From the early colidar systems all laser rangefinders, laser altimeters and lidar units are derived. The first practical terrestrial application of a colidar system was the "Colidar Mark II", a large rifle-like laser rangefinder produced in 1963 which had a range of 7 miles and an accuracy of 15 feet, to be used for military targeting.The first mention of lidar as a stand-alone word in 1963 suggests it originated as a portmanteau of "light" and "radar": "Eventually the laser may provide an extremely sensitive detector of particular wavelengths from distant objects. Meanwhile, it is being used to study the moon by 'lidar' ..." The Oxford English Dictionary supports this etymology.
Lidar's first applications came in meteorology, where the National Center for Atmospheric Research used it to measure clouds and pollution. The general public became aware of the accuracy and usefulness of lidar systems in 1971 during the Apollo 15 mission, when astronauts used a laser altimeter to map the surface of the moon.
Although the English language no longer treats "radar" as an acronym, and printed texts universally present the word uncapitalized, the word "lidar" became capitalized as "LIDAR" or "LiDAR" in some publications beginning in the 1980s. Currently, no consensus exists on capitalization, reflecting uncertainty about whether or not "lidar" is an acronym, and if it is an acronym, whether it should appear in lower case, like "radar" and "sonar". Various publications refer to lidar as "LIDAR", "LiDAR", "LIDaR", or "Lidar". The USGS uses both "LIDAR" and "lidar", sometimes in the same document; the New York Times predominantly uses "lidar" for staff-written articles, although contributing news feeds such as Reuters may use Lidar.
General description
Lidar uses ultraviolet, visible, or near infrared light to image objects. It can target a wide range of materials, including non-metallic objects, rocks, rain, chemical compounds, aerosols, clouds and even single molecules. A narrow laser beam can map physical features with very high resolutions; for example, an aircraft can map terrain at resolution or better.The essential concept of lidar was originated by EH Synge in 1930, who envisaged the use of powerful searchlights to probe the atmosphere. Indeed, lidar has since been used extensively for atmospheric research and meteorology. Lidar instruments fitted to aircraft and satellites carry out surveying and mapping – a recent example being the U.S. Geological Survey Experimental Advanced Airborne Research Lidar. NASA has identified lidar as a key technology for enabling autonomous precision safe landing of future robotic and crewed lunar-landing vehicles.
Wavelengths vary to suit the target: from about 10 micrometers to approximately 250 nm. Typically, light is reflected via backscattering, as opposed to pure reflection one might find with a mirror. Different types of scattering are used for different lidar applications: most commonly Rayleigh scattering, Mie scattering, Raman scattering, and fluorescence. Suitable combinations of wavelengths can allow for remote mapping of atmospheric contents by identifying wavelength-dependent changes in the intensity of the returned signal.
Design
The two kinds of lidar detection schemes are "incoherent" or direct energy detection and coherent detection. Coherent systems generally use optical heterodyne detection. This is more sensitive than direct detection and allows them to operate at much lower power, but requires more complex transceivers.Both types employ pulse models: either micropulse or high energy. Micropulse systems utilize intermittent bursts of energy. They developed as a result of ever-increasing computer power, combined with advances in laser technology. They use considerably less energy in the laser, typically on the order of one microjoule, and are often "eye-safe", meaning they can be used without safety precautions. High-power systems are common in atmospheric research, where they are widely used for measuring atmospheric parameters: the height, layering and densities of clouds, cloud particle properties, temperature, pressure, wind, humidity, and trace gas concentration.
Components
Lidar systems consist of several major components.Laser
600–1000 nm lasers are most common for non-scientific applications. The maximum power of the laser is limited, or an automatic shut off system which turns the laser off at specific altitudes is used in order to make it eye safe for the people on the ground.One common alternative, 1550 nm lasers, are eye-safe at relatively high power levels since this wavelength is not strongly absorbed by the eye, but the detector technology is less advanced and so these wavelengths are generally used at longer ranges with lower accuracies. They are also used for military applications because 1550 nm is not visible in night vision goggles, unlike the shorter 1000 nm infrared laser.
Airborne topographic mapping lidars generally use 1064 nm diode pumped YAG lasers, while bathymetric systems generally use 532 nm frequency doubled diode pumped YAG lasers because 532 nm penetrates water with much less attenuation than does 1064 nm. Laser settings include the laser repetition rate. Pulse length is generally an attribute of the laser cavity length, the number of passes required through the gain material, and Q-switch speed. Better target resolution is achieved with shorter pulses, provided the lidar receiver detectors and electronics have sufficient bandwidth.
Flash lidar
In flash lidar, the entire field of view is illuminated with a wide diverging laser beam in a single pulse. This is in contrast to conventional scanning lidar, which uses a collimated laser beam that illuminates a single point at a time, and the beam is raster scanned to illuminate the field of view point-by-point. This illumination method requires a different detection scheme as well. In both scanning and flash lidar, a time-of-flight camera is used to collect information about both the 3-D location and intensity of the light incident on it in every frame. However, in scanning lidar, this camera contains only a point sensor, while in flash lidar, the camera contains either a 1-D or a 2-D sensor array, each pixel of which collects 3-D location and intensity information. In both cases, the depth information is collected using the time of flight of the laser pulse, which requires the pulsing of the laser and acquisition by the camera to be synchronized. The result is a camera that takes pictures of distance, instead of colors.As with all forms of lidar, the onboard source of illumination makes flash lidar an active sensor. The signal that is returned is processed by embedded algorithms to produce a nearly instantaneous 3-D rendering of objects and terrain features within the field of view of the sensor. The laser pulse repetition frequency is sufficient for generating 3-D videos with high resolution and accuracy. The high frame rate of the sensor makes it a useful tool for a variety of applications that benefit from real-time visualization, such as highly precise remote landing operations. By immediately returning a 3D elevation mesh of target landscapes, a flash sensor can be used to identify optimal landing zones in autonomous spacecraft landing scenarios.
Seeing at a distance requires a powerful burst of light. The power is limited to levels that do not damage human retinas. Wavelengths must not affect human eyes. However, low-cost silicon imagers do not read light in the eye-safe spectrum. Instead, gallium-arsenide imagers are required, which can boost costs to $200,000. Gallium-arsenide is the same compound used to produce high-cost, high-efficiency solar panels usually used in space applications.
Phased arrays
A phased array can illuminate any direction by using a microscopic array of individual antennas. Controlling the timing of each antenna steers a cohesive signal in a specific direction.Phased arrays have been used in radar since the 1950s. The same technique can be used with light. On the order of a million optical antennas are used to see a radiation pattern of a certain size in a certain direction. The system is controlled by timing the precise flash. A single chip replace a $75,000 electromechanical system, drastically reducing costs.
Several companies are working on developing commercial solid-state lidar units, including the company Quanergy which is designing a 905 nm solid state device, although they appear to be having some issues in development.
The control system can change the shape of the lens to enable zoom in/zoom out functions. Specific sub-zones can be targeted at sub-second intervals.
Electromechanical lidar lasts for between 1,000 and 2,000 hours. By contrast, solid-state lidar can run for 100,000 hours.
Microelectromechanical machines
Microelectromechanical mirrors are not entirely solid-state. However, their tiny form factor provides many of the same cost benefits. A single laser is directed to a single mirror that can be reoriented to view any part of the target field. The mirror spins at a rapid rate. However, MEMS systems generally operate in a single plane. To add a second dimension generally requires a second mirror that moves up and down. Alternatively, another laser can hit the same mirror from another angle. MEMS systems can be disrupted by shock/vibration and may require repeated calibration. The goal is to create a small microchip to enhance innovation and further technological advances.Scanner and optics
Image development speed is affected by the speed at which they are scanned. Options to scan the azimuth and elevation include dual oscillating plane mirrors, a combination with a polygon mirror and a dual axis scanner. Optic choices affect the angular resolution and range that can be detected. A hole mirror or a beam splitter are options to collect a return signal.Photodetector and receiver electronics
Two main photodetector technologies are used in lidar: solid state photodetectors, such as silicon avalanche photodiodes, or photomultipliers. The sensitivity of the receiver is another parameter that has to be balanced in a lidar design.Position and navigation systems
Lidar sensors mounted on mobile platforms such as airplanes or satellites require instrumentation to determine the absolute position and orientation of the sensor. Such devices generally include a Global Positioning System receiver and an inertial measurement unit.Sensor
Lidar uses active sensors that supply their own illumination source. The energy source hits objects and the reflected energy is detected and measured by sensors. Distance to the object is determined by recording the time between transmitted and backscattered pulses and by using the speed of light to calculate the distance traveled. Flash LIDAR allows for 3-D imaging because of the camera's ability to emit a larger flash and sense the spatial relationships and dimensions of area of interest with the returned energy. This allows for more accurate imaging because the captured frames do not need to be stitched together, and the system is not sensitive to platform motion resulting in less distortion.3-D imaging can be achieved using both scanning and non-scanning systems. "3-D gated viewing laser radar" is a non-scanning laser ranging system that applies a pulsed laser and a fast gated camera. Research has begun for virtual beam steering using Digital Light Processing technology.
Imaging lidar can also be performed using arrays of high speed detectors and modulation sensitive detector arrays typically built on single chips using complementary metal–oxide–semiconductor and hybrid CMOS/Charge-coupled device fabrication techniques. In these devices each pixel performs some local processing such as demodulation or gating at high speed, downconverting the signals to video rate so that the array can be read like a camera. Using this technique many thousands of pixels / channels may be acquired simultaneously. High resolution 3-D lidar cameras use homodyne detection with an electronic CCD or CMOS shutter.
A coherent imaging lidar uses synthetic array heterodyne detection to enable a staring single element receiver to act as though it were an imaging array.
In 2014, Lincoln Laboratory announced a new imaging chip with more than 16,384 pixels, each able to image a single photon, enabling them to capture a wide area in a single image. An earlier generation of the technology with one fourth that number of pixels was dispatched by the U.S. military after the January 2010 Haiti earthquake; a single pass by a business jet at 3,000 meters over Port-au-Prince was able to capture instantaneous snapshots of 600-meter squares of the city at a resolution of, displaying the precise height of rubble strewn in city streets. The Lincoln system is 10x faster. The chip uses indium gallium arsenide, which operates in the infrared spectrum at a relatively long wavelength that allows for higher power and longer ranges. In many applications, such as self-driving cars, the new system will lower costs by not requiring a mechanical component to aim the chip. InGaAs uses less hazardous wavelengths than conventional silicon detectors, which operate at visual wavelengths.
Types
Based on orientation
Lidar can be oriented to nadir, zenith, or laterally.For example, lidar altimeters look down, an atmospheric lidar looks up, and lidar-based collision avoidance systems are side-looking.
Based on platform
Lidar applications can be divided into airborne and terrestrial types. The two types require scanners with varying specifications based on the data's purpose, the size of the area to be captured, the range of measurement desired, the cost of equipment, and more.Spaceborne platforms are also possible, see satellite laser altimetry.
Airborne
Airborne lidar is when a laser scanner, while attached to an aircraft during flight, creates a 3-D point cloud model of the landscape. This is currently the most detailed and accurate method of creating digital elevation models, replacing photogrammetry. One major advantage in comparison with photogrammetry is the ability to filter out reflections from vegetation from the point cloud model to create a digital terrain model which represents ground surfaces such as rivers, paths, cultural heritage sites, etc., which are concealed by trees. Within the category of airborne lidar, there is sometimes a distinction made between high-altitude and low-altitude applications, but the main difference is a reduction in both accuracy and point density of data acquired at higher altitudes. Airborne lidar can also be used to create bathymetric models in shallow water.The main constituents of airborne lidar include digital elevation models and digital surface models. The points and ground points are the vectors of discrete points while DEM and DSM are interpolated raster grids of discrete points. The process also involves capturing of digital aerial photographs. In order to interpret deep-seated landslides for example, under the cover of vegetation, scarps, tension cracks or tipped trees airborne lidar is used. Airborne lidar digital elevation models can see through the canopy of forest cover, perform detailed measurements of scarps, erosion and tilting of electric poles.
Airborne lidar data is processed using a toolbox called Toolbox for Lidar Data Filtering and Forest Studies for lidar data filtering and terrain study software. The data is interpolated to digital terrain models using the software. The laser is directed at the region to be mapped and each point's height above the ground is calculated by subtracting the original z-coordinate from the corresponding digital terrain model elevation. Based on this height above the ground the non-vegetation data is obtained which may include objects such as buildings, electric power lines, flying birds, insects, etc. The rest of the points are treated as vegetation and used for modeling and mapping. Within each of these plots, lidar metrics are calculated by calculating statistics such as mean, standard deviation, skewness, percentiles, quadratic mean, etc.
Airborne lidar bathymetry
The airborne lidar bathymetric technological system involves the measurement of time of flight of a signal from a source to its return to the sensor. The data acquisition technique involves a sea floor mapping component and a ground truth component that includes video transects and sampling. It works using a green spectrum laser beam. Two beams are projected onto a fast rotating mirror, which creates an array of points. One of the beams penetrates the water and also detects the bottom surface of the water under favorable conditions.The data obtained shows the full extent of the land surface exposed above the sea floor. This technique is extremely useful as it will play an important role in the major sea floor mapping program. The mapping yields onshore topography as well as underwater elevations. Sea floor reflectance imaging is another solution product from this system which can benefit mapping of underwater habitats. This technique has been used for three-dimensional image mapping of California's waters using a hydrographic lidar.
Drones are now being used with laser scanners, as well as other remote sensors, as a more economical method to scan smaller areas. The possibility of drone remote sensing also eliminates any danger that crews of a manned aircraft may be subjected to in difficult terrain or remote areas.
Terrestrial
Terrestrial applications of lidar happen on the Earth's surface and can be either stationary or mobile. Stationary terrestrial scanning is most common as a survey method, for example in conventional topography, monitoring, cultural heritage documentation and forensics. The 3-D point clouds acquired from these types of scanners can be matched with digital images taken of the scanned area from the scanner's location to create realistic looking 3-D models in a relatively short time when compared to other technologies. Each point in the point cloud is given the colour of the pixel from the image taken located at the same angle as the laser beam that created the point.Mobile lidar is when two or more scanners are attached to a moving vehicle to collect data along a path. These scanners are almost always paired with other kinds of equipment, including GNSS receivers and IMUs. One example application is surveying streets, where power lines, exact bridge heights, bordering trees, etc. all need to be taken into account. Instead of collecting each of these measurements individually in the field with a tachymeter, a 3-D model from a point cloud can be created where all of the measurements needed can be made, depending on the quality of the data collected. This eliminates the problem of forgetting to take a measurement, so long as the model is available, reliable and has an appropriate level of accuracy.
Terrestrial lidar mapping involves a process of occupancy grid map generation. The process involves an array of cells divided into grids which employ a process to store the height values when lidar data falls into the respective grid cell. A binary map is then created by applying a particular threshold to the cell values for further processing. The next step is to process the radial distance and z-coordinates from each scan to identify which 3-D points correspond to each of the specified grid cell leading to the process of data formation.
Applications
There are a wide variety of applications for lidar, in addition to the applications listed below, as it is often mentioned in National lidar dataset programs.Agriculture
s have been used for a variety of purposes ranging from seed and fertilizer dispersions, sensing techniques as well as crop scouting for the task of weed control.Lidar can help determine where to apply costly fertilizer. It can create a topographical map of the fields and reveal slopes and sun exposure of the farmland. Researchers at the Agricultural Research Service used this topographical data with the farmland yield results from previous years, to categorize land into zones of high, medium, or low yield. This indicates where to apply fertilizer to maximize yield.
Lidar is now used to monitor insects in the field. The use of Lidar can detect the movement and behavior of individual flying insects, with identification down to sex and species. In 2017 a patent application was published on this technology in the United States of America, Europe and China.
Another application is crop mapping in orchards and vineyards, to detect foliage growth and the need for pruning or other maintenance, detect variations in fruit production, or count plants.
Lidar is useful in GPS-denied situations, such as nut and fruit orchards, where foliage blocks GPS signals to precision agriculture equipment or a driverless tractor. Lidar sensors can detect the edges of rows, so that farming equipment can continue moving until GPS signal is reestablished.
Plant species classification
Controlling weeds requires identifying plant species. This can be done by using 3-D lidar and machine learning. Lidar produces plant contours as a "point cloud" with range and reflectance values. This data is transformed, and features are extracted from it. If the species is known, the features are added as new data. The species is labelled and its features are initially stored as an example to identify the species in the real environment. This method is efficient because it uses a low-resolution lidar and supervised learning. It includes an easy-to-compute feature set with common statistical features which are independent of the plant size.Archaeology
Lidar has many uses in archaeology, including planning of field campaigns, mapping features under forest canopy, and overview of broad, continuous features indistinguishable from the ground. Lidar can produce high-resolution datasets quickly and cheaply. Lidar-derived products can be easily integrated into a Geographic Information System for analysis and interpretation.Lidar can also help to create high-resolution digital elevation models of archaeological sites that can reveal micro-topography that is otherwise hidden by vegetation. The intensity of the returned lidar signal can be used to detect features buried under flat vegetated surfaces such as fields, especially when mapping using the infrared spectrum. The presence of these features affects plant growth and thus the amount of infrared light reflected back. For example, at Fort Beauséjour – Fort Cumberland National Historic Site, Canada, lidar discovered archaeological features related to the siege of the Fort in 1755. Features that could not be distinguished on the ground or through aerial photography were identified by overlaying hill shades of the DEM created with artificial illumination from various angles. Another example is work at Caracol by Arlen Chase and his wife Diane Zaino Chase. In 2012, lidar was used to search for the legendary city of La Ciudad Blanca or "City of the Monkey God" in the La Mosquitia region of the Honduran jungle. During a seven-day mapping period, evidence was found of man-made structures. In June 2013, the rediscovery of the city of Mahendraparvata was announced. In southern New England, lidar was used to reveal stone walls, building foundations, abandoned roads, and other landscape features obscured in aerial photography by the region's dense forest canopy. In Cambodia, lidar data were used by Demian Evans and Roland Fletcher to reveal anthropogenic changes to Angkor landscape
In 2012, Lidar revealed that the Purépecha settlement of Angamuco in Michoacán, Mexico, had about as many buildings as today's Manhattan; while in 2016, its use in mapping ancient Maya causeways in northern Guatemala, revealed 17 elevated roads linking the ancient city of El Mirador to other sites. In 2018, archaeologists using lidar discovered more than 60,000 man-made structures in the Maya Biosphere Reserve, a "major breakthrough" that showed the Maya civilization was much larger than previously thought.
Autonomous vehicles
may use lidar for obstacle detection and avoidance to navigate safely through environments. Point cloud output from the lidar sensor provides the necessary data for robot software to determine where potential obstacles exist in the environment and where the robot is in relation to those potential obstacles. Singapore's Singapore-MIT Alliance for Research and Technology is actively developing technologies for autonomous lidar vehicles. Examples of companies that produce lidar sensors commonly used in vehicle automation are Ouster and Velodyne. Examples of obstacle detection and avoidance products that leverage lidar sensors are the Autonomous Solution, Inc. Forecast 3-D Laser System and Velodyne HDL-64E. Lidar simulation models are also provided in autonomous car simulators.The very first generations of automotive adaptive cruise control systems used only lidar sensors.
Object detection for transportation systems
In transportation systems, to ensure vehicle and passenger safety and to develop electronic systems that deliver driver assistance, understanding vehicle and its surrounding environment is essential. Lidar systems play an important role in the safety of transportation systems. Lots of electronic systems which add to the driver assistance and vehicle safety such as Adaptive Cruise Control, Emergency Brake Assist, and Anti-lock Braking System depend on the detection of a vehicle's environment to act autonomously or semi-autonomously. Lidar mapping and estimation achieve this.Basics overview: Current lidar systems use rotating hexagonal mirrors which split the laser beam. The upper three beams are used for vehicle and obstacles ahead and the lower beams are used to detect lane markings and road features. The major advantage of using lidar is that the spatial structure is obtained and this data can be fused with other sensors such as radar, etc. to get a better picture of the vehicle environment in terms of static and dynamic properties of the objects present in the environment. Conversely, a significant issue with lidar is the difficulty in reconstructing point cloud data in poor weather conditions. In heavy rain, for example, the light pulses emitted from the lidar system are partially reflected off of rain droplets which adds noise to the data, called 'echoes'.
Below mentioned are various approaches of processing lidar data and using it along with data from other sensors through sensor fusion to detect the vehicle environment conditions.
GRID based processing using 3-D lidar and fusion with radar measurement
In this method, proposed by Philipp Lindner and Gerd Wanielik, laser data is processed using a multidimensional occupancy grid. Data from a four-layer laser is pre-processed at the signal level and then processed at a higher level to extract the features of the obstacles. A combination two- and three-dimensional grid structure is used and the space in these structures is tessellated into several discrete cells. This method allows a huge amount of raw measurement data to be effectively handled by collecting it in spatial containers, the cells of the evidence grid. Each cell is associated with a probability measure that identifies the cell occupation. This probability is calculated by using the range measurement of the lidar sensor obtained over time and a new range measurement, which are related using Bayes' theorem. A two-dimensional grid can observe an obstacle in front of it, but cannot observe the space behind the obstacle. To address this, the unknown state behind the obstacle is assigned a probability of 0.5. By introducing the third dimension or in other terms using a multi-layer laser, the spatial configuration of an object could be mapped into the grid structure to a degree of complexity. This is achieved by transferring the measurement points into a three-dimensional grid. The grid cells which are occupied will possess a probability greater than 0.5 and the mapping would be color-coded based on the probability. The cells that are not occupied will possess a probability less than 0.5 and this area will usually be white space. This measurement is then transformed to a grid coordinate system by using the sensor position on the vehicle and the vehicle position in the world coordinate system. The coordinates of the sensor depend upon its location on the vehicle and the coordinates of the vehicle are computed using egomotion estimation, which is estimating the vehicle motion relative to a rigid scene. For this method, the grid profile must be defined. The grid cells touched by the transmitted laser beam are calculated by applying Bresenham's line algorithm. To obtain the spatially extended structure, a connected component analysis of these cells is performed. This information is then passed on to a rotating caliper algorithm to obtain the spatial characteristics of the object. In addition to the lidar detection, RADAR data obtained by using two short-range radars is integrated to get additional dynamic properties of the object, such as its velocity. The measurements are assigned to the object using a potential distance function.; Advantages and disadvantages
The geometric features of the objects are extracted efficiently, from the measurements obtained by the 3-D occupancy grid, using rotating caliper algorithm. Fusing the radar data to the lidar measurements give information about the dynamic properties of the obstacle such as velocity and location of the obstacle with respect to the sensor location which helps the vehicle or the driver decide the action to be performed in order to ensure safety. The only concern is the computational requirement to implement this data processing technique. It can be implemented in real time and has been proven efficient if the 3-D occupancy grid size is considerably restricted. But this can be improved to an even wider range by using dedicated spatial data structures that manipulate the spatial data more effectively, for the 3-D grid representation.
Fusion of 3-D lidar and color camera for multiple object detection and tracking
The framework proposed in this method by Soonmin Hwang et al., is split into four steps. First, the data from the camera and 3-D lidar is input into the system. Both inputs from lidar and camera are parallelly obtained and the color image from the camera is calibrated with the lidar. To improve the efficiency, horizontal 3-D point sampling is applied as pre-processing. Second, the segmentation stage is where the entire 3-D points are divided into several groups per the distance from the sensor and local planes from close plane to far plane are sequentially estimated. The local planes are estimated using statistical analysis. The group of points closer to the sensor are used to compute the initial plane. By using the current local plane, the next local plane is estimated by an iterative update. The object proposals in the 2-D image are used to separate foreground objects from background. For faster and accurate detection and tracking Binarized Normed Gradients for Objectness Estimation at 300fps is used. BING is a combination of normed gradient and its binarized version which speeds up the feature extraction and testing process, to estimate the objectness of an image window. This way the foreground and background objects are separated. To form objects after estimating the objectness of an image using BING, the 3-D points are grouped or clustered. Clustering is done using DBSCAN algorithm which could be robust due to its less-parametric characteristic. Using the clustered 3-D points, i.e. 3-D segment, more accurate region-of-interests are generated by projecting 3-D points on the 2-D image. The third step is detection, which is broadly divided into two parts. First is object detection in 2-D image which is achieved using Fast R-CNN as this method doesn't need training and it also considers an image and several regions of interest. Second is object detection in 3-D space that is done by using the spin image method. This method extracts local and global histograms to represent a certain object. To merge the results of 2-D image and 3-D space object detection, same 3-D region is considered and two independent classifiers from 2-D image and 3-D space are applied to the considered region. Scores calibration is done to get a single confidence score from both detectors. This single score is obtained in the form of probability. The final step is tracking. This is done by associating moving objects in present and past frame. For object tracking, segment matching is adopted. Features such as mean, standard deviation, quantized color histograms, volume size and number of 3-D points of a segment are computed. Euclidean distance is used to measure differences between segments. To judge the appearance and disappearance of an object, similar segments from two different frames are taken and the physical distance and dissimilarity scores are calculated. If the scores go beyond a range for every segment in the previous frame, the object being tracked is considered to have disappeared.; Advantages and disadvantages
The advantages of this method are using 2-D image and 3-D data together, F l-score, average precision are higher than that when only 3-D data from lidar is used. These scores are conventional measurements which judge the framework. The drawback of this method is the usage of BING for object proposal estimation as BING predicts a small set of object bounding boxes.
Obstacle detection and road environment recognition using lidar
This method proposed by Kun Zhou et al. not only focuses on object detection and tracking but also recognizes lane marking and road features. As mentioned earlier the lidar systems use rotating hexagonal mirrors that split the laser beam into six beams. The upper three layers are used to detect the forward objects such as vehicles and roadside objects. The sensor is made of weather-resistant material. The data detected by lidar are clustered to several segments and tracked by Kalman filter. Data clustering here is done based on characteristics of each segment based on object model, which distinguish different objects such as vehicles, signboards, etc. These characteristics include the dimensions of the object, etc. The reflectors on the rear edges of vehicles are used to differentiate vehicles from other objects. Object tracking is done using a 2-stage Kalman filter considering the stability of tracking and the accelerated motion of objects Lidar reflective intensity data is also used for curb detection by making use of robust regression to deal with occlusions. The road marking is detected using a modified Otsu method by distinguishing rough and shiny surfaces.; Advantages
Roadside reflectors that indicate lane border are sometimes hidden due to various reasons. Therefore, other information is needed to recognize the road border. The lidar used in this method can measure the reflectivity from the object. Hence, with this data road border can also be recognized. Also, the usage of sensor with weather-robust head helps detecting the objects even in bad weather conditions. Canopy Height Model before and after flood is a good example. Lidar can detect high detailed canopy height data as well as its road border.
Lidar measurements help identify the spatial structure of the obstacle. This helps distinguish objects based on size and estimate the impact of driving over it.
Lidar systems provide better range and a large field of view which helps detecting obstacles on the curves. This is one major advantage over RADAR systems which have a narrower field of view. The fusion of lidar measurement with different sensors makes the system robust and useful in real-time applications, since lidar dependent systems can't estimate the dynamic information about the detected object.
It has been shown that lidar can be manipulated, such that self-driving cars are tricked into taking evasive action.
Biology and conservation
Lidar has also found many applications in forestry. Canopy heights, biomass measurements, and leaf area can all be studied using airborne lidar systems. Similarly, lidar is also used by many industries, including Energy and Railroad, and the Department of Transportation as a faster way of surveying. Topographic maps can also be generated readily from lidar, including for recreational use such as in the production of orienteering maps. Lidar has also been applied to estimate and assess the biodiversity of plants, fungi, and animals.In addition, the Save the Redwoods League has undertaken a project to map the tall redwoods on the Northern California coast. Lidar allows research scientists to not only measure the height of previously unmapped trees, but to determine the biodiversity of the redwood forest. Stephen Sillett, who is working with the League on the North Coast lidar project, claims this technology will be useful in directing future efforts to preserve and protect ancient redwood trees.
Geology and soil science
High-resolution digital elevation maps generated by airborne and stationary lidar have led to significant advances in geomorphology. The lidar abilities to detect subtle topographic features such as river terraces and river channel banks, to measure the land-surface elevation beneath the vegetation canopy, to better resolve spatial derivatives of elevation, and to detect elevation changes between repeat surveys have enabled many novel studies of the physical and chemical processes that shape landscapes.In 2005 the Tour Ronde in the Mont Blanc massif became the first high alpine mountain on which lidar was employed to monitor the increasing occurrence of severe rock-fall over large rock faces allegedly caused by climate change and degradation of permafrost at high altitude.
Lidar is also used in structural geology and geophysics as a combination between airborne lidar and GPS for the detection and study of faults, for measuring uplift. The output of the two technologies can produce extremely accurate elevation models for terrain – models that can even measure ground elevation through trees. This combination was used most famously to find the location of the Seattle Fault in Washington, United States. This combination also measures uplift at Mt. St. Helens by using data from before and after the 2004 uplift. Airborne lidar systems monitor glaciers and have the ability to detect subtle amounts of growth or decline. A satellite-based system, the NASA ICESat, includes a lidar sub-system for this purpose. The NASA Airborne Topographic Mapper is also used extensively to monitor glaciers and perform coastal change analysis.
The combination is also used by soil scientists while creating a soil survey. The detailed terrain modeling allows soil scientists to see slope changes and landform breaks which indicate patterns in soil spatial relationships.
Atmosphere
Initially, based on ruby lasers, lidar for meteorological applications was constructed shortly after the invention of the laser and represent one of the first applications of laser technology. Lidar technology has since expanded vastly in capability and lidar systems are used to perform a range of measurements that include profiling clouds, measuring winds, studying aerosols, and quantifying various atmospheric components. Atmospheric components can in turn provide useful information including surface pressure, greenhouse gas emissions, photosynthesis, fires, and humidity. Atmospheric lidars can be either ground-based, airborne or satellite depending on the type of measurement.Atmospheric lidar remote sensing works in two ways –
- by measuring backscatter from the atmosphere, and
- by measuring the scattered reflection off the ground or other hard surface.
Doppler lidar systems are also now beginning to be successfully applied in the renewable energy sector to acquire wind speed, turbulence, wind veer, and wind shear data. Both pulsed and continuous wave systems are being used. Pulsed systems use signal timing to obtain vertical distance resolution, whereas continuous wave systems rely on detector focusing.
The term, eolics, has been proposed to describe the collaborative and interdisciplinary study of wind using computational fluid mechanics simulations and Doppler lidar measurements.
The ground reflection of an airborne lidar gives a measure of surface reflectivity at the lidar wavelength, however, the ground reflection is typically used for making absorption measurements of the atmosphere. "Differential absorption lidar" measurements utilize two or more closely spaced wavelengths to factor out surface reflectivity as well as other transmission losses, since these factors are relatively insensitive to wavelength. When tuned to the appropriate absorption lines of a particular gas, DIAL measurements can be used to determine the concentration of that particular gas in the atmosphere. This is referred to as an Integrated Path Differential Absorption approach, since it is a measure of the integrated absorption along the entire lidar path. IPDA lidars can be either pulsed or CW and typically use two or more wavelengths. IPDA lidars have been used for remote sensing of carbon dioxide and methane.
Synthetic array lidar allows imaging lidar without the need for an array detector. It can be used for imaging Doppler velocimetry, ultra-fast frame rate imaging, as well as for speckle reduction in coherent lidar. An extensive lidar bibliography for atmospheric and hydrospheric applications is given by Grant.
Scheimpflug principle
Another lidar technique for atmospheric remote sensing has emerged. It is based on Scheimpflug principle, referred to as Scheimpflug lidar."The implication of the Scheimpflug principle is that when a laser beam is transmitted into the atmosphere, the backscattering echo of the entire illuminating probe volume is still in focus simultaneously without diminishing the aperture as long as the object plane, image plane and the lens plane intersect with each other". A two dimensional CCD/CMOS camera is used to resolve the backscattering echo of the transmitted laser beam.
Thus as in the case of conventional lidar technologies continuous wave light sources such as diode lasers can be employed for remote sensing instead of using complicated nanosecond pulse light sources. The SLidar system is also a robust and inexpensive system based on compact laser diodes and array detectors.
A continuous-wave NO2 DIAL system based on the Scheimpflug principle has been developed by employing a compact high-power CW multimode 450 nm laser diode as the light source. Laser emissions at the on-line and off-line wavelengths of the NO2 absorption spectrum are implemented by tuning the injection current of the laser diode. Lidar signals are detected by a 45° tilted area CCD image sensor satisfying the Scheimpflug principle. Range-resolved NO2 concentrations on a near-horizontal path are obtained by the NO2 DIAL system in the range of 0.3–3 km and show good agreement with those measured by a conventional air pollution monitoring station. A detection sensitivity of ± 0.9 ppbv at 95% confidence level in the region of 0.3–1 km is achieved with 15-minute averaging and 700 m range resolution during hours of darkness, which allows accurate concentration measurement of ambient NO2. The low-cost and robust DIAL system demonstrated in this work opens up many possibilities for field NO2 remote sensing applications.
Law enforcement
s are used by the police to measure the speed of vehicles for speed limit enforcement purposes. Additionally, it is used in forensics to aid in crime scene investigations. Scans of a scene are taken to record exact details of object placement, blood, and other important information for later review. These scans can also be used to determine bullet trajectory in cases of shootings.Military
Few military applications are known to be in place and are classified, but a considerable amount of research is underway in their use for imaging. Higher resolution systems collect enough detail to identify targets, such as tanks. Examples of military applications of lidar include the Airborne Laser Mine Detection System for counter-mine warfare by Areté Associates.A NATO report evaluated the potential technologies to do stand-off detection for the discrimination of biological warfare agents. The potential technologies evaluated were Long-Wave Infrared, Differential Scattering, and Ultraviolet Laser Induced Fluorescence. The report concluded that : Based upon the results of the lidar systems tested and discussed above, the Task Group recommends that the best option for the near-term application of stand-off detection systems is UV-LIF , however, in the long-term, other techniques such as stand-off Raman spectroscopy may prove to be useful for identification of biological warfare agents.
Short-range compact spectrometric lidar based on Laser-Induced Fluorescence would address the presence of bio-threats in aerosol form over critical indoor, semi-enclosed and outdoor venues such as stadiums, subways, and airports. This near real-time capability would enable rapid detection of a bioaerosol release and allow for timely implementation of measures to protect occupants and minimize the extent of contamination.
The Long-Range Biological Standoff Detection System was developed for the U.S. Army to provide the earliest possible standoff warning of a biological attack. It is an airborne system carried by a helicopter to detect synthetic aerosol clouds containing biological and chemical agents at long range. The LR-BSDS, with a detection range of 30 km or more, was fielded in June 1997.
Five lidar units produced by the German company Sick AG were used for short range detection on Stanley, the autonomous car that won the 2005 DARPA Grand Challenge.
A robotic Boeing AH-6 performed a fully autonomous flight in June 2010, including avoiding obstacles using lidar.
Mining
For the calculation of ore volumes is accomplished by periodic scanning in areas of ore removal, then comparing surface data to the previous scan.Lidar sensors may also be used for obstacle detection and avoidance for robotic mining vehicles such as in the Komatsu Autonomous Haulage System used in Rio Tinto's Mine of the Future.
Physics and astronomy
A worldwide network of observatories uses lidars to measure the distance to reflectors placed on the moon, allowing the position of the moon to be measured with millimeter precision and tests of general relativity to be done. MOLA, the Mars Orbiting Laser Altimeter, used a lidar instrument in a Mars-orbiting satellite to produce a spectacularly precise global topographic survey of the red planet. Laser altimeters produced global elevation models of Mars, the Moon Mercury, NEAR–Shoemaker Laser Rangefinder. Future missions will also include laser altimeter experiments such as the Ganymede Laser Altimeter as part of the Jupiter Icy Moons Explorer mission.In September, 2008, the NASA Phoenix Lander used lidar to detect snow in the atmosphere of Mars.
In atmospheric physics, lidar is used as a remote detection instrument to measure densities of certain constituents of the middle and upper atmosphere, such as potassium, sodium, or molecular nitrogen and oxygen. These measurements can be used to calculate temperatures. Lidar can also be used to measure wind speed and to provide information about vertical distribution of the aerosol particles.
At the JET nuclear fusion research facility, in the UK near Abingdon, Oxfordshire, lidar Thomson Scattering is used to determine Electron Density and Temperature profiles of the plasma.
Rock mechanics
Lidar has been widely used in rock mechanics for rock mass characterization and slope change detection. Some important geomechanical properties from the rock mass can be extracted from the 3-D point clouds obtained by means of the lidar. Some of these properties are:- Discontinuity orientation
- Discontinuity spacing and RQD
- Discontinuity aperture
- Discontinuity persistence
- Discontinuity roughness
- Water infiltration
THOR
THOR is a laser designed toward measuring Earth's atmospheric conditions. The laser enters a cloud cover and measures the thickness of the return halo. The sensor has a fiber optic aperture with a width of 7.5 inches that is used to measure the return light.
Robotics
Lidar technology is being used in robotics for the perception of the environment as well as object classification. The ability of lidar technology to provide three-dimensional elevation maps of the terrain, high precision distance to the ground, and approach velocity can enable safe landing of robotic and manned vehicles with a high degree of precision. Lidar are also widely used in robotics for simultaneous localization and mapping and well integrated into robot simulators. Refer to the Military section above for further examples.Spaceflight
Lidar is increasingly being utilized for rangefinding and orbital element calculation of relative velocity in proximity operations and stationkeeping of spacecraft. Lidar has also been used for atmospheric studies from space. Short pulses of laser light beamed from a spacecraft can reflect off tiny particles in the atmosphere and back to a telescope aligned with the spacecraft laser. By precisely timing the lidar 'echo,' and by measuring how much laser light is received by the telescope, scientists can accurately determine the location, distribution and nature of the particles. The result is a revolutionary new tool for studying constituents in the atmosphere, from cloud droplets to industrial pollutants, which are difficult to detect by other means."Laser altimetry is used to make digital elevation maps of planets, including the Mars Orbital Laser Altimeter mapping of Mars, the Lunar Orbital Laser Altimeter and Lunar Altimeter mapping of the Moon, and the Mercury Laser Altimeter mapping of Mercury.
Surveying
Airborne lidar sensors are used by companies in the remote sensing field. They can be used to create a DTM or DEM ; this is quite a common practice for larger areas as a plane can acquire 3–4 km wide swaths in a single flyover. Greater vertical accuracy of below 50 mm can be achieved with a lower flyover, even in forests, where it is able to give the height of the canopy as well as the ground elevation. Typically, a GNSS receiver configured over a georeferenced control point is needed to link the data in with the WGS.LiDAR are also in use in hydrographic surveying. Depending upon the clarity of the water LiDAR can measure depths from 0.9m to 40m with a vertical accuracy of 15 cm and horizontal accuracy of 2.5m.
Forestry
Lidar systems have also been applied to improve forestry management. Measurements are used to take inventory in forest plots as well as calculate individual tree heights, crown width and crown diameter. Other statistical analysis use lidar data to estimate total plot information such as canopy volume, mean, minimum and maximum heights, and vegetation cover estimates. Aerial LiDAR has been used to map the bush fires in Australia in early 2020. The data was manipulated to view bare earth, and identify healthy and burned vegetation.
Transport
Lidar has been used in the railroad industry to generate asset health reports for asset management and by departments of transportation to assess their road conditions. CivilMaps.com is a leading company in the field. Lidar has been used in adaptive cruise control systems for automobiles. Systems such as those by Siemens, Hella, Ouster and Cepton use a lidar device mounted on the front of the vehicle, such as the bumper, to monitor the distance between the vehicle and any vehicle in front of it. In the event, the vehicle in front slows down or is too close, the ACC applies the brakes to slow the vehicle. When the road ahead is clear, the ACC allows the vehicle to accelerate to a speed preset by the driver. Refer to the Military section above for further examples. A lidar-based device, the Ceilometer is used at airports worldwide to measure the height of clouds on runway approach paths.Wind farm optimization
Lidar can be used to increase the energy output from wind farms by accurately measuring wind speeds and wind turbulence. Experimental lidar systems can be mounted on the nacelle of a wind turbine or integrated into the rotating spinner to measure oncoming horizontal winds, winds in the wake of the wind turbine, and proactively adjust blades to protect components and increase power. Lidar is also used to characterise the incident wind resource for comparison with wind turbine power production to verify the performance of the wind turbine by measuring the wind turbine's power curve. Wind farm optimization can be considered a topic in applied eolics. Another aspect of Lidar in wind related industry is to use computational fluid dynamics over Lidar-scanned surfaces in order to assess the wind potential, which can be used for optimal wind farms placement.Solar photovoltaic deployment optimization
Lidar can also be used to assist planners and developers in optimizing solar photovoltaic systems at the city level by determining appropriate roof tops and for determining shading losses. Recent airborne laser scanning efforts have focused on ways to estimate the amount of solar light hitting vertical building facades, or by incorporating more detailed shading losses by considering the influence from vegetation and larger surrounding terrain.Video games
Recent simulation racing games such as rFactor Pro, iRacing, Assetto Corsa and Project CARS increasingly feature race tracks reproduced from 3-D point clouds acquired through Lidar surveys, resulting in surfaces replicated with centimeter or millimeter precision in the in-game 3-D environment.The 2017 exploration game Scanner Sombre, by Introversion Software, uses Lidar as a fundamental game mechanic.