This blog was written by Clayton Crawford, Product Engineer on the 3D Analyst team in ESRI’s Software Products group in Redlands.
Lidar is promoted as an accurate form of elevation data. To a large degree, this comes from its dense sample interval. Some refer to lidar as ‘painting the ground with measurements’. Indeed, it’s commonplace to get sub-meter sampling. This facilitates canopy penetration which improves the accuracy of ground models in forested areas. The high sample density also improves results for certain applications such as floodplain delineation. That said, lidar is not a silver bullet for all surface modeling activities. Two areas that tend to be problematic are contour derivation and slope analysis.
Messy contours and steep slopes
Getting a computer to produce nice looking contours from traditional data sources is hard enough. It's even harder to produce them from lidar. Contours produced from full resolution lidar tend to be jagged with many twists and turns and isolated closed rings (Figure 1).
Slope assessment with lidar is also problematic. If one examines the average slope from full resolution lidar they will find it’s unusually high. This is true even over relatively flat ground as shown in Figure 2.
Some people misinterpret these issues with contours and slope as simply a result of lidar being more accurate than other forms of surface data. At large enough scales almost any surface is rough, right? While there’s some truth to this a significant part of the problem really stems from the relationship between horizontal sample density and vertical accuracy. As with any measurement technology lidar is not perfectly accurate. Its vertical accuracy generally ranges from 12 to 15cm. That provides a random height difference between two adjacent points of 24 to 30cm. When the points are only one meter or less apart horizontally that height differential becomes significant. From a signal processing perspective this is called high frequency noise.
Techniques to reduce noise
In order to get less jagged contours and more reasonable slope estimates you have to remove the noise from the lidar and lose as little real information as possible while doing so. While there’s no way to prevent any loss, harm can be minimized. The terrain dataset offers a couple of tools to facilitate this. One is intelligent point thinning; the other is a high quality interpolator.
Point thinning occurs during the terrain pyramid building process. Some people think pyramids are strictly a visualization tool, used solely to speed up drawing. While this is true for raster pyramids, the same does not apply to terrains. Terrains were designed knowing that lidar is better suited for some applications when it’s been generalized.
There are two terrain point thinning algorithms: z-tolerance and windowsize. Both have something to offer and are arguably better than random nth point filters used by other solutions that, while fast and reasonable for visualization, are not what you’d want to use for analysis.
The z-tolerance filter employs a TIN based algorithm to find a subset of points sufficient to create a surface that’s within a given vertical distance to the full resolution surface. TerraScan, an industry leading lidar processing package, uses a similar technique for the selection of what it refers to as ‘model key’ points. The use of this filter is recommended when a quantifiable measure of vertical accuracy is needed.
The windowsize filter selects points within a given horizontal sample distance. Every so many units in XY (i.e., the sample window size) the points within that area are examined and one or two are selected depending on which option you chose: the one closest to the mean of the other points in the sample window, the highest or lowest of those points, or both the highest and lowest. As stated earlier, much of the noise problem results from a bad ratio of high horizontal sample density to vertical accuracy. The windowsize filter enables you to improve that ratio. While it’s hard to quantify the accuracy of the thinned data, empirical evidence has shown this approach works well.
Thinning the data
Lidar points are thinned when a terrain dataset is built. The filter algorithm applied is based on the type of pyramid selected for the terrain. Fortunately, the same name is used for pyramid type as filter algorithm so there’s no ambiguity. The pyramid type is selected via the terrain wizard found on the feature dataset context menu in ArcCatalog (Figure 3).
The next step is to specify the pyramid levels. For the sake of noise reduction we’re interested in that first pyramid level which is one step removed from full resolution (Figure 4). A z-tolerance equal to the vertical accuracy of the data is reasonable for that level. This will eliminate as much noise as possible while decreasing the accuracy of the result as little as possible. If you start out with 15cm vertical accuracy, you end up with approximately 30cm accuracy and if you go on to produce contours, set their interval to at least double this, namely 60cm or approximately 2 feet. When building a windowsize pyramid a sample distance equal to twice the nominal point spacing is reasonable.
After having built a terrain, the thinned points reside in its pyramid. You can extract a TIN made from these thinned points via the Terrain to TIN tool and from this generate contours and slope. The results will be less noisy than if made from the un-thinned points. One drawback to this approach is that TINs have a size restriction. Ten million points per TIN is about as large as you can go. With lidar projects this can mean that you have to extract multiple TINs from a single terrain, even when using thinned data, if you’re working on a large area. Another drawback is that contours and slope estimates made from these TINs will still be more angular and discontinuous than necessary. A recommended alternative is to go through a raster.
Note, in ArcGIS 9.4 you will be able to derive contour and slope output directly from terrains without needing to go through TINs.
Contour and slope output can be improved by deriving them from a raster that’s made from a terrain. Engineers have a traditional bias towards working directly on TINs, so they may not like the idea of going through a raster, but when dealing specifically with contour and slope production from lidar this bias is unwarranted. For one thing, lidar points are essentially collected in a random distribution and the resulting triangles are not handpicked or guaranteed to fit some mathematical model made inside a CAD package (e.g., for road design). Additionally, the linear piecewise surface defined by planar triangle faces is not smooth. A smoother surface can be made using the Terrain To Raster tool with the natural neighbors interpolation option. An added benefit of using Terrain To Raster is that you can rasterize an entire terrain in one shot and avoid the size constraint associated with TIN extraction (Figure 5).
You then use the Contour and Slope tools on the derived raster. Figures 6 and 7 illustrate the difference between full resolution and generalized contour and slope derivatives.
Lidar is noisy because of a low vertical accuracy relative to horizontal sample distance. This noise translates into poor quality contours and excessively high average slope rates. Noise can be reduced via point thinning and smoothing without also losing much of the accuracy and detail. The terrain dataset provides these tools via pyramiding and natural neighbor interpolation.
That’s it for this installment of the Lidar Solutions in ArcGIS series. Subscribe to this blog or check back in a month or so for a look at business partner offerings.