digital terrains

Embed Size (px)

Citation preview

  • 7/30/2019 digital terrains

    1/42

    CRC for Spatial Information

    [email protected]

    Report on

    Performance of DEM GenerationTechnologies in Coastal

    Environments

    Clive Fraser and Mehdi RavanbakshCooperative Research Centre for Spatial Information

  • 7/30/2019 digital terrains

    2/42

    2

    Table of ContentsPage

    Executive Summary 3

    1. Introduction 5

    2. Project Overview 5

    3. Overview of DEM Generation Technologies 5

    3.1 Technology Options 5

    3.2 Accuracy Considerations 6

    3.3 LiDAR 7

    3.4 Photogrammetry 8

    3.5 IfSAR 8

    4. Project Work Plan 9

    5. Test Area Locations 11

    6. Specifications for DEM Data Sets 15

    7. Benchmark Elevation Data 16

    7.1 Permanent Survey Marks 16

    7.2 GPS Survey of Height Profiles 16

    7.3 Comparison of GPS and Ground Survey Elevations 16

    7.4 GPS Heighting versus LiDAR DEMs 20

    8. Analysis of Different DEMs Against LiDAR Reference DEM 23

    8.1 Discrepancies in Elevation 23

    8.2 SRTM DEM 26

    8.3 SPOT5 DEM 28

    8.4 Topo DEM (from 1:25,000 map data) 28

    8.5 Airborne IfSAR DEM 28

    8.6 ADS40 DEM 31

    9. Impact of Land Cover on DEM Accuracy 31

    9.1 Urban areas 31

    9.2 Open Rural Areas 35

    9.3 Forest/Bushland Areas 36

    9.4 Mixed Coastal Land Cover 38

    10. Influence of Terrain Slope 39

    11. Conclusions 40

  • 7/30/2019 digital terrains

    3/42

    3

    Executive Summary

    Reliable digital elevation models (DEMs) are vital to better understand and prepare for the

    impacts of sea level rise and storm surges caused by climate change. A number of satellite

    and airborne remote sensing technologies can be used to generate digital elevation models,

    however each technology possesses its own advantages and limitations. The primary aim of

    this project has been to evaluate the performance of different technologies for the generation

    of digital elevation models, specifically in coastal environments. The accuracy characteristics

    of six such technologies have been assessed within four test areas on the mid north coast of

    New South Wales. These test sites were chosen as being representative of low-lying coastal

    zones of differing land cover, topography and geomorphology.

    The DEM technologies investigated were:

    Airborne LiDAR (airborne laser scanning)

    Airborne IfSAR (interferometric synthetic aperture radar)

    SPOT5 HRS satellite imagery

    1-second SRTM-based national DEM

    Aerial photography:o with the DEM sourced from existing 1:25,000 digital topographic mappingo with the DEM derived from recent ADS40 digital imagery

    An objective of the project was to look beyond differences in vertical resolution, cost and

    productivity, and to consider the overall performance of different DEMs in the context of

    fulfilling anticipated requirements for fit-for-purpose elevation data in Australias vulnerable

    coastal zones. Outcomes of the project can be used to inform the development of future

    guidelines covering optimal DEM generation technologies for programs such as UDEM and

    the National Digital Elevation Framework (NEDF).

    Recent forecasts of sea level rise are in the range of 0.5m to upwards of 1m over the

    remainder of this century. Digital elevation modelling in support of prediction andmonitoring of the inundation impacts of sea level rise and storm surges will therefore require

    vertical resolution at the sub-metre and even decimetre level. A principal finding of this

    project has been to reinforce the prevailing view that LiDAR is the optimal DEM generation

    technology for this application. While DEMs produced photogrammetrically from aerial

    imagery can match the vertical resolution of LiDAR, namely around 10cm, they are

    invariably more expensive and exhibit significant shortcomings in bare-earth elevation

    modelling if automated classification and filtering are solely relied upon.

    Beyond highlighting the recognized superiorities of LiDAR, this project has identified DEM

    characteristics that are perhaps not as widely appreciated, but are nevertheless important in

    the context of producing accurate bare-earth DEMs of coastal terrain. One of these concerns

    the accuracy gap between LiDAR DEMs and those derived from airborne IfSAR and aerial

    photography. Comparing LiDAR accuracy (10cm) to IfSAR and ADS40 accuracy (50-

    100cm), one would expect LiDAR DEMs to be at least 5 times better. However the

    difference are accentuated by shortcomings in the automated classification and filtering of

    both vegetation and, to a lesser extent, man-made structures, within the process of producing

    a bare-earth DEM from the latter two technologies. Multiple-return LiDAR on the other hand

    displays significant advantages by way of last-pulse ground definition, which cannot be

  • 7/30/2019 digital terrains

    4/42

    4

    matched in densely vegetated areas by radar and photogrammetry techniques, except through

    skill-intensive and expensive manual editing processes.

    The results obtained for DEM performance in open areas, largely free of trees and buildings,

    highlighted the fact that distinctions in DEM accuracy are as much due to different terrain

    and land cover, and consequently to filtering, as to differences in the basic metric resolution

    of DEM technologies. In the case of open pasture, sub-metre accuracy was obtained for theSRTM DEM while the 1:25,000 mapping, the IfSAR DEM and the aerial imagery DEM all

    displayed sub-half metre accuracy. Although the accuracy of all these lower-resolution

    DEMs exceeded specifications, they are nevertheless still not likely to fulfill requirements for

    fit-for-purpose high-resolution elevation models for decision support and risk analysis

    associated with sea level rise.

  • 7/30/2019 digital terrains

    5/42

    5

    1. Introduction

    This report summarises the objectives, work plan, conduct and outcomes of the project

    Performance of DEM Generation Technologies in Coastal Environments, which formed a

    research project under the Urban Digital Elevation Modelling in High Priority Regions

    Program(UDEM). The focus of the analyses required to evaluate the performance of different

    Digital Elevation Model (DEM) technologies has been upon detailed assessments of the

    heighting accuracy produced by six DEM generation technologies within four test areas

    representing typical low-lying Australian coastal environments, with land cover types

    including urban, rural and forest. Outcomes of the project will provide an increased

    understanding of the characteristics of different elevation data technologies and how well

    they perform in Australian coastal environments. Results will therefore inform the

    development of guidelines covering optimal DEM generation technologies for vulnerable

    coastal zones. Results will also be of benefit to producers of DEMs, particularly to those

    producing elevation models under the National Digital Elevation Framework (NEDF) and

    UDEM.

    2. Project Overview

    The objective of this project has been to investigate the performance of different DEM

    generation technologies within a range of coastal environments. The quality of DEMs is a

    function not only of the data acquisition and subsequent data processing, but also of the

    characteristics of the terrain being mapped, especially in regard to topography and vegetation,

    and to the presence of cities and urban land cover. DEMs produced from different imaging

    and ranging sensors need to be analysed in order to better understand their characteristics and

    accuracy, and also their cost-benefit ratios in relation to producing fit-for-purpose elevation

    models for coastal assessments.

    3. Overview of DEM Generation Technologies

    3.1 Technology Options

    It was initially envisaged that the research would investigate the accuracy capabilities of four

    categories of DEMs/DEM generation technologies, namely:

    Airborne LiDAR (Light Detection and Ranging technology).

    The new national Australian mid-resolution SRTM 1 DEM (derived from the ShuttleRadar Topography Mission IfSAR data).

    The mid-resolution SPOT DEM (derived from approximately 5m resolutionstereoscopic SPOT5 HRS satellite imagery).

    High-resolution airborne IfSAR.

    In addition to comparing DEMs, and in cases DSMs (digital surface models), generated fromthese four data sources, the work plan was extended to also encompass analysis of DEMs

    derived from the following sources:

    Photogrammetrically derived DEMs from digital aerial imagery, and specificallyautomatically produced DEMs from the Leica ADS40 3-line scanning system.

    DEM data from current 1:25,000 topographic mapping, the data having beengenerated over several decades, originally from manual stereo-compilation of analog

  • 7/30/2019 digital terrains

    6/42

    6

    aerial imagery and subsequently from digitisation of the resulting contour maps. This

    elevation model is referred to throughout this report as the Topo DEM.

    Photogrammetrically derived DEMs from 3-line ALOS PRISM satellite imagery.

    IfSAR derived DEMS from the TerraSAR-X/TanDEM-X satellite radar system.

    DEM data from six of the above technologies was successfully sourced for the project.

    LiDAR, ADS40, airborne IfSAR and Topo DEM data was made available by LPMA (NSW

    Dept. of Lands), and Geoscience Australia supplied the SPOT5 and SRTM DEMs. The

    project team was unable to source both ALOS PRISM and TanDEM-X DEM data. In the

    case of tandem-X, the system is still within its initial commissioning phase, with commercial

    operations not anticipated to commence for several months. The ALOS satellite unfortunately

    ceased to operate in late April, 2011, which lessened the imperative to examine this DEM

    data source.

    In the absence of PRISM and TanDEM-X data, it is still possible to infer to some degree the

    overall performance of these two technologies from the results obtained for the SPOT5 HRS

    and airborne IfSAR DEMs, respectively. However, in the IfSAR case, TanDEM-X is

    anticipated to produce vertical accuracies at the 2m level as opposed to the 0.5-1m expected

    accuracy for airborne IfSAR DEMs. Also ALOS PRISM DEMs should display higher overall

    accuracy than those from SPOT5 HRS, since PRISM has double the spatial resolution and 3-

    line scanner geometry.

    Prior to describing the methodology and workflow of the project, it is useful to recall the

    accuracy associated with each DEM technology and salient characteristics of the DEM

    sources considered. These are briefly summarized in the following sections.

    3.2 Accuracy Considerations

    Associated with each DEM technology is an accuracy specification. This is generally

    expressed as a bound, since DEM accuracy is a function of both sensor and topographic/land

    cover characteristics, and in the case of LiDAR and photogrammetry it can vary according toproject design requirements. More will be said in the following paragraphs about the

    accuracy specifications for each of the DEM technologies considered in this investigation,

    but it is initially useful to appreciate the range of accuracy anticipated, this range being

    shown in Figure 1. The figure shows representative 1-sigma accuracy bounds (68%

    confidence level) for each of the six DEM technologies. The bounds shown are indicative

    only, and their purpose is to highlight the order of magnitude and more difference between

    the representative 15cm vertical accuracy of LiDAR and the 8-9m accuracy of the SPOT5

    HRS and SRTM DEMs. The considerable variation in vertical resolution needs to be kept in

    mind when comparing the merits of different DEM generation options.

    Another important aspect related to DEM quality is the presence or absence of height bias,

    which can be local, for example as a consequence of incomplete filtering of above-groundfeatures in the DSM-to-DEM conversion process, or large-area, as a consequence of

    systematic errors in sensor positioning and orientation. Through reference to Figure 1 the

    reader can visualize that whereas a DEM may have high precision, of let us say a standard

    error (1-sigma value) of +/- 15cm, it may be inaccurate by the extent of the bias, which is

    indicated by the dashed line in the figure.

  • 7/30/2019 digital terrains

    7/42

    7

    Figure 1. Representative 1-sigma vertical accuracy bounds for DEM technologies.

    3.3 LiDAR

    Airborne laser scanning or LiDAR is today the clear technology of choice for the generation

    of high-resolution DEMs with post spacings of of 1-3m. The advantages of LiDAR centre

    upon its relatively high-accuracy of generally 10-15cm in height and around 1/2000th

    of the

    flying height in the horizontal, and upon the very high mass point density of nowadays

    around 4 points/m2. This high point density greatly assists in filtering out non-groundartefacts in the conversion from the directly acquired DSM to the final bare-earth DEM.

    Moreover, LiDAR has high productivity of around 300 km2

    of coverage per hour, and it can

    be operated locally, day or night. In practice, data acquisition is generally confined to

    daylight hours since most LiDAR units nowadays come with dedicated digital cameras

    (usually medium format), the resulting imagery being used both to assist in the artefact

    removal process and for orthoimage production.

    One of the most significant attributes of LiDAR is multiple-return sensing, where the first

    return of a pulse indicates the highest point encountered and the last the lowest point. There

    may also be mid pulse returns. As a consequence, LIDAR has the ability to see through all

    but thick vegetation. Whereas it might not be certain from where in the canopy the first pulse

    was reflected, it can be safely assumed that a good number of the last returns will be frombare earth. This greatly simplifies the DSM-to-DEM conversion process in vegetated areas.

    Whereas aerial photogrammetry techniques can yield DSMs of vertical accuracy equivalent

    to LiDAR, it is generally not economical to opt for photogrammetry over LiDAR for DEM

    generation at vertical resolutions of 10-20cm. Thus, in the context of UDEM, LiDAR stands

    alone in most practical respects as the most accurate and comprehensive means to produce

    highest resolution DEMs of coastal environments. For this reason, LiDAR has been chosen in

  • 7/30/2019 digital terrains

    8/42

    8

    this project as the standard against which the other DEM generation technologies are

    compared. In order to quantify the accuracy of LiDAR against ground-truth, elevation data

    from a kinematic GPS survey of several thousand points has been used, along with data from

    permanent survey marks.

    3.4 Photogrammetry

    As a tool for topographic mapping, photogrammetry has a long history. Traditionally

    elevation data was extracted from stereo aerial photography in the form of contours, as

    exemplified in this project by the Topo DEM, which was obtained from 1:25,000 topographic

    map data. DSM generation was automated with the advent of analytical stereoplotters and

    then further process automation accompanied the introduction of digital aerial imagery. The

    generation of a DSM from digital aerial or satellite imagery is today a fully automatic batch

    process, with the resulting elevation model often being employed to support orthoimage

    generation.

    Broad area DEM generation via photogrammetry is presently not the preferred approach,

    except in special circumstances such as very high accuracy DSMs for 3D city modelling. The

    latter is exemplified to some extent by current programs to create high definition,photorealistic models of major cities. For example, one approach employs the Vexcel

    Ultracam digital camera flown in a block configuration of 80% forward overlap and 60% side

    overlap at an imaging scale that yields a 15cm ground sample distance (GSD). DSM and

    subsequently DEMs to around 30cm vertical and 2-3m horizontal resolution can be generated

    with a high degree of automation through such a process. For the present project, DEM data

    generated from 50 cm GSD imagery recorded by LPMAs Leica ADS40 line scanning

    camera to a nominal vertical resolution of 0.5-1m has been adopted as representative of the

    capabilities of fully automated DSM production from digital aerial imagery, followed up with

    initial stage automated DSM-to-DEM conversion. However, it is noteworthy that the final

    stage, manually intensive classification and filtering was not carried out, and thus the ADS40

    data should be thought of as constituting a bare-earth DEM over open terrain, and to some

    extent in urban areas, but only as a smoothed DSM for land cover comprising densevegetation.

    Satellite imaging systems have gained popularity for DSM generation at vertical resolutions

    within the range of 1m to 10m. For example, the GeoEye-1 and World View-1 and -2

    satellites have a 50cm GSD, which will support DSM extraction to around 1-2m vertical

    accuracy. Also, the dedicated DEM generation program of SPOT Image, which uses the

    SPOT 5 HRS system, yields DEMs with a nominal 5-10m height accuracy (1-sigma) and 20-

    30m horizontal resolution. All satellite imaging systems used for 3D terrain modelling use

    line scanner technology, with the 2.5m resolution ALOS PRISM satellite having a 3-line

    scanner geometry similar to that in the ADS40 aerial camera. DEMs produced from ALOS

    PRISM can be expected to display height accuracies of around 3-5m.

    3.5 IfSAR

    Synthetic Aperture Radar (SAR) has been employed for a few decades as an imaging

    technology in remote sensing. Through an augmentation of a conventional airborne or

    spaceborne SAR system with a second receiving antenna, spatially separated from the first, it

    has been possible to utilise the principles of interferometry to extend SAR from a 2D imaging

    system to a 3D topographic modelling technology. The resulting Interferometric Synthetic

    Aperture Radar(IfSAR) system determines the relative heights of imaged ground points as a

  • 7/30/2019 digital terrains

    9/42

    9

    function of the phase difference of the coherently combined signals received at the two

    antennas. The first commercial IfSAR system for DEM generation, the Intermap STAR 3i

    system, appeared in the mid 1990s and global focus was brought onto the capabilities of

    IfSAR to produce DEMs with the successful completion of the Shuttle Radar Topography

    Mission (SRTM) in 2000.

    There have been a number of refinements made to the SRTM DEM of Australia over the pastfew years, to the point where an updated DTED 2 DEM with a post spacing of 1 second

    (30m) and a nominal vertical accuracy in the range of 6-12m has recently been released. This

    new SRTM DEM data was accessed for the current project from Geoscience Australia.

    Beyond the heavily built-up areas of major cities and very rough mountainous areas, the

    Australian terrain can be characterized as being ideal for DEM generation via airborne radar.

    Intermap Technologies have recently completed a large project within the Murray Darling

    Basin with their STAR system and produced DSMs with a stated 0.5 - 1m vertical accuracy,

    and a post spacing of 5m. Moreover, use of stereo radar imagery as a complement to the

    process allows for semi-automated DSM-to-DEM conversion. Airborne IfSAR can record

    data at the very rapid rate of around 100-200 km2

    per minute, which is some 10-20 times the

    area acquisition rate of LiDAR (the IfSAR swath width is generally 8-20km). Moreover, datacollection in not impeded by clouds. Over the past two or three years, there has been a

    considerable upsurge in 1m-accurate DEM generation via IfSAR, with national DEMs being

    commercially available through Intermaps Nextmap product line.

    A second source of radar DEMs is single-pass spaceborne IfSAR. Under the TanDEM-X

    program of Germanys DLR and the Infoterra company, the current TerraSAR-X satellite has

    been joined in space by a second X-band SAR unit. With the orbits of the two satellites being

    tightly controlled, single-pass IfSAR operation is possible, as is vegetation removal using

    new techniques for polarmetric radar interferometry. The intended elevation model product

    from TanDEM-X is a global DTED3 DEM of 12m post spacing and 2m vertical accuracy. As

    at late May 2011, full commercial operation of TanDEM-X had not commenced, though

    initial results from the system are reported as being very encouraging. From the standpoint of

    a DEM generation system that can economically provide 2m accuracy elevation models of

    Australia to horizontal resolutions of 10m, TanDEM-X has considerable potential.

    4. Project Work Plan

    Shown in Figure 2 is the workflow designed for the DEM analysis. Given that a main focus

    of this analysis is upon DEM accuracy, it is useful to keep in mind that the DEMs being

    compared have accuracy ranges that differ by more than an order of magnitude. Recall that

    nominal vertical resolutions of the DEMS are: approximately 5-15m for SRTM and SPOT5

    HRS, 3-5m for the 1:25000 Topo DEM (referred to here as the Topo DEM), 0.51m for

    airborne IfSAR and the ADS40 DEM, and 15cm for LiDAR. Thus, the principal aim of the

    analysis to be conducted is to better characterize the performance of these different DEM

    technologies within a typical Australian coastal environment, rather than to reinforce well-

    recognised differences in resolution and accuracy.

    Also shown in Figure 2 is the work flow adopted for the production of the reference LiDAR

    DEM from the measured mass points. Automated classification and filtering was based upon

    analysis of the multiple-pulse returns, after which interpolation was adopted to generate the

    final grid of 2m horizontal spacing.

  • 7/30/2019 digital terrains

    10/42

    10

    As can be seen from Figure 2, the initial step in the accuracy assessment and analysis of

    differently sourced DEMs of varying resolution against the reference dataset, which is taken

    to be LiDAR data, involves bringing all DEM datasets into a uniform reference coordinate

    system. Especially important is uniformity within the height datum. All current DEM data

    acquisition technologies utilize GPS for absolute positioning and consequently the DEM

    datum is initially referenced to the WGS84 ellipsoid. A height conversion from ellipsoidal toorthometric is then carried out using both geoid height information from AusGEOID09 and,

    where applicable and if known, the local relationship between AusGEOID09 and the

    Australian Height Datum (AHD) to facilitate a transformation of the DEM to AHD. In the

    conversion of height data recorded in the kinematic GPS survey conducted as part of the

    project, AusGEOID09 was employed to facilitate a one-step WGS84-to-AHD reference

    datum conversion.

    It is noteworthy that there can be discrepancies in actual local MSL and AHD amounting to

    70cm or more as a consequence of sea-surface topography. However, localized distortions in

    AHD will have no significant impact in the accuracy analysis for two reasons. Firstly, height

    differences are being determined, which nullifies the effect of absolute biases in the datum, at

    least when all DEMs are nominally referenced to AHD. Secondly, the anticipated localizedMSL versus AHD biases can be anticipated to be very small in relation to the overall error

    budget for all DEM data other than the LiDAR reference data.

    Figure 2. Project workflow.

    In order to compare height values from different DEMs at specific positions, interpolation is

    needed because of the multiple horizontal resolutions (post spacings) involved. Within the

    current project the principle adopted is that the interpolation should occur in the higher

  • 7/30/2019 digital terrains

    11/42

    11

    resolution DEM. Thus, height comparisons require interpolation within the 2m horizontal

    resolution LiDAR DEM, this interpolation being bilinear as opposed to bicubic, in order to

    minimize smoothing effects. A result of this approach is that the number of sample points

    will vary proportionally to the horizontal resolution of the DEM being compared to the

    LiDAR reference data.

    In accordance with the different height resolutions of the DEMs being considered, differentlevels of initial artefact removal and filtering have been applied in the DSM-to-DEM

    conversion, with automated processes alone being largely relied upon. It is important to keep

    in mind that the characteristics of both the underlying terrain and the particular sensor

    technology will dictate the degree of complexity of the DSM-to-DEM process. Issues

    include, for example, the fact that photogrammetry techniques beneficially support manual

    artefact removal in a visual 3D environment, whereas removal of above ground features in

    LiDAR DSMs is greatly aided by both the high density and vertical resolution of the mass

    points and the provision of multiple returns (ranges) which allow penetration of the

    vegetation layer. Also, IfSAR DSMs can be accompanied by intensity images that support

    stereo visual interpretation to aid in the DSM-to-DEM conversion.

    5. Test Area Locations

    Two criteria governed selection of geographic location for the DEM analysis: 1) suitability in

    the context of overall assessment of coastal zone vulnerability to climate change; and 2)

    availability of elevation model coverage from as many data acquisition sources as possible.

    Fulfilment of the latter criterion turned out to be the factor that most influenced the selection

    of test area locations, since the choice was essentially limited to the mid north coast of NSW,

    where there had been recent production of medium- and high-resolution DEMs from airborne

    IfSAR, LiDAR and photogrammetry (from ADS40 aerial imagery). Moreover, there was

    coverage from SRTM, 1:25000 topographic mapping and SPOT5 HRS.

    Following the selection of the general test area based on data availability, it was necessary to

    select specific test sites, which in combination fulfilled the following requirements:

    1) Coastal zone with mixed vegetation, ranging from grassland to scrubland and forest.

    2) Topographic variation, ranging from floodplains, to undulating low-level coastal sanddunes to low- and medium level hills.

    3) Variation in landcover, from urban to rural to bushland and forests.

    4) Containing extensive areas below 10m elevation and open to the coastline.

    A principal aim of the project was to assess the influence of both man-made structures in an

    urban environment, and different land and vegetation cover, on the accuracy and integrity of

    bare-earth DEMs. Although there have been a number of published reports on the

    performance of different DEM generation techniques in different topography, especially as afunction of ground slope, this factor has only been briefly analysed here. The reasons for this

    are, firstly, that by its very nature the vulnerable coastal zone is low-lying, with only mild

    topographic variation; and, secondly, the metadata necessary to comprehensively consider

    slope and aspect for the IfSAR, ADS40 and LiDAR were not available. The analysis was thus

    limited to gridded DEM data only.

    Shown in Figure 3 are the four selected test areas: Area 1 (128 km2) extends from South West

    Rocks to the Stuarts Point/Grassy Head area and comprises varied coastal topography and

  • 7/30/2019 digital terrains

    12/42

    12

    vegetation cover. Area 2 (76 km2), which is centred on the town of Kempsey, constitutes the

    sample low-lying urban area. Area 3 (24 km2) covers Crescent Head and this was selected

    based on the varying terrain of the headland. Area 4 (72 km2

    ) was added to the initial three in

    order to provide further coverage of dense coastal forest areas, as well as an additional urban

    area, namely the settlement of Scotts Head. The DEMs within each of the test areas are

    shown in Figure 4, and Figure 5 highlights the areas below 10m elevation within each of the

    four test sites.

    Figure 3: Test areas, with locations shown for 9 permanent survey marks used as GPS checkpoints.

    Area 1

    Area 3

    Area 2

    Area 4

    10 km

  • 7/30/2019 digital terrains

    13/42

    13

    (a) Area 1 (b) Area 2

    (c) Area 3 (d) Area 4

    Figure 4: LiDAR DEMs for each test area.

  • 7/30/2019 digital terrains

    14/42

    14

    (a) Area 1 (b) Area 2

    (c) Area 3 (d) Area 4

    Figure 5: Areas below 10m elevation (black areas are >10m or outside area).

  • 7/30/2019 digital terrains

    15/42

    15

    6. Specifications of DEM Datasets

    Shown in Table 1 is a summary of the specification for the different DEM datasets employed

    in the project. With the exception of the 1-second SRTM and SPOT5 data, which were made

    available by Geoscience Australia (GA) , all DEM data was kindly provided to the project by

    the Land and Property Management Authority (LPMA) of the NSW Department of Lands.

    The project is indebted to LPMA and GA for this support, which was crucial to realization of

    the project objectives.

    Table 1. Specifications of DEM Datasets and GPS survey data.

    Dataset Technology Data format

    Horizontal

    accuracy

    (RMSExy)

    Vertical

    accuracy

    (RMSEz)

    SPOT5 DEMSpace

    photogrammetry

    ESRI binary

    30m grid

    (.ADF)

    10m 5-10m

    ADS40 DSM

    Aerial

    photogrammetry

    (50 cm GSD)

    ERDAS

    8 m grid

    (.IMG)

    0.5m 0.5-1m

    Airborne IfSAR

    DEM

    Intermap STAR 3

    & 4 IfSAR

    5 m grid

    (.BIL)1.5m 0.5 - 1m

    SRTM DEM Space-borne IfSAR

    ESRI binary

    30m grid

    (.ADF)

    7m 6-12m

    LiDAR mass pointsAirborne Laser

    Scanning

    2m grid

    (.LAS)0.3m 0.15m

    Topo DEM

    Aerial

    photogrammetry,

    1:25000 mapping

    ERDAS 25m

    Grid (.IMG)6m 3m

    Ground check points Kinematic GPS ASCII 0.03m 0.03m

    7. Benchmark Elevation Data

    7.1 Permanent Survey Marks

    The reference elevation model against which DEMs from different data sources are compared

    is taken as the LiDAR DEM. In order to assess the quality of the LiDAR standard against

    ground survey data that is directly referenced to AHD to a nominal accuracy of better than

    10cm, surveyed benchmark data within the test area was accessed. The elevations of nine

    benchmarks, the locations of which are indicated in Figure 3, were used in a comparison of

    GPS-derived AHD heights versus those of the permanent survey marks. It had originally been

    intended to employ additional benchmarks as ground checkpoints, however time constraintsand difficulties imposed in locating the permanent survey marks beyond township areas

    meant that the number of checkpoints was restricted to nine. This number would be sufficient

    to indicate the presence of any localized biases in the AHD reference system that were not

    modeled via the AusGEOID09 Geoid model.

  • 7/30/2019 digital terrains

    16/42

    16

    7.2 GPS Survey of Height Profiles

    Prior to the adoption of airborne LiDAR as the highest accuracy master elevation data set

    against which other DEM generation technologies are compared, it was necessary to validate

    the absolute accuracy of the LiDAR DEM. This is by no means a simple matter in practise,

    since the only available basis for comparison is elevation data acquired from ground surveys,

    either via GPS or standard surveying techniques of spirit or trigonometric levelling. As willbe explained in a following section, there are practical limitations to utilization of thinly

    distributed benchmark and permanent survey mark data as an accurate base against which to

    assess LiDAR DEMs. Not only are there uncertainties of several cm in the height relationship

    between the ellipsoidal WGS84 and AHD reference systems, but there is also the inherent

    accuracy limitation, again several cm, of the ground surveyed elevations.

    The only feasible approach for assessing the absolute accuracy of LiDAR DEM data covering

    the UDEM test areas is through the provision of GPS surveyed bare-earth elevations. The

    most practical way of acquiring such data is through the use of real-time kinematic GPS

    (RTKGPS) surveying where a GPS receiver is mounted in a vehicle and 3D positions to an

    accuracy of a few cm are determined through the use of either a nearby radio-linked base

    station or a CORS network. For the present project, RTKGPS surveys were conducted in five

    areas: Scotts Head, Stuarts Point, South West Rocks, Kempsey and Crescent Head.

    The surveyed height profiles were mainly restricted to areas in or near townships, for two

    reasons. Firstly, the mode of operation was to utilize a base station that broadcast corrections

    to the vehicle-borne roving receiver via a radio link, and the effective maximum distance for

    radio reception was about 4km depending upon topography. Secondly, beyond townships,

    roads tended to be covered by overhanging trees, which blocked reception of the GPS

    signals. This accounts for most of the broken height profiles shown in Figures 6-11.

    Notwithstanding these shortcomings, some 27,000 elevation readings at generally 3-5m

    intervals were made to 2-4cm accuracy over the roads indicated in the figures. One benefit of

    being restricted to open roadways was that heights to the same points would have been

    readily recorded within the LiDAR survey. An illustration of the problems posed by

    vegetation in the RTKGPS surveys is provided in Figure 12, which shows favourable and

    unfavourable areas for data collection within the Stuarts Point area. The vegetation cover is

    indicative of most of the native forest areas within the region, with only the low-lying test

    area around Kempsey (Area 2) being largely free of forest cover.

    7.3 Comparison of GPS and Ground Survey Elevations

    In order to ascertain the absolute accuracy of the LiDAR DEM data, comparisons were to be

    made with the elevation data recorded within the vehicle-borne Real-Time Kinematic GPS

    (RTKGPS) Survey. Both technologies yield elevations, in the first instance, within an

    ellipsoidal height reference system, namely WGS84. In this sense, discrepancies between the

    RTKGPS heights and those determined from LiDAR yield an indication of the accuracy ofthe LiDAR system free of the effects of uncertainty in the relationship between the ellipsoidal

    and the orthometric height datums. For the conversion of both LiDAR and GPS surveyed

    heights to elevations referenced to AHD, it is necessary to apply a geoid correction, in this

    case via the AUSGeoid09 correction model. GPS surveyed AHD heights can then be directly

    compared to elevations of benchmarks (BMs) and permanent survey marks (PMs), which

    have traditionally been established via spirit leveling.

  • 7/30/2019 digital terrains

    17/42

    17

    Figure 6. Elevation profiles recorded by real-time kinematic GPS in the four test areas.

    Figure 7. Elevation profiles recorded by kinematic GPS in Scotts Head (Area 4).

    Scotts Head

    SW Rocks

    Stuarts Point

    Kempsey

    Crescent Head

  • 7/30/2019 digital terrains

    18/42

    18

    Figure 8. Elevation profiles recorded by kinematic GPS in Stuarts Point (Area 4).

    Figure 9. Elevation profiles recorded by kinematic GPS in South West Rocks (Area 1).

    Figure 10. Elevation profiles recorded by kinematic GPS in Kempsey (Area 2).

  • 7/30/2019 digital terrains

    19/42

    19

    Figure 11. Elevation profiles recorded by kinematic GPS in Crescent Head (Area 3).

    Figure 12. Constraints on kinematic GPS surveying: unfavourable vegetation conditions (left) and generally

    favourable conditions (right).

    A principal cause of discrepancies between GPS surveyed AHD heights and those for

    BMs/PMs can be anticipated to be localized biases in Geoid modeling. In the case of the test

    areas considered, the geoid correction value N varies by 1.1m over the 50km from Crescent

    Head to Scotts Head, from 30.7m to 31.8m, and by 0.4m over the 20km from Crescent Head

    to Kempsey. This fact, coupled with the anticipated accuracy (95% confidence) level of only

    5-8cm for ground surveyed BM/PM elevations suggests that RMS discrepancies in the order

    of 10cm might well be expected between GPS and ground surveyed elevations.

    Table 2 lists the results of the GPS to BM/PM height data comparison for nine survey marks

    in the Kempsey, Scotts Head and Crescent Head areas. The overall RMS value of height

    discrepancies is 9.4cm and it is noteworthy that there is a systematic trend in the discrepancy

    values at two of the locations. These are indicative of either one or two factors: firstly,

    localized biases in AUSGeoid09 or, secondly, systematic errors in the BM/PM data. Either

    way, the results suggest that in order to independently ascertain the accuracy of the LiDAR

    data, it is more appropriate to use RTK GPS heights rather than benchmark data.

  • 7/30/2019 digital terrains

    20/42

    20

    For the purposes of this study it suffices to note that the level of agreement between BM and

    PM data and RTKGPS is of a similar magnitude to the 1-sigma elevation accuracy

    anticipated from airborne LiDAR data, namely 10-15cm. Subsequent comparisons of the

    LiDAR DEM heights to ground surveyed data will utilize only RTKGPS data.

    Table 2. Comparison between GPS surveyed and published elevations for nine benchmarks/permanent survey

    marks (units are metres).

    7.4 GPS Heighting versus LiDAR DEMsThe LiDAR DEM has been adopted as the reference DEM in view of its significantly higher

    accuracy, and generally also resolution, as compared to the other DEM generation

    technologies. In order to validate, as far as was practical, the absolute accuracy of the LiDAR

    derived elevations, a comparison with the profiles of RTKGPS data described above was

    conducted. Within this process, some 27,000 individual RTKGPS height measurements were

    compared to elevations interpolated from the gridded LiDAR DEM via bilinear interpolation.

    The resulting discrepancies in elevation are summarized in Tables 3 and 4, where the

    heighting bias of LiDAR (-ve value indicates higher LiDAR elevation), the RMS

    discrepancy, the bias-free standard deviation of the discrepancies H and the size of the

    sample within each of the four test areas is listed. Table 3 shows results when all RTKGPS

    points are included, whereas Table 4 lists the corresponding results when height discrepancy

    values of greater than three times the standard deviation (ie 99% confidence level) of H

    values are omitted.

    In assessing the heighting discrepancies between the RTKGPS and corresponding points

    from the LiDAR DEM, it should be kept in mind that given the 2-3cm accuracy of the laser

    ranging component, and the fact that both data sets were transformed from ellipsoidal to

    orthometric heights via the AusGeoid09 geoid model, elevation differences will primarily be

    a function of:

    GPS

    PointPM

    Ellipsoidal

    Height

    from GPS

    Geoid

    Separation

    Orthometric

    height

    H from

    Levelling

    Height of

    PM from

    GPS

    True

    height of

    PM

    Error in

    Height

    (m)

    Kempsey

    GPS06 PM25886 54.24 31.06 23.18 0.70 23.88 23.89 -0.01

    GPS10 PM25983 39.92 31.12 8.80 -0.77 8.03 8.02 0.01

    GPS16 PM26032 41.36 31.08 10.28 -0.35 9.93 9.86 0.07

    Scotts Head

    GPS10 PM56083 36.39 31.78 4.61 1.46 6.07 6.11 -0.04

    GPS12 PM93242 61.63 31.79 29.84 -0.19 29.65 29.77 -0.12

    GPS13 PM72384 45.40 31.77 13.63 0.10 13.73 13.77 -0.04

    Crescent Head

    GPS11 PM12869 41.66 30.75 10.91 0.04 10.95 11.07 -0.12

    GPS14 PM12867 34.39 30.76 3.63 -0.09 3.54 3.64 -0.10

    GPS17 PM12884 120.78 30.74 90.04 1.61 91.65 91.83 -0.18

  • 7/30/2019 digital terrains

    21/42

    21

    discrepancies in the GPS surveying of platform positions, airborne and terrestrial; and

    errors in the filtering of the LiDAR data, ie in the removal of above bare-ground features.

    In many respects the comparison of elevations along roadways would be expected to yield

    optimal results, since the filtering issue is minimized. However, in the case of the test areas

    considered, there were instances were roadside vegetation appeared to influence localized

    filtering results. This issue will be addressed following a general summary of the results ofthe RTKGPS versus LiDAR comparison.

    Table 3. Comparison between RTK GPS surveyed elevations and those from the LiDAR DEM; all GPS

    points included (Units are metres).

    Table 4. Comparison between RTK GPS surveyed elevations and those from the LiDAR DEM; GPS points

    where H is greater than 3 times the standard deviation are omitted (Units are metres).

    In the context of validating the LiDAR DEM via RTKGPS data, the results listed in Tables 3

    and 4 are quite encouraging for two of the test areas, Areas 1 and 2, where neither the bias

    value nor the standard deviation of height discrepancies is significant given the 1-sigma

    accuracy of the LiDAR of around 15cm. However, the level of compatibility is less than

    expected within the remaining two areas, in Area 3 because of a higher than expected positive

    height bias for the LiDAR, and in Area 4 because of a high RMS discrepancy value. It is also

    noteworthy in Table 4 that, for the Stuarts Point and Kempsey test fields, only 1% of

    discrepancy values fell outside 3-sigma error bounds, which is consistent with a normal

    distribution. The corresponding figures for rejected points (H >3 in Areas 3 and 4 are

    much higher at 11% and 8%, respectively.

    It is difficult to definitively establish the reasons for the larger mean LiDAR heighting bias in

    Crescent Head, though preliminary analysis suggests that it may in fact be due to a

    combination of both errors in the LiDAR DEM and lower than expected accuracy within the

    RTKGPS data. Shown in Figure 13 are plots of the positions of RTKGPS points, with the

    height discrepancy at each point being indicated by a coloured dot. White indicates within 1-

    standard deviation ofH (ie within 1-sigma), blue between 1- and 2-sigma, green between 2-

    and 3-sigma, and red greater than 3-sigma, the sigma values being those listed in Table 4.

    Test Area

    Mean elevation

    discrepancy

    (heighting bias)

    RMS elevation

    discrepancy

    Std. deviation

    of HNo. of points

    1, Stuarts Pt 0.06 0.12 0.10 4880

    2, Kempsey 0.02 0.12 0.11 10188

    3, Crescent Hd -0.14 0.16 0.07 5740

    4, Scotts Hd -0.09 0.24 0.22 6311

    Test Area

    Mean elevation

    discrepancy

    (heighting bias)

    RMS elevation

    discrepancy

    Std. deviation

    of HNo. of points

    % of points

    removed

    (H >3

    1, Stuarts Pt 0.06 0.10 0.07 4842 1%

    2, Kempsey 0.02 0.04 0.04 10130 1%

    3, Crescent Hd -0.13 0.14 0.05 5097 11%

    4, Scotts Hd -0.04 0.12 0.12 5831 8%

  • 7/30/2019 digital terrains

    22/42

    22

    Note in the upper two of the three images how GPS errors are suggested by distinctly

    different H values being obtained in overlapping runs of the vehicle borne GPS survey. This

    is particularly apparent in the right-hand image covering a road roundabout. On the other

    hand, the lower image of Figure 13 shows systematic error in a double run along the edge of

    what is essentially a cliff face, and here one could infer that the heighting error is more likely

    to have arisen within the LiDAR processing.

    Figure 13. Sample discrepancies in LiDAR DEM versus RTKGPS elevation data, Crescent Head. White

    indicates within 1-sigma; blue, 1-2 sigma; green, 2-3 sigma; and red, >3 sigma.

    A further example of where the height discrepancies are more likely attributable to

    shortcomings in LiDAR classification and filtering is shown in Figure 14. Note how the

    discrepancies increase for a double-run RTKGPS survey exactly at the transition between an

    open urban area and a heavily forested area. The elevation cross section through the LiDAR

    DEM, at the position indicated by the yellow line, is also shown. A final example, which

    needs no explanation, is indicated by Figure 15. This shows the error arising when the

    vehicle borne GPS crosses a railway bridge, some 4-5m above the underlying DEM.

    It can be difficult to accurately attribute errors in the determination of absolute elevation to

    the LiDAR DEM versus the RTKGPS data. However, there is the consolation in this

    investigation that height discrepancies are of a sufficiently small magnitude where they are

    consistent overall with the 1-sigma vertical accuracy specification of around 15cm for the

    LiDAR DEM. Given that the next highest resolution DEM to be considered has a nominal

    vertical accuracy of 50cm, the LiDAR DEM can be safely taken as the benchmark against

  • 7/30/2019 digital terrains

    23/42

    23

    which to assess the remaining DEM generation technologies. Notwithstanding the acceptance

    of this benchmark status, the RTKGPS versus LiDAR DEM analysis has highlighted

    practical issues that still hinder the acquisition of DEMs with vertical accuracies of better

    than, say, 10cm. This analysis has indicated that remaining shortcomings in the DSM-to-

    DEM conversion for LiDAR data are most apparent in the classification and filtering of

    vegetation as opposed to man-made, above-ground structures such as buildings.

    Figure 14. Sample discrepancies in LiDAR DEM versus RTKGPS elevation data, South West Rocks. White

    indicates within 1-sigma; and blue 1-2 sigma. Also shown is the cross section height profile corresponding to

    the yellow line.

    Figure 15. Discrepancies in LiDAR DEM versus RTKGPS elevation data at bridge crossing, Kempsey. White

    indicates within 1-sigma; and red greater than 3-sigma.

    8. Analysis of Different DEMs against LiDAR Reference DEM

    8.1 Discrepancies in Elevation

    Shown in Tables 5 and 6 are results from initial comparisons of DEMs against the LiDARstandard, for each different data acquisition technology investigated. The areas of

    comparison have been restricted to those indicated in Figure 5, ie to areas with an elevation

    of 10m or less, which are deemed most vulnerable to the impact of rising sea level and storm

    surges. The results represent an initial summary of overall accuracy in these regions, as

    quantified by both the Root Mean Square height discrepancy/Error value (RMSE) and the

    estimated standard error (h), both being relative to the LiDAR DEM. The distinction

    between these two measures is that the RMSE includes the error arising from systematic

    height biases, whereas the h is free of the overall mean bias. Thus, h will always be equal

  • 7/30/2019 digital terrains

    24/42

    24

    to or smaller than the RMSE, with the two estimates being equal when there is no mean

    height bias.

    Table 5. Accuracy evaluation result against LiDAR derived reference DEM. Only height differences below

    listed thresholds were included and those above removed, as per the %-removed column.

    DatasetHeight

    bias (m)

    RMSE

    (m)h(m)

    Sample Size % removed

    SRTM DEM (Area 1,Threshold=15m)

    0.5 3.4 3.4 85740 0.06

    SRTM DEM (Area 2,

    Threshold=15m)-0.6 2.2 2.1 51625 0.03

    SRTM DEM (Area 3,

    Threshold=15m)1.9 4.1 3.6 13075 0

    SRTM DEM (Area 4,

    Threshold=15m)2.4 4.3 3.6 20927 0.8

    SPOT5 DEM (Area 1,

    Threshold=15m)4.3 5.1 2.8 79077 7.8

    SPOT5 DEM (Area 2,

    Threshold=15m)4.3 4.7 1.9 51443 0.4

    SPOT5 DEM (Area 3,Threshold=15m)

    4.8 5.5 2.8 12359 5.5

    SPOT5 DEM (Area 4,

    Threshold=15m) 5.3 6.0 3.0 18962 10.1Topo DEM (Area 1,

    Threshold=10m)0.8 2.3 2.2 123454 0.01

    Topo DEM (Area 2,Threshold=10m)

    2.0 3.3 2.7 74354 0.02

    Topo DEM (Area 3,Threshold=10m)

    2.5 3.2 1.9 18906 0.1

    Topo DEM (Area 4,Threshold=10m)

    1.4 2.6 2.2 30377 0.2

    IfSAR DEM (Area 1,

    Threshold=5m)0.0 1.4 1.4 3038739 2.0

    IfSAR DEM (Area 2,Threshold=5m)

    0.1 0.8 0.8 1855859 0.3

    IfSAR DEM (Area 3,

    Threshold=5m)0.4 1.1 1.0 473092 0.7

    IfSAR DEM (Area 4,Threshold=5m)

    0.3 1.5 1.5 701487 8.7

    ADS40 DSM (Area 1,Threshold=5m)

    0.8 1.9 1.8 852493 29.5

    ADS40 DSM (Area 2,

    Threshold=5m)0.3 0.9 0.9 695044 4.4

    ADS40 DSM (Area 3,

    Threshold=5m)0.9 1.7 1.4 116038 37.6

    ADS40 DSM (Area 4,

    Threshold=5m)0.6 1.6 1.5 179993 39.9

    The distinction between Tables 5 and 6 lies in the adopted threshold for classification of

    particular height discrepancy values as outliers, or gross errors. These are removed from the

    computation of the RMSE and standard deviation values. The outlier thresholds (cut-off

    values) in Table 6 impose a tighter tolerance on data acceptance than those of Table 5, and

    the different threshold values afford an indication of the extent of noise within each DEM

    data set. The cut-off height discrepancy values in Table 5 were set at 15m for SRTM and

    SPOT5 data, 10m for the Topo DEM and 5m for both the IfSAR and ADS40 DEMs. These

    values correspond roughly to multiples of three to five times the respective standard

    deviations. The area that was most noise-free, as expected, was Area 2 and the ADS40 DEM

    constituted the noisiest data. Some 40% of ADS40 data points in Area 4 were classed as

  • 7/30/2019 digital terrains

    25/42

    25

    outliers, which is no doubt attributable to incomplete classification and filtering within

    forested areas.

    Table 6. Accuracy evaluation result against LiDAR derived reference DEM. Only height differences below

    listed thresholds were included and those above removed, as per the %-removed column.

    Dataset Heightbias (m)

    RMSE(m)

    h(m)

    Sample Size % removed

    SRTM DEM (Area 1,

    Threshold=10m)0.4 3.2 3.2 84665 1.3

    SRTM DEM (Area 2,

    Threshold=10m)-0.6 2.2 2.1 51508 0.3

    SRTM DEM (Area 3,Threshold=10m)

    1.7 3.7 3.3 12735 2.6

    SRTM DEM (Area 4,Threshold=10m)

    1.9 3.5 3.0 19908 5.6

    SPOT5 DEM (Area 1,Threshold=10m)

    4.0 4.6 2.3 75763 11.7

    SPOT5 DEM (Area 2,

    Threshold=10m)4.2 4.5 1.7 50700 1.8

    SPOT5 DEM (Area 3,

    Threshold=10m) 4.3 4.8 2.1 11655 10.9SPOT5 DEM (Area 4,

    Threshold=10m)4.6 5.1 2.1 17325 17.8

    Topo DEM (Area 1,

    Threshold=5m)0.7 2.2 2.0 119997 2.8

    Topo DEM (Area 2,

    Threshold=5m)1.3 2.5 2.2 62730 15.7

    Topo DEM (Area 3,

    Threshold=5m)2.3 2.8 1.6 17461 7.7

    Topo DEM (Area 4,

    Threshold=5m)1.1 2.2 1.9 28473 6.5

    IfSAR DEM (Area 1,

    Threshold=3m)0.0 1.1 1.1 2889079 6.8

    IfSAR DEM (Area 2,

    Threshold=3m)0.1 0.7 0.7 1828944 1.7

    IfSAR DEM (Area 3,Threshold=3m)

    0.4 0.9 0.8 458564 3.7

    IfSAR DEM (Area 4,

    Threshold=3m)0.1 1.1 1.1 651056 15.2

    ADS40 DSM (Area 1,

    Threshold=3m)0.4 1.3 1.2 723999 40.1

    ADS40 DSM (Area 2,

    Threshold=3m)0.3 0.8 0.8 683764 6.0

    ADS40 DSM (Area 3,

    Threshold=3m)0.6 1.1 0.9 103057 44.5

    ADS40 DSM (Area 4,

    Threshold=3m)0.3 1.0 1.0 160370 46.4

    A feature to note is that due to the restriction of the analysis to elevations of less than 10m,

    there is limited initial consideration of DEM performance within urban environments, sincemost of the town of Kempsey, as well as significant parts Southwest Rocks, Crescent Head

    and Scotts Head, all lie at elevations above 10m. DEM performance in urban areas will be

    addressed in a later section of this report.

    The results in Tables 5 and 6, coupled with the plots in Figures 16-20 showing height

    discrepancies above given thresholds, reveal a number of characteristics, some unique to

    particular DEM data acquisition technologies and others common to all. In the latter

    category, findings could be briefly summarizes as follows:

  • 7/30/2019 digital terrains

    26/42

    26

    The accuracy associated with each DEM technology, as assessed via the RMSE and

    h values was basically consistent with or better than suggested by specifications. In

    the case of the SRTM data the RMSE values of around 2 - 4m were significantly

    lower than anticipated, whereas the standard error of the SPOT5 DEM displayed

    lower than expected standard error values of 2 - 3m, but a disturbing, persistent height

    bias of close to 5m. The accuracy of the Topo DEM was close to specifications,namely around 3m, whereas the IfSAR and ADS40 DEMs displayed an accuracy

    level in the range of 0.7m to 1.5m, which is equal to or slightly below expectations.

    As anticipated, both heighting biases and height RMSE values are generally larger forAreas 1, 3 and 4 than for Area 2. The lack of forest cover in the extensive open

    floodplain area around Kempsey accounts to a large degree for this characteristic,

    since the positive bias effect of the DEM being in reality more of a canopy DSM in

    forest areas is absent. This enhances the prospect for a better fit to the bare-earth

    LiDAR DEM. It can be seen that the bias and RMSE values follow this trend for the

    SRTM, IfSAR and ADS40 DEMs, but not for the SPOT5 and Topo DEMs. In the

    case of the SPOT5 DEM there is a relatively uniform bias of 4-5m across all three

    areas, with corresponding uniform RMSE values of 4.5-5.5m.

    Also as anticipated, the distribution of RMSE values and standard errors for each caseare correlated to the presence or absence of forest. There should be an expectation that

    automated DSM-to-DEM conversion will yield better results for IfSAR versus

    photogrammetrically derived DEMs generated through image matching because of

    the ability of radar to penetrate vegetation, at least to a moderate extent. It is

    noteworthy that the mean biases for the SRTM and airborne IfSAR DEMs are 0.9m

    and 0.3m, respectively. In the case of the Topo DEM, where extensive manual

    filtering has been carried out, the systematic errors in DEM heights, although

    influenced by the presence of forest, tend to be concentrated in a small number of

    areas, as opposed to being distributed widely throughout forested regions.

    Based on results obtained in the foregoing analysis, as summarized in Tables 5 and 6, the

    following general summaries of DEM accuracy can be offered:

    8.2 SRTM DEM

    When assessed against the basic accuracy specifications for the 1-second SRTM DEM, the

    achieved RMSE, standard error (1-sigma) and mean height bias values are very impressive.

    Instead of finding an RMSE in the range of 6-12m, the values instead range from 2.2m for

    Area 2 to 4.3m for the heavily forested Area 4. The corresponding 1-sigma values are 2.1m

    and 3.6m. The number of points with height discrepancies exceeding 10m (roughly 3-sigma)

    reaches 5.6% in the worst case (Area 4) and 0.3% in the best (Area 2). At a 15m or approx. 5-

    sigma threshold the number of rejected points falls below 1%. A further encouraging feature

    of the SRTM DEM, which can be seen for Areas 1, 3 and 4 in Figure 16, is that the

    distribution of height discrepancies exceeding the 10m threshold is characterized by

    concentrations in a few, mainly forested locations, with the majority of the area being free

    from rejected points. It is also noteworthy that there is a concentration of outlier points both

    within vegetated valley areas, which increase with increasing elevation, and along two

    watercourses. Heighting blunders exceeding 15m are confined to a small number of local

    vegetation clusters in Area 4. The main conclusion regarding the GA-supplied 1-second

  • 7/30/2019 digital terrains

    27/42

    27

    SRTM DEM is that within the coastal areas considered it is more accurate than specifications

    would suggest, and it is free of significant height biases when assessed against RMSE values.

    Area 1 Area 2

    Area 3 Area4

    Figure 16: Points within the SRTM DEM with height discrepancies greater than threshold values when

    compared to the LiDAR reference DEM. Red areas representing a 10m threshold are overlaid by blue areas

    representing a 15m cutoff (areas not to scale).

  • 7/30/2019 digital terrains

    28/42

    28

    8.3 SPOT5 DEM

    In the absence of height biases, DEMs generated from SPOT5 HRS imagery could be

    expected to show a standard error in elevation within the range of 5-10m. It is encouraging to

    see that with a point rejection threshold of 15m, the resulting standard errors for the SPOT5DEM are 3m or just under in Areas 1, 3 and 4, and just below 2m in Area 2. This is well

    within specifications. Of concern, however, is the very significant height bias of over 4 - 5m

    in all four test areas, which results in RMSE values ranging from 4.7 to 6.0m. One can only

    speculate as to the cause of the systematic heighting error. For example, it is could arise in

    large part in this case from errors in the exterior orientation of the stereo satellite imagery,

    perhaps as a consequence of insufficient or inaccurate ground control within the block

    adjustment process. Alternatively, it might be attributable to shortcomings in the filtering of

    vegetation within the DSM-to-DEM conversion. The latter assumption is supported to some

    degree by the percentages of the rejected points where the height error exceeded a 15m

    threshold, there being over 8% in Area 1, 6% in Area 3, 10% in Area 4, and a predictably

    lower 0.4% in Area 2, which is largely devoid of forest cover. The rejections grow to greater

    than 10% in Areas 1 and 3, and to 18% in Area 4, when the threshold is reduced to 10m, withthe distribution of the rejected points being shown in Figure 17. The rejected points are

    concentrated mainly in areas of dense coastal forest. Initial indications are that whereas the

    precision of relative heights is within specifications for SPOT5 data, the DEM exhibits

    degraded accuracy due to the presence of significant height biases, even in the absence of

    vegetation.

    8.4 Topo DEM (from 1:25,000 map data)

    The vertical accuracy specification typically associated with 1:25,000 topographic mapping is

    3m, corresponding to a third of the contour interval of 10m. Initial expectations for the Topo

    DEM would then be an RMSE at the 3m level, with localized occurrences of height biases as

    opposed to the area wide bias seen in the SPOT5 DEM. The mean height biases obtained forthe Topo DEM, with a 10m removal threshold for height discrepancies, were 0.8m in Area 1,

    2m in Area 2, 2.5m in Area 3 and 1.4m in Area 4. While the biases in Areas 2 and 3 are

    higher than one would anticipate for a 3m-accurate DEM, they are not viewed as significant

    given the corresponding 1-sigma values, which had a range of 1.9 - 2.7m. The number of

    points with height discrepancies greater than the 10m cutoff (nominal 3-sigma value) was

    0.2% or less for all four areas. This is consistent with the expectation that the Topo DEM

    should have fewer filtering errors and thus fewer %-removals because of the map compilation

    process being based on manual stereoplotting from aerial photography. The higher %-

    removal values shown for a 5m cutoff in Table 6 can be discounted somewhat because the

    threshold is set too tight at only 2-sigma, but it is nevertheless interesting that the points

    removed are concentrated in localized, mainly forested areas, as shown in Figure 18.

    8.5 Airborne IfSAR DEM

    With the relatively coarse rejection threshold value of 5m or approximately 5-sigma assigned

    to the airborne IfSAR DEM, resulting RMSE values were 1.4m in Area 1, 0.8m in Area 2,

    1.1m in Area 3 and 1.5m in Area 4. The corresponding 1-sigma values were basically the

    same as a consequence of the modest bias values of 0.5m or less. Unlike the three lower

    resolution DEMs discussed above, the attained accuracy of the IfSAR DEM was not well

    within specifications.

  • 7/30/2019 digital terrains

    29/42

    29

    Area 1 Area 2

    Area 3 Area4

    Figure 17: Points within the SPOT5 DEM with height discrepancies greater than threshold values when

    compared to the LiDAR reference DEM. Red areas representing a 10m threshold are overlaid by blue areas

    representing a 15m cutoff (areas not to scale).

  • 7/30/2019 digital terrains

    30/42

    30

    Area 1 Area 2

    Area 3 Area4

  • 7/30/2019 digital terrains

    31/42

    31

    Figure 18: Points within the Topo DEM (1:25,000 map data) with height discrepancies greater than threshold

    values when compared to the LiDAR reference DEM. Red areas representing a 5m threshold are overlaid by

    blue areas representing a 10m cutoff (areas not to scale).

    Instead, the accuracy was generally consistent with expectations and even a little worse than

    anticipated. The accuracy indicators of RMSE and standard error changed marginally when

    the rejection threshold was lowered from 5m to 3m, and significantly more points were

    removed. The %-removal values climbed to 7% and 15% in Areas 1 and 4, respectively, and

    to 2% and 4%, respectively, for Areas 2 and 3. As can be seen in Figure 19, the regions with

    most rejected points correspond to hilly terrain with steeper slopes, and to a lesser extent to

    forested areas. Generally speaking, the results obtained with the IfSAR DEM were in

    accordance with accuracy expectations, with the technology performing best in low lying

    areas.

    8.6 ADS40 DEM

    The DEM derived from ADS40 digital 3-line scanner aerial imagery was in fact a

    smoothed DSM that had undergone some initial automated classification and filtering. The

    first indication of the partial filtering of the ADS40 DSM is indicated in Figure 20, where it

    can be seen that the majority of the elevations within forested areas were rejected as outliers,

    their associated discrepancy values against the LiDAR data being greater than 5m or roughly5-sigma. Some 40% of the height discrepancy values in Area 4 were rejected. The

    assumption that the RMSE values were inflated by an incomplete DSM-to-DEM conversion

    is reinforced by the results of the mostly forest free Area 2, where the RMSE value for the

    5m threshold falls from the near 2m level of Areas 1, 3 and 4 to 1m, and the %-removal value

    drops from 30% or more to 4%. The height bias for Area 2 is also reduced to 0.3m from

    closer to 1m for the remaining areas. Given the incomplete filtering, it is difficult to

    characterize the accuracy of the ADS40 DEM (actually DSM), but it is encouraging to see

    results in Area 2 which are consistent with accuracy specifications, ie an RMSE value of less

    than 1m.

    9. Impact of Land Cover on DEM Accuracy

    Based on the results obtained in the analysis of performance of the five DEMs against the

    LIDAR reference DEM, it is apparent that a significant factor limiting vertical accuracy in

    the generation of supposedly bare-earth DEMs is the automated classification and filtering in

    forest and urban areas, with vegetation cover appearing as a more significant issue than the

    presence of buildings and other man-made structures. In order to gain further insight into the

    impact of different land cover on the DEM technologies considered, analyses were carried

    out for samples of four specific land cover types: urban, forest/bushland, open farm land, and

    mixed coastal cover of vegetated dunes and housing. Once again, elevation bias, RMSE and

    standard error of height discrepancies were quantified using the LIDAR data as the reference

    DEM.

    9.1 Urban AreasFigure 21 shows three sample urban areas: (a) a part of the coastal settlement of Scotts

    Head (taken from Test Area 4), (b) the commercial centre of South West Rocks (Area 1), and

    (c) a low-lying residential area of West Kempsey (Area 2). The results of the analysis for

    these three test sites are shown in Table 7, which has the same structure as the earlier Tables

    5 and 6.

  • 7/30/2019 digital terrains

    32/42

    32

    Area 1 Area 2

    Area 3 Area4

    Figure 19: Points within the airborne IfSAR DEM with height discrepancies greater than threshold values when

    compared to the LiDAR reference DEM. Red areas representing a 3m threshold are overlaid by blue areas

    representing a 5m cutoff (areas not to scale).

  • 7/30/2019 digital terrains

    33/42

    33

    Area 1 Area 2

    Area 3 Area4

    Figure 20: Points within the ADS40 DEM with height discrepancies greater than threshold values when

    compared to the LiDAR reference DEM. Red areas representing a 3m threshold are overlaid by blue areas

    representing a 5m cutoff (areas not to scale).

  • 7/30/2019 digital terrains

    34/42

    34

    (a)

    (b)

    Table 7. Accuracy evaluation result against LiDAR derived reference DEM for three Urban Test Areas. Only height

    differences below listed thresholds were included and those above removed, as per the %-removed column. Sample labels

    correspond with those in Figure 21.

    DatasetHeight

    bias (m)

    RMSE

    (m)h(m)

    Sample Size % removed

    SRTM DEM (Area a,Threshold=15m)

    1.1 2.5 2.3 273 0

    SRTM DEM (Area b,

    Threshold=15m)1.4 2.9 2.5 150 0

    SRTM DEM (Area c,

    Threshold=15m)1.1 1.9 1.5 403 0

    SPOT5 DEM (Area a,Threshold=15m)

    6.1 6.6 2.6 273 0

    SPOT5 DEM (Area b,

    Threshold=15m)7.2 7.7 3 150 0

    SPOT5 DEM (Area c,

    Threshold=15m)6.7 6.9 1.6 403 0

    TopoDEM (Area a,Threshold=10m)

    -0.8 3.8 3.7 243 11

    Topo DEM (Area b,Threshold=10m)

    -1.7 3 2.5 150 0

    Topo DEM (Area c,

    Threshold=10m)0.1 1.9 1.9 403 0

    IfSAR DEM (Area a,

    Threshold=5m) 0.3 1.6 1.5 10043 0.9IfSAR DEM (Area b,

    Threshold=5m)-0.8 1.4 1.2 5512 0.1

    IfSAR DEM (Area c,

    Threshold=5m)-1 1.4 1 14800 0

    ADS40 DSM (Area a,Threshold=5m)

    0 1.3 1.3 3828 1.1

    ADS40 DSM (Area b,

    Threshold=5m)-0.4 1.2 1.1 2135 0.5

    ADS40 DSM (Area c,

    Threshold=5m)0.5 1 0.9 5725 0.4

    (c)

    Figure 21: Urban test areas, (a) Scotts Head, (b)

    South West Rocks and (c) West Kempsey.

  • 7/30/2019 digital terrains

    35/42

    35

    The first feature of note in Table 7 is that for the SRTM and SPOT5 DEMs, the bias value

    has increased over that listed in Tables 5 and 6. In the case of SRTM, it is safe to assume that

    this is attributable to an incomplete removal of buildings in the DSM-to-DEM conversion.

    The South West Rocks town centre, Figure 21b, is characterized by buildings taller than a

    single story and it is thus not unexpected to see a more significant bias being present. The

    bias value for SPOT5, at between 6m and 7m, is not at all consistent with a shortcoming in

    building classification and filtering. Instead, it is a gross positive height error likelyattributable to a failure to utilize local ground control in the exterior orientation determination

    for the HRS imagery. Upon compensation for the bias, both SRTM and SPOT5 yield

    standard errors of height discrepancies in the range of 1.5m to 3m.

    The results achieved for the three urban areas for the Topo, IfSAR and ADS40 DEMs show

    an overall reduction in height bias, which is indicative of a more successful filtering of

    buildings in the automated DSM-to-DEM conversion. In terms of accuracy, the RMSE values

    obtained are largely consistent with those obtained in the full-area evaluations.

    9.2 Open Rural Areas

    Figure 22 shows the three selected open rural area sites: (a) open grassland with thinlydistributed houses and trees in West Kempsey, (b) open fields near Yarrahappini and (c)

    ploughed fields south of Stuarts Point. The first two areas are gently undulating, while the

    third is flat. The results of the analysis for these test sites are shown in Table 8, where it can

    be immediately seen that the DEM accuracy improves significantly when the need for

    extensive filtering is removed from the DSM-to-DEM transformation.

    (a)

    (b)

    Shortcomings in the DSM filtering required in the area shown in Figure 21a, which

    comprises a relatively small number of houses and trees, is enough to significantly inflate the

    RMSE value compared to that for the bare-ground areas of Figs. 21b and 21c, for all five

    (c)

    Figure 22: Open rural test areas, (a) West

    Kempsey, (b) Yarrahappini and (c) Stuarts Point

  • 7/30/2019 digital terrains

    36/42

    36

    DEMs. In the open areas, the accuracy of SRTM, as expressed through the RMSE, is better

    than 1m, and the corresponding values for the IfSAR and ADS40 DEMs are between 0.4m

    and 0.8m. The absolute accuracy for all DEMs is within specifications for all three test sites.

    Table 8. Accuracy evaluation result against LiDAR derived reference DEM for three Open Rural Test Areas.

    Only height differences below listed thresholds were included and those above removed, as per the %-removed

    column. Sample labels correspond with those in Figure 22.

    DatasetHeight

    bias (m)RMSE

    (m)h(m)

    Sample Size % removed

    SRTM DEM (Area a,Threshold=10m)

    -1.6 2.3 1.7 703 0

    SRTM DEM (Area b,Threshold=10m)

    -0.4 0.6 0.5 374 0

    SRTM DEM (Area c,Threshold=10m)

    0.7 0.9 0.6 286 0

    SPOT5 DEM (Area a,Threshold=10m)

    4.2 4.4 1.5 702 0.1

    SPOT5 DEM (Area b,

    Threshold=10m)3.3 3.5 1.2 374 0

    SPOT5 DEM (Area c,

    Threshold=10m)

    3.3 3.4 1 286 0

    Topo DEM (Area a,Threshold=5m)

    0.2 1.6 1.6 691 1.7

    Topo DEM (Area b,

    Threshold=5m)0.8 0.9 0.6 374 0

    Topo DEM (Area c,

    Threshold=5m)-2.3 2.4 0.4 286 0

    IfSAR DEM (Area a,

    Threshold=3m)-0.2 0.6 0.6 25087 1.3

    IfSAR DEM (Area b,

    Threshold=3m)-0.1 0.4 0.4 13802 0

    IfSAR DEM (Area c,Threshold=3m)

    -0.6 0.8 0.6 9726 0.3

    ADS40 DSM (Area a,Threshold=3m)

    0.9 1 0.6 9860 0.1

    ADS40 DSM (Area b,Threshold=3m) 0.2 0.3 0.2 5376 0

    ADS40 DSM (Area c,

    Threshold=3m)-0.3 0.7 0.6 3854 0

    Given that the cultivated area shown in Figure 21c was likely bushland at the time the Topo

    DEM was produced, the probable reason for the bias figure of -2.3m is land clearing and

    subsequent earthworks to create the cultivated fields. Also exhibiting a large positive bias is,

    once again, the SPOT5 DEM. Given the largely insignificant height biases and RMSE values

    that are within specifications, it is not surprising to see so few points classified as outliers,

    with virtually all of these being found in the DEMs covering the scene with houses and trees.

    9.3 Forest/Bushland AreasFigure 23 shows the three selected forest/bushland sites: (a) Dense tall (>10m) eucalypt forest

    at Yarrahappini, (b) Tall forest near Grassy Head and (c) scrubland covering a coastal dune at

    Stuarts Point, including an area of mangroves. The results of the analysis for these test sites

    are shown in Table 9. The table indicates a number of interesting features worthy of note.

    Firstly, in the heavily forested area, Figure 23a, the accuracy of the SPOT5 DEM is no better

    than 10m in absolute terms. Indeed, it can be seen that some 91% of the sample points are

  • 7/30/2019 digital terrains

    37/42

    37

    rejected as outliers, meaning they are in error by more than 10m, the cause no doubt being a

    combination of the already referred to exterior orientation bias and an inadequate removal of

    vegetation from the DSM.

    (a) (b) (c)

    Figure 23: Forest/Bushland test areas (a) Yarrahappini, (b) Grassy Head Road, and (c) Stuarts Point Beach.

    Table 9. Accuracy evaluation result against LiDAR derived reference DEM for three forest/bushland areas.

    Only height differences below listed thresholds were included and those above removed, as per the %-removed

    column. Sample labels correspond with those in Figure 23.

    DatasetHeight

    bias (m)

    RMSE

    (m)h(m)

    Sample Size % removed

    SRTM DEM ( Area a,

    Threshold=15m)2.2 2.9 1.8 240 0

    SRTM DEM (Area b,

    Threshold=15m)2.2 2.2 0.5 60 0

    SRTM DEM (Area c,

    Threshold=15m)-2.3 2.7 1.4 395 0

    SPOT5 DEM (Area a,

    Threshold=15m)9.7 10.2 3.3 21 91

    SPOT5 DEM (Area b,

    Threshold=15m)4 4.1 1.2 60 0

    SPOT5 DEM (Area c,

    Threshold=15m)4.3 4.8 2 395 0

    Topo DEM (Area a,Threshold=10m)

    1.5 1.6 0.5 240 0

    Topo DEM (Area b,

    Threshold=10m)-3.2 3.2 0.4 60 0

    Topo DEM (Area c,Threshold=10m)

    -0.4 1.3 1.3 395 0

    IfSAR DEM (Area a,Threshold=5m)

    0.4 0.5 0.2 8910 0

    IfSAR DEM (Area b,Threshold=5m)

    1.5 1.7 0.7 1888 0

    IfSAR DEM (Area c,Threshold=5m)

    -0.8 1.4 1.2 15017 0.8

    ADS40 DSM (Area a,Threshold=5m)

    3.9 4 0.9 5 99.9

    ADS40 DSM (Area b,

    Threshold=5m)1.8 2.4 1.6 529 28

    ADS40 DSM (Area c,

    Threshold=5m)0.1 1.7 1.7 5907 0.7

    Another DEM showing a high bias value in thick forest was that from ADS40 imagery,

    though this was to be anticipated given the low level of filtering undertaken with this data.

    Contrasting to the poor accuracy of the SPOT5 and ADS40 DEMs is the result for the

    airborne IfSAR DEM in the same area, where agreement with the LIDAR DEM is 0.5m

    RMS.

  • 7/30/2019 digital terrains

    38/42

    38

    Also noteworthy in Table 9 are the negative height biases of the SRTM DEM in the coastal

    scrubland and mangrove environment of Figure 23c and the Topo DEM in the tall bushland

    of Figure 23b. Neither systematic error is immediately explainable. Overall, the results for the

    forested areas are consistent with expectations, namely that the RMSE is higher than

    specifications for the DEM technologies would suggest, with the achievable accuracy being

    inversely proportional to vegetation density.

    9.4 Mixed Coastal Land Cover

    The final land cover type sampled could be characterized as mixed coastal dunes, scrubland,

    bush and built-up urban area. The chosen test area shown in Figure 24 is representative of

    much of the low-lying coastal environment along Australias eastern seaboard that is

    vulnerable to sea level rise and storm surges.

    Figure 24: Coastal area of mixed land cover, Arakoon.

    The results shown in Table 10 show largely the same characteristics as those presented in

    Table 5 for the full test areas. The RMSE of the SRTM elevations is a commendable 3m,

    with a modest bias, whereas the SPOT5 elevations show an RMSE of 6m, largely due to the

    now quite familiar large bias of also close to 6m. The Topo DEM also has a larger than

    expected bias given that the test area is right on the coast, with its RMSE value being

    marginally higher than expected. Both the ADS40 and IfSAR DEMs have small biases,

    though RMSE values which are outside specifications by approximately 0.7m. Given these

    results it is observed that the particular combination of land cover types does not reveal any

    distinctive performance characteristics which might not be apparent in the data covering the

    broader test areas.

    Table 10. Accuracy evaluation result against LiDAR derived reference DEM for coastal area of mixed land

    cover, as shown in Fig. 24. Only height differences below listed thresholds were included and those above

    removed, as per the %-removed column.

    DatasetHeight

    bias (m)

    RMSE

    (m)h(m)

    Sample Size % removed

    Smoothed SRTM DEM

    (Sample 1, Threshold=15m) -1.2 2.9 2.6 244 0

    SPOT5 DEM (Sample 1,

    Threshold=15m)5.6 6 2.1 233 4.51

    Topo DEM (Sample 1,

    Threshold=10m)2.7 3.7 2.5 244 0

    IfSAR DEM (Sample 1,

    Threshold=5m)0.2 1.8 1.7 8557 9.39

    Smoothed ADS40 DSM

    (Sample 1, Threshold=5m)0.1 1.7 1.7 3281 10.87

  • 7/30/2019 digital terrains

    39/42

    39

    10. Influence of Terrain Slope

    It is well established that the performance of DEM generation technologies is generally

    degraded as a function of increasing terrain slope. Steep terrain adversely impacts especially

    upon image matching in stereo photogrammetric techniques and upon radar interferometry.

    Whereas, the impact of slope might not be of prime importance within low-lying coastal

    topography potentially affected by sea-level rise, the current project offered a favourableopportunity to examine how different DEM technologies behaved in areas of differing

    topography. Shown in Figure 25a-d are plots of the variation of RMSE values for DEM

    elevations for slopes from 50

    to 500

    within the four test areas. The values shown represent

    discrepancies between the LiDAR reference elevations and the SRTM, SPOT5, Topo,

    airborne IfSAR and ADS40 DEMs.

    (a) Area 1 (b) Area 2

    (c) Area 3 (d) Area 4

    Figure 25. Plots of RMSE values against LiDAR elevations for different DEM technologies for different terrain

    slope. Error cut-off thresholds of 15m apply for the SRTM, SPOT5 and Topo DEMs, whereas a cut-off of 5m

    applies to the IfSAR and ADS40 data.

    The accuracy degradation with slope is clearly apparent, being most pronounced in Area 1,

    which contains both the steepest and most forested terrain. Whereas the impact of forest is

    not anticipated to significantly influence the character of the plots, it is interesting to note that

    the mean errors, essentially heighting biases, are impacted in the cases of SRTM and SPOT5.

    The mean error for SPOT5 derived elevations increases from around 4.5m in near-flat areas

    to 7.5m in areas displaying slopes of between 250

    and 500. The biases are considerably less

    for SRTM, but a value of 5m is obtained for Area 4. Another feature of the plots is that in

    addition to the airborne IfSAR and ADS40 DEMs displaying significantly higher accuracy

  • 7/30/2019 digital terrains

    40/42

    40

    than the SPOT5, SRTM and Topo DEMs, their performance is less influenced by terrain

    slope, though a mild fall off in accuracy with increasing slope is apparent.

    Overall, in the context of terrain modelling within low-lying coastal zones, the results of the

    analysis of the influence of slope on DEM technologies has served to further emphasise that

    of the five DEMs considered, only the airborne IfSAR and the ADS40 DEMs, as well as the

    LiDAR reference data of course, are of sufficient accuracy and reliability. Yet, as can be seenfrom Figure 25 and from Table 11, RMSE values for IfSAR and the ADS40 DEMS are

    nevertheless at a higher than desired level of 1-2m.

    Table 11. RMSE values for DEMs assessed against LiDAR DEM at different ground slopes. Units are metres.

    Slope category % 0-5 5-15 15-25 25-50

    DEMHeight

    biasRMSE

    Heightbias

    RMSEHeight

    biasRMSE

    Heightbias

    RMSE

    SRTM (Area 1) 0.5 3.1 1.2 4.2 1 5.6 1.7 7

    SRTM (Area 2) -0.8 2.3 0 3.2 0.4 3.9 1.4 4.7

    SRTM (Area 3) 1.4 3.5 2.8 4.8 0.3 5.9 0 7.1

    SRTM (Area 4) 1.6 3.4 2.3 5.1 2.7 6.4 4.9 8SPOT5 (Area 1) 4.2 5.1 4.8 5.5 4.9 6 4.5 7.2

    SPOT5 (Area 2) 4.2 4.6 5.1 5.4 5.5 6.1 6.5 7.4

    SPOT5 (Area 3) 4.7 5.4 5.2 5.9 5.1 6.4 5.9 8

    SPOT5 (Area 4) 4.9 5.7 4.9 5.8 4.9 6.4 5.5 7.7

    Topo (Area 1) 0.9 2.3 0 2.7 -1.5 3.4 -1.7 4.7

    Topo (Area 2) 1.2 3.6 -0.8 3.4 -1.3 3.5 -2.2 4.1

    Topo (Area 3) 2.9 3.4 1.4 3 -0.3 4.2 -0.3 5

    Topo (Area 4) 1.3 2.9 -0.9 3.7 -1.7 4.8 -0.8 5.1

    IfSAR (Area 1) 0 1.2 0.1 1.5 -0.5 1.8 -0.9 2.1

    IfSAR (Area 2) -0.2 0.9 -0.4 1.4 -0.2 1.7 -0.1 1.8

    IfSAR (Area 3) 0.4 1 0.3 1.4 0 2.2 0.5 2.6

    IfSAR (Area 4) 0.2 1.2 0.3 1.7 0.4 1.9 0.3 2.2

    ADS40 (Area 1) 0.4 1.8 1.2 2 0.6 1.7 0.6 2.1

    ADS40 (Area 2) 0.3 0.9 0.5 1.2 0.6 1.5 0.4 1.7

    ADS40 (Area 3) 0.8 1.5 1 1.8 0.8 1.9 1 2.2

    ADS40 (Area 4) 0.5 1.3 0.8 1.8 0.6 1.9 0.8 2.3

    Conclusions

    In most respects, the findings from the evaluation of the performance of DEM generation

    technologies are consistent with expectations regarding both accuracy and recognisedattributes and limitations of the different DEM data sources considered. However, this project

    has also revealed characteristics of the different DEMs that are perhaps not as widely

    recognized, but are nevertheless important in the context of producing accurate bare-earth

    DEMs of coastal terrain vulnerable to the impact of climate change.

    The accuracy gap between LiDAR DEMs and those from airborne IfSAR and ADS40 aerial

    photography at 50cm GSD might only be a factor of three to four according to specifications,

    eg 15-25cm elevation accuracy for LiDAR versus 0.5-1m for IfSAR and the ADS40.

  • 7/30/2019 digital terrains

    41/42

    41

    However, this difference is accentuated by shortcomings in the automated classification and

    filtering of both vegetation and, to a lesser extent, man-made structures within the DSM-to-

    DEM conversion of the radar and photogrammetrically produced DEMs. Multiple-return

    LiDAR displays significant advantages by way of last-pulse ground definition, which cannot

    be matched in densely vegetated areas by radar and photogrammetry techniques, except

    through skill-intensive and expensive manual editing processes.

    The residual systematic elevation errors attributable to incomplete filtering of DEMs have the

    potential to compromise the integrity of bare-earth elevation models in low-lying coastal

    areas that are either heavily vegetated or urbanized. Such land cover accounts for the majority

    of the populated coastal regions of Australia. As a consequence of the classification/filtering

    issues, and to a lesser extent, the difference in vertical resolution between different DEM data

    sources, it can be concluded that LiDAR is very much the preferred option for DEM

    generation in coastal regions vulnerable to sea level rise and storm surges.

    The results listed in Table 8 for DEM performance in open areas, largely free of trees and

    buildings, highlight the fact that distinctions in DEM accuracy are as much due to different

    terrain and land cover, and consequently to filtering, as to differences in basic metric

    resolution of the different technologies. In the case of the open pasture (Area b), sub-metre

    RMSE values were obtained for the SRTM and Topo DEMs, and the IfSAR and ADS40

    DEMs showed sub-half metre RMSE values.

    In regard to the five DEM generation technologies evaluated against LiDAR within the

    project, each produced localized, relative vertical accuracy within specifications, and indeed

    in the case of the 1-second SRTM accuracy significantly exceeded specifications. When

    corrected for bias, the SPOT5 DEM also produced a relative accuracy well within

    specifications, but here the bias problem was very significant, to the point where this DEM

    has little utility for higher resolution terrain modeling. Elevation biases are generally

    attributable to incomplete filtering of vegetation and buildings, and are therefore generally

    positive in sign. In the case of the SPOT5 DEM, a 4-7m systematic elevation error was

    present, irrespective of land cover. This bias likely arises due to accuracy shortcomings in the

    exterior orientation determination for the SPOT5 line scanner imagery, which was performed

    without the use of local ground control. The systematic error effects then flowed through to

    the image matching and object point triangulation phases.

    Finally, the following short summaries of the performance of each of the DEM generation

    technologies in the four selected test sites on the mid north coast of NSW are offered:

    The integrity if the LiDAR master DEM was validated through checks against the27,000-point kinematic GPS survey, which revealed an overall RMS height

    discrepancy value of close to 0.1m. This is entirely consistent with the anticipated

    RMS elevation accuracy of the LiDAR DEM of 0.15m. It must be kept in mind,

    however, that all checkpoints were positioned along open roads where issues with

    filtering in the DSM-to-DEM conversion do not arise. Whereas the removal ofvegetation and building from the DSM through automated classification and filtering

    can be expected to be more complete with multiple-return LiDAR than with radar or

    Topo DEMs, the presence of residual height errors over dense vegetation cover and

    low-level man-made features can be anticipated to some extent, though there is an

    absence of available tools to assess the extent of such systematic errors.

  • 7/30/2019 digital terrains

    42/42

    The 1-second SRTM DEM, with its RMSE value range of 2.2 to 4.1m, appears to be amore accurate bare-earth elevation model than its accuracy specifications would

    suggest.

    The SPOT5 DEM also produces an accuracy, as quantified by an RMSE value ofaround 5m, which is inside specifications, but it displays a disturbingly high

    systematic height bias averaging around 5m.

    The Topo DEM derived from 1:25,000 topographic map data is internally quiteconsistent and displays an accuracy in accordance with its 3m specification. This

    DEM displays localized areas of systematic height bias, in some cases due to changes

    in land cover.

    The airborne IfSAR DEM displays an accuracy at the high end of its anticipated 0.5 to1m range, and has optimal accuracy in low-lying areas with sparse vegetation

    coverage.

    Given that the ADS40 elevation model was really a smoothed, partially filtered DSM,a comprehensive DEM accuracy analysis was precluded. However, it is noteworthy

    that within Area 2, which has only minor coverage of either scrubland or forest, thebetter than 1m height accuracy attained in the smoothed DSM is consistent with

    accuracy specifications.

    Acknowledgments

    This work was funded by the Australian Governme