Interactive comment on “ Quantifying uncertainty of remotely sensed topographic surveys for ephemeral gully channel monitoring

Spatio-temporal measurements of landform evolution provide the basis for process-based theory formulation and validation. Overtime, field measurement of landforms has increased significantly worldwide, driven primarily by the availability 10 of new surveying technologies. However, there is not a standardized and/or coordinated effort within the scientific community to collect morphological data in a dependable and reproducible manner, specifically when performing long-term small-scale process investigation studies. Measurements of the same site using identical methods and equipment, but performed at different time periods can lead to incorrect estimates of landform change as a result of three-dimensional registration errors. This work evaluated measurements of an ephemeral gully channel located in agricultural land using multiple independent survey techniques for 15 locational accuracy and their applicability to generate information for model development/validation. Terrestrial photogrammetry and un-maned aerial vehicle-based platforms were compared to terrestrial LiDAR, defined herein as the reference dataset. Given the small-scale of the landform measured, alignment and ensemble equivalence between data sources were addressed sine qua non through post-processing. Utilization of ground control points were prerequisite to three-dimensional registration between datasets and improved confidence in the morphology information generated. Not one of the methods was without limitation; however, 20 careful attention to project pre-planning and data nature will ultimately guide temporal efficacy and practicality of management decisions.


Introduction
Spatio-temporal measurements of landform evolution provide the basis for process-based theory formulation and validation.Field 25 measurement of landforms has increased significantly worldwide, driven by the availability of new surveying technologies.Recent improvements to existing systems, such as mobile light detection and ranging (LiDAR) systems (Kukko et al., 2012;Eitel et al., 2016;Hawdon et al., 2016), integrated Un-manned Aerial Vehicles (UAV) (Bachrach et al., 2012;Bry et al., 2015;Honkavaara et al., 2016), Real Time Kinematics (RTK) (Rietdorf et al., 2006) and photogrammetry techniques (James and Robson, 2014;Marzolff, 2016), and low-cost and/or freeware structure from motion (SfM) photogrammetric software (Castillo et al., 2016;30 Piermattei et al., 2016).However, there is not a standardized and/or coordinated effort within the scientific community to collect morphological data in a dependable and reproducible manner, specifically when performing long-term process investigation studies.
Ephemeral gullies are often defined as small channels, on the order of a few centimeters in depth, predominantly in agricultural fields (SSSAJ, 2008).The emergence, evolution and persistence of these concentrated flow path erosion features are controlled by 35 the combined effects of flow, slope, soil properties, topography and vegetation characteristics (Zevenbergen, 1989).The term Earth Surf.Dynam. Discuss., doi:10.5194/esurf-2017-3, 2017 Manuscript under review for journal Earth Surf.Dynam.Published: 20 January 2017 c Author(s) 2017.CC-BY 3.0 License.ephemeral refers to the fact that agricultural producers often erase these channels during regular farming operations (Foster, 2005).
The combination of a highly dynamic lifespan with the relatively small-scale channel features require high accuracy measurements with high temporal and spatial resolution.
Many studies have been conducted to assess topographic accuracy of ephemeral/classical gully morphological measurements using a wide range of systems (i.e. Casalli, et al., 2004).Among them, LiDAR studies primarily focused on quantifying locational error 5 from datasets generated by air-borne systems, where locational variations are the result of coalesced errors generated by inaccuracies of Global Positional Systems (GPS), aircraft Inertial Measurement Unit (IMU) and overall timing of the system (Hodgson and Bresnahan, 2004).LiDAR positional errors can also be the result of interaction between the laser pulse and features with sharp relief change or occlusions that result in multiple returns from one laser pulse (Milenković et al., 2015).Evaluations of the accuracy of topographic information using airborne LiDAR are often compared with discrete sample locations and/or man-10 made targets with known coordinates (Hodgson and Bresnahan, 2004;Csany et al, 2005).Despite the large number of studies and methods developed to quantify positional errors in traditional airborne LiDAR surveys, this type of survey does not offer the temporal and spatial resolution necessary for quantitative monitoring of small-scale geomorphological characteristics (i.e.ephemeral gullies) necessary for process description.
At a finer scale, investigation of ground-based/terrestrial LiDAR has demonstrated a high locational accuracy (~2mm) and noted 15 the importance of appropriate spatial sampling density for ephemeral and classical gully investigation (Momm et al., 2013a).
Topographic representation of gully channels requires datasets with a specific minimum sampling density, which is dependent on site-specific topographic characteristics (Castillo et al., 2012;Momm et al., 2013a).Overlapping the same area by multiple scans increases the overall sampling density to overcome this limitation.
Studies involving various surveying techniques of concentrated flow paths revealed a wide range of quality, accuracy, cost, and 20 field campaign effort (Momm et al., 2011(Momm et al., , 2013b;;Castillo et al., 2012;Wells et al., 2016).Among the surveying techniques considered, photogrammetry has been shown to provide simple but robust measurement of small-scale changes in geomorphologic characteristics within agricultural fields (Castillo et al., 2012;Gesch et al., 2015;Wells et al., 2016).Further, a wide variety of platforms and techniques have been used to capture images, including kites (Marzolff, et al., 2003), backpacks (Wells et al., 2016) and UAVs (Ries and Marzolff, 2003;James and Robson, 2014;Bachrach et al., 2012;Cook, 2017).Erosion monitoring programs 25 based on photogrammetry have several advantages when compared to other surveying techniques.Photogrammetric field surveys do not interfere with farming operations, as it is non-obstructive; field campaigns are extremely efficient; and often they do not require specialized technical skillsets to implement.However, photogrammetric results can vary as a function of controlling parameters used during data collection and processing (Eltner et al., 2016).
The conversion of irregularly sampled point clouds into regular grids, referred to as Digital Elevation Models (DEMs), is extremely 30 common as most flow routing algorithms and Geographic Information Systems (GIS)-based soil erosion modeling technology are designed to work using these digital representations.As a result, a large number of studies have been conducted to evaluate DEM representation as affected by sampling intervals and interpolation algorithms (Aguilar et al., 2005;Ziadat, 2007;Bater and Coops, 2009;James and Robson, 2012;James and Robson, 2014), and by DEM spatial resolution (Zhang and Montgomery, 1994;Kienzle, 2004;Momm et al., 2013a).35 The majority of previous studies have focused on accuracy evaluation of a specific photogrammetric survey method at a single time period.Varying sensors, platforms, and processing methods can yield different results (variations in sampling densities, gaps, and noise).Furthermore, measurements of the same site using identical methods and equipment, but performed at different time periods can also lead to three-dimensional registration errors.Therefore, the scope of this work was to evaluate multiple survey techniques for measuring ephemeral gully channels.Three surveying methods with varying parameters were conducted 40 Earth Surf.Dynam. Discuss., doi:10.5194/esurf-2017-3, 2017 Manuscript under review for journal Earth Surf.Dynam.Published: 20 January 2017 c Author(s) 2017.CC-BY 3.0 License.
independently and evaluated for locational accuracy and their applicability in generating information for model development/validation.The objectives of this study are twofold: to quantify the overall accuracy of the different survey methods with varying configurations and to develop practical guidelines for the design and implementation of future ephemeral gully monitoring studies.

Study Site
The study site was located in the northwest corner of Webster County, Iowa, U.S.A. (Figure 1).Farming is the dominant enterprise in Webster County.The crop rotation was a corn-soybean rotation.Total annual precipitation is about 873 mm, of which, 70 percent usually falls between April and September.The area of interest (AOI) within the field survey was a small reach (1.9 m x 1.3 m; 2.47m 2 ) of a 150 m long ephemeral gully, oriented south to northwest, eroding Clarion loam (fine-loamy, mixed, 10 superactive, mesic Typic Hapludolls) at the upper (south) extent, Terril loam (fine-loamy, mixed, superactive, mesic Cumulic Hapludolls) on the intermediate slopes, and Webster clay loam (fine-loamy, mixed, superactive, mesic Typic Endoquolls) within the lower relief section of the field.Within the AOI, the soil was Clarion loam.

Field Survey
Field surveys were conducted using three independent modes: ground-based/terrestrial LiDAR, ground-based photogrammetry, 15 and airborne photogrammetry.The surveys yielded twelve datasets (Table 1; dashed lines represent delineation between survey modes).The terrestrial LiDAR was considered the reference dataset, due to perceived superior accuracy.All surveys were run independently of each other and completed on the same day.Each dataset was represented using the NAD83 UTM15N coordinate system.
In this study, the terrestrial LiDAR point cloud was generated using TopCon ScanMaster software 20 (https://www.topconpositioning.com/software/mass-data-collection/scanmaster),all ground-based photogrammetric point clouds were generated using PhotoModeler Scanner software (www.photomodeler.com/products/scanner/default.html) and all airborne photogrammetric point clouds were generated using Pix4Dmapper Pro software (https://pix4d.com/pix4dmapper-pro/).It is acknowledged that the selection of input parameters influences the sampling intensity and local elevation variance; however, the quantification of the influence of input parameters on the output is beyond the scope of this study.Here, similar survey methods 25 used the same input parameters to generate point clouds.

differential GPS of Ground Control Points
Site preparation began by locating a state monument point (Figure 2) and laying out 406 x 406 mm quad-triangle, white-on-black sheet GCPs, considering the long and short axes of the field as well as the high and low elevations within the field boundary, herein considered to be the field GCPs (10 total).One additional set of GCPs with rad coded targets was arranged along the gully channel 30 (four pins at the location; Wells et al., 2016), herein considered to be the channel GCPs (4 total).All GCPs were surveyed using TopCon GR-3 dGPS survey equipment (TopCon Corporation, Tokyo, Japan) to obtain relative position in reference to the state monument point.A static occupation (6hrs) was initiated with the base station, then all GCPs (field, channel and state monument) were surveyed with the rover (20 sec collection interval).All survey data was corrected using an OPUS (National Geodetic Survey) Earth Surf.Dynam. Discuss., doi:10.5194/esurf-2017-3, 2017 Manuscript under review for journal Earth Surf.Dynam.Published: 20 January 2017 c Author(s) 2017.CC-BY 3.0 License.solution of the base station location, processed with reference to the state monument point.Both (field and channel) GCP positions were used to adjust point clouds from the LiDAR and photogrammetric surveys of the site.

Terrestrial LiDAR Survey
The terrestrial LiDAR survey was conducted using a TopCon GLS 1500 (TopCon Corporation, Tokyo, Japan).The system operates in similar fashion to standard total stations.For each laser pulse, the system records X, Y and Z coordinate values with respect to 5 the position of the scanner sensor (local coordinate system), the intensity of the returned signal (reflectance) and spectral information from an integrated digital camera within the instrument.Local coordinates are transformed into global coordinates during post-processing by entering the external geometry coordinates (i.e.absolute position determined from Post-Processed Kinematic survey) of GCPs.The AOI (demarked by channel GCPs; 2.47 m 2 ) within the field boundaries, covering the gully channel, was surveyed with one scan resulting in a total of 5,613,334 scan points. 10 Given the level of user control of input parameters and the high locational accuracy of terrestrial LiDAR systems, this survey method was selected as the reference to which all other survey methods were compared.However, it is important to acknowledge that this survey method does have limitations.
In surveys with high sampling intensity, it is common for the same location on the ground to be hit by multiple laser pulses.This yields datasets with high sampling intensity but with a range of elevation values for the same location (i.e.fluff) given the vertical 15 accuracy of the system.In this study, this locational variability is estimated to be approximately ±2 mm.The sensor operates in the near-infrared portion of the electromagnetic spectrum (1,535 ηm) and in this spectral range; electromagnetic energy is absorbed by water (Aronof, 2005).In locations with water features, the sensor emits laser pulses but no laser pulse is reflected back to the sensor, preventing the range calculation for that particular pulse.During data collection for this survey, a shallow film of water was present in the gully channel (Figure 3A) and, as result, no points were recorded in the water-covered region (Figure 3B).20 Sampling gaps in LiDAR surveys can also be attributed to vertical features that limit the sensor line-of-site, referred to as shadowing (Figure 3C).The basic principle of LiDAR technology is to measure the time needed for an individual laser pulse to travel from the transmitter to the target and back to the receiver allowing the range distance to be calculated (Wehr and Lohr, 1999).However, in certain situations, as the scanner moves along the scan arc, the laser footprint hits an area just past the edge of a surface where the next return appears to be from a distance greater than expected (i.e.occlusion).In this case, the gap is linearly 25 filled by equally spaced points (Figure 3C highlight).A shadow of the obstruction appears in the dataset.These artificial points may be filtered by intensity and multiple scan positions may be used to discriminate the features.Here, the AOI was slightly decreased to omit GCP occlusions from the dataset.
Multiple images were collected around the channel GCPs, including views from each corner and all sides.Still images captured by the camera were transformed to point clouds using PhotoModeler Scanner photogrammetric software.Initial data processing included: aerial triangulation and bundle adjustment, camera position and exterior orientation.Following initial processing, channel 35 GCP positions were included to optimize point cloud accuracy.

UAV-based Photogrammetric Survey
Still images captured by the UAVs were transformed to point clouds using Pix4DMapper Pro photogrammetric software.Initial data processing included: camera calibration, aerial triangulation and bundle adjustment, camera position and exterior orientation.
Following initial processing, field GCP positions were included to optimize point cloud accuracy.
Both, fixed-wing and quadrotor, UAV systems were deployed with a fixed path and common photograph overlap percentage.All 5 missions/deployments were preplanned using flight planning and control software provided by the manufacturer.A mission block and specific area or point of interest were selected, including preferred ground resolution, camera head angle (quadrotor only) and flight altitude, then flight lines for aerial coverage, circular paths with horizontal plane around objects of interest (quadrotor only), image capture points and waypoints were generated prior to deployment.Key flight parameters were displayed in real-time, along with battery level and image acquisition progress, while the autopilot continuously analyzed onboard control data to optimize the 10 flight.

Fixed-wing UAV
The fixed-wing platform had an 18.2 MP nadir camera (i.e.belly mount) and was deployed to capture the entire field boundary (Figure 2) by throwing the craft in the air, then the craft flies, captures images and lands itself.Two altitudes were flown: 122 m and 61 m. 15

Quadrotor UAV
The quadrotor platform had a 38 MP camera mounted within an 180 o vertical range head and was deployed to capture both, the extent of the gully within the field and specific points of interest (i.e.gully cross sections; Gesch et al. 2015;Wells et al. 2016).
The quadrotor was deployed through mission planning software.The craft takes off, flies and captures images, then lands itself.
Two altitudes were flown: 35 m and 20 m. 20

Dataset Alignment
To ensure the highest three-dimensional alignment among all point clouds and consistent spatial coverage by all methods, a threestep preprocessing approach was developed.First, the set of four channel GCPs (white square targets in Figure 2B) were used to generate a rectangular polygon to subset the point clouds in all surveys, assuring that all surveys cover exactly the same ground position.Second, a smaller polygon subset was performed by generating a polygon with a 5 cm reduction on all sides.This was 25 performed to exclude areas close to the channel GCPs and assure the elimination of shadowing and occlusion created by the slightly elevated channel GCPs.Third, the sampling void within the LiDAR dataset, created from the presence of a thin film of water within the channel (Figure 3A), was manually digitized into another polygon and used to remove points from all photogrammetricgenerated datasets to assure uniformity between all datasets.Essentially, instead of using interpolated data within this void in the reference dataset, we simply placed a void in all datasets; therefore, we do not introduce bias into the calculations with regard to 30 the water film void in the LiDAR data.
Subsequently, since each surveying method was performed using the same set of GCPs, manual inspection of measured points located coincident with channel GCPs (white square targets in Figure 2B) were used to generate planes, yielding ten planes, one for each dataset with the exception of the Fixed_61m and Fixed_122m datasets, in which no points were located on top of the channel GCPs.The four GPS surveyed coordinates of the center location of the channel GCPs were used to fit a reference plane to be matched by all surveys (black squares in Figure 4).Three-dimensional locational differences between the reference plane generated using the GPS survey (black squares in Figure 4) and the planes of each surveyed dataset (LiDAR and photogrammetry) were calculated using the Iterative Closest Point (ICP) algorithm (Besl and Mckey, 1994) implemented in MATLAB (MathWorks Inc., Natick, Massachusetts).The ICP algorithm minimizes the locational differences between two sets of three-dimensional point clouds and outputs a 3x3 rotation angle matrix, , and a 1x3 translation vector,  (Eq.1).These matrices were used to three-5 dimensionally transform, through rotation and translation, the measured point clouds to best match the reference plane.

Error Metrics
One of the problems in looking at the data in the original point cloud format was the large difference between the total number of points within datasets (hundreds to millions), which tends to bias the results; therefore, the photogrammetry data were gridded to 10 introduce a common means of discussing differences between the survey methods.Furthermore, this type of data structure (i.e.raster grid) is a common format used to estimate soil loss volumes and generate cross-sections for modeling exercises (Dabney et al., 2014).
In the point cloud analysis, each point within the photogrammetry surveys was compared to that within the LiDAR survey.The analysis was carried out in two ways: point normal to the plane (each photogrammetry point was projected normal to a fitted plane 15 of LiDAR points at the nearby position) and spot elevation to TIN (each photogrammetry point was projected up or down to intersect a TIN surface of the LiDAR points).

Sampling Intensity, Local Variance, and Spatial Pattern
Sampling intensity is defined as the number of points per unit of area.Investigation of the sampling intensity spatial variation can reveal over/under sampled locations.Under sampled locations can potentially be sources of positional errors, especially in surfaces 20 with high relief.Sampling intensity was evaluated using the quadrant method (Dodd, 2011), where a virtual regular grid of 1 cm was imposed on each dataset and the number of LiDAR and photogrammetry points falling within each grid was counted and recorded.Point pattern analysis was performed using the G Function (distribution of nearest neighbor distance differences) and F Function (distribution of nearest neighbor to virtual grid centroid distance differences) to evaluate potential irregularities in sampling, such as over sampling one area while under sampling another (Bailey and Gatrell, 1995), and overall spatial distribution 25 of the data sampling (i.e.clustered or disperse).
Similarly, the local elevation variance was evaluated by calculating the elevation range (difference between maximum and minimum) within each grid.The local elevation variance is a function of the terrain characteristics, sampling intensity, survey method, and post-processing parameters.

2.4.2
Vertical and Horizontal Displacement using Point Tangential Projection to Plane 30 Vertical and horizontal displacement between the LiDAR and photogrammetry measurements in each dataset were quantified using the normal projection of each photogrammetry point into a plane fitted to the nearest LiDAR points (Figure 5A).For each photogrammetry point (green circle in Figure 5A), the nearest (within 2.5cm sphere) LiDAR points were selected (black dots in the plane, the coordinates of the intersection point (red circle in Figure 5A) recorded and statistics were generated (Table 2).This analysis was performed for all datasets using an in-house developed software package, where (2)

Vertical Displacement using Spot Elevation
The three-dimensional point cloud representing the reference dataset (LiDAR) was converted into a Triangular Irregular Network (TIN) (Figure 5B).Each point in the photogrammetry (photogrammetry) dataset (green circle in Figure 5B) was compared to the LiDAR TIN by fixing the photogrammetry X and Y coordinates while varying the Z-coordinate up or down until the point intersected the TIN (red circle in Figure 5B).The Z-coordinate at intersection was recorded.This analysis was performed using 10 ArcGIS (Esri, 2011).
The vertical displacement at a location is calculated by subtracting the photogrammetry elevation from the LiDAR elevation of the TIN.Descriptive elevation statistics were generated based on all raster grid cells (Table 3).

Gridded Surface Assessment 15
All point clouds were converted into 5x5 mm regular raster grids using linear interpolation.The two upstream channel GCPs served as the base of the rectangular grid; roughness of individual datasets was highly dependent upon sampling intensity.Volume difference calculations were performed between the LiDAR raster grid and the photogrammetry raster grid.Two metrics were calculated, volume difference and absolute volume difference (Eqs.4 & 5; Table 4).
where  9GF1' is the reference elevation,  8<=>= is photogrammetry elevation,  is the raster grid cell size (0.005), and  is the total number of raster grid cells.

Gridded Cross-Section Assessment
Gully modeling technologies often use cross-sections as basic modeling units.With the objective to assess the error introduced by 25 each survey to cross-section analysis (Figure 10), the raster grid surfaces were used to generate eleven (11) cross-sections.Area above the curve was selected as one of the metrics to quantify cross-section accuracy, given as where  L is the area for cross-section  in square meters,  G is the elevation at point  in the cross-section, and ∆ FG2> is the distance between points in the cross-section (0.005 m).The constant (353.00) was used to adjust all cross-section elevations, due to local 30 elevation relation to mean sea level, thereby truncating area values.The deviation of the area estimates from the LiDAR were calculated using: Quantification of cross-section elevation descriptive statistics (Table 5; Figure 11) and fitted linear models between LiDAR and photogrammetry elevations for individual cross-sections were also used as comparison metrics (Table 6).

Dataset Scoring
Since there is no true standard of judging the performance of the measurements provided herein, a system of scoring was developed to grade the photogrammetry data with regard to the LiDAR data (Table 7).Scores between 1 and 11 were assigned to each 5 evaluation category using both, point cloud and gridded data.For example, the difference in absolute volume was assigned decreasing scores (1→11) for increasing volume difference and, correlation coefficients were assigned decreasing scores (1→11) for decreasing correlation.Simply, if a variable had a positive impact, it received a higher score.Each score is defined in Table 7.

Sampling Intensity Evaluations 10
The point clouds presented here had a large variability in sampling intensity (from 1 to > 250 points per cm 2 ).The difference in point sampling influences micro-topography and apparent roughness, and may lead to bias in volume estimation.If the surface is rough, the effect will be greater (Figure 11).In Figure 6, point counts are very low for the fixed-wing flights in comparison to the other methods and the sparse point count leads to interpolation (filling) during raster gridding.Variance shows roughness is similar for most of the surveys, with the exception of Fixed_122 where the variance is extremely low (0.0005; Table 5).This result merely 15 points out that the Fixed_122 is extremely smooth in comparison to the other surveys, due to the low sampling density (Figure 8) and enhanced interpolation between points for the high spatial resolution raster grid (0.005m cell size).

Vertical and Horizontal Displacement Evaluations
Graphical representation of both spot elevation to TIN and normal to fitted plane for the fixed-wing flight at 122 m altitude (Fixed_122 (Figure 6)), and the four photo-pair (Ground_8A (Figure 7)) are provided for comparative purposes.20 Very similar results were obtained with these two methods.All datasets had negligible mean displacement in the X and Y-directions (Table 2).The standard error of the normal to plane analysis (Table 2) ranged from 0.3 mm (Ground_8A, Ground_6A, Ground_4B, Ground_4C, Ground_2B) to 2.9 mm (Fixed_122).The mean displacement for the normal to plane analysis in the Z-direction (Table 2) ranged from 0.2 mm (Quad_35) to 7.4 mm (Fixed_61).For the vertical spot to TIN analysis (Table 3), standard error ranged from 0.4 mm (Ground_8A, Ground_6A, Ground_4B, Ground_2B) to 2.1 mm (Fixed_122), and mean displacement ranged from 25 4.5 mm (Ground_8A) to 35 mm (Fixed_122).The mean elevation difference between photogrammetry and LiDAR was approximately 5mm, with the exceptions of the fixed-wing surveys (10 and 35 mm).Perhaps this was simply because they were not pre-processed (affine transformation was not applied); however, this result does show the importance of common reference points between surveys.
Most of what is reported here is due to point cloud alignment during the preprocessing step discussed earlier (i.e.GCP alignment 30 for each, except fixed-wing flights).The mean elevation difference was approximately 5mm for all datasets (Tables 2 and 3).
Positive values indicate that the LiDAR data was, on average, higher than the photogrammetry data and negative values indicate that the LiDAR data was, on average, lower than the photogrammetry data.Similarly, when contrasting the photogrammetry elevation with the elevation of the normal point through linear regression (Table 2), the slope of the fitted line is very close to unity for all methods except Fixed_122, indicating spatially variable discrepancies (higher elevation differences at one region than the rest of the study site).

Gridded Surface Evaluations
The conversion of point clouds with irregularly spaced points, and spatially varying sampling intensity point clouds, into regular raster grids affected each dataset differently.For example, the LiDAR dataset contained a high sampling intensity (> 100 points 5 per cm 2 ) with relatively large elevation variability of the points within a raster grid cell.Therefore, the interpolation procedure generated a significantly smoothed surface.Conversely, the fixed-wing surveys had a low sampling intensity and the interpolation procedure linearly filled the gaps, potentially generating a surface that differs from natural-looking surfaces.
One of the most important measurements for gully monitoring is the volume difference between surfaces (Table 4).Given the small scale of this type of erosional feature (on the order of few cm), it is vital to have a good understanding of the expected error 10 for each method.Among similar collection methods (ground-photogrammetry), the absolute volume difference (Table 4) ranged from 0 to 52%, although these differences were extremely small in reality (i.e.range of 0.0062 to 0.0105 m 3 ).Performance ranking, in terms of absolute volume difference, was Quad_35, Ground_8A, Ground_4A, Ground_6A, Quad_20, Ground_4C, Ground_4B, Ground_2B, Ground_2A, Fixed_61 and Fixed_122 (10 to 165% absolute volume difference for Ground_8A and Fixed_122, respectively, in comparison to Quad_35; Table 4).Variance values in elevation difference between the LiDAR and 15 photogrammetry data were all quite similar (Table 5), with the exception of the fixed-winged flights (effect of interpolation).In terms of the elevation range of the data, most were smaller than 2.5 cm per cm 2 and appeared to be very similar (Figure 7), with the exceptions of Ground_2A (too smooth) and LiDAR (too rough).

Gridded Cross-section Evaluations
Comparing elevation information (Tables 5 and 6) between photogrammetric cross-sections and LiDAR cross-sections through 20 linear regression, indicates a coefficient of determination larger than 0.95 for all datasets, except the two fixed-wing systems.The standard error of this regression was less than 10 mm for Quad_20, Quad_35, Ground_2B, Ground_4B, Ground_4C, Ground_6A, and Ground_8A.Following that, Ground_2A and Ground_4A had a standard error of approximately 20 mm and the two fixed wings of 25 mm.The average wet area percent difference for all cross-sections were within 1.5%, while the two fixed wings had 3 and 7%, for Fixed_61 and Fixed_122, respectively.It is important to mention that the range of wet area percent difference is 25 within ± 2%, while the fixed-wing systems had up to 15% percent difference.The error is huge for instance if this dataset was intended to be used for the purpose of development/calibration/validation of a soil erosion model.

Dataset Scoring Evaluations
Gridded data performance was led by Ground_8A and Quad_35.Combined category score points ranged from 49 (Ground_8A) to 6 (Fixed_122), ground-photogrammetry ranged from 49 to 20, quadrotor ranged from 42 to 35, and the fixed-wing ranged from 30 9 to 6 (Table 8).For the point cloud analysis, scoring results were, for the most part, very similar.The ground-photogrammetry surveys all score very high, with the quadrotor falling in the middle and the fixed-wing at the bottom.Overall, scoring ranged from 140 to 24, with the ground-photogrammetry leading the group.As the number of photos increased, so too did the sample density; however, the four photo-pair (Ground_8A) was less dense than either the six photo-pair (Ground_6A) or two photo-pair (Ground_4B, Ground_4C), which may be associated with higher accuracy in pixel matching or the addition of inferior image/s to Earth Surf.Dynam. Discuss., doi:10.5194/esurf-2017-3, 2017 Manuscript under review for journal Earth Surf.Dynam.Published: 20 January 2017 c Author(s) 2017.CC-BY 3.0 License. the project.However, it is noteworthy to add that sampling intensity did increase as the UAV altitude decreased, although the Quad_35 did outperform the Quad_20 in a number of categories.

Methods Comparison
An alarming concern in this analysis was the realization that rotation and translation (Figure 4) was required to ensure that all data 5 was properly aligned.The LiDAR global coordinates were the same as those used for the fixed-wing and quadrotor flights (i.e.field GCPs) and, the channel GCPs were also utilized to optimize the LiDAR point cloud solution.In all ground-photogrammetry point cloud solutions, the same set of global coordinates (channel GCPs) were used.One might expect that the solutions would converge without the need to manipulate the point clouds in post-processing; however, not one of the solutions contained the exact positions of the channel GCPs, including solutions generated using the same platform but with varying processing parameters.For 10 example, three-dimensional registration discrepancies were LiDAR between solutions from the quadrotor platform at 20 and 35 meters, from the fixed-wing platform at 61 and 122 meters, and from the terrestrial photogrammetry surveys.This realization does offer extreme difficulty in temporal studies of ephemeral erosion processes, no matter the choice of resolution, platform, and/or processing parameters.
Initially, an attempt was made to analyze all datasets in their original form; however, two limitations to the approach were noted: 15 the lack of three-dimensional registration between the datasets skewed efforts to quantify individual point accuracy and, more importantly, reduced confidence in the geomorphology information generated; and the difference in point sampling density, ranging from hundreds (fixed-wing platform) to millions (LiDAR), biased results.Therefore, the discussion presented herein relates to solutions that have been altered from the original solutions produced by the respective software packages.And, two point pattern analysis techniques (F and G functions; see Sec. 4.2.1) were completed to evaluate the spatial distribution of the data 20 sampling (clustered or disperse).Comparison of measured F and G functions with estimated theoretical spatial distribution under the Complete Spatial Randomness assumption suggested that all datasets did not present any spatial clustering, therefore indicating that the study site was sampled uniformly (regularly spatially distributed throughout the study site).These point pattern analyses were performed using the spatstat package in the R software package (Bivand et al., 2008, Baddley andTurner, 2005).Furthermore, points with the same X, Y and Z to the fifth decimal place were removed from the LiDAR dataset, indicating collection of 25 redundancy information.
Another interesting finding was the difference between solutions from ground-photogrammetry (varying number and/or orientation of photo pairs).Solutions from Ground_2A and Ground_2B datasets both used only one photo pair, however, results from the ranking analysis indicate a superior solution generated from the Ground_2B (i.e.upstream/downstream orientation; Table 1) pairing.This could be potentially attributed to the orientation of the images in relation to the channel, in which differences in 30 illumination could hamper the photogrammetric process of automated pixel matching between each photo pair (Marzoff and Poesen, 2009).Additionally, increasing the number of photo pairs used in the solution, does seem to yield improved solutions; however, the small differences between Ground_6A and Ground_8A suggest a potential threshold in the number of photo pairs to which including additional photo pairs adds marginally to the final quality of the solution.

Monitoring Guidelines
Two photogrammetric software packages (Pix4DMapper Pro and PhotoModeler Scanner) were used to generate solutions for the UAV platform and ground-based photogrammetry surveys.Pix4DMapper Pro uses a larger number (>=3) of overlapping photos, while PhotoModeler Scanner can offer solutions with only two overlapping photos.These software packages differ in the level of user control options for processing and point cloud generation and, point clouds processed by different software packages and/or 5 users could yield very different solutions.
Long-term photogrammetric monitoring of ephemeral gullies should be performed with systems and procedures that: (1) provide a minimum sampling density to capture the overall and local terrain characteristics based upon study objectives (i.e.temporal headcut migration process understanding may require sub-centimeter resolution data, while temporal channel meander process understanding may only require decimeter resolution), (2) utilize static ground control points visible in comparable photo pairs in 10 all time-step surveys (fixed known points within the scene provide checks to assure proper three-dimensional registration of temporal data), (3) collect the same number of photo pairs using the same sensor and with the same orientation in all time-step surveys (i.e.data collection strategies should not vary temporally and new sensors must be carefully calibrated to preexisting datasets), and (4) process and generate photogrammetric solutions using the same software package and similar input processing parameters (perhaps a topic for further study; however, temporal studies should be processed using the same input parameters).15

Conclusions
Comparative evaluations were completed using terrestrial LiDAR and photogrammetry, both ground and aerial (UAV).None of these methods are without limitation and the ultimate goal of the data collection effort should guide the planning phase of the project.One cautionary note: without GCP there is no reasonable expectation that temporal activities will be successful.Although GCP may increase the workload during data acquisition, this is the only realization that will assure global alignment, minimize 20 project error and enhance process theory development.In terms of survey choice, all results point to a financial and temporal question.What is the project goal?What are the data expectations?Temporal assessment of gully channels and most geomorphic process descriptions can be accomplished with a camera and a few GCPs, whether on the ground or airborne.Each of the survey methods provided herein performed very well and, although the scoring was not very spectacular, the Fixed_61 data would be satisfactory for most static model evaluations.As expectations rise, so too will the planning and technology.25 Earth Surf.Dynam.Discuss., doi:10.5194/esurf-2017-3,2017   Manuscript under review for journal Earth Surf.Dynam.Published: 20 January 2017 c Author(s) 2017.CC-BY 3.0 License.

Table 1 :
Datasets generated by three distinct surveying methods for the purpose of quantifying locational uncertainty in gully studies.right with corner left and right photo pair Grd_4B Upstream and downstream with corner left and right photo pair Grd_4C Upstream and downstream with channel left and right photo pair Grd_6A Upstream, downstream, channel left and right with corner left and right photo pair Grd_8A Upstream, downstream, channel left and right with both corner photo pairs Quad_20m Quadrotor flight at 20 m above ground surface Quad_35m Quadrotor flight at 35 m above ground surface Fixed_61m Fixed-wing flight at 61 m above ground surface Fixed_122m Fixed-wing flight at 122 m above ground surface LiDAR Terrestrial LiDAR survey (reference) *Dashed lines represent delineation between survey modes.Earth Surf.Dynam.Discuss., doi:10.5194/esurf-2017-3,2017 Manuscript under review for journal Earth Surf.Dynam.Published: 20 January 2017 c Author(s) 2017.CC-BY 3.0 License.

Figure 1 :
Figure 1: Study site location used in the evaluation of close-range photogrammetric surveys of ephemeral gully channels.

Figure 2 :
Figure 2: Study site with field ground control points (GCPs; circles), channel GCPs (square) and state monument (cross) on bridge in upper right corner (A).Selected area for detailed surveys and comprehensive evaluation (B).

Figure 3 :
Figure 3: Limitations of ground-based LiDAR survey used as reference dataset.(A) Photograph of AOI shows water in the channel, which limits laser pulse return to the sensor causing sampling gaps in the point cloud (B), and presence of high relief features (GCPs) in the DEM (C) where sharp edges that cause the generation of multiple laser pulse returns due to split footprint effect.

Figure 5 :
Figure 5: Schematic representation of positional accuracy analysis.Black dots represent reference dataset (LiDAR) and green circle represents point being evaluated.Red circles represent the normal projection of green point onto the tangential plane fitted to the LiDAR dataset (A) and the vertical projection of the green point into the three-dimensional LiDARgenerated triangular irregular network (TIN) (B).

Figure 8 :
Figure 8: Sampling intensity using the quadrat method for each dataset considered.Individual color represents point sampling count intervals within a 1x1cm virtual grid.Points located in the channel were removed to match the area covered by the LiDAR dataset (herein considered as reference).

Figure 9 :
Figure 9: Elevation range (difference between minimum and maximum elevation) represented as individual colors within a 1x1cm virtual grid.Points located in the channel were removed to match the area covered by the LiDAR dataset (herein considered as reference).

Figure 10 : 5 Earth
Figure 10: Illustration of interpolating three-dimensional point clouds into raster grid for volume and cross-section analysis of gully monitoring and geomorphologic quantification.

Figure 11 :
Figure 11: Selected cross-sections generated from interpolating point-clouds into 5x5 mm raster grid file.

Table 6 : Cross-section evaluation comparison between photogrammetry and LiDAR. Fitting linear model between photogrammetry and LiDAR Dataset Slope Intercept r-value p-value Standard error Wet area* percent difference Minimum wet area* percent difference Maximum wet area* percent difference
* Wet area calculations used a horizontal reference elevation of 353 meters.