• Local Tilt

Reference Peak

Local Tilt Investigation

Aims and Objectives

Purpose of test: To investigate the magnitude and mechanism of local tilting of the DTM with respect to the truth topography in the region of the reference peak.

Additional Objectives:

• Calculate the relative magnitudes of tilt and induced vertical error with image resolution and maplet ground sample distance;
• Investigate the effect of the order in which overlap constraints are applied to maplets;
• Study a larger evaluation area centered on the peak and containing multiple peak and crater features;
• Compare tilt behavior across tests, comparing results for test Over11I against Over11S, T and U, the spacecraft zenith variation test suite.

Acronyms

Definitions and Disambiguation

Background

A local tilt in the region of the reference peak is apparent upon inspection of the DTM, specifically the 4mx4m bigmap which is centered on the peak. Furthermore, tilt of bigmaps with respect to truth topographies has been observed across SPC tests. The purpose of this investigation is to measure the magnitude and direction of tilt after each tiling and iterating step and to explore the mechanism by which the DTM tilts with respect to the truth topography.

A possible source of systematic error and tilt is the procedure for constraining the solution contained in each individual maplet by considering the local solutions of all overlapping maplets and applying an 'overlap constraint'. The standard procedure for tiling bigmaps and iterating maplets with regards to overlap constraint is as follows.

Tiling a bigmap:

• The region in which the bigmap is contained is tiled with a suite of new maplets which will have a Ground Sample Distance (GSD) smaller than (higher resolution than) the GSD of the existing maplets contained in the bigmap area.
• Center coordinates for the new maplets are generated based on the maplet GSD, the Q size, and the overlap percentage.
• Starting from the North-West-most maplet center coordinate, the first new maplet is created.
• Images are added, eliminated and registered for the new maplet. A template photoclinometric solution is generated from the images. Once sufficiently mature, the template is conditioned using differential stereo, constrained using the local information of overlapping maplets and used to build the topographic model. A stereogrammetric correction is then applied causing adjustment to the vertical and horizontal location of the maplet. During tiling, the overlapping maplets are the pre-existing larger GSD (lower resolution) maplets.
• The maplet center coordinate immediately to the East of the first new maplet is then used to create the next new maplet.
• Again, images are added, eliminated and registered, a template photoclinometric solution is generated from the images, conditioned using differential stereo, overlap-constrained and used to build topography, finally the maplet location is adjusted by applying a stereogrammetric correction. Now the overlapping maplets will include the first new same-GSD maplet, which overlaps only along one edge of the new maplet.
• This procedure continues until the Northernmost row of new maplets has been created and overlap constrained.
• The process is then repeated to create the row of maplets immediately South of the Northernmost row of new maplets.
• Tiling is complete when the South-Eastern-most new maplet has been generated.
• As new maplets are created, their names are appended to USED_MAPS.TXT. Since these maplets are contained in a 20mx20m region, they have the same lat/wlong identifier (in this test-case, EE) and they are distinguished by an incrementing numeric identifier, e.g. EE0001 and EE0002.

Iterating the maplets contained in a bigmap:

• The maplets are processed in the order in which they appear in USED_MAPS.TXT, in this test-case, the order in which they were created (EE0001, EE0002, etc.). Thus each GSD suite of maplets is iterated in turn, starting from the largest GSD (lowest resolution) suite and ending with the smallest GSD (highest resolution) suite. The maplets in each GSD suite are iterated from the North-Western-most maplet to the South-Eastern-most maplet in rows traveling West to East.
• For each maplet, S/C position and camera pointing is adjusted to increase image-template correlation (adjustment of image registry), the template photoclinometric solution is generated from the images, conditioned using differential stereo, constrained using the local information of overlapping maplets and used to build the topographic model. During iteration, the overlapping maplets will include iterated higher GSD (lower resolution) maplets, the not-yet-iterated lower GSD maplets, iterated same-GSD maplets to the North and West of the maplet, and not-yet-iterated same-GSD maplets to the East and South of the maplet.

This study therefore intends to investigate the contribution of the order in which overlap constraints are applied, to the tilt local to the reference peak, calculate the magnitude and direction of tilt at various GSD tiling and iterating steps, calculate the vertical displacement introduced by tilt and observe systematic error in a larger evaluation area around the reference peak.

Methodology

Test Over11I was chosen for the investigation since it has been the focus of user-to-user and server-to-server repeatability and has a low number of images, enabling high relative speed and ease in processing.

The following tests were conducted:

• Baseline: Test Over11I tiled at a GSD of 10cm and 5cm. Iterated 100 times at a GSD of 5cm.

• Repeat: A same-user, same-server, equal-number-of-core-processors repeat of the baseline test Over11I was conducted to obtain a measure of uncertainty.

• Backwards Tiling/Iterating: The baseline test was re-run with the only difference being the order in which maplets were tiled and iterated. The standard order was reversed; maplets were processed starting with the South-East-most maplet, traveling East to West along each row, finishing with the North-West-most maplet. (There was no change in the order in which GSD maplet suites were processed; as per standard processing, maplet suites were processed starting with the largest GSD (lowest resolution) maplets and finishing with the smallest GSD (highest resolution) maplets.

• Higher Resolution Tiling: The Baseline and Backwards Tiling/Iterating Tests were tiled at smaller GSDs (higher resolutions), starting from the last tiling step, thus their processing steps were: Tiled at 10cm, 5cm, 2cm, and 1cm, with no iterations.

The overlap ratio was consistent across GSD tilings giving an overlap of approximately 62%.

Images, from left to right, 4m x 4m bigmap centered on the reference peak, and 10m x 10m bigmap centered on the reference peak.

Two evaluation areas were used to obtain evaluation statistics, CompMapVec heat plots and FlatMapVec central transits, as illustrated above. The 4m x 4m evaluation area is consistent with the full suite of Over11 Reference Peak test evaluations. An additional 10m x 10m area centered on the reference peak was evaluated in order to consider systematic errors, tilt and relative evaluation statistics for a larger terrain around the reference peak which contains multiple peak and crater features. As per the Over11 test suite, the evaluation area GSD was 1cm across tests.

Local to the reference peak, the truth terrain exhibits an underlying gradient of approximately 0.3°. The local tilt error (herein referred to as 'local tilt') is therefore calculated as the difference between the model and truth local gradients. In turn, the magnitudes of the local truth and DTM gradients were calculated from the respective heights of their terrain at the edges of the 4m x 4m flatmap central transits running due North and due East as illustrated in the above diagram. Local tilt therefore refers to the gradient of the underlying DTM landscape with respect to the truth landscape extending only 2m from the center of the reference peak.

The local tilt induced vertical error was calculated by obtaining the vertical displacement due to tilt across 4m, as illustrated in the above diagram.

Results and Discussion

Repeatability

The results of the repeatability test are stored here: Local Tilt Repeatability Results

The Over11I baseline and repeat test demonstrate high repeatability:

• CompMapVec, Compare OBJ and Compare OBJ with Optimal Translation and Rotation RMS errors repeat within 2% for tiling steps and early iterations, and repeat within 7% beyond 10 iterations;

• Formal Uncertainties repeat within 1% for tiling steps, but vary by up to 30% with iteration steps;

• Baseline and repeat CompMapVec heat plots and FlatMapVec central transits are indistinguishable from one another.

The differences in RMS errors, CompMapVec heat plots and central transits reported herein are therefore due almost entirely to the order in which the maplets are processed and overlap constraints applied.

Dynamic Tilt Behavior

To view animations illustrating the dynamic tilting behavior over tiling and iteration steps, please follow the links:

Local Tilt Heat Plot Animations

Local Tilt Central Transit Animations

The static images are stored here:

Local Tilt Heat Plots

Local Tilt Central Transits

The dynamic tilt behavior is different depending on the order by which the maplets have been processed and overlap constraints applied.

For both the standard ('forwards') and the 'backwards' tiled/iterated tests, 10cm tiling results in a significant tilt across the 10m evaluation area which decreases with smaller GSD (higher resolution) tiling and iteration, although the entire DTM surface drifts vertically, away from the truth.

Interestingly the direction of tilt comparing the standard (forward) and backward tiling/iteration results is broadly similar West to East but is not consistent North to South, but neither is it opposite in direction as would perhaps be expected given the reversed order of processing maplets and applying overlap constraints.

In terms of the central transits, the West-East central transits for standard (forward) and backward tiling/iteration are not dissimilar, with differences in tilt magnitude after the 10cm and 5cm tiling diminishing through iteration to reach near-conformity by 50 iterations. The North-South transits however remain dissimilar in tilt magnitude and direction throughout the 10cm and 5cm tiling and iteration steps.

Tilt is no longer visible for the 2cm and 1cm tilings and the standard (forward) and backward tiling/iteration tests demonstrate close conformity.

Dynamic tilt behavior is also recognizably similar across tests Over11I, S, T and U, which differ only in the number of images and the viewing conditions.

Magnitude of Tilt and Scalability with DTM Ground Sample Distance

The results of the tilt magnitude and vertical displacement calculations are stored here: Local Tilt Magnitude Results

Test Over11I Standard (Forwards) and Backwards Tiling/Iterating:

• North-South transit local tilt: The direction of tilt is opposite for the forward and backward tiling/iterating tests throughout tiling and iterating steps.

  • Forwards Tiling/Iterating: Tilt increases from -0.02° at the 10cm tiling step to to -0.5° at the 5cm tiling step, oscillates around -0.4° throughout 5cm iteration, and resolves to -0.02° at 2cm and 1cm tiling steps.

  • Backwards Tiling/Iterating: Tilt decreases from 0.6° at the 10cm tiling step to 0.3° at the 5cm tiling step, decreasing slowly throughout 5cm iteration to 0.02°, but only decreases to 0.1° with 2cm and 1cm tiling steps.

• West-East transit local tilt: The direction of tilt is the same but the magnitude different for the forward and backward tiling/iterating tests throughout tiling and iterating steps.

  • Forwards Tiling/Iterating: Tilt decreases from -0.7° at the 10cm tiling step, to -0.4° at the 5cm tiling step, resolves to zero throughout 5cm iteration, only becoming non-zero (0.1°) at the 1cm tiling step.

  • Backwards Tiling/Iterating: Tilt decreases from -1.1° at the 10cm tiling step, to -0.6° at the 5cm tiling step, resolves to -0.1° throughout 5cm iteration and the 2cm and 1cm tiling steps.

The magnitude and direction of tilt through tiling and iterating steps is broadly similar across tests Over11I, S, T, and U. Tilt magnitude and direction does not therefore appear to be significantly affected by the number of images or the viewing conditions.

The local tilt magnitude and the induced vertical displacement across 4m scales with DTM ground sample distance as follows:

• 10cm tiling: local tilt (local gradient error) of up to 2.2°, induced vertical displacement of up to 15.0cm.

• 5cm tiling: local tilt (local gradient error) of up to 0.6°, induced vertical displacement of up to 4.0cm.

• 5cm iteration 50: local tilt (local gradient error) of up to 0.4°, induced vertical displacement of up to 2.4cm.

• 2cm tiling: local tilt (local gradient error) of up to 0.2°, induced vertical displacement of up to 1.0cm.

• 1cm tiling: local tilt (local gradient error) of up to 0.2°, induced vertical displacement of up to 1.0cm.

Evaluation Statistics: Standard (Forwards) vs Backwards Bigmap Tiling and Iterating

Link to charts: Local Tilt Stats

Comparing the evaluation statistics between standard (forwards) and backwards tiling/iteration gives a variety of results in terms of trends and differences in formal uncertainty and RMS error magnitudes.

Formal Uncertainties demonstrate a very similar trend although the inherent oscillatory behavior causes deltas of up to 6% at 10cm/5cm tiling, up to 35% during 5cm iteration and up to 14% at 2cm/1cm tiling. These deltas are similar to the Over11I repeatability results, therefore the formal uncertainty is not considered to be affected by the order in which the maplets are processed and overlap constrained.

The CompMapVec RMS error is vastly different depending on the order in which maplets have been processed and overlap constrained. Although the trend is somewhat similar, deltas of up to 90% only decrease to <5% after more than 80 iterations at a GSD of 5cm. Deltas decrease to only 26% with 2cm and 1cm tiling. However, CompMapVec RMS errors remain within the GSD throughout the tests, as follows:

Order of maplet processing and overlap constraint also impacts the magnitude of Compare OBJ RMS errors. Compare OBJ RMS error with and without optimal translation and rotation demonstrate both similar trends and similar deltas with respect to the order in which maplets are processed and overlap constrained. Tiling steps give deltas of less than 16%, but 5cm iteration increase deltas to up to 45% without optimal translation and rotation, settling to 25% after 60 iterations, and up to 30% with optimal translation and rotation. However, CompareOBJ RMS errors remain within the GSD throughout the tests, as follows:

A significant increase in RMS error uncertainty is therefore introduced by changing the order in which maplets are processed and overlap constrained. This is a significant finding with regards to the statistics used to evaluate the quality of the DTM produced using the SPC software suite when quoting statistics below the GSD of the DTM.

10m Evaluation Area Statistics

Link to charts: 10m TAG vs 4m Peak Stats

Evaluation statistics are compared for two evaluation sites: a 4mx4m area centered on the reference peak, and a 10mx10m area centered on the reference peak. The larger evaluation site includes a higher number of topographic features.

In general, RMS errors increase with an increased evaluation area indicating that the reference peak is not the most error-prone feature in the local terrain. RMS error trends with tiling and iterating steps are broadly similar across the two evaluation site areas.

With an increase in evaluation from 4mx4m to 10mx10m:

• CompMapVec RMS errors are multiplied by up to 4, with the value peeking above the GSD in the 2cm standard tiling test (the backward 2cm tiling test maintains an RMS error below the GSD);

• CompareOBJ RMS errors (with and without optimal translation and rotation) are multiplied by up to 2.2.

  Baseline Test Statistics for 4mx4m and 10mx10m Evaluation Areas:

These findings are broadly consistent across Over11I, S, T, and U tests.

Conclusions and Recommendations

Uncertainty in instantaneous gradients in the DTM should be considered as part of the assessment of the stereophotoclinometry software suite for OSIRIS-REx safety-mapping and TAG site selection. Local tilt errors have been measured as part of this investigation for an artificial, simplified terrain containing peaks with 20° slopes and various-sized parabolic craters. The tilt magnitude (of the DTM with respect to the truth topography) and induced vertical displacements have been found to scale with the ground sample distance of the DTM. Local gradient errors (as measured across 4m) have been found to be small (<0.4° at a DTM GSD of 5cm) and the induced vertical displacements found to be within the vertical uncertainty obtained in similar tests (<4cm at a DTM GSD of 5cm). However, gradient errors measured across smaller distances may be greater and it is not clear how dependent the local tilt behavior is on the local terrain and topographic features.

The order in which maplets are processed and overlap constraints applied causes significant changes to the dynamic behavior of the DTM tilt with respect to the truth topography and the magnitude of local gradient errors, vertical displacements, and the statistics used to assess the quality of the DTM. Although formal uncertainty is not affected by maplet processing order, deltas of up to 45% have been found in CompareOBJ RMS errors (with and without optimal translation and rotation) and up to 90% in CompMapVec RMS errors when processing the maplets in the standard order and in reverse order. It should be noted that all RMS errors remain below the GSD of the DTM (CompMapVec RMS errors below 50% of the GSD, CompareOBJ RMS errors below 80% of the GSD). This finding is significant therefore when quoting statistics below the GSD of the DTM.

By altering the order and manner in which maplets are processed and overlap constraints are applied, it may be possible to decrease the systematic errors which are undoubtably influencing the nature and magnitude of local tilting of the DTM with respect to the truth topography. Maplets could be processed and overlap constrained in a random order which changes from processing step to processing step. Errors will certainly remain, but will be more randomly distributed and less amenable to systematic propagation through the DTM.

A further parameter for study is the maplet overlap percentage used when tiling a bigmap. This full suite of tests has used a consistent overlap percentage (approximately 62%) which has since been deemed to be higher than the recommended overlap percentage (30%). A decrease in the degree by which same-GSD maplets overlap may alter the significance of overlap constraint affects found herein.