Test 05: OptimalF2

Aim and Objectives

Purpose of test: To obtain highest possible topographic accuracy and resolution for TAG site 1, given perfect camera position and pointing and optimal SPC image suite.

Additional Objectives

• To obtain user-in-the-loop and server-in-the-loop RMS error variations.
• To obtain user-in-the-loop procedural variation.
• To obtain beginner/expert user variation.

Acronyms

Definitions and Disambiguation

Methodology

Data

• Data was generated by Imager_mg using shape3.8 on ormacsrv1.
• PT10A - Centered at the middle of TAG site one.
• Image Suite: Optimal flight path, image resolutions: 50cm, 20cm, 10cm, 5cm.
  • S/C:
    • Every 20° azimuth angle, 45° zenith angle
  • Sun:
    • 0, 90°, 180°, 270° azimuth angle, 45 zenith angle
    • 135° azimuth angle, 30° zenith angle

Spacecraft:

Sun:

User/Server-in-the-loop:

Bigmaps

The following TAG1 bigmaps were tiled/iterated and evaluated:

Starting topography defined from: START1.MAP:

• GSD = 25cm;
• Q size = 150;
• width = 75m;
• center lat/wlong = -8.027, 262.768.

DTM Bigmap parameters:

Evaluation Bigmap parameters:

• GSD = 5cm;
• Q size = 500;
• width = 50m;

Tiling Parameters

Tiling parameters may have varied between users:

alpha-numerics in brackets refer to Lithos/LithosP menu options.

Iteration Parameters

Iteration parameters varied between users:

alpha-numerics in brackets refer to Lithos/LithosP menu options.

Results

Please see Tables and Figures.

Discussion

User Differences in Tiling and Iteration Parameters

The only differences in tiling parameters pertain to the auto-elimination of images during the initial 35cm tiling. The maximum permissible emission angle varied between 50 and 60 degrees. The minimum allowable ration between image scale and misplace varied between 0 and 0.25. All other tiling step parameters were the same between users.

Iteration parameters however varied greatly between users, with no two users applying an identical set of parameters. All users calculated the central vector (v1), however not all users reset albedo/slopes, conditioned with differential stereo and/or conditioned with shadows.

User Differences in Processing Steps

Since users had different criteria for triggering the cessation of iteration and continuing to the next tiling step (such as 2 iterations per tiling step, or, no further change in evaluation statistics), users iterated solutions a varied number of times.

Number of iterations conducted at each GSD step:

User Errors

Additionally, two users applied incorrect or non-standard processing procedures for some portion of the test related to their level of experience and training:

• DLambert calculated statistics on an incorrectly sized evaluation bigmap during the 35cm tiling and first iteration leading to loss of statistics data for these processing steps;
• DLambert failed to eliminate low correlating images during the first three 35cm iteration steps, leading to initial problems in the topography which are reflected in the formal uncertainties and RMS errors. The errors receded at the 18cm tiling step.
• KDrozd used different parameters in the utility program FITS2OBJ, upstream of obtaining the CompareOBJ RMS error, as other team members, leading to differing RMS error statistics which do not necessarily reflect differences in the evaluation bigmap compared to other users' evaluation bigmaps.

User Differences in Evaluation Statistics

CompareOBJ RMS errors show large variation from user to user. Some of these variations can be directly linked to user procedural differences, for example the high RMS errors through the 35cm iterations for DLambert and KDrozd are directly related to incorrect and non-standard processing (see above section). Other differences reflect some combination of:

• variation in iteration parameters (see above section);
• variation in number of iterations conducted;
• variation in user skill and experience in manually registering/eliminating problem images;
• server-based variation;
• number-of-core-processors-based variation:
• SPC uncertainty due to random seeds utilized in maplet slope and height calculations.

Formal uncertainties were more similar between users, differences can be directly linked to user procedural differences, specifically incorrect and non-standard processing covered in the previous section.

Overall SPC Performance

All users obtain formal uncertainties below the highest image resolution (< 5cm).

Final CompareOBJ RMS errors vary. The two final solutions which reached a GSD of 5cm obtained an RMS error of 12.3cm and 4.5cm. Slight improvements in CompareOBJ RMS can be gained by using the CompareOBJ optimal translation/rotation utilities.

Conclusions and Recommendations

Formal uncertainties performed well in respect to both between-user variation and final performance measures. However, the CompareOBJ RMS error did not, showing high variability user-to-user and high variability in final performance measures.

User-to-user differences can be minimized in future testing through a combination of standardization of tiling/iteration parameters and iteration cessation criteria, and increase in user skill and experience.

Tiling and iteration parameters should be investigated to find the best-practice for standardized seed file sets. This recommendation is addressed in Test06 - FindHeights.

To further investigate the performance of SPC with perfect camera position and pointing and optimal viewing conditions, a suite of higher-resolution images will be added and the best performing 5cm DTM will be tiled and iterated at higher ground sample distances. This recommendation led to the implementation of Test08 - OptimalUltraF2.

Tables and Figures

RMS Distance (cm) - Compare OBJ (with no translation/rotation)

RMS Distance (cm) - Compare OBJ (with optimal translation/rotation)

Formal Uncertainty (cm) - RESIDUALS

35cm GSD Maplets - 50cm Resolution Images

18cm GSD Maplets - 20cm Resolution Images

9cm GSD Maplets - 10cm Resolution Images