> == CompareOBJ RMS == CompareOBJ does not appear to be affected largely by the quality of imaging conditions. We have come to realize that RMS is not a sufficient means of evaluating topography for accuracy in and of itself, but rather should be used in addition to other criteria. '''CompareOBJ RMS:''' ||'''Sub-Test'''||'''RMS (cm)'''||'''Optimal Trans-Rot RMS (cm)'''|| ||F3B1||62.418||18.120|| ||F3B2||62.028||17.529|| ||F3B3||62.927||19.363|| ||F3B4||63.519||17.879|| ||F3B5||63.273||17.696|| There are a few caveats to the above results that are important to understand. First, we now know that there is a shift of approximately 2m of the coordinate system between the truth model and the evaluation model. This does not have an effect on the topography, just its location in 3D space. This means that the plain CompareOBJ RMS is fundamentally calculating RMS in the wrong place, and thus the values in the second column of the above table are fairly meaningless. Next, it is also known that CompareOBJ's translation and rotation optimization algorithm does not always find the correct location, which is indeed what is happening here. We have used other tools (primarily Meshlab) to verify that these translations are indeed incorrect. This means that the RMS values in the third column above, while more accurate than those in column two, are still incorrect. <

> == Normalized Cross Correlation == A normalized cross correlation (NCC) analysis was done both on the evaluation bigmap (20x20 m) as a whole, and the individual 5 cm/pixel maplets that form the 50x50 m study region. The full bigmaps are pictured below with their respective correlation values. Visually it is easy to see that the topography is drastically affected by the number of images. Features on the surface move about, and the bigmap clarity is worse for fewer images. <

> === F3B1 === {{attachment:EV20B1.png}} <

>Correlation Score: 0.7486 <

>Number of Images: 4 <

><

> === F3B2 === {{attachment:EV20B2.png}} <

>Correlation Score: 0.7489 <

>Number of Images: 7 <

><

> === F3B3 === {{attachment:EV20B3.png}} <

>Correlation Score: 0.7842 <

>Number of Images: 14 <

><

> === F3B4 === {{attachment:EV20B4.png}} <

>Correlation Score: 0.7822 <

>Number of Images: 70 <

><

> === F3B5 === {{attachment:EV20B5.png}} <

>Correlation Score: 0.7698 <

>Number of Images: 140 <

><

> Next the statistics for each maplet in the 50x50 m study region are below. The correlation value was plotted for each maplet in terms of x-y position on the truth bigmap to show where the topography performs better or worse. In addition, the individual translations are plotted to show the distance each maplet had to move to where the correlation was found. Seeing strong agreement in the translations is very encouraging because we know we have a bias error of approximately 2m in our model. From these translations we are able to come up with exclusion criteria for maplets which correlated in an incorrect location. We set a standing 100 pixel translation for a fail criteria, and a 5 pixel deviation from average as a marginal boundary. This is useful because while a maplet may have a high correlation, if it is in the wrong place, this correlation is meaningless. The statistics for each subtest are outlined in the table below, and all values were calculated after the outliers (moved more than 100 pixels) were removed from the set. ||'''Sub-Test'''||'''Average Translation (px)'''||'''Standard Deviation (px)'''||'''Pass/Marg/Fail (%)'''|| ||F3B1||39.170||0.661||98.78/0.61/0.61|| ||F3B2||39.371||0.671||99.00/0.50/0.50|| ||F3B3||40.424||0.777||100.00/0.00/0.00|| ||F3B4||42.786||0.790||99.11/0.44/0.44|| ||F3B5||42.023||1.100||99.11/0.44/0.44|| <

> === F3B1 === {{attachment:F3B1cor.png}} {{attachment:F3B1trans.png}} <

><

> === F3B2 === {{attachment:F3B2cor.png}} {{attachment:F3B2trans.png}} <

><

> === F3B3 === {{attachment:F3B3cor.png}} {{attachment:F3B3trans.png}} <

><

> === F3B4 === {{attachment:F3B4cor.png}} {{attachment:F3B4trans.png}} <

><

> === F3B5 === {{attachment:F3B5cor.png}} {{attachment:F3B5trans.png}}