= TestF3G - Results = == Definitions == '''CompareOBJ RMS:''' The root mean square of the distance from each bigmap pixel/line location to the nearest facet of the truth OBJ.<
> '''PTG:''' The formal camera pointing uncertainty.<
> '''SCOBJ:''' The S/C position vector from the center of the asteroid.<
> '''VSO:''' The formal SCOBJ uncertainty.<
> == Additional Tests == === S/C position and pointing uncertainties === The Detailed Survey PolyCam F3G data set had large S/C position and pointing uncertainties, unsuitable for Detailed Survey Baseball Diamond trajectories: * VSO = 1km; * PTG = 1mrad. An additional suite of tests was therefore run with a duplicate Detailed Survey PolyCam dataset with S/C position and pointing uncertainties set to one-sigma: * VSO = 6.4m; * PTG = 0.217mrad. Results for both data sets are presented herein. === Model shift === In the process of generating the 35cm Preliminary Shape Model a shift in the body center with respect to the inertial center occurred. The magnitude of the shift was approximately 2m. Since the final S/C position across F3G subtests lay in a region 2m to 8m from the true S/C position, the effect of the body center shift came into question. To investigate, an alternative 35cm Preliminary Shape Model with no body-center shift was generated. One subtest was re-run using the alternative start model. == Key Findings == The results show no significant difference in the final S/C position (SCOBJ) and model accuracy (as measured with CompareOBJ RMS) with respect to the magnitude of S/C position and pointing perturbation within the ranges tested, indicating that the SPC-driven modeling is immune to S/C position and pointing perturbation up to three standard deviations. It should be noted that S/C position perturbation was divided equally between the SCOBJ components, resulting in a distance from the truth position which was a multiple of the standard deviation of 6.4m. Therefore: * maximum lateral perturbation was a multiple of 3.7m (6.4m/sqrt(3)); * maximum normal perturbation (wrt body center) was 3.7m (6.4m/sqrt(3)). It is assumed that the worst case scenario is a 3 x sigma (19.2m) lateral perturbation. The maximum possible lateral perturbation tested was 3 x 3.7m = 11.1m. In all cases, the final SPC-derived S/C position is within 8m of the true S/C position, but in only two cases is within 2m of the true S/C position. The actual distance from the true S/C position is not dependent on the initial perturbed position - for example in the 0.25 x sigma case, the SPC-derived S/C position (distance from truth: 1.6m to 6.9m) in most cases moves further away from the true S/C position than its initial position (distance from truth: 1.6m). The final SPC-derived S/C positions appear to be clustered around an incorrect solution 2m-8m distant from the true S/C position. == CompareOBJ RMS == Three CompareOBJ RMS values are presented for each subtest and each S/C position and camera pointing uncertainty: * The largest CompareOBJ RMS (approx. 65cm across subtests) is obtained by running CompareOBJ on the untranslated and unrotated evaluation model. * The second smallest CompareOBJ RMS (approx. 15cm across subtests) is obtained by running CompareOBJ with its optimal translation and rotation option. * The smallest CompareOBJ RMS (approx. 9cm across subtests) is obtained by manually translating the evaluation model and searching for a local CompareOBJ RMS minimum. The CompareOBJ optimal translation routine is not optimized for the evaluation model scale (5cm pix/line resolution). Manual translations of the bigmap were therefore conducted in an attempt to find a minimum CompareOBJ RMS. The manually translated evaluation models gave the smallest CompareOBJ RMSs. The CompareOBJ RMS (for the final 5cm resolution 20m x 20m evaluation bigmap) does not appear to be affected by the magnitude of S/C position and pointing perturbation within the ranges tested. CompareOBJ RMSs differ slightly with S/C position and camera pointing uncertainties: * approx. 5cm difference for CompareOBJ RMS without translation/rotation; * approx. 2cm difference for CompareOBJ RMS with optimal translation and rotation; * approx. 0.4cm difference for CompareOBJ RMS with manual translation. {{attachment:CompareOBJ_VSO_1km_resized60pt.png}} {{attachment:CompareOBJ_VSO_1sigma_resized60pt.png}} '''CompareOBJ Optimal Translations:''' ||'''Sub-Test'''||'''S/C Position Uncertainty'''||'''Perturbation Magnitude'''||'''RMS (cm)'''||||||'''Translation (cm)'''||'''Translated Distance (cm)'''|| ||F3G7||1km||0.25 x sigma||15.7849||85.0698||62.3596||-14.3765||106.4532|| ||F3G6||1km||0.50 x sigma||15.1147||84.5538||61.6624||-15.3434||105.7687|| ||F3G5||1km||0.75 x sigma||14.3150||95.8438||59.9313||-21.6901||115.1011|| ||F3G3||1km||1.00 x sigma||13.6399||106.4870||58.2162||-27.3527||124.4057|| ||F3G4||1km||1.50 x sigma||15.8949||79.1224||63.2865||-19.6432||103.2056|| ||F3G2||1km||2.00 x sigma||13.6938||110.1339||58.9454||-23.2403||127.0596|| ||F3G1||1km||3.00 x sigma||14.6271||93.5937||61.6997||-26.8422||115.2698|| ||'''Sub-Test'''||'''S/C Position Uncertainty'''||'''Perturbation Magnitude'''||'''RMS (cm)'''||||||'''Translation (cm)'''||'''Translated Distance (cm)'''|| ||F3G7||6.4m||0.25 x sigma||15.9614||95.5555||66.8688||-17.6793||117.9613|| ||F3G6||6.4m||0.50 x sigma||16.0260||93.7254||66.9687||-17.2331||116.4743|| ||F3G5||6.4m||0.75 x sigma||16.4387||88.3626||69.0785||-12.1628||112.8172|| ||F3G3||6.4m||1.00 x sigma||9.5449||167.5406||57.3951||-27.0780||179.1571|| ||F3G4||6.4m||1.50 x sigma||16.0525||93.4737||70.0956||-15.0998||117.808=1|| ||F3G2||6.4m||2.00 x sigma||16.1591||94.1139||69.1165||-19.95051||118.4514|| ||F3G1||6.4m||3.00 x sigma||15.2938||101.8134||70.3467||-20.3179||125.4091|| '''CompareOBJ Manual Translations:''' ||'''Sub-Test'''||'''S/C Position Uncertainty'''||'''Perturbation Magnitude'''||'''RMS (cm)'''||||||'''Translation (cm)'''||'''Translated Distance (cm)'''|| ||F3G7||1km||0.25 x sigma||8.0140||187.5||47.5||-30||195.74|| ||F3G6||1km||0.50 x sigma||7.9343||187.5||47.5||-30||195.74|| ||F3G5||1km||0.75 x sigma||7.9579||187.5||47.5||-30||195.74|| ||F3G3||1km||1.00 x sigma||7.6749||187.5||47.5||-30||195.74|| ||F3G4||1km||1.50 x sigma||7.9964||187.5||47.5||-40||197.52|| ||F3G2||1km||2.00 x sigma||7.8591||187.5||47.5||-40||197.52|| ||F3G1||1km||3.00 x sigma||7.6739||187.5||47.5||-50||209.19|| ||'''Sub-Test'''||'''S/C Position Uncertainty'''||'''Perturbation Magnitude'''||'''RMS (cm)'''||||||'''Translation (cm)'''||'''Translated Distance (cm)'''|| ||F3G7||6.4m||0.25 x sigma|| ||F3G6||6.4m||0.50 x sigma|| ||F3G5||6.4m||0.75 x sigma|| ||F3G3||6.4m||1.00 x sigma|| ||F3G4||6.4m||1.50 x sigma|| ||F3G2||6.4m||2.00 x sigma|| ||F3G1||6.4m||3.00 x sigma|| == Image Footprints == The first graph shows footprints for all Detailed Survey PolyCam pictures which were included in the model. The second graph shows the four pictures down-selected due to their coverage of the 20m x 20m evaluation region, and their almost complete containment within the iterated 100m x 100m region. {{attachment:vertices_all_resized.png}} {{attachment:vertices_eval_resized.png}} == Distance SCOBJ(truth) to SCOBJ(solution) == The distance of the final SPC-derived S/C position from the true S/C position is plotted for the full Detailed Survey PolyCam image set for each magnitude of perturbation. The evaluation images are plotted in red. 3D graphs of final SPC-derived SCOBJ and true SCOBJ are then plotted for each picture. The first four are the down-selected evaluation pictures, the rest of the image set is included for comparison. The pattern of final SPC-derived SCOBJ is broadly consistent across magnitudes of perturbation. The position correction is mostly a normal correction with lateral movement, bringing the modeled S/C position within an approx. 8m-radius sphere around the true position (or, in the case of perturbations<8m, pushing SCOBJ outwards up to 8m). {{attachment:scobj_distance_resized.png}} === Evaluation Pictures === '''Example nominal SCOBJs:''' {{attachment:P601293751G3_nominal_resized.png}} ''' Final solution SCOBJs:''' {{attachment:P601293751G3_final_resized.png}} {{attachment:P601372862G2_final_resized.png}} {{attachment:P601372868G2_final_resized.png}} {{attachment:P601372874G2_final_resized.png}} === Remaining Detailed Survey PolyCam Pictures === {{attachment:P601293196G2_final_resized.png}} {{attachment:P601293757G3_final_resized.png}} {{attachment:P601372298G3_final_resized.png}} {{attachment:P601372304G3_final_resized.png}} {{attachment:P601372310G3_final_resized.png}} {{attachment:P601372316G3_final_resized.png}} {{attachment:P601372856G2_final_resized.png}}