Differences between revisions 7 and 37 (spanning 30 versions)

F3E Sub-Tests

The goal of these sub-tests are to determine whether limbs shots at better resolution help generate a more detailed SPC product. The sub-tests involve testing 100cm, 50cm, 15cm, and 5cm limb images. Each sub-test will be run twice. Once with high stereo data and the other with low stereo data.

Important Note

The entirety of this test will take place within the /SPC_Test/F3E-Limbs directory on ormacsrv1. When ever an action is performed during the test, commands will be logged for future reference. To do this type the following commands

script 
date
pwd 
ls -l

Then continue with a procedural step. When done with a procedural step cancel the script and place it into the /SPC_Test/F3E-Limbs/log directory.bWith script still enabled do the following to end it while in the test directory.

exit
mv typescript ../log/<name/path>

Procedure

The procedures shown can be repeated for every sub-test in the table.

Sub-Test Number	Base image set	Limb image resolution	Methods	Evaluation
F3E-1	Low Stereo	100cm	Iteration x5 GEOMETRY 2	Spacecraft & Landmark position
F3E-2	Low Stereo	50cm	Iteration x5 GEOMETRY 2	Spacecraft & Landmark position
F3E-3	Low Stereo	15cm	Iteration x5 GEOMETRY 2	Spacecraft & Landmark position
F3E-4	Low Stereo	5cm	Iteration x5 GEOMETRY 2	Spacecraft & Landmark position
F3E-5	High Stereo	100cm	Iteration x5 GEOMETRY 2	Spacecraft & Landmark position
F3E-6	High Stereo	50cm	Iteration x5 GEOMETRY 2	Spacecraft & Landmark position
F3E-7	High Stereo	15cm	Iteration x5 GEOMETRY 2	Spacecraft & Landmark position
F3E-8	High Stereo	5cm	Iteration x5 GEOMETRY 2	Spacecraft & Landmark position

1. Setting up the test directory

Depending on the subtest, the high or low stereo prep directory is being copied over to the test directory with the test directory being emptied first.

Test F3E Test Directory Set-Up

2. Symbolically Linking the Correct List of Images to PICTLIST.TXT

While in the test directory symbolically link the PICTLIST.TXT file to the correct file in the support directory for the respected test. For example, for the 100cm limb shot test using low stereo data do the following while in the test directory:

rm PICTLIST.TXT
ln -s ../lsupport/PICTLISTF3Elow_1.txt PICTLIST.TXT

3. Register Sub-Test Limb Images

Copy the correct registering file in in the lsupport directory for the respected test as make_scriptR.in in the test directory. For example, for the 100cm limb shot test do the following while in the test directory.

cp ../lsupport/F3E1Registering.txt make_scriptR.in

Then execute the following procedural block

Test F3E Register

4. Run 5 iterations

Next, run 5 iterations one after one. Each iteration should be done by following the procedural block below.

Test F3E Iterate -- Use the 100 cm area scenario

5. Obtain 5cm Landmark Point Cloud and Evaluate its obj to the Truth Model

First enter the following:

bigmap < support/EVAL20.in

This will output a list of the landarks within the evaluation region of the tag site. Copy and paste this list into LMKFILES/LMRKEvalList.txt

While in the test directory input the following commands.

grep "0.0500" MAPINFO.TXT | awk '{ print $1 }' >> LMKFILES/LMRK05.txt        # Makes a text file of all 5cm landmark names
cd /SPC_Test/F3E-Limbs/test/LMKFILES
while read line; do f=`grep $line LMRKEvalList.txt | wc -l`; if [ $f -gt 0 ]; then echo $line >> EvalLMRK05List.txt; fi; done < LMRK05.txt     # Makes a list of 5 cm landmarks in 20 by 20 meter area around tagsites
while read line; do grep "VLM" $line* | awk '{ print $1, $2, $3 }' >> PC05.txt; done < EvalLMRK05List.txt        # Makes the point cloud
cp PC05.txt LMRK05.txt ../../results/<subtest name>/                            # Copy point cloud files to results
cp PC05.txt ~/send/                                                          # Copy point cloud to send directory so it can be evaluated on personal machine

Now, these are the exact commands I enter to evaluate the point cloud on my machine.

cp send/PC05.txt send/CompareOBJfiles/
cd /Users/kdrozd/Dropbox\ \(OREx\)/SPC-ORex/eval_tools/altwg-2016.01.25-macosx-x64/bin
vi ~/send/CompareOBJfiles/PC05.txt
:%s/D/E/g
ZZ
./ComparePointCloudToOBJ --compute-optimal-translation-and-rotation ~/send/CompareOBJfiles/PC05.txt truthTAG1-5cm-1.1K.obj

The Copy the output of the last command into a file in the the subtest directory located in the /SPC_Test/F3E-Limbs/results directory.

6. Run Geometry 2

Next run Geometry to just update the spacecraft position and pointing (not landmark heights). To do this follow the procedural block below, but when Geometry is run make sure only option 2 and 0 are selected under the geometry section.

Test F3E Geometry 2

7. Evaluate Position and Pointing of Spacecraft

First step is to make a text file that contains a list of the names of the 4 high or 4 low stereo images appended with the tests respected 4 limb shot images. This text file can be made anyway and does not require a direct procedure. Just make sure the textfile is a list only. Call the textile imgname.txt.

Then follow Diane's procedural block below. Replace listSum, with imgname.txt. Also when, logging the SCOBJ and point vector files put them in the test's respected directory in the /SPC_Test/F3E-Limbs/results directory.

Test F3E Procedure Block - Capture SCOBJ & Pointing

-  ⇤ ← Revision 7 as of 2016-03-11 09:52:24 → 
  Size: 2106
  Editor: KristoferDrozd
  Comment:
+   ← Revision 37 as of 2016-03-16 17:08:27 → ⇥
  Size: 5675
  Editor: KristoferDrozd
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-The goal of these sub-tests are determine whether limbs shots at better resolution help generate a more detailed SPC product. The sub-tests involve testing 100cm, 50cm, 15cm, and 5cm limb images. Each sub-test will be run twice. Once with high stereo data and the other with low stereo data.
+The goal of these sub-tests are to determine whether limbs shots at better resolution help generate a more detailed SPC product. The sub-tests involve testing 100cm, 50cm, 15cm, and 5cm limb images. Each sub-test will be run twice. Once with high stereo data and the other with low stereo data.

== Important Note ==

The entirety of this test will take place within the /SPC_Test/F3E-Limbs directory on ormacsrv1. When ever an action is performed during the test, commands will be logged for future reference. To do this type the following commands

{{{
script 
date
pwd 
ls -l
}}}

Then continue with a procedural step. When done with a procedural step cancel the script and place it into the /SPC_Test/F3E-Limbs/log directory.bWith script still enabled do the following to end it while in the test directory.

{{{
exit
mv typescript ../log/<name/path>
}}}
-Line 7:
+Line 26:
-=== Setting up the test directory ===
+The procedures shown can be repeated for every sub-test in the table.
-Line 9:
+Line 28:
-Basically, depending on the subtest, the high or low stereo prep directory is being copied over to the test directory with the test directory being emptied first. Following the 1B procedural block for test F3-G made by Diane Lambert does this exactly. Just replace the orig directory with the low or high stereo prep directory located in /SPC_Test/F3E-Limbs/PrepDirectories.
+||Sub-Test Number||Base image set||Limb image resolution||Methods||Evaluation||
||F3E-1||Low Stereo||100cm||Iteration x5 GEOMETRY 2||Spacecraft & Landmark position||
||F3E-2||Low Stereo||50cm||Iteration x5 GEOMETRY 2||Spacecraft & Landmark position||
||F3E-3||Low Stereo||15cm||Iteration x5 GEOMETRY 2||Spacecraft & Landmark position||
||F3E-4||Low Stereo||5cm||Iteration x5 GEOMETRY 2||Spacecraft & Landmark position||
||F3E-5||High Stereo||100cm||Iteration x5 GEOMETRY 2||Spacecraft & Landmark position||
||F3E-6||High Stereo||50cm||Iteration x5 GEOMETRY 2||Spacecraft & Landmark position||
||F3E-7||High Stereo||15cm||Iteration x5 GEOMETRY 2||Spacecraft & Landmark position||
||F3E-8||High Stereo||5cm||Iteration x5 GEOMETRY 2||Spacecraft & Landmark position||
-Line 11:
+Line 38:
-[[Test F3G Procedures Block-1B - TEST DIRECTORY SET-UP - TEST]]<<BR>>
-Line 13:
+Line 39:
-=== Symbolically Linking the Correct List of Images to PICTLIST.TXT ===
+=== 1. Setting up the test directory ===

Depending on the subtest, the high or low stereo prep directory is being copied over to the test directory with the test directory being emptied first. 

[[Test F3E Test Directory Set-Up]]

=== 2. Symbolically Linking the Correct List of Images to PICTLIST.TXT ===
-Line 22:
+Line 54:
-=== Register Sub-Test Limb Images ===
+=== 3. Register Sub-Test Limb Images ===
-Line 32:
+Line 64:
-Execute [[Block 3 - Register]]
+[[Test F3E Register]]
-Line 34:
+Line 66:
-=== Run 5 iterations ===
+=== 4. Run 5 iterations ===
-Line 38:
+Line 70:
-[[Block 5 - Iterate]] -- Use the 100 cm area scenario
+[[Test F3E Iterate]] -- Use the 100 cm area scenario
-Line 40:
+Line 72:
-=== Obtain 5cm Landmark Point Cloud and Evaluate its obj to the Truth Model ===
+=== 5. Obtain 5cm Landmark Point Cloud and Evaluate its obj to the Truth Model ===
-Line 42:
+Line 74:
-=== Run Geometry ===
+First enter the following:
-Line 44:
+Line 76:
-Perform the procedural block below to update the spacecraft pointing.
+{{{
bigmap < support/EVAL20.in
}}}
-Line 46:
+Line 80:
-[[Block 8 - Geometry/Residuals]]
+This will output a list of the landarks within the evaluation region of the tag site. Copy and paste this list into LMKFILES/LMRKEvalList.txt
-Line 48:
+Line 82:
-=== Evaluate Position and Pointing of Spacecraft ===
+While in the test directory input the following commands.
{{{
grep "0.0500" MAPINFO.TXT | awk '{ print $1 }' >> LMKFILES/LMRK05.txt        # Makes a text file of all 5cm landmark names
cd /SPC_Test/F3E-Limbs/test/LMKFILES
while read line; do f=`grep $line LMRKEvalList.txt | wc -l`; if [ $f -gt 0 ]; then echo $line >> EvalLMRK05List.txt; fi; done < LMRK05.txt     # Makes a list of 5 cm landmarks in 20 by 20 meter area around tagsites
while read line; do grep "VLM" $line* | awk '{ print $1, $2, $3 }' >> PC05.txt; done < EvalLMRK05List.txt        # Makes the point cloud
cp PC05.txt LMRK05.txt ../../results/<subtest name>/                            # Copy point cloud files to results
cp PC05.txt ~/send/                                                          # Copy point cloud to send directory so it can be evaluated on personal machine
}}}

Now, these are the exact commands I enter to evaluate the point cloud on my machine.

{{{
cp send/PC05.txt send/CompareOBJfiles/
cd /Users/kdrozd/Dropbox\ \(OREx\)/SPC-ORex/eval_tools/altwg-2016.01.25-macosx-x64/bin
vi ~/send/CompareOBJfiles/PC05.txt
:%s/D/E/g
ZZ
./ComparePointCloudToOBJ --compute-optimal-translation-and-rotation ~/send/CompareOBJfiles/PC05.txt truthTAG1-5cm-1.1K.obj 
}}}

The Copy the output of the last command into a file in the the subtest directory located in the /SPC_Test/F3E-Limbs/results directory.

=== 6. Run Geometry 2 ===

Next run Geometry to just update the spacecraft position and pointing (not landmark heights). To do this follow the procedural block below, but when Geometry is run make sure only option 2 and 0 are selected under the geometry section.

[[Test F3E Geometry 2 ]]

=== 7. Evaluate Position and Pointing of Spacecraft ===

First step is to make a text file that contains a list of the names of the 4 high or 4 low stereo images appended with the tests respected 4 limb shot images. This text file can be made anyway and does not require a direct procedure. Just make sure the textfile is a list only. Call the textile imgname.txt.

Then follow Diane's procedural block below. Replace listSum, with imgname.txt. Also when, logging the SCOBJ and point vector files put them in the test's respected directory in the /SPC_Test/F3E-Limbs/results directory.

[[Test F3E Procedure Block - Capture SCOBJ & Pointing]]<<BR>>