Differences between revisions 65 and 66
Revision 65 as of 2019-08-30 14:47:38
Size: 12958
Editor: JohnWeirich
Comment:
Revision 66 as of 2019-09-03 09:58:54
Size: 13008
Editor: EricPalmer
Comment: Freeze the Orbit B and ROI development proceedures
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from Block 17 - NFT Features

Describe Block 17 - NFT Features here.

Setup

Extra item names

  • A 15cm DEM for evaluation
  • B... and following
  • Z is a single maplet over the region (22cm/Q=49)
  • Y is the tiling bigmap (22cm/Q=116)
  • X is the matching maplet that provides extra "umph" to the 35cm images (22cm/Q=81 - just a touch bigger)
  • W is the working maplet. We use this for all of our processing. A is a published version and not the starting thing.
  • V is a small bitmap that just covers the ROI

bigmap

Q

GSD

Width

Purpose

A...

67

15cm

20m

10m radius, published to the team

B, C...

200

5cm

20m

10m radius, published to the team

V

60

5cm

6m

Just barely covers the ROI, assuming 6m radius

W

350

5cm

35m

Baseline width we use for the ROI to cover outlying regions

X

99

5cm

10m

Central reference maplet. Used to denote the middle of the object for lithos, CSPLOT, etc

Y

250

5cm

35cm

The region to tile at 14 and 5cm

Z

49

5cm

5m

Temporary maplet, used in evaluation scripts -- dynamically built.

Tiling using XXXXXY, <idY>. Iterate over XXXXX, <idA>

22 cm Features

We need 22cm features for Lockheed's NFT.

Assumptions

  • Get rid of NavCam and FB0

    • Such a high phase doesn't make good features for much
  • Get rid of 14cm landmarks.
    • We tiled them too early and introduced more artifacts than they are worth

Make a Map

ID=00875

bigmap < mapConfig/nft-${ID}W.in
cd MAPFILES
relink.sh $ID.MAP XXXXXX.MAP
cd ..
echo y | cp USED_MAPS.TXT lsupport/nftMapList-${ID}W
relink.sh lsupport/nftMapList-${ID}W make_script.in

Prep Maplets

Do this for all the landmarks in the bigmap

Eliminate Images

  • Ensure the resolution and emission angle is correct
  • Ensure no NavCam images

120cm

75cm

35 cm

22 cm

14cm

0 50 .5 .5 .1 .75

0 50 .5 .5 .1 1.1

0 50 .5 .5 .25 2.2

0 50 .5 .5 .25 2.5

0 50 .5 .5 .5 3

Check fill

  • Check to see if shape or ZMAPS are better than current. During processing, especially with weak data, you can end up with a poor feature. It is better to use the same fill for a feature, but I haven't committed to that in all cases (I am still doing it by hand to evaluate if it is okay to do)

0
0
1

m
0
0
1

b
y
n
ZS0824
0
0
1

Batch Run

Update the ID for localLoop.sh

  vi localLoop.sh

Consider changing USRMX to something other than 12. If there are < 24 landmarks, then using N as user max will be faster. While each process takes longer to run, it is faster than doing two maplets in succession.

Run a batch -- suggest using monitor and daemon with looping

  make_scriptP | tee run.sh
  sh run.sh
  monitor

Check the following

  • Daemon is running
  • Terminate is unset (if you want more than one)
  • localLoop.sh is set (map=<id>)

  • interationCount is a useful number. Whatever number is set in "iterationCount" when monitor is start will be increment and used as the $id variable in localLoop.sh. Thus, it is useful to set it to something new if you are doing a different suite of iterations.

For the first 2 full iterations (i.e. over ALL maplets), make sure there are no images with a correlation <0.1 before starting a full iteration. If there are images that are getting a correlation <0.1 even after several iterations, delete if emission angle is >50, illumination coverage (after x-0.025) < 0.5, or phase less than 8 or 9. They are weak images and will not contribute much anyway.

After you have done 2 full iterations, clean according to the criteria for a correlation <0.1, but apply it to correlations <0.3. Again, if after many iterations they still are <0.3, they are not strong anyway.

The final check for an MLN is to use bigEval.sh <Feature Name>A

  • If the resulting bigmap has SigmaScore >2, you are done. Make sure you've run at least 2 full iterations (all maplets) to make sure everything has stabilized.

  • More iterations may help (stop if the avg sigmaScore is going down) or you may need more images, a fill, or some other technique.


Tiling

When the 35cm (and above) are cleaned up, then tile at higher GSD. Suggest 22cm or 14cm. 22cm seem to be too small to use the 5cm well; however, that is the size needed by the final NFT feature.

Tiling

GSD

Width

Q

22cm

51m

116

14cm

45m

160

What Size Maplets

Width

Width

Width

Width

Grid Size

Q required

5cm Maplets

14cm Maplets

22cm Maplets

35cm Maplets

4x4

125

12.5M

35m

55m

88m

5x5

160

16.0m

45m

70m

112m

6x6

196

19.6m

55m

86m

137m

ID=00056

mkdir -p log/$ID/
bigmap < mapConfig/tile-${ID}W.in
cd MAPFILES
relink.sh ${ID}Y.MAP XXXXXX.MAP
cd ..
echo XXXXXX | showmap


convert XXXXXX.pgm log/$ID/low-$ID.jpg

map_coverage
XXXXXX
0 .00014
convert coverage_m.pgm log/$ID/pre-$ID.jpg

Build the tile file and run

echo N | make_tilefile > tmpOut
echo XXXXXX > lsupport/bigmap_tile.in
echo scripts/XXX014.seed >> lsupport/bigmap_tile.in
sed 1d tmpOut >> lsupport/bigmap_tile.in
relink.sh lsupport/bigmap_tile.in make_scriptT.in

make_scriptT
nohup sh run_script.b
finished tiling-done

Check

map_coverage
XXXXXX
0 .00014
convert coverage_m.pgm log/$ID/post-$ID.jpg

bigmap < mapConfig/tile-${ID}W.in
echo XXXXXX | showmap
convert XXXXXX.pgm log/$ID/high-$ID.jpg
score=`tail -2 SIGMAS.TXT | head -1 | cut -c 23-30`
mScore=`echo "scale=2; $score * 1000" | bc | cut -c -4`
convert SIGMAS.pgm   -fill white -gravity North -pointsize 15 -annotate +0+10 Max:${mScore}m   log/$ID/sig-$ID.jpg

blockFinish nft.$ID

Use basic tiling steps. For the tiling bigmap, use ${ID}Y to ensure it doesn't get confused with anything else.


Iterate

Some special rules for iterations

ID=00397

Note: You can change config/iterationCount to be ID*100

echo map=$ID > config/nftID
echo "Getting read for iteration $ID" >> notes
date >> notes
echo $ID | tee -a notes
echo ${ID}W > tmpRun
cat mapConfig/nft-${ID}W.in >> tmpRun
cat tmpRun >> notes
bigMapRef < tmpRun
tail -2 SIGMAS.TXT | head -1 | tee -a notes
echo y | cp -v USED_MAPS.TXT LIST.TXT
duplicates

relink.sh scripts/Piterate1St.seed make_scriptP.seed | tee -a notes
ls -l make_scriptP.seed >> notes
MAKE_LMRKLISTX 


make_scriptP | tee run.sh
echo "Running iteration at " `date` >> notes
sh run.sh
monitor

bigMapRef < tmpRun
echo "Post run Sigma" >> notes
tail -2 SIGMAS.TXT | head -1 | tee -a notes

Checking

Create a maplet at the bigmap's center to see how it is performing. Use all "decent" imagery (FB2 and FB7 without NavCam or FB0). Note, don't keep this maplet around when you build the final bigmap because it will only have resampled the bigmap and will have no iterations (because it is only for test).

lithos
c
<ID>Z
m
<ID>W
79
0.00022 49
i
y
.5
n
e
n
0 50 .5 .5 .5 2
1
0
3
n
0
y
1
0
1
n
0
y
u
1
q

landmarkEval.sh ${ID}Z

lithos
d
<id>Z
1
y

ROI Tile 14cm

ID=RTM11

mkdir -p log/$ID/
bigmap < mapConfig/tile14-${ID}Y.in
cd MAPFILES
relink.sh ${ID}Y.MAP XXXXXX.MAP
cd ..
echo XXXXXX | showmap


convert XXXXXX.pgm log/$ID/low14-$ID.jpg

map_coverage
XXXXXX
0 .00014
convert coverage_m.pgm log/$ID/pre14-$ID.jpg

Build the tile file and run

echo N | make_tilefile > tmpOut
echo XXXXXX > lsupport/bigmap_tile.in
echo scripts/XXXROI14.seed >> lsupport/bigmap_tile.in
sed 1d tmpOut >> lsupport/bigmap_tile.in
relink.sh lsupport/bigmap_tile.in make_scriptT.in

make_scriptT
nohup sh run_script.b
finished tiling-done

Check

map_coverage
XXXXXX
0 .00014
convert coverage_m.pgm log/$ID/post14-$ID.jpg

echo ${ID}Y > tmpRun
cat mapConfig/tile14-${ID}Y.in >> tmpRun
bigMapRef < tmpRun


echo XXXXXX | showmap
convert XXXXXX.pgm log/$ID/high14-$ID.jpg
score=`tail -2 SIGMAS.TXT | head -1 | cut -c 23-30`
mScore=`echo "scale=2; $score * 1000" | bc | cut -c -4`
convert SIGMAS.pgm   -fill white -gravity North -pointsize 15 -annotate +0+10 Max:${mScore}m   log/$ID/sig-$ID.jpg

Use basic tiling steps. For the tiling bigmap, use ${ID}Y to ensure it doesn't get confused with anything else.


ROI Tile 5cm

ID=RTM11

mkdir -p log/$ID/
bigmap < mapConfig/tile05-${ID}Y.in
cd MAPFILES
relink.sh ${ID}Y.MAP XXXXXX.MAP
cd ..
echo XXXXXX | showmap


convert XXXXXX.pgm log/$ID/low05-$ID.jpg

map_coverage
XXXXXX
0 .00005
convert coverage_m.pgm log/$ID/pre05-$ID.jpg

Build the tile file and run

echo N | make_tilefile > tmpOut
echo XXXXXX > lsupport/bigmap_tile.in
echo scripts/XXX005.seed >> lsupport/bigmap_tile.in
sed 1d tmpOut >> lsupport/bigmap_tile.in
relink.sh lsupport/bigmap_tile.in make_scriptT.in

make_scriptT
nohup sh run_script.b
finished tiling-done

Check

map_coverage
XXXXXX
0 .00005
convert coverage_m.pgm log/$ID/post05-$ID.jpg

echo ${ID}Y > tmpRun
cat mapConfig/tile05-${ID}Y.in >> tmpRun
bigMapRef < tmpRun


echo XXXXXX | showmap
convert XXXXXX.pgm log/$ID/high05-$ID.jpg
score=`tail -2 SIGMAS.TXT | head -1 | cut -c 23-30`
mScore=`echo "scale=2; $score * 1000" | bc | cut -c -4`
convert SIGMAS.pgm   -fill white -gravity North -pointsize 15 -annotate +0+10 Max:${mScore}m   log/$ID/sig-$ID.jpg

Use basic tiling steps. For the tiling bigmap, use ${ID}Y to ensure it doesn't get confused with anything else.


ROI Iterate

bigmap < mapConfig/ROI-${ID}W.in

Be sure you have made the bigmap before pasting below

echo map=$ID > config/nftID
echo "Getting read for iteration $ID" >> notes
date >> notes
echo $ID | tee -a notes
echo ${ID}W > tmpRun
cat mapConfig/ROI-${ID}W.in >> tmpRun
cat tmpRun >> notes
bigMapRef < tmpRun
tail -2 SIGMAS.TXT | head -1 | tee -a notes
echo y | cp -v USED_MAPS.TXT LIST.TXT
duplicates

relink.sh scripts/Piterate1St.seed make_scriptP.seed | tee -a notes
ls -l make_scriptP.seed >> notes
MAKE_LMRKLISTX 


make_scriptP | tee run.sh
echo "Running iteration at " `date` >> notes
sh run.sh
monitor

bigMapRef < tmpRun
echo "Post run Sigma" >> notes
tail -2 SIGMAS.TXT | head -1 | tee -a notes


Evaluation

  • corrEval.sh -- This reads the *.OOT files generated from an iteration. It calculates the low, average, stdev and sigma-score for every landmark that was run. This can be run in any directory that has OOT files. Note, it would like to read MAPINFO.TXT to get the lat/lon and GSD of the landmark. If it is not there, it will have 0 0 0. You can "cheat" by running ln -s ../MAPINFO.TXT . (or whatever the path is to the original). It runs awk on landmarkOOT.awk.

You can give it a GSD so that it will only compute that GSD. Thus, 35cm landmarks would be 0.3500 (it uses string matches from MAPINFO.TXT)

  • corrEval.sh (0.1400)

    name

    resolution

    lat

    lon

    minimum

    average

    sigma score

    percent of images that passed 0.6

---

  • landmarkEval.sh -- This uses lithos to load and evaluate the correlation score for a landmark. It will load the given landmark and run a 1 0 1 to get the scores. It doesn't change anything, but it is better to run in "wk" to avoid collision with gray.pgm.

    landmarkEval.sh <lmkname>

    name

    minimum

    maximum

    average

    stdev

    sigma score

---

  • bigEval.sh - This does a bunch of things to get the state of the MNL
    • Builds a bigmap using bigMapRef. Because it is using a reference bigmap, you need to run normal bigmap first

    • It links the new bigmap to XXXXXX.MAP so it can be loaded in lithos. Map files that start with 0 do not get loaded. When NFT features have leading numbers other than 0, then we won't need this step
    • Creates the Z maplet. This maplet is a simple 22cm with Q=49 that matches the size of the Orbit B MLNs. We create the maplet, load the just-created bigmap, eliminate out of range images, correct alignment and get the correlation values.
    • Calculates the cardinal values for this landmark (which is close to representing the landmark (Q is 49 rather than 79)
    • Displays the SIGMA.TXT value for the bigmap
    • Copies CSPLOT.pgm to ~/send/cs-name.jpg
    • Calculates the correlation score using landmarkEval.sh

    • Deletes the Z maplet so it doesn't mess anything up.

Block 17 - NFT Features-V2.0 (last edited 2019-09-03 09:58:54 by EricPalmer)