|
Size: 13388
Comment:
|
← Revision 13 as of 2019-09-26 13:23:11 ⇥
Size: 1900
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 22: | Line 22: |
Each week, we will receive a NFT feature description of less than 20 features. We must take these definitions and fold them into a CSV file that includes the following || NFT ID || Picture name (raw) || SPC Image Name || Sample || Line || GSD [cm] || Q (half width) || Notes || || 10021 || 20191028T121532.002.fits || P601324232A3 || 150|| 801 || 6.3 || 100 || || etc. || Our goal will be to generate a set of maplets over the NFT site plus 1/2 maplet boarder at a GSD that is 1/2 of the requested NavCam pixel size. Obviously, images 27-30 will not work. We will create one CSV file for each week. Then this file will be exported and run through a program (TBD) that will create the input files for bigmap and make_scriptT. The general format will be string-ID-GSD.sufix. * NFT-10021-8.IN with the bigmap called <idGSD.MAP> (e.g. MAPFILES/100218.MAP and MAPFILES/100216.MAP) * tileSeed-10021-8.seed where the 1st line uses the GSD from the CSV file When we run the program (TBD), it will generate the bigmap input and seed files for all records in the file (which should be about 18). Then we can use these as the inputs for the base tiling, intermediate tiling, and final tiling. We assume that 14cm can jump to 5cm and that 14cm exists everywhere. Assuming the standard for NFT features of width of 100, we will use a Q=50 (width of 101). We will tile at a Q=150, which would give 1/2 a maplet over all the edges. The core NFT region would be covered by a 3x3 and with the "margin" it will become 5x5, which means there is a full four maplets over the exact NFT unit. Steps of resolution -- Note, we will use the exact decimal value for the final tiling || GSD|| Base || Low || Medium || Final || ||16|| || || || 8 || ||14|| || || || 7 || ||12|| || || || 6 || ||10|| || || || 5 || ||8|| || || || 4 || ||7|| || || || 3.5 || ||6|| || || || 3 || ||5||5 || || || 2.5 || ||4||5 || || || 2 || ||3.5||5 || 2 || || 1.75 || ||3 ||5 || 2 || || 1.5 || ||2.5||5 || 2 || || 1.25 || ||2 ||5 || 2 || || 1 || ||1.5||5 || 2 || 1 || 0.75 || ||1 ||5 || 2 || 1 || 0.5 || ||.5 ||5 || 2 || 1 || 0.5 || |
|
| Line 90: | Line 50: |
| ==== Make a Map ==== {{{ ID=RDL09 bigmap < mapConfig/nft-${ID}W.in cd MAPFILES relink.sh $ID.MAP XXXXXX.MAP cd .. echo y | cp USED_MAPS.TXT lsupport/nftMapList-${ID}W relink.sh lsupport/nftMapList-${ID}W make_script.in }}} |
|
| Line 108: | Line 53: |
==== Tiling ==== When the 35cm (and above) are cleaned up, then tile at higher GSD. Suggest 22cm or 14cm. 22cm seem to be too small to use the 5cm well; however, that is the size needed by the final NFT feature. Tiling || GSD || Width || Q|| || 22cm || 51m || 116 || || 14cm || 45m || 160|| || 8cm || || 5cm || || 2cm || ===== What Size Maplets ===== || || || Width || Width || Width || Width || || Grid Size || Q required || 5cm Maplets || 14cm Maplets || 22cm Maplets || 35cm Maplets || || 4x4 || 125 || 12.5M || 35m || 55m || 88m || || 5x5 || 160 || 16.0m || 45m || 70m || 112m || || 6x6 || 196 || 19.6m || 55m || 86m || 137m || {{{ ID=RDL09 mkdir -p log/$ID/ bigmap < mapConfig/tile-${ID}W.in cd MAPFILES relink.sh ${ID}Y.MAP XXXXXX.MAP cd .. echo XXXXXX | showmap convert XXXXXX.pgm log/$ID/low-$ID.jpg map_coverage XXXXXX 0 .00005 convert coverage_m.pgm log/$ID/pre-$ID.jpg }}} Build the tile file and run {{{ echo N | make_tilefile > tmpOut echo XXXXXX > lsupport/bigmap_tile.in echo scripts/XXX014.seed >> lsupport/bigmap_tile.in sed 1d tmpOut >> lsupport/bigmap_tile.in relink.sh lsupport/bigmap_tile.in make_scriptT.in make_scriptT nohup sh run_script.b finished tiling-done }}} Check {{{ map_coverage XXXXXX 0 .00005 convert coverage_m.pgm log/$ID/post-$ID.jpg bigmap < mapConfig/tile-${ID}W.in echo XXXXXX | showmap convert XXXXXX.pgm log/$ID/high-$ID.jpg score=`tail -2 SIGMAS.TXT | head -1 | cut -c 23-30` mScore=`echo "scale=2; $score * 1000" | bc | cut -c -4` convert SIGMAS.pgm -fill white -gravity North -pointsize 15 -annotate +0+10 Max:${mScore}m log/$ID/sig-$ID.jpg blockFinish nft.$ID }}} Use basic tiling steps. For the tiling bigmap, use ${ID}Y to ensure it doesn't get confused with anything else. ----- == Iterate == ==== Batch Run ==== Check the following * Daemon is running * Terminate is unset (if you want more than one) * localLoop.sh is set (map=<id>) * interationCount is a useful number. Whatever number is set in "iterationCount" when monitor is start will be increment and used as the $id variable in localLoop.sh. Thus, it is useful to set it to something new if you are doing a different suite of iterations. For the first 2 full iterations (i.e. over ALL maplets), make sure there are no images with a correlation <0.1 before starting a full iteration. If there are images that are getting a correlation <0.1 even after several iterations, delete if emission angle is >50, illumination coverage (after x-0.025) < 0.5, or phase less than 8 or 9. They are weak images and will not contribute much anyway. After you have done 2 full iterations, clean according to the criteria for a correlation <0.1, but apply it to correlations <0.3. Again, if after many iterations they still are <0.3, they are not strong anyway. The final check for an MLN is to use bigEval.sh <Feature Name>A * If the resulting bigmap has SigmaScore >2, you are done. Make sure you've run at least 2 full iterations (all maplets) to make sure everything has stabilized. * More iterations may help (stop if the avg sigmaScore is going down) or you may need more images, a fill, or some other technique. Note: Consider changing USRMX to something other than 12. If there are < 24 landmarks, then using N as user max will be faster. While each process takes longer to run, it is faster than doing two maplets in succession. {{{ ID=RDL09 }}} Note: You can change config/iterationCount to be ID*100 {{{ echo map=$ID > config/nftID echo "Getting read for iteration $ID" >> notes date >> notes echo $ID | tee -a notes echo ${ID}W > tmpRun cat mapConfig/nft-${ID}W.in >> tmpRun cat tmpRun >> notes bigMapRef < tmpRun tail -2 SIGMAS.TXT | head -1 | tee -a notes echo y | cp -v USED_MAPS.TXT LIST.TXT duplicates relink.sh scripts/Piterate1St.seed make_scriptP.seed | tee -a notes ls -l make_scriptP.seed >> notes MAKE_LMRKLISTX make_scriptP | tee run.sh echo "Running iteration at " `date` >> notes sh run.sh monitor }}} {{{ bigMapRef < tmpRun echo "Post run Sigma" >> notes tail -2 SIGMAS.TXT | head -1 | tee -a notes }}} ==== Checking ==== Create a maplet at the bigmap's center to see how it is performing. Use all "decent" imagery (FB2 and FB7 without NavCam or FB0). Note, don't keep this maplet around when you build the final bigmap because it will only have resampled the bigmap and will have no iterations (because it is only for test). {{{ lithos c <ID>Z m <ID>W 79 0.00022 49 i y .5 n e n 0 50 .5 .5 .5 2 1 0 3 n 0 y 1 0 1 n 0 y u 1 q landmarkEval.sh ${ID}Z lithos d <id>Z 1 y }}} ------- == ROI Tile 5cm == {{{ ID=RDL09 mkdir -p log/$ID/ bigmap < mapConfig/tile05-${ID}Y.in cd MAPFILES relink.sh ${ID}Y.MAP XXXXXX.MAP cd .. echo XXXXXX | showmap convert XXXXXX.pgm log/$ID/low05-$ID.jpg map_coverage XXXXXX 0 .00005 convert coverage_m.pgm log/$ID/pre05-$ID.jpg }}} Build the tile file and run {{{ echo N | make_tilefile > tmpOut echo XXXXXX > lsupport/bigmap_tile.in echo scripts/XXX005.seed >> lsupport/bigmap_tile.in sed 1d tmpOut >> lsupport/bigmap_tile.in relink.sh lsupport/bigmap_tile.in make_scriptT.in make_scriptT nohup sh run_script.b finished tiling-done }}} Check {{{ map_coverage XXXXXX 0 .00005 convert coverage_m.pgm log/$ID/post05-$ID.jpg echo ${ID}Y > tmpRun cat mapConfig/tile05-${ID}Y.in >> tmpRun bigMapRef < tmpRun echo XXXXXX | showmap convert XXXXXX.pgm log/$ID/high05-$ID.jpg score=`tail -2 SIGMAS.TXT | head -1 | cut -c 23-30` mScore=`echo "scale=2; $score * 1000" | bc | cut -c -4` convert SIGMAS.pgm -fill white -gravity North -pointsize 15 -annotate +0+10 Max:${mScore}m log/$ID/sig-$ID.jpg }}} Use basic tiling steps. For the tiling bigmap, use ${ID}Y to ensure it doesn't get confused with anything else. ------- == ROI Iterate == {{{ bigmap < mapConfig/ROI-${ID}W.in }}} Be sure you have made the bigmap before pasting below {{{ echo map=$ID > config/nftID echo "Getting read for iteration $ID" >> notes date >> notes echo $ID | tee -a notes echo ${ID}W > tmpRun cat mapConfig/ROI-${ID}W.in >> tmpRun cat tmpRun >> notes bigMapRef < tmpRun tail -2 SIGMAS.TXT | head -1 | tee -a notes echo y | cp -v USED_MAPS.TXT LIST.TXT duplicates relink.sh scripts/Piterate1St.seed make_scriptP.seed | tee -a notes ls -l make_scriptP.seed >> notes MAKE_LMRKLISTX make_scriptP | tee run.sh echo "Running iteration at " `date` >> notes sh run.sh monitor }}} {{{ bigMapRef < tmpRun echo "Post run Sigma" >> notes tail -2 SIGMAS.TXT | head -1 | tee -a notes }}} ------- == Evaluation == * corrEval.sh -- This reads the *.OOT files generated from an iteration. It calculates the low, average, stdev and sigma-score for every landmark that was run. This can be run in any directory that has OOT files. Note, it would like to read [[MAPINFO.TXT]] to get the lat/lon and GSD of the landmark. If it is not there, it will have 0 0 0. You can "cheat" by running ln -s ../MAPINFO.TXT . (or whatever the path is to the original). It runs awk on landmarkOOT.awk. You can give it a GSD so that it will only compute that GSD. Thus, 35cm landmarks would be 0.3500 (it uses string matches from MAPINFO.TXT) corrEval.sh (0.1400) || name || resolution || lat || lon || minimum || average || sigma score || percent of images that passed 0.6 || --- * landmarkEval.sh -- This uses [[lithos]] to load and evaluate the correlation score for a landmark. It will load the given landmark and run a 1 0 1 to get the scores. It doesn't change anything, but it is better to run in "wk" to avoid collision with gray.pgm. * This copies the [[LMRK_DISPLAY1.pgm]] over to ~/send/ so you can see what it looks like landmarkEval.sh <lmkname> || name || minimum || maximum || average || stdev || sigma score || --- * bigEval.sh - This does a bunch of things to get the state of the MNL * Builds a bigmap using [[bigMapRef]]. Because it is using a reference bigmap, you need to run normal bigmap first * It links the new bigmap to XXXXXX.MAP so it can be loaded in lithos. Map files that start with 0 do not get loaded. When NFT features have leading numbers other than 0, then we won't need this step * Creates the Z maplet. This maplet is a simple 22cm with Q=49 that matches the size of the Orbit B MLNs. We create the maplet, load the just-created bigmap, eliminate out of range images, correct alignment and get the correlation values. * Calculates the cardinal values for this landmark (which is close to representing the landmark (Q is 49 rather than 79) * Displays the SIGMA.TXT value for the bigmap * Copies CSPLOT.pgm to ~/send/cs-name.jpg * Calculates the correlation score using [[landmarkEval.sh]] * Deletes the Z maplet so it doesn't mess anything up. |
Describe Block 17 - NFT Features here.
Previous version
== Intro ==
We are expecting the following levels of tiling.
Image Number
GSD of last image
Pre-tile Now
Post-Recon A
Post-Recon B
Area
1-11
8cm
14cm
NA
NA
Global'ish
11-17
5cm
5cm
NA
NA
Several 100m
18-23
3.4cm
5cm
NA
NA
100x100m TAG
Check Point
23-26
2.3cm
5cm
2cm
NA
50x50m TAG
Match Point
27-29
0.6cm
5cm
2cm
1cm
25x25m TAG
General
Setup
Extra item names
- A 15cm DEM for evaluation
- B... and following
- Z is a single maplet over the region (22cm/Q=49)
- Y is the tiling bigmap (22cm/Q=116)
- X is the matching maplet that provides extra "umph" to the 35cm images (22cm/Q=81 - just a touch bigger)
- W is the working maplet. We use this for all of our processing. A is a published version and not the starting thing.
- V is a small bitmap that just covers the ROI
bigmap |
Q |
GSD |
Width |
Purpose |
A... |
67 |
15cm |
20m |
10m radius, published to the team |
B, C... |
200 |
5cm |
20m |
10m radius, published to the team |
V |
60 |
5cm |
6m |
Just barely covers the ROI, assuming 6m radius |
W |
350 |
5cm |
35m |
Baseline width we use for the ROI to cover outlying regions |
X |
99 |
5cm |
10m |
Central reference maplet. Used to denote the middle of the object for lithos, CSPLOT, etc |
Y |
250 |
5cm |
35cm |
The region to tile at 14 and 5cm |
Z |
49 |
5cm |
5m |
Temporary maplet, used in evaluation scripts -- dynamically built. |
Tiling using XXXXXY, <idY>. Iterate over XXXXX, <idA>