Differences between revisions 15 and 44 (spanning 29 versions)
Revision 15 as of 2019-10-24 15:21:14
Size: 3890
Editor: JohnWeirich
Comment:
Revision 44 as of 2019-10-29 10:28:48
Size: 5168
Editor: EricPalmer
Comment:
Deletions are marked like this. Additions are marked like this.
Line 43: Line 43:
ID=RDL09 ID=00000
Line 49: Line 49:
checkVar
Line 52: Line 53:
== ROI Iterate == == ROI Iterate over new maplets (Note: Uses LMRKLIST1.TXT) ==
Line 56: Line 57:
checkVar
Line 64: Line 66:
echo y | cp -v USED_MAPS.TXT LIST.TXT echo y | cp -v LMRKLIST1.TXT LIST.TXT
Line 82: Line 84:
checkVar
Line 89: Line 92:
checkVar
nftBlockFinish.sh nft.$ID
sh rem_script.b
}}}
Line 90: Line 97:
== Run AUTOREGISTER before kicking off remaining iterations ==

{{{
checkVar
grep P6 USED_PICS.TXT > make_script.in
echo END >> make_script.in
echo "Running AUTOREGISTER" > notes

relink.sh support/make_scriptA_jrwLowerC2.seed make_scriptA.seed
make_scriptAP > run.sh
sh run.sh

autoEval.sh

checkVar
Line 93: Line 115:
== Run AUTOREGISTER before kicking off 10 iterations == == If AUTOREGISTER excludes many images, do another iterate over the new maplets ==
Line 96: Line 118:
grep P6 USED_PICS.TXT > make_script.in
echo END >> make_script.in

relink.sh support/make_scriptA_jrwLowerC2.seed make_scriptA.seed
make_scriptA
nohup sh run_script.b
autoEval.sh
Line 104: Line 121:
== Setup Batch Iterate NFT Feature == == Make sure you have 3 BBD stations for each image ==

{{{

# Needs new maplets in MAPINFO.TXT; Run if RESIDUALS has not been run since tiling
echo "5 .001 .001" | RESIDUALS


getImagesInLandmarks2.sh LMRKLIST1.TXT
head -1 tmpjrw
sort -k16,16 tmpjrw | head

}}}

== Setup Batch Iterate NFT Feature (Note: Uses USED_MAPS.TXT) ==
Line 108: Line 139:
ID=RDL09
Res=5

checkVar
echo "Getting ready for iteration $ID-$Res" >> notes
date >> notes
echo $ID$Res | tee -a notes
echo ${ID}$Res > tmpRun
cat nftConfig/nftBigmap-${ID}-$Res.in >> tmpRun
cat tmpRun >> notes
bigMapRef < tmpRun
tail -2 SIGMAS.TXT | head -1 | tee -a notes
echo y | cp -v USED_MAPS.TXT LIST.TXT
duplicates

relink.sh scripts/Piterate1St.seed make_scriptP.seed | tee -a notes
ls -l make_scriptP.seed >> notes
MAKE_LMRKLISTX
Line 113: Line 158:
echo 0 > config/iterationCount echo 1 > config/iterationCount
Line 117: Line 162:

make_scriptP | tee run.sh
echo "Running iteration at " `date` >> notes
sh run.sh
monitor


Line 130: Line 183:
sh rem_script.b
Line 131: Line 186:

== Tuck Log files into logNFT ==

{{{
mkdir -p logNFT
mkdir -p logNFT/$ID
mv log/* logNFT/$ID/
}}}

Iterate

Loop as required. Do nftBlockFinish when done with iterations.

Log the following. localLoop.sh will handle this

  • sigma after each iteration
  • nofit "no correlation" flags
    • Stop on a real flag and email
  • Number of nofits
  • corrEval results

Batch Run

Check the following

  • Daemon is running
  • Terminate is unset (if you want more than one)
  • localLoop.sh is set (map=<ID><Res>)

  • interationCount is a useful number. Whatever number is set in "iterationCount" when monitor is start will be increment and used as the $id variable in localLoop.sh. Thus, it is useful to set it to something new if you are doing a different suite of iterations.

For the first 2 full iterations (i.e. over ALL maplets), make sure there are no images with a correlation <0.1 before starting a full iteration. If there are images that are getting a correlation <0.1 even after several iterations, delete if emission angle is >50, illumination coverage (after x-0.025) < 0.5, or phase less than 8 or 9. They are weak images and will not contribute much anyway.

After you have done 2 full iterations, clean according to the criteria for a correlation <0.1, but apply it to correlations <0.3. Again, if after many iterations they still are <0.3, they are not strong anyway.

The final check for an MLN is to use bigEval.sh <Feature Name>A

  • If the resulting bigmap has SigmaScore >2, you are done. Make sure you've run at least 2 full iterations (all maplets) to make sure everything has stabilized.

  • More iterations may help (stop if the avg sigmaScore is going down) or you may need more images, a fill, or some other technique.

Note: Consider changing USRMX to something other than 12. If there are < 24 landmarks, then using N as user max will be faster. While each process takes longer to run, it is faster than doing two maplets in succession.

Setup ROI Iterate

NOTE - Be sure you have made the bigmap before pasting below.

Because bigmap provides sigmas in km, it lacks the resolution needed for good statistics. Thus, we need to use bigmapRef, which outputs SIGMA.TXT in units of meters. However, bigmapRef needs an additional bigmap name as the first line of the file to define the normal plane. Thus, we have to run normal bigmap before we run bigmapRef

ID=00000
Res=5

logP=log/$ID$Res
mkdir -p $logP

checkVar
bigmap < nftConfig/nftBigmap-$ID-$Res.in

ROI Iterate over new maplets (Note: Uses LMRKLIST1.TXT)

checkVar
echo "Getting ready for iteration $ID-$Res" >> notes
date >> notes
echo $ID$Res | tee -a notes
echo ${ID}$Res > tmpRun
cat nftConfig/nftBigmap-${ID}-$Res.in >> tmpRun
cat tmpRun >> notes
bigMapRef < tmpRun
tail -2 SIGMAS.TXT | head -1 | tee -a notes
echo y | cp -v LMRKLIST1.TXT LIST.TXT
duplicates

relink.sh scripts/Piterate1St.seed make_scriptP.seed | tee -a notes
ls -l make_scriptP.seed >> notes
MAKE_LMRKLISTX 


make_scriptP | tee run.sh
echo "Running iteration at " `date` >> notes
sh run.sh
monitor

checkVar
bigMapRef < tmpRun
echo "Post run Sigma" >> notes
tail -2 SIGMAS.TXT | head -1 | tee -a notes

checkVar
nftBlockFinish.sh nft.$ID
sh rem_script.b

Run AUTOREGISTER before kicking off remaining iterations

checkVar
grep P6 USED_PICS.TXT > make_script.in
echo END >> make_script.in
echo "Running AUTOREGISTER" > notes

relink.sh support/make_scriptA_jrwLowerC2.seed make_scriptA.seed
make_scriptAP > run.sh
sh run.sh

autoEval.sh

checkVar
nftBlockFinish.sh nft.$ID

If AUTOREGISTER excludes many images, do another iterate over the new maplets

autoEval.sh

Make sure you have 3 BBD stations for each image

# Needs new maplets in MAPINFO.TXT; Run if RESIDUALS has not been run since tiling
echo "5 .001 .001" | RESIDUALS


getImagesInLandmarks2.sh LMRKLIST1.TXT
head -1 tmpjrw
sort -k16,16 tmpjrw | head 

Setup Batch Iterate NFT Feature (Note: Uses USED_MAPS.TXT)

checkVar
echo "Getting ready for iteration $ID-$Res" >> notes
date >> notes
echo $ID$Res | tee -a notes
echo ${ID}$Res > tmpRun
cat nftConfig/nftBigmap-${ID}-$Res.in >> tmpRun
cat tmpRun >> notes
bigMapRef < tmpRun
tail -2 SIGMAS.TXT | head -1 | tee -a notes
echo y | cp -v USED_MAPS.TXT LIST.TXT
duplicates

relink.sh scripts/Piterate1St.seed make_scriptP.seed | tee -a notes
ls -l make_scriptP.seed >> notes
MAKE_LMRKLISTX 

echo nft=$ID > config/nftID
echo res=$Res >> config/nftID
echo 1 > config/iterationCount
echo 10 > config/stopCount

relink.sh /opt/local/spc/bin/nftLoop.sh localLoop.sh

make_scriptP | tee run.sh
echo "Running iteration at " `date` >> notes
sh run.sh
monitor

Run the normal NFT iterate steps to start the running.

Remember to:

  • Turn off terminate
  • Run localLoop
  • Start the deamon

nftBlockFinish.sh nft.$ID
sh rem_script.b

Tuck Log files into logNFT

mkdir -p logNFT
mkdir -p logNFT/$ID
mv log/* logNFT/$ID/

NFT-3-iterate (last edited 2020-06-09 07:40:15 by JohnWeirich)