Differences between revisions 35 and 47 (spanning 12 versions)

find_nofit

(Compiled by DL)

Purpose: find_nofit searches batch processing output files to identify landmarks or images which may need further work.

find_nofit is a useful utility for identifying landmarks which may require further work following a batch processing task such as tasks which use make_scriptF or make_scriptR.

Requires

make_script.in - input file for batch processes such as make_scriptF, make_scriptR, make_scriptA or make_scriptAP.

Either:

<LMRKNM>.OOT files - output by a batch process such as make_scriptF.

Or:

<PICNM>.OOT files - output by a batch process such as make_scriptR, make_scriptA or make_scriptAP.

Output

Either:

redo.txt - list of landmarks which terminated in error, do not correlate with at least one image, or correlate weakly with at least one image.

Or:

REMOVED.TXT - list of landmarks which have been eliminated from at least one image due to low or no correlation.

Description

The batch processes initiated by make_scriptF, make_scriptR, make_scriptA and make_scriptAP produce output files <LMKNM>.OOT or <PICNM>.OOT, basically listings of the outputs of procedures lithos, register, autoregister and autoregisterP respectively for each input landmark or image. find_nofit looks through these output files for each landmark or picture listed in the make_script.in file to identify landmarks or images which may need further work.

If the entries in the .in file are landmarks (make_script.infor lithos runs), the find_nofit procedure identifies:

!  cat <LMKNM>.OOT - process still running or terminated in error.
*  cat <LMKNM>.OOT - maplets not correlating with at least one image.
+  cat <LMKNM>.OOT - maplets correlating weakly with at least one image.
.  cat <LMKNM>.OOT - no correlation for at least one overlap.
   cat <LMKNM>.OOT - low correlation for at least one overlap.

The cat <LMKNM>.OOT will produce a listing of the output file for more complete diagnostics. The program also produces a redo.txt file listing those landmarks that failed one of the first three tests. The limit for weak correlation is set by the 'NOFIT=' keyword in INIT_LITHOS.TXT.

If the entries in the .in file are images (make_script.in for register, autoregister and autoregisterP runs), the find_nofit procedure identifies:

.  cat <PICNM>.OOT  image not correlating with at least one maplet.

The cat <PICNM>.OOT will produce a listing of the output file for more complete diagnostics. The program also produces a REMOVED.TXT file listing those landmarks that were eliminated from one of the images for low or no correlation.

Example output from find_nofit:

*  cat EE0017.OOT              <- maplets not correlating with at least one image
   cat EE0019.OOT              <- low correlation for at least one overlap
   cat EE0022.OOT
.  cat EE0027.OOT              <- no correlation for at least one overlap
+  cat EE0034.OOT              <- maplets correlating weakly with at least one image
*  cat EE0042.OOT
*  cat EE0044.OOT
+  cat EE0045.OOT
!  cat EE0051.OOT              <- process still running or terminated in error
!  cat EE0053.OOT
+  cat EE0054.OOT
   cat EE0055.OOT
!  cat EE0063.OOT
   cat EE0067.OOT
   cat EE0069.OOT
   cat EE0081.OOT

Example redo.txt file:

* EE0017              <- maplets not correlating with at least one image
+ EE0034              <- maplets correlating weakly with at least one image
* EE0042
* EE0044
+ EE0045
EE0051                <- process still running or terminated in error
EE0053
+ EE0054
EE0063
END

Example REMOVED.TXT file:

AA0001                <- maplet eliminated from an image for low or no correlation
AF0023

-  ⇤ ← Revision 35 as of 2016-01-25 10:51:50 → 
  Size: 1780
  Editor: KristoferDrozd
  Comment:
+   ← Revision 47 as of 2016-01-26 16:42:46 → ⇥
  Size: 3984
  Editor: DianeLambert
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-= nofitT/nofitP/nofit =
+= find_nofit =
(Compiled by DL)
-Line 5:
+Line 6:
-Compiled by KD
+'''Purpose:''' find_nofit searches batch processing output files to identify landmarks or images which may need further work.

find_nofit is a useful utility for identifying landmarks which may require further work following a batch processing task such as tasks which use [[make_scriptF]] or [[make_scriptR]].

== Requires ==
 * [[make_script.in]] - input file for batch processes such as [[make_scriptF]], [[make_scriptR]], [[make_scriptA]] or [[make_scriptAP]].
Either:
 * <LMRKNM>.OOT files -  output by a batch process such as [[make_scriptF]].
Or:
 * <PICNM>.OOT files - output by a batch process such as [[make_scriptR]], [[make_scriptA]] or [[make_scriptAP]].

== Output ==
Either:
 * [[redo.txt]] - list of landmarks which terminated in error, do not correlate with at least one image, or correlate weakly with at least one image.
Or:
 * [[REMOVED.TXT]] - list of landmarks which have been eliminated from at least one image due to low or no correlation.
-Line 9:
+Line 25:
-At the end of each tiling or iteration, a form of find_nofit (T or P) is used to check for bad landmarks.
+The batch processes initiated by [[make_scriptF]], [[make_scriptR]], [[make_scriptA]] and [[make_scriptAP]] produce output files <LMKNM>.OOT or <PICNM>.OOT, basically listings of the outputs of procedures [[lithos]], [[register]], [[autoregister]] and [[autoregisterP]] respectively for each input landmark or image.  find_nofit looks through these output files for each landmark or picture listed in the [[make_script.in]] file to identify landmarks or images which may need further work.
-Line 11:
+Line 27:
- * find_nofitT - this routine shows which maplets created during tiling that are faulty.
+If the entries in the .in file are landmarks ([[make_script.in]]for [[lithos]] runs), the find_nofit procedure identifies:
{{{
!  cat <LMKNM>.OOT - process still running or terminated in error.
*  cat <LMKNM>.OOT - maplets not correlating with at least one image.
+  cat <LMKNM>.OOT - maplets correlating weakly with at least one image.
.  cat <LMKNM>.OOT - no correlation for at least one overlap.
   cat <LMKNM>.OOT - low correlation for at least one overlap.
}}}
The cat <LMKNM>.OOT will produce a listing of the output file for more complete diagnostics.  The program also produces a [[redo.txt]] file listing those landmarks that failed one of the first three tests.  The limit for weak correlation is set by the 'NOFIT=' keyword in [[INIT_LITHOS.TXT]].
-Line 13:
+Line 37:
- * find_nofitP - this routine shows which maplet created during iteration that are faulty.
+If the entries in the .in file are images ([[make_script.in]] for [[register]], [[autoregister]] and [[autoregisterP]] runs), the find_nofit procedure identifies:
{{{
.  cat <PICNM>.OOT  image not correlating with at least one maplet.
}}}
The cat <PICNM>.OOT will produce a listing of the output file for more complete diagnostics.  The program also produces a [[REMOVED.TXT]] file listing those landmarks that were eliminated from one of the images for low or no correlation.
-Line 15:
+Line 43:
-The routines listed above determine faulty maplets by going through every .OOT file in the working directory, which are produced at the end of tiling and iteration when run_script.sh or run.sh are used.


=== Input ===
 * <3 digit number>.OOT files in the working directory

=== Output ===
 * ! - currently running (logfile isn't complete) or there was an error and the landmark was deleted
 * (blank space)  LMKNM - minor issue
 * * LMKNM - major issue (at least one image didn't correlate at all the with the map)
 * . LMKNM - minor issue (at least one maplet overlap error)
 * + LMKNM - moderate issue (at least one image had a correlation below the value set in init_lithos - usually .3 or .4)

== Example of Output ==
+'''Example output from find_nofit:'''
-Line 30:
+Line 45:
-EQ0004:   cat 004.OOT 
EP0002:   cat 005.OOT 
EP0003:   cat 006.OOT 
EO0003:   cat 007.OOT 
EN0003: * cat 008.OOT 
EN0004: * cat 009.OOT 
EM0003: . cat 010.OOT 
EM0004: * cat 011.OOT 
132 deleted
   !  cat 132.OOTThe
+*  cat EE0017.OOT              <- maplets not correlating with at least one image
   cat EE0019.OOT              <- low correlation for at least one overlap
   cat EE0022.OOT
.  cat EE0027.OOT              <- no correlation for at least one overlap
+  cat EE0034.OOT              <- maplets correlating weakly with at least one image
*  cat EE0042.OOT
*  cat EE0044.OOT
+  cat EE0045.OOT
!  cat EE0051.OOT              <- process still running or terminated in error
!  cat EE0053.OOT
+  cat EE0054.OOT
   cat EE0055.OOT
!  cat EE0063.OOT
   cat EE0067.OOT
   cat EE0069.OOT
   cat EE0081.OOT
-Line 42:
+Line 63:
+Example [[redo.txt]] file:
{{{
* EE0017              <- maplets not correlating with at least one image
+ EE0034              <- maplets correlating weakly with at least one image
* EE0042
* EE0044
+ EE0045
EE0051                <- process still running or terminated in error
EE0053
+ EE0054
EE0063
END
}}}
-Line 43:
+Line 77:
-== John's notes ==
!ZN0205 cat 023.OOT  process still running (okay) or process didn’t finish normally (bummer).

ZN0205 cat 023.OOT no correlation in at least one maplet overlap.  Probably no big deal.

* ZN0205 cat 023.OOT small or no correlation in at least one maplet-image overlap.  Should be checked by hand in LITHOS.

----
CategoryPrograms
+Example [[REMOVED.TXT]] file:
{{{
AA0001                <- maplet eliminated from an image for low or no correlation
AF0023
}}}