Differences between revisions 24 and 25
⇤ ← Revision 24 as of 2016-01-26 17:16:54 →
Size: 3807
Editor: DianeLambert
Comment:
← Revision 25 as of 2016-01-29 14:46:04 → ⇥
Size: 3899
Editor: BMittan
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
(Compiled by DL)
Line 4: Line 3:
'''Purpose:''' Duplicates writes an ordered list of unique landmark names to [[make_script.in]], the .in file required by batch processes such as [[lithosP]] processing using [[make_scriptP]]. Duplicates also checks that the landmarks are included in [[LMRKLIST.TXT]], and that LMKFILES and MAPFILES exist for each entry. == Description ==
Line 6: Line 5:
The duplicates procedure serves two functions:
 * First, some lists of landmarks to be processed may contain duplicate names. The files [[check.txt]] and [[MAPCHK.TXT]] produced by [[residuals]] are examples. duplicates outputs only one of each to the file [[make_script.in]];
 * Second, in iterating a set of landmarks, it is believed to be more reliable to proceed from lower to higher resolution maplets. The program does this ordering using the Heapsort algorithm (Numerical Recipes, p.231) as part of the process. 		This program writes an ordered list of unique landmark names to [[make_script.in]], the .in file required by batch processes such as [[lithosP]] processing using [[make_scriptP]]. It also checks that the landmarks are included in [[LMRKLIST.TXT]], and that LMKFILES and MAPFILES exist for each entry.
Line 10: Line 7:
The input file for the procedure duplicates is LIST.TXT, consisting of a list of landmark names followed by END as the final record. The output file [[make_script.in]] has the same form. The '''duplicates''' program serves two functions:
 * First, some lists of landmarks to be processed may contain duplicate names. The files [[check.txt]] and [[MAPCHK.TXT]] produced by [[residuals]] are examples. '''duplicates''' outputs only one of each to the file [[make_script.in]].
 * Second, in iterating a set of landmarks, it is believed to be more reliable to proceed from lower to higher resolution maplets. '''duplicates''' does this ordering using the Heapsort algorithm (Numerical Recipes, p. 231) as part of the process.
Line 12: Line 11:
=== Requires === The input file for '''duplicates''' is LIST.TXT, which consists of a list of landmark names followed by END as the final record. The output file [[make_script.in]] has the same form.

=== Required Files ===
Line 14: Line 15:
 * LIST.TXT - Temporary list of the subset of landmarks which the user wishes to process appended with 'END', for sorting and removal of duplicates using the program duplicates.  * LIST.TXT - Temporary list of the subset of landmarks which the user wishes to process appended with 'END', for sorting and removal of duplicates using this program.
Line 18: Line 19:
=== Output ===
 * [[make_script.in]] - .in file required by script-makers such as [[make_scriptP]] for generating the run scripts required for batch processing. 		=== Output  Files ===
 * [[make_script.in]] - the .in file required by script-makers such as [[make_scriptP]] for generating the run scripts required for batch processing.

----------
Line 23: Line 26:
=== Create Input File === === Step 1. Create Input File ===
Line 25: Line 28:
Create LIST.TXT, a list of landmark names for the landmarks intended for batch processing, appended with the end-of-file identifier, 'END', see example below. Create LIST.TXT, a list of landmark names for the landmarks intended for batch processing, appended with the end-of-file identifier, 'END'. See sample file below.
Line 27: Line 30:
=== Run duplicates === === Step 2. Run duplicates ===
Line 29: Line 32:
duplicates processes landmark names as follows:
 * for every landmark listed in [[LMRKLIST.TXT]]:
  * duplicates checks that the MAPFILE exists;
  * duplicates obtains the maplet's Q-size and Ground Sample Distance (GSD) from the LMKFILE;
  * duplicates sorts the list by decreasing GSD and increasing name (lat/wlong identifier and numeric identifier);
  * duplicates discards duplicate entries;
 * for every landmark listed in LIST.TXT:
  * duplicates checks that the landmark is also listed in [[LMRKLIST.TXT]], if so the landmark is included in the output;
 * duplicates writes the sub-set of ordered landmarks to [[make_script.in]].		 '''duplicates''' processes landmark names as follows:
 * For every landmark listed in [[LMRKLIST.TXT]], '''duplicates'''
  * checks that the MAPFILE exists
  * obtains the maplet's Q-size and Ground Sample Distance (GSD) from the LMKFILE
  * sorts the list by decreasing GSD and increasing name (lat/wlong identifier and numeric identifier)
  * discards duplicate entries
 * For every landmark listed in LIST.TXT, '''duplicates'''
  * checks that the landmark is also listed in [[LMRKLIST.TXT]]; if it is, the landmark is included in the output
 * '''duplicates''' writes the sub-set of ordered landmarks to [[make_script.in]].
Line 39: Line 42:
Note:
 * The LANDMARK and MAP files for the landmarks included in [[LMRKLIST.TXT]] must exist in [[LMKFILES]]/ and [[MAPFILES]]/;
 * duplicates only considers landmarks contained in LIST.TXT, that are also contained in [[LMRKLIST.TXT]].		 {{{#!wiki note
'''Note'''
 * The LANDMARK and MAP files for the landmarks included in [[LMRKLIST.TXT]] must exist in [[LMKFILES]]/ and [[MAPFILES]]/.
Line 43: Line 46:
=== Example ===  * '''duplicates''' only considers landmarks contained in LIST.TXT, that are also contained in [[LMRKLIST.TXT]].
}}}
Line 45: Line 49:
'''Example LMRKLIST.TXT:''' === Sample Files ===

Example '''LMRKLIST.TXT''' file:
Line 70: Line 76:
'''Example LIST.TXT:''' Example '''LIST.TXT''' file:
Line 98: Line 104:
Note that the example LIST.TXT contains a disordered list of landmark names, and many duplicate entries. Further note that the FF* landmarks are not included in the example LMRKLIST.TXT, and that the example LIST.TXT does not include any landmarks ending in 1, 2, or 3. (This scenario is included for illustrative purposes only.) Note that the example LIST.TXT contains a disordered list of landmark names and many duplicate entries. Also, the FF* landmarks are not included in the example LMRKLIST.TXT and the example LIST.TXT does not include any landmarks ending in 1, 2, or 3. (This scenario is included for illustrative purposes only.)
Line 100: Line 106:
'''Example make_script.in''' generated using duplicates: Example '''make_script.in''' file generated using the program:
Line 118: Line 124:

-----
''(Compiled by DL)''

duplicates

Description

This program writes an ordered list of unique landmark names to make_script.in, the .in file required by batch processes such as lithosP processing using make_scriptP. It also checks that the landmarks are included in LMRKLIST.TXT, and that LMKFILES and MAPFILES exist for each entry.

The duplicates program serves two functions:

  • First, some lists of landmarks to be processed may contain duplicate names. The files check.txt and MAPCHK.TXT produced by residuals are examples. duplicates outputs only one of each to the file make_script.in.

  • Second, in iterating a set of landmarks, it is believed to be more reliable to proceed from lower to higher resolution maplets. duplicates does this ordering using the Heapsort algorithm (Numerical Recipes, p. 231) as part of the process.

The input file for duplicates is LIST.TXT, which consists of a list of landmark names followed by END as the final record. The output file make_script.in has the same form.

Required Files

  • LMRKLIST.TXT - Full list of landmarks, appended with 'END'.

  • LIST.TXT - Temporary list of the subset of landmarks which the user wishes to process appended with 'END', for sorting and removal of duplicates using this program.

  • LMKFILES/ - Directory containing all the LMKFILES listed in LMRKLIST.TXT.

  • MAPFILES/ - Directory containing all the MAPFILES for landmarks listed in LMRKLIST.TXT.

Output Files

  • make_script.in - the .in file required by script-makers such as make_scriptP for generating the run scripts required for batch processing.

Using duplicates

Step 1. Create Input File

Create LIST.TXT, a list of landmark names for the landmarks intended for batch processing, appended with the end-of-file identifier, 'END'. See sample file below.

Step 2. Run duplicates

duplicates processes landmark names as follows:

  • For every landmark listed in LMRKLIST.TXT, duplicates

    • checks that the MAPFILE exists
    • obtains the maplet's Q-size and Ground Sample Distance (GSD) from the LMKFILE
    • sorts the list by decreasing GSD and increasing name (lat/wlong identifier and numeric identifier)
    • discards duplicate entries

  • For every landmark listed in LIST.TXT, duplicates

    • checks that the landmark is also listed in LMRKLIST.TXT; if it is, the landmark is included in the output

  • duplicates writes the sub-set of ordered landmarks to make_script.in.

Note

  • The LANDMARK and MAP files for the landmarks included in LMRKLIST.TXT must exist in LMKFILES/ and MAPFILES/.

  • duplicates only considers landmarks contained in LIST.TXT, that are also contained in LMRKLIST.TXT.

Sample Files

Example LMRKLIST.TXT file: 

EE0001
EE0002
EE0003
EE0004
EE0005
EE0006
EE0007
EE0008
EE0009
EE0010
EF0001
EF0002
EF0003
EF0004
EF0005
EF0006
EF0007
EF0008
EF0009
EF0010
END

Example LIST.TXT file: 

EF0006
EF0006
EF0006
FF0006
EE0004
FF0009
EE0007
EE0010
EE0009
EE0006
EF0010
EF0004
EE0005
EF0005
EF0008
FF0008
EE0008
FF0005
FF0010
EF0009
EF0007
FF0004
FF0007
END

Note that the example LIST.TXT contains a disordered list of landmark names and many duplicate entries. Also, the FF* landmarks are not included in the example LMRKLIST.TXT and the example LIST.TXT does not include any landmarks ending in 1, 2, or 3. (This scenario is included for illustrative purposes only.)

Example make_script.in file generated using the program: 

EE0004
EE0005
EE0006
EE0007
EE0008
EE0009
EE0010
EF0004
EF0005
EF0006
EF0007
EF0008
EF0009
EF0010
END

(Compiled by DL)

duplicates (last edited 2016-07-24 08:19:15 by BMittan)