Differences between revisions 7 and 29 (spanning 22 versions)
⇤ ← Revision 7 as of 2016-01-18 19:09:40 →
Size: 2318
Editor: DianeLambert
Comment:
← Revision 29 as of 2016-07-24 08:19:15 → ⇥
Size: 3915
Editor: BMittan
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
(Compiled by DL)
Line 4: Line 3:
'''Purpose:''' . ||Category B||Version 3.0||
Line 6: Line 5:
== Description ==
Line 7: Line 7:
This program writes an ordered list of unique landmark names to [[make_script.in]], the .in file required by batch processes such as [[lithosP]] processing using [[make_scriptP]]. It also checks that the landmarks are included in [[LMRKLIST.TXT]], and that LMKFILES and MAPFILES exist for each entry.
Line 8: Line 9:
=== Requires ===
 * [[LMRKLIST.TXT]] - Full list of landmarks.
 * LIST.TXT - Temporary list of the subset of landmarks which the user wishes to process, for sorting and removal of duplicates using the program duplicates.
 * [[LMKFILES]] - Directory containing all the LMKFILES listed in LMRKLIST.TXT and LIST.TXT.
 * [[MAPFILES]] - Directory containing all the MAPFILES for landmarks listed in LMRKLIST.TXT and LIST.TXT.		 The '''duplicates''' program serves two functions:
 * First, some lists of landmarks to be processed may contain duplicate names. The files [[check.txt]] and [[MAPCHK.TXT]] produced by [[residuals]] are examples. '''duplicates''' outputs only one of each to the file [[make_script.in]].
 * Second, in iterating a set of landmarks, it is believed to be more reliable to proceed from lower to higher resolution maplets. '''Duplicates''' does this ordering using the Heapsort algorithm (Numerical Recipes, p. 231) as part of the process.
Line 14: Line 13:
=== Output ===
 * make_script.in - .in file required by [[make_scriptP]] for generating the run scripts required to iterate the landmarks using [[lithosP]]. 		The input file for '''duplicates''' is LIST.TXT, which consists of a list of landmark names followed by END as the final record. The output file [[make_script.in]] has the same form.

=== Required Files ===
 * [[LMRKLIST.TXT]] - Full list of landmarks, appended with 'END'.
 * LIST.TXT - Temporary list of the subset of landmarks which the user wishes to process appended with 'END', for sorting and removal of duplicates using this program.
 * [[LMKFILES]]/ - Directory containing all the LMKFILES listed in [[LMRKLIST.TXT]].
 * [[MAPFILES]]/ - Directory containing all the MAPFILES for landmarks listed in [[LMRKLIST.TXT]].

=== Output Files ===
 * [[make_script.in]] - the .in file required by script-makers such as [[make_scriptP]] for generating the run scripts required for batch processing.

----------
Line 19: Line 28:
duplicates processes landmark names as follows:
 * for every landmark listed in [[LMRKLIST.TXT]]:
  * duplicates checks that the MAPFILE exists;
  * duplicates obtains the maplet's Q-size and Ground Sample Distance (GSD) from the LMKFILE;
  * duplicates sorts the list by decreasing GSD and increasing name (lat/wlong identifier and numeric identifier);
  * duplicates discards duplicate entries;
 * for every landmark listed in LIST.TXT:
  * duplicates checks that the landmark is also listed in [[LMRKLIST.TXT]], if so the landmark is included in the output;
 * duplicates writes the sub-set of ordered landmarks to make_script.in.		  1. Create Input File
Line 29: Line 30:
NB:
 * The LANDMARK and MAP files for the landmarks included in [[LMRKLIST.TXT]] must exist in [[LMKFILES]]/ and [[MAPFILES]]/;
 * duplicates only considers landmarks contained in LIST.TXT, that are also contained in [[LMRKLIST.TXT]].		 Create LIST.TXT, a list of landmark names for the landmarks intended for batch processing, appended with the end-of-file identifier, 'END'. See sample file below.
Line 33: Line 32:
=== Example ===  1.#2 Run duplicates
Line 35: Line 34:
'''Example LMRKLIST.TXT:''' '''duplicates''' processes landmark names as follows:
 * For every landmark listed in [[LMRKLIST.TXT]], '''duplicates'''
  * checks that the MAPFILE exists
  * obtains the maplet's Q-size and Ground Sample Distance (GSD) from the LMKFILE
  * sorts the list by decreasing GSD and increasing name (lat/wlong identifier and numeric identifier)
  * discards duplicate entries
 * For every landmark listed in LIST.TXT, '''duplicates'''
  * checks that the landmark is also listed in [[LMRKLIST.TXT]]; if it is, the landmark is included in the output
 * '''duplicates''' writes the sub-set of ordered landmarks to [[make_script.in]].

 (!) The LANDMARK and MAP files for the landmarks included in [[LMRKLIST.TXT]] must exist in [[LMKFILES]]/ and [[MAPFILES]]/.

 (!) '''duplicates''' only considers landmarks contained in LIST.TXT, that are also contained in [[LMRKLIST.TXT]].


Here is a sample '''LMRKLIST.TXT''' file:
Line 60: Line 74:
'''Example LIST.TXT:''' Here is a sample '''LIST.TXT''' file:
Line 65: Line 79:
FF0002
Line 69: Line 82:
EF0002
EE0002
Line 76: Line 87:
EE0001
EE0001
Line 79: Line 88:
EE0001
Line 83: Line 91:
EE0003
Line 86: Line 93:
FF0001
Line 89: Line 95:
FF0003
Line 92: Line 97:
EF0003
Line 95: Line 99:
EE0003
EF0001
Line 100: Line 102:
'''Example make_script.in:'''  /!\ Note that the sample LIST.TXT contains a disordered list of landmark names and many duplicate entries. Also, the FF* landmarks are not included in the sample LMRKLIST.TXT and the sample LIST.TXT does not include any landmarks ending in 1, 2, or 3. (This scenario is included for illustrative purposes only.)

Here is a sample [[make_script.in]] file generated using '''duplicates''':
Line 102: Line 106:
EE0001
EE0002
EE0003
Line 112: Line 113:
EF0001
EF0002
EF0003
Line 124: Line 122:

----------
''(Compiled by DL)''

CategoryPrograms

duplicates

Category B

Version 3.0

Description

This program writes an ordered list of unique landmark names to make_script.in, the .in file required by batch processes such as lithosP processing using make_scriptP. It also checks that the landmarks are included in LMRKLIST.TXT, and that LMKFILES and MAPFILES exist for each entry.

The duplicates program serves two functions:

  • First, some lists of landmarks to be processed may contain duplicate names. The files check.txt and MAPCHK.TXT produced by residuals are examples. duplicates outputs only one of each to the file make_script.in.

  • Second, in iterating a set of landmarks, it is believed to be more reliable to proceed from lower to higher resolution maplets. Duplicates does this ordering using the Heapsort algorithm (Numerical Recipes, p. 231) as part of the process.

The input file for duplicates is LIST.TXT, which consists of a list of landmark names followed by END as the final record. The output file make_script.in has the same form.

Required Files

  • LMRKLIST.TXT - Full list of landmarks, appended with 'END'.

  • LIST.TXT - Temporary list of the subset of landmarks which the user wishes to process appended with 'END', for sorting and removal of duplicates using this program.

  • LMKFILES/ - Directory containing all the LMKFILES listed in LMRKLIST.TXT.

  • MAPFILES/ - Directory containing all the MAPFILES for landmarks listed in LMRKLIST.TXT.

Output Files

  • make_script.in - the .in file required by script-makers such as make_scriptP for generating the run scripts required for batch processing.

Using duplicates

  1. Create Input File

Create LIST.TXT, a list of landmark names for the landmarks intended for batch processing, appended with the end-of-file identifier, 'END'. See sample file below.

  1. Run duplicates

duplicates processes landmark names as follows:

  • For every landmark listed in LMRKLIST.TXT, duplicates

    • checks that the MAPFILE exists
    • obtains the maplet's Q-size and Ground Sample Distance (GSD) from the LMKFILE
    • sorts the list by decreasing GSD and increasing name (lat/wlong identifier and numeric identifier)
    • discards duplicate entries

  • For every landmark listed in LIST.TXT, duplicates

    • checks that the landmark is also listed in LMRKLIST.TXT; if it is, the landmark is included in the output

  • duplicates writes the sub-set of ordered landmarks to make_script.in.

    (!) The LANDMARK and MAP files for the landmarks included in LMRKLIST.TXT must exist in LMKFILES/ and MAPFILES/.

    (!) duplicates only considers landmarks contained in LIST.TXT, that are also contained in LMRKLIST.TXT.

Here is a sample LMRKLIST.TXT file: 

EE0001
EE0002
EE0003
EE0004
EE0005
EE0006
EE0007
EE0008
EE0009
EE0010
EF0001
EF0002
EF0003
EF0004
EF0005
EF0006
EF0007
EF0008
EF0009
EF0010
END

Here is a sample LIST.TXT file: 

EF0006
EF0006
EF0006
FF0006
EE0004
FF0009
EE0007
EE0010
EE0009
EE0006
EF0010
EF0004
EE0005
EF0005
EF0008
FF0008
EE0008
FF0005
FF0010
EF0009
EF0007
FF0004
FF0007
END

  • /!\ Note that the sample LIST.TXT contains a disordered list of landmark names and many duplicate entries. Also, the FF* landmarks are not included in the sample LMRKLIST.TXT and the sample LIST.TXT does not include any landmarks ending in 1, 2, or 3. (This scenario is included for illustrative purposes only.)

Here is a sample make_script.in file generated using duplicates: 

EE0004
EE0005
EE0006
EE0007
EE0008
EE0009
EE0010
EF0004
EF0005
EF0006
EF0007
EF0008
EF0009
EF0010
END

(Compiled by DL)

CategoryPrograms

duplicates (last edited 2016-07-24 08:19:15 by BMittan)