= duplicates =

||Category B||Version 3.0||

== Description ==

This program writes an ordered list of unique landmark names to [[make_script.in]], the .in file required by batch processes such as [[lithosP]] processing using [[make_scriptP]]. It also checks that the landmarks are included in [[LMRKLIST.TXT]], and that LMKFILES and MAPFILES exist for each entry.

The '''duplicates''' program serves two functions:
 * First, some lists of landmarks to be processed may contain duplicate names.  The files [[check.txt]] and [[MAPCHK.TXT]] produced by [[residuals]] are examples.  '''duplicates''' outputs only one of each to the file [[make_script.in]].
 * Second, in iterating a set of landmarks, it is believed to be more reliable to proceed from lower to higher resolution maplets.  '''Duplicates''' does this ordering using the Heapsort algorithm (Numerical Recipes, p. 231) as part of the process.  

The input file for '''duplicates''' is LIST.TXT, which consists of a list of landmark names followed by END as the final record.  The output file [[make_script.in]] has the same form.

=== Required Files  ===
 * [[LMRKLIST.TXT]] - Full list of landmarks, appended with 'END'.
 * LIST.TXT - Temporary list of the subset of landmarks which the user wishes to process appended with 'END', for sorting and removal of duplicates using this program.
 * [[LMKFILES]]/ - Directory containing all the LMKFILES listed in [[LMRKLIST.TXT]].
 * [[MAPFILES]]/ - Directory containing all the MAPFILES for landmarks listed in [[LMRKLIST.TXT]].

=== Output  Files ===
 * [[make_script.in]] - the .in file required by script-makers such as [[make_scriptP]] for generating the run scripts required for batch processing. 

----------

== Using duplicates ==

 1. Create Input File

Create LIST.TXT, a list of landmark names for the landmarks intended for batch processing, appended with the end-of-file identifier, 'END'. See sample file below.

 1.#2 Run duplicates 

'''duplicates''' processes landmark names as follows:
 * For every landmark listed in [[LMRKLIST.TXT]], '''duplicates'''
  * checks that the MAPFILE exists
  * obtains the maplet's Q-size and Ground Sample Distance (GSD) from the LMKFILE
  * sorts the list by decreasing GSD and increasing name (lat/wlong identifier and numeric identifier)
  * discards duplicate entries
 * For every landmark listed in LIST.TXT, '''duplicates'''
  * checks that the landmark is also listed in [[LMRKLIST.TXT]]; if it is, the landmark is included in the output
 * '''duplicates''' writes the sub-set of ordered landmarks to [[make_script.in]].

 (!) The LANDMARK and MAP files for the landmarks included in [[LMRKLIST.TXT]] must exist in [[LMKFILES]]/ and [[MAPFILES]]/.

 (!) '''duplicates''' only considers landmarks contained in LIST.TXT, that are also contained in [[LMRKLIST.TXT]].


Here is a sample '''LMRKLIST.TXT''' file:
{{{
EE0001
EE0002
EE0003
EE0004
EE0005
EE0006
EE0007
EE0008
EE0009
EE0010
EF0001
EF0002
EF0003
EF0004
EF0005
EF0006
EF0007
EF0008
EF0009
EF0010
END
}}}

Here is a sample '''LIST.TXT''' file:
{{{
EF0006
EF0006
EF0006
FF0006
EE0004
FF0009
EE0007
EE0010
EE0009
EE0006
EF0010
EF0004
EE0005
EF0005
EF0008
FF0008
EE0008
FF0005
FF0010
EF0009
EF0007
FF0004
FF0007
END
}}}

 /!\ Note that the sample LIST.TXT contains a disordered list of landmark names and many duplicate entries. Also, the FF* landmarks are not included in the sample LMRKLIST.TXT and the sample LIST.TXT does not include any landmarks ending in 1, 2, or 3. (This scenario is included for illustrative purposes only.)

Here is a sample [[make_script.in]] file generated using '''duplicates''':
{{{
EE0004
EE0005
EE0006
EE0007
EE0008
EE0009
EE0010
EF0004
EF0005
EF0006
EF0007
EF0008
EF0009
EF0010
END   
}}}

----------
''(Compiled by DL)''

CategoryPrograms