Differences between revisions 17 and 18

Export and Import

These two programs will allow you to export landmarks from multiple machines, which can then be added to another machine. Instead of merging two or more datasets, you can achieve a similar result by adding all landmarks to a "main" or "head" machine.

While this is useful for combining data from multiple users, it was originally intended to allow a user to tile an object/bigmap simultaneously on multiple machines and then combine the results onto a single machine. Hence it uses LMRKLIST1.TXT, which is a list of newly added landmarks.

User Warnings

$/!\$ You must ensure that LMRKLIST1.TXT only has the recently created landmarks. If it has older ones, trim the top of the file.
$/!\$ You should also ensure that there aren't any maplets or landmarks remaining in NEW_FILES. If NEW_FILES does not exist, you must create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.
$/!\$ The imported landmarks will be renamed based upon region! This renaming procedure ensures that no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.

Export

Requred Files

MAPFILES/ - a directory containing the full suite of maplets
LMKFILES/ - a directory containing the full suite of landmarks
LMRKLIST1.TXT - a list of the new landmarks

Output Files

EXPORT.TXT - Script that will execute the EXPORT.B script and clean up the previous version of NEW_FILES
EXPORT.B - Script that will cp new maplet and landmark files to NEW_FILE

Using Export

To run export, you must generate a script that will copy the new landmarks and put them into a tar ball.

Make EXPORT.B and the script EXPORT.TXT. Note that EXPORT.TXT will run the script EXPORT.B.
$/!\$ You must ensure that LMRKLIST1.TXT only has the recently created landmarks. If it has older ones, trim the top of the file.
$/!\$ You should also ensure that there aren't any maplets or landmarks remaining in NEW_FILES. If NEW_FILES does not exist, you must create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.
Enter the following at the command line:

/usr/local/bin/EXPORT

This will create the directory NEW_FILES and populate it with copies of the newly created mapfiles and landmarks.

The script EXPORT.TXT looks this:

 cp LMRKLIST1.TXT NEW_FILES/LMRKLIST.TXT
 cd NEW_FILES/LMKFILES
 rm -f *.LMK
 cd ../..
 cd NEW_FILES/MAPFILES
 rm -f *.MAP
 cd ../..
 chmod +x EXPORT.b
 ./EXPORT.b
 tar -cf NEW_FILES.tar NEW_FILES
 rm -f LMRKLIST1.TXT

Run this script by entering the following at the command line:

sh EXPORT.TXT

Change the name of NEW_FILES.tar to something distinctive like
- new_<bigmapName>.tar
Transfer the tar file to the other computer.

Import

Required Files

NEW_FILES/LMRKLIST.TXT - List of new landmarks
NEW_FILES/MAPILES/ - a directory containing the new maplets
NEW_FILES/LMKFILES/ - a directory containing the new landmarks

Output Files

MAPFILES/ - a directory containing the full suite of maplets
LMKFILES/ - a directory containing the full suite of landmarks
LMRKLIST.TXT - updated landmark list
NEWLIST.TXT - list of new landmarks with the new name

Using Import

Untar to recover NEW_FILES directory. If you didn't rename the tar file, from the working directory type the following at the command line:

tar -xf NEW_FILES.TAR

This will create the directory NEW_FILE if it doesn't already exist. If you are concerned about overwriting data in the current directory, use "tar" to check the location the data will be written to.

Once you have confirmed that you will be writing to an empty directory, type the following at the command line of the working directory:

/usr/local/bin/IMPORT

This will import the new landmarks.
$/!\$ The imported landmarks will be renamed based upon region! This renaming procedure ensures that no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.

(Modified by JRW from text written by EEP)

-  ⇤ ← Revision 17 as of 2016-01-26 12:07:47 → 
  Size: 3576
  Editor: JohnWeirich
  Comment:
+   ← Revision 18 as of 2016-01-29 16:13:29 → ⇥
  Size: 4508
  Editor: BMittan
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-(Modified by JRW from text written by EEP)
+These two programs will allow you to export landmarks from multiple machines, which can then be added to another machine. Instead of merging two or more datasets, you can achieve a similar result by adding all landmarks to a "main" or "head" machine.
 Line 5:
-These two programs will allow you to export landmarks from multiple machines, which can then be added to another machine. Instead of merging two or more datasets, a similar end result is achieved by adding all landmarks to a "main" or "head" machine.
+While this is useful for combining data from multiple users, it was originally intended to allow a user to tile an object/bigmap simultaneously on multiple machines and then combine the results onto a single machine. Hence it uses [[LMRKLIST1.TXT]], which is a list of newly added landmarks.
 Line 7:
-While this is useful for combining data from multiple users, it was originally intended to be a tool which would allow someone to tile an object/bigmap simultaneously on multiple machines, then combine the results onto a single machine. Hence it uses [[LMRKLIST1.TXT]], which is a list of newly added landmarks.
+== User Warnings ==
 ''''' /!\ You must ensure that LMRKLIST1.TXT only has the recently created landmarks.  If it has older ones, trim the top of the file.  

 /!\ You should also ensure that there aren't any maplets or landmarks remaining in [[NEW_FILES]]. If [[NEW_FILES]] does not exist, you must create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.'''''

 ''''' /!\ The imported landmarks will be renamed based upon region! This renaming procedure ensures that no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.'''''
-Line 11:
+Line 17:
-=== REQUIRES ===
 * [[MAPFILES]]/ - a directory containing the full suite of maplets;
 * [[LMKFILES]]/ - a directory containing the full suite of landmarks;
 * [[LMRKLIST1.TXT]] - a list of the new landmarks;
+=== Requred Files ===
 * [[MAPFILES]]/ - a directory containing the full suite of maplets
 * [[LMKFILES]]/ - a directory containing the full suite of landmarks
 * [[LMRKLIST1.TXT]] - a list of the new landmarks
-Line 16:
+Line 22:
-=== OUTPUT ===
 *  EXPORT.TXT - Script that will execute the EXPORT.B script & clean up the previous version of NEW_FILES
+=== Output Files ===
 *  EXPORT.TXT - Script that will execute the EXPORT.B script and clean up the previous version of NEW_FILES
-Line 20:
+Line 26:
-We generate a script that will copy the new landmarks and put them into a tar ball.  
Makes EXPORT.B and the script EXPORT.TXT.  If you notice, EXPORT.TXT will run the script EXPORT.B.  The key thing you need to ensure is that LMRKLIST1.TXT only has the recently created landmarks.  If it has older ones, then you might need to trim the top of the file.  You should also ensure that there aren't any maplets or landmarks remaining in [[NEW_FILES]]. If [[NEW_FILES]] does not exist, you will need to create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.
+----------
== Using Export ==

To run export, you must generate a script that will copy the new landmarks and put them into a tar ball.  

 1. Make EXPORT.B and the script EXPORT.TXT. Note that EXPORT.TXT will run the script EXPORT.B.  

 ''''' /!\ You must ensure that LMRKLIST1.TXT only has the recently created landmarks.  If it has older ones, trim the top of the file.     /!\ You should also ensure that there aren't any maplets or landmarks remaining in [[NEW_FILES]]. If [[NEW_FILES]] does not exist, you must create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.'''''

 1.#2 Enter the following at the command line:
-Line 28:
+Line 45:
-The script EXPORT.TXT looks this, but you don't need to understand it to use it:
+The script EXPORT.TXT looks this:
-Line 43:
+Line 60:
-Run this script by
+.#3 Run this script by entering the following at the command line:
-Line 48:
+Line 65:
-Change name of NEW_FILES.tar to something distinctive like
+.#4 Change the name of NEW_FILES.tar to something distinctive like
-Line 52:
+Line 69:
-and transfer the tar file to the other computer.
+.#5 Transfer the tar file to the other computer.
-Line 56:
+Line 73:
-=== REQUIRES ===
+=== Required Files ===
-Line 58:
+Line 75:
- * NEW_FILES/MAPILES/ - a directory containing the new maplets;
 * NEW_FILES/LMKFILES/ - a directory containing the new landmarks;
+ * NEW_FILES/MAPILES/ - a directory containing the new maplets
 * NEW_FILES/LMKFILES/ - a directory containing the new landmarks
-Line 61:
+Line 78:
+=== Output Files ===
 * [[MAPFILES]]/ - a directory containing the full suite of maplets
 * [[LMKFILES]]/ - a directory containing the full suite of landmarks
 * [[LMRKLIST.TXT]] - updated landmark list
 * NEWLIST.TXT - list of new landmarks with the new name
-Line 62:
+Line 84:
-=== OUTPUT ===
 * [[MAPFILES]]/ - a directory containing the full suite of maplets;
 * [[LMKFILES]]/ - a directory containing the full suite of landmarks;
 * [[LMRKLIST.TXT]] - updated landmark list;
 * NEWLIST.TXT - List of new landmarks with the new name;
+----------
-Line 68:
+Line 86:
-Untar to recover NEW_FILES directory. If you didn't rename the tar file, from the working directory all you need to do is type...
+== Using Import ==

 1. Untar to recover NEW_FILES directory.   If you didn't rename the tar file, from the working directory type the following at the command line:
-Line 73:
+Line 94:
- which should create the directory NEW_FILE if it doesn't already exist. If you are concerned about overwriting data in the current directory, use "tar" to check the location the data will be written to. Once confirmed, from the working directory type ...
+  This will create the directory NEW_FILE if it doesn't already exist. If you are concerned about overwriting data in the current directory, use "tar" to check the location the data will be written to. 

 1.#2 Once you have confirmed that you will be writing to an empty directory, type the following at the command line of the working directory:
-Line 79:
+Line 102:
- which will import the new landmarks. '''NOTE:''' The imported landmarks will be renamed based upon region! This renaming procedure ensures no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.
+ This will import the new landmarks. 

 ''''' /!\ The imported landmarks will be renamed based upon region! This renaming procedure ensures that no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.'''''

-----

''(Modified by JRW from text written by EEP)''