Differences between revisions 10 and 33 (spanning 23 versions)
⇤ ← Revision 10 as of 2016-01-22 15:05:22 →
Size: 2564
Editor: JohnWeirich
Comment:
← Revision 33 as of 2021-05-21 06:33:59 → ⇥
Size: 5300
Editor: DalyTerik
Comment: Clarified which LMKs to have in LMRKLIST1.TXT
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
(Modified by JRW from text written by EEP) ||Category B||Version 3.0||
Line 5: Line 5:
These two programs will allow you to export landmarks from multiple machines, which can then be added to another machine. Instead of merging two or more datasets, a similar end result is achieved by adding all landmarks to a "main" or "head" machine. These two programs will allow you to export landmarks from multiple machines, which can then be added to another machine. Instead of merging two or more datasets, you can achieve a similar result by adding all landmarks to a "main" or "head" machine.
Line 7: Line 7:
While this is useful for combining data from multiple users, it was originally intended to be a tool which would allow someone to tile an object/bigmap simultaneously on multiple machines, then combine the results onto a single machine. Hence it uses [[LMRKLIST1.TXT]], which is a list of newly added landmarks. While this is useful for combining data from multiple users, it was originally intended to allow a user to tile an object/bigmap simultaneously on multiple machines and then combine the results onto a single machine. Hence it uses [[LMRKLIST1.TXT]], which is a list of newly added landmarks.

== User Warnings ==
 ''' /!\ You must ensure that LMRKLIST1.TXT only has the recently created landmarks (i.e., contains only the LMKs you wish to export and import into a new model). If LMRKLIST1.TXT has other LMKs you don't wish to bring into a different model, remove them from the top of the LMRKLIST1.TXT.'''

 ''' /!\ You should also ensure that there aren't any maplets or landmarks remaining in [[NEW_FILES]]. If [[NEW_FILES]] does not exist, you must create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.'''

 ''' /!\ The imported landmarks will be renamed based upon region! This renaming procedure ensures that no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.'''
Line 11: Line 19:
We generate a script that will copy the new landmarks and put them into a tar ball.
Makes EXPORT.B and the script EXPORT.TXT. If you notice, EXPORT.TXT will run the script EXPORT.B. The key thing you need to ensure is that LMRKLIST1.TXT only has the recently created landmarks. If it has older ones, then you might need to trim the top of the file. You should also ensure that there aren't any maplets or landmarks remaining in [[NEW_FILES]]. If [[NEW_FILES]] does not exist, you will need to create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.		 === Required Files ===
 * [[MAPFILES]]/ - a directory containing the full suite of maplets
 * [[LMKFILES]]/ - a directory containing the full suite of landmarks
 * [[LMRKLIST1.TXT]] - a list of the new landmarks

=== Output Files ===
 * EXPORT.TXT - Script that will execute the EXPORT.B script and clean up the previous version of NEW_FILES
 * EXPORT.B - Script that will cp new maplet and landmark files to NEW_FILE

----------
== Using Export ==

To run export, you must generate a script that will copy the new landmarks and put them into a tar ball.

 1. Make EXPORT.B and the script EXPORT.TXT. Note that EXPORT.TXT will run the script EXPORT.B.

 ''' /!\ You must ensure that LMRKLIST1.TXT only has the recently created landmarks. If it has older ones, trim the top of the file.'''

 ''' /!\ You should also ensure that there aren't any maplets or landmarks remaining in [[NEW_FILES]]. If [[NEW_FILES]] does not exist, you must create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.'''

 1.#2 Enter the following at the command line:
 
Line 14: Line 42:
mkdir -p NEW_FILES
mkdir -p NEW_FILES/LMKFILES
mkdir -p NEW_FILES/MAPFILES
Line 16: Line 47:
This will create the directory NEW_FILES and populate it with copies of the newly created mapfiles and landmarks. This will populate NEW_FILES with copies of the newly created mapfiles and landmarks.
Line 19: Line 50:
The script EXPORT.TXT looks this, but you don't need to understand it to use it: The script EXPORT.TXT looks this:
Line 34: Line 65:
Run this script by  1.#3 Run this script by entering the following at the command line:
Line 39: Line 70:
Change name of NEW_FILES.tar to something distinctive like  1.#4 Change the name of NEW_FILES.tar to something distinctive like
Line 43: Line 74:
and transfer the tar file to the other computer.  1.#5 Transfer the tar file to the other computer.

---------------------
Line 47: Line 80:
=== Required Files ===
 * NEW_FILES/LMRKLIST.TXT - List of new landmarks
 * NEW_FILES/MAPILES/ - a directory containing the new maplets
 * NEW_FILES/LMKFILES/ - a directory containing the new landmarks
Line 48: Line 85:
Untar to recover NEW_FILES directory. If you didn't rename the tar file, from the working directory all you need to do is type... === Output Files ===
 * [[MAPFILES]]/ - a directory containing the full suite of maplets
 * [[LMKFILES]]/ - a directory containing the full suite of landmarks
 * [[LMRKLIST.TXT]] - updated landmark list
 * NEWLIST.TXT - list of new landmarks with the new name

----------

== Using Import ==

1. Untar to recover NEW_FILES directory.
 If you didn't rename the tar file, from the working directory type the following at the command line:
Line 50: Line 98:
tar -xf NEW_FILES.TAR tar -xvf NEW_FILES.TAR
Line 53: Line 101:
  This will create the directory NEW_FILE if it doesn't already exist. If you are concerned about overwriting data in the current directory, use "tar" to check the location the data will be written to.
Line 54: Line 103:
2. Once you have confirmed that you will be writing to an empty directory, type the following at the command line of the working directory:

{{{
Line 55: Line 107:
}}}
Line 56: Line 109:
 This will import the new landmarks.

 ''' /!\ The imported landmarks will be renamed based upon region! This renaming procedure ensures that no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.'''

3. Import will update and move all the files to be integrated into the SPC working directory. However, it needs an iteration to run the renaming portion and connect the overlaps with the existing maplets.

{{{
Line 57: Line 117:
}}}
Line 73: Line 135:
To clean up any old runs, use these commands (copy into window) Copy into window:
Line 75: Line 137:
sh rem_script.b
rm TESTFILES/*
rm TESTFILES1/*
~/bin/make_scriptF.e
./run_script.b &		 ./rem_script.b
rm -f ./TESTFILES/*
rm -f ./TESTFILES1/*
make_scriptF
sh ./run_script.b &
Line 82: Line 144:
(!) Type this from time to time to see how it is going, [[import]]. Check progress with:
Line 84: Line 146:
~/bin/find_nofitT (this didn't seem to work, but find_nofitP.e gave output that made sense) find_nofit
Line 86: Line 148:

----------

''(Modified by JRW from text written by EEP)''

CategoryPrograms

Export and Import

Category B

Version 3.0

These two programs will allow you to export landmarks from multiple machines, which can then be added to another machine. Instead of merging two or more datasets, you can achieve a similar result by adding all landmarks to a "main" or "head" machine.

While this is useful for combining data from multiple users, it was originally intended to allow a user to tile an object/bigmap simultaneously on multiple machines and then combine the results onto a single machine. Hence it uses LMRKLIST1.TXT, which is a list of newly added landmarks.

User Warnings

  • /!\ You must ensure that LMRKLIST1.TXT only has the recently created landmarks (i.e., contains only the LMKs you wish to export and import into a new model). If LMRKLIST1.TXT has other LMKs you don't wish to bring into a different model, remove them from the top of the LMRKLIST1.TXT.

    /!\ You should also ensure that there aren't any maplets or landmarks remaining in NEW_FILES. If NEW_FILES does not exist, you must create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.

    /!\ The imported landmarks will be renamed based upon region! This renaming procedure ensures that no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.

Export

Required Files

  • MAPFILES/ - a directory containing the full suite of maplets

  • LMKFILES/ - a directory containing the full suite of landmarks

  • LMRKLIST1.TXT - a list of the new landmarks

Output Files

  • EXPORT.TXT - Script that will execute the EXPORT.B script and clean up the previous version of NEW_FILES
  • EXPORT.B - Script that will cp new maplet and landmark files to NEW_FILE

Using Export

To run export, you must generate a script that will copy the new landmarks and put them into a tar ball.

  1. Make EXPORT.B and the script EXPORT.TXT. Note that EXPORT.TXT will run the script EXPORT.B.

    /!\ You must ensure that LMRKLIST1.TXT only has the recently created landmarks. If it has older ones, trim the top of the file.

    /!\ You should also ensure that there aren't any maplets or landmarks remaining in NEW_FILES. If NEW_FILES does not exist, you must create NEW_FILES, as well as NEW_FILES/LMKFILES and NEW_FILES/MAPFILES.

  2. Enter the following at the command line: 

mkdir -p NEW_FILES
mkdir -p NEW_FILES/LMKFILES
mkdir -p NEW_FILES/MAPFILES
/usr/local/bin/EXPORT

This will populate NEW_FILES with copies of the newly created mapfiles and landmarks.

The script EXPORT.TXT looks this: 

 cp LMRKLIST1.TXT NEW_FILES/LMRKLIST.TXT
 cd NEW_FILES/LMKFILES
 rm -f *.LMK
 cd ../..
 cd NEW_FILES/MAPFILES
 rm -f *.MAP
 cd ../..
 chmod +x EXPORT.b
 ./EXPORT.b
 tar -cf NEW_FILES.tar NEW_FILES
 rm -f LMRKLIST1.TXT
  1. Run this script by entering the following at the command line: 

sh EXPORT.TXT
  1. Change the name of NEW_FILES.tar to something distinctive like

    • new_<bigmapName>.tar

  2. Transfer the tar file to the other computer.

Import

Required Files

  • NEW_FILES/LMRKLIST.TXT - List of new landmarks
  • NEW_FILES/MAPILES/ - a directory containing the new maplets
  • NEW_FILES/LMKFILES/ - a directory containing the new landmarks

Output Files

  • MAPFILES/ - a directory containing the full suite of maplets

  • LMKFILES/ - a directory containing the full suite of landmarks

  • LMRKLIST.TXT - updated landmark list

  • NEWLIST.TXT - list of new landmarks with the new name

Using Import

1. Untar to recover NEW_FILES directory.

  • If you didn't rename the tar file, from the working directory type the following at the command line: 

tar -xvf NEW_FILES.TAR
  • This will create the directory NEW_FILE if it doesn't already exist. If you are concerned about overwriting data in the current directory, use "tar" to check the location the data will be written to.

2. Once you have confirmed that you will be writing to an empty directory, type the following at the command line of the working directory: 

/usr/local/bin/IMPORT
  • This will import the new landmarks.

    /!\ The imported landmarks will be renamed based upon region! This renaming procedure ensures that no existing landmarks will be overwritten. It also means you may end up with duplicate landmarks if you didn't make sure to exclude them when you created LMRKLIST1.TXT.

3. Import will update and move all the files to be integrated into the SPC working directory. However, it needs an iteration to run the renaming portion and connect the overlaps with the existing maplets. 

cp NEWLIST.TXT make_script.in

For make_scriptF.seed use: 

n
n
g
u
1
o
RECENT
n
1
q
END

Copy into window: 

./rem_script.b
rm -f ./TESTFILES/*
rm -f ./TESTFILES1/*
make_scriptF
sh ./run_script.b &

Check progress with: 

find_nofit

(Modified by JRW from text written by EEP)

CategoryPrograms

Export_Import (last edited 2021-05-21 06:33:59 by DalyTerik)