Training notes for JAXA session in support of MMX

Goals

Student Prep

Because class time is limited, it is important to get the software and data installed before you show up to the conference. I will be available during the conference to help troubleshoot problems and help get your system functional before the class starts Wednesday morning. Once the class starts, I won't be able to ignore everyone else in the class to fix an install.

To maximize your learning in the class, I provide some information that will be beneficial to review.

Required Tasks To Complete Before Class Starts

Extra software that is useful

Day 1

Day 1 Practical - It's all about LITHOS

The core of SPC is lithos. That tool lets you create landmarks, associate images, align images, determine the 3D position of the landmark and create topography.

Fixing landmarks, using Input landmark

Aligning landmarks, using Align landmarks

Building a landmark Create new landmark and How To Build A Landmark

Template

Topography

Once all the images are aligned (the center pixel of every image is at the exact same feature of the surface), then you can build topography using photoclinometry with some conditions. Find heights

toposcript

State

A key diagnostic tool for SPC is understanding the state of the system. By state, I mean what are the conditions (the health or quality) of the landmark. There is no one parameter that tells you all of this, but it is looking at a wide set of details that tells you if there are problems.

Align, Extra and Solve (AES) Iteration

Because SPC is an iterative process, each time the AES process is done, more data is extracted from the images and the solution moves towards a stability point.

Block 5 - Iterate v2

See Notes from day 1 -- These are things brought up in class or requested from the previous class on SPC. They are provided here because they are useful, but don't fit in another category.

Day 2

New files for today

Use branch "B-register" from GitHUB. If you are using GitHUB Desktop, it will ask to if you should bring the changes over or stash them. Select stash. If you are downloading this as a separate file (such as a .zip), then just do your work in that new directory.

Clean some of the landmarks

See toDoB in the working directory. Clean the landmarks listed at the top of the file.

"Clean" means to check the state of the landmark and fix any problems.

Bring in Images

Register

Now register the images that are listed in the toDoB.

Autoregister

autoregister takes an image and assigns landmarks to it. This is the opposite of lithos which adds images to a landmark. When you start autoregister, you give it an image name and then tell it if you want to try to add more landmarks. Most times, the answer is yes, but sometimes you want to use the abilities of autoregister without adding more landmarks -- that is done when you've eliminated some landmarks that SPC thinks is value but they are not ready to be used for that image.

* Prepare the quick lookup file LMRKLISTX.TXT

* Run autoregister

* This will take you to the main menu. You will want to removed images that fall out of parameters (emission angle, coverage, resolution limits)

* Now, you will want to check the status of the list of landmarks (review the correlation scores)

* Set the image to be used (i.e. remove the star) for all the landmarks

* Exit autoregister

Now practice autoregister the images that are listed in the toDoB.

AES Iterate

Now that we've pulled in extra information, we need to iterate to get the information propagated to all the files

This is the first time you're working with batch processing, so a few comments:

Create a higher resolution

Here we want to cover an entire region (a bigmap) with landmarks. We will use a pre-made bigmap that is located in the lsupport directory -- lsupport is local support where you can put local and temporary files.

This is the second batch process we will be using. Much of it is similar the the others with one big exception. The seed file does not have a standard format, but is defined within make_scriptT.in.

Tiling is the one procedure that requires you do know a bit more about the structure of the control files than most of the other programs. In the other programs, you make a list of items and you do a make_scriptP command and all the input files are created.

For tiling, we use the file make_scriptT.in. This file contains the name of the BIGMAP that provides the location reference. It also contains the name of the seed file. Then it has a list of all of the x/y position in which to create landmarks, typically every 50 pixels. Note, if an entry has a # symbol, that is a comment and that location will be skipped. The top part of an example file is below:

make_scriptT.in 

BIGEQ1
lsupport/bigmap-XXX120.seed
     100  150
#    150  150
     200  150
     250  150
#    300  150
     350  150
...

When you run make_scriptT it will generate a bunch of inputs files (starting with 001.INN and counting up) and the run_script.b that you need to run it. For this work, I will email a file to be used as for both make_scriptT.in and the seed file. Put both of the files into "lsupport".

For this special case of tiling, we will be using a bigmap named "BIGEQ1" that we will have to create. We will run the bigmap routine and view it. Once done, we will use the pregenerated files so you can see how it works and let it run overnight.

Day 3

Please switch to D---postTiling branch of the repository.

Review Tiling

Yesterday, we launched a tasks that can run for a long time. I took steps to keep it from being too long, but the typical time needed to tile a medium sized BIGMAP with as many images as we have could be 5 hours. I have run the system and will review the results, which is in branch "D---postTiling".

We will look at the commands:

Errors in SPC

There are times when SPC has a glitch and something needs to be fixed. I've created this page as a list of problems and known solutions. If you find things that should be added, please let me know.

Fixing Glitches in SPC

Daily Practice

Batch Processing

Batch Register

We are going to improve the registration of some of the images by running batch register. You will find the list in todo-register in branch D---postTiling.

Batch Autoregister

We are not going to practice batch autoregister. These instructions are here so you can try it yourself. It is pretty straight forward and works just like batch [[register]

Blocks

We have provided some procedural blocks for different tasks. They include some additional log and configuration that will be different from you, but the main SPC steps are the same. Additionally, Bob has some "PROCEDURES" directory with his source code. There are instructions there and copy/paste commands.

Note: These were crafted for the OSIRIS-REx mission to ensure that the shape model was created with known and understood procedures. It is acceptable to deviate from them, but we wanted to have most of the work being done with established and tested procedures for consistency. Included in these blocks are specific data tracking steps that are not needed by SPC. They were put in so we recorded what the state was for the shape model as it progressed. These do not have to be done. However, many of these steps do provide a record for tracking what was done so that if there is an error, it can be identified and fixed.

SPC Blocks, V2

Build a shape model

Building the shape model, or global DEM, is one of main goals of SPC. Fundamentally, it is just resampling of the maplets that you've already created into a global model. SPC uses an uncommon, but highly efficient format called Interconnected Quadrangles(ICQ). Rather than the typical large set of vectors and triangular plates, these define a systematic series of rectangles in a much smaller file format.

System Evaluation

To evaluate the model run:

You will get the following output files

Conditioning of Solutions

In SPC, we can take a large number of inputs to help improve the solution of the location of the landmarks, the spacecraft and its pointing. I'm not going into this very much because it is a sophisticated concept, but more information about this can be found:

Solution Conditioning

Support Files

Demo of other tools

monitor and monitord blockFinish.sh uberLithos

Bonus -- Phobos

While working with Ryugu is great, our mission is to Phobos. Dr. Ernst and Dr. Daly have graciously provided us with a copy of their SPC main directory that includes all the images, SPICE files and the working directory. This allows us to continue their work where they left off. We will be getting much higher resolution data, but this provides an excellent opportunity to become familiar with SPC using the data we will be starting with.

I have added you all to a new repository called Phobos-2022. This is using their shape model data as of March 2022. This has already been given to the MMX project, but I can also provide the full copy. This system is very large -- too large to be easily used. As such, for this repository, I have removed much of the images, keeping them over one region that we can use for practice. This makes it so you can download and use it easily. Then, when you are ready, you can get the rest of the data and continue your training and testing.

https://github.com/StereoPhotoClinometry/Phobos-2022

I have provided a list of landmarks that need to be fixed (cleaned). They are called todo-landmarks. These are actual misalignments (or weak shape topography) that need some attention. I suggest that you use the skills you've learned here to practice on this.

Possible practice

JAXA MMX Training Notes (last edited 2023-03-07 23:22:42 by EricPalmer)