Training notes for JAXA session in support of MMX

Goals

• Bring the students to the "SPC Training Level 2" TrainingLevels

  • Provide enough SPC hand-on skills that basic shape models can be built
  • Provides enough background so that additional practice and instruction can occur
  • It will be insufficient to fully train new personal to a level in which they could run an entire mission without support
• Specific things to learn
  • Be able to create landmarks and add images
  • Basic trouble shooting and fixing mis-registration
  • Perform tiling (both globally and BIGMAP)
  • Do iterations
  • Run residuals, geometry
  • Register images to shape model or maps
  • Autoregister
• Practice. A key component of this training is to practice. The practice allows for key concepts to become understood at a sufficient level to allow for more advanced concepts to make sense.

Student Prep

Because class time is limited, it is important to get the software and data installed before you show up to the conference. I will be available during the conference to help troubleshoot problems and help get your system functional before the class starts Wednesday morning. Once the class starts, I won't be able to ignore everyone else in the class to fix an install.

To maximize your learning in the class, I provide some information that will be beneficial to review.

• Stuff to read (links below)

  • Palmer et al., 2022

  • Gaskell's 2008

Required Tasks To Complete Before Class Starts

• Sign the Licensing Agreement. This must be on file before the software can be distributed. //Complete//

• Download and install SPC software via GitHub

  • Installing SPC Instructions

  • SPC-GUI-tools - The web tools

• Download sample data. The data volume is large, so it is important to do it ahead of time. Some of the files need to be expanded and moved.
  • Day 1 - Ryugu
  • Day 2 - Ryugu
  • Day 3A - Ryugu
  • Day 3B - Phobos

  • Starting directory & SPICE (600MB)

    • You can put this directory anywhere. It is the base directory for the SPC work.
  • Images:

    • GitHub - https://github.com/StereoPhotoClinometry/SPC-Class-data

  • Support Tools:
    • Over time, I have found that there are many nice helpers for SPC that do well if organized into its own area. They include scripts, programs and image lists. I've included some here. I suggest putting these two directories in /opt/local/spc/

    • GitHUB - https://github.com/StereoPhotoClinometry/SPC-Class-data

Extra software that is useful

• Software - These are helpful tools. There are other tools you can use, but these are ones that we have found to be very useful.

  • gnuplot - Installing using MacPorts, "sudo port install gnuplot"

  • imagemagick Installing using MacPorts, "sudo port install ImageMagick"

• SPC tools - (TBD) This is a suite of tools that make measuring pixel/line locations and offsets easier. This is not required and class time won't be used for installing this. Install instructions are coming soon
  • apache -- The is installed as default for most UNIX (and Mac) systems.
  • spc_tool javascript libraries
  • updateDisplay.sh - script to convert SPC output to a web-accessible format

  • How to install and run - SPC-GUI-tools

Day 1

• Introductions
• Discussion of the Wiki/User’s manual

• Unix file layout and paths -- File System Setup

  • executables
  • source code
  • helpers
  • scripts
  • support
• SPC files
  • working
  • mapfile/landmark files
  • kernels
  • images

• Previous presentation on, Theory of SPC -- https://sbib.psi.edu/spc_wiki/Training_JAXA_Aug2016?action=AttachFile&do=view&target=Lesson_1.pdf

Day 1 Practical - It's all about LITHOS

• Topics
  • display landmark
  • landmark management
    • create
    • delete
    • rename
  • images
    • add images
    • remove images - auto and manual
    • select useable, correlation and manual
    • adjust image position, auto, manual
  • global shift
  • build template
  • find height
    • options

The core of SPC is lithos. That tool lets you create landmarks, associate images, align images, determine the 3D position of the landmark and create topography.

Fixing landmarks, using Input landmark

Aligning landmarks, using Align landmarks

Building a landmark Create new landmark and How To Build A Landmark

• Name: DD0002
• Lat/Lon: 20.072 / 290.164
• Res: 0.0012
• Q-size: 49

Template

• Find template

Topography

Once all the images are aligned (the center pixel of every image is at the exact same feature of the surface), then you can build topography using photoclinometry with some conditions. Find heights

toposcript

State

A key diagnostic tool for SPC is understanding the state of the system. By state, I mean what are the conditions (the health or quality) of the landmark. There is no one parameter that tells you all of this, but it is looking at a wide set of details that tells you if there are problems.

Align, Extra and Solve (AES) Iteration

Because SPC is an iterative process, each time the AES process is done, more data is extracted from the images and the solution moves towards a stability point.

Block 5 - Iterate v2

See Notes from day 1 -- These are things brought up in class or requested from the previous class on SPC. They are provided here because they are useful, but don't fit in another category.

Day 2

New files for today

Use branch "B-register" from GitHUB. If you are using GitHUB Desktop, it will ask to if you should bring the changes over or stash them. Select stash. If you are downloading this as a separate file (such as a .zip), then just do your work in that new directory.

Clean some of the landmarks

See toDoB in the working directory. Clean the landmarks listed at the top of the file.

"Clean" means to check the state of the landmark and fix any problems.

• Input the landmark into lithos "i"
• Check the alignment of the landmarks (use 1 0 1 n 0 n) and look for an images that do not correlate
• Align any images that are off
  • You can do automatic, " 0. Auto align" to let the computer try to find a good fit. Typically try a space (search radius) from 1 to 4.
  • You can do manual, "1. Individual shift". Use SPC Web Tools or count pixels to identify the x/y that is needed to get the image close. Once it is close, you can use Auto align to snap it into place.
  • You can do other picture, "3. Align with picture". This lets you select one image that is very good and get the other images to align to it. Note, if you use then, you have to use option "0 Find Template" to turn all of the images back on.
• Check to see if all of the images are selected for building the template -- i.e. when you go into option 0. Find template, none of the images have a star by their name.
  • If images have a star, then toggle them if they are aligned.

• If you've made changes, then run the toposcript to pull the updated data into the shape model.

• Save the model (u 1)

Iterate

Now that we've pulled in extra information, we need to iterate to get the information propagated to all the files

Block 5 - Iterate v2

• Iteration
  • Setup

    • INIT_LITHOS.TXT -- Update the number of processors on your computer. You set USRMX=

    • make_lmrklistX -- This make the LMRKLISTX.TXT which pre-generates the positions of landmarks to make SPC run faster

    • cp LMRKLIST.TXT LIST.TXT

    • duplicates -- This reads list, sorts the file, removes duplicates and creates make_script.in

    • ln -s <whatever> make_scriptP.seed

    • make_scriptP | tee run.sh

  • Running
    • sh run.sh
    • find_nofitP

Bring in Images

• Converting images into SPC format

  • process_fits -- the generic information

  • PROCESS_FITS_MMX -- Will be written soon

  • make_sumfiles.in -- This file has a line for each image. It contains the FITS name and the SPC name.

• Converting geometric information into body-fixed frame

  • make_sumfiles -- This program creates a SUMFILES and NOMINALS for each image, and adds the image name to PICTLIST.TXT

Register

• Each images should be aligned with the shape model at a course level.

  • Run register.

  • Provide it the image name
  • When asked for "reference", select "s" for shape
  • When asked to "enter scale (km/px)", use ".1"
• You will then see the register menu. You should look at "TEMPFILE.pgm" and "TEMPFILE.ppm" to determine if the image is aligned to the shape model at that scale.
  • For SPC tools, you run the "register" link and "Reload" the window. You should see the image next to the model.
• If you need to make a correction:
  • Select 3. Shift unknown (LEFT/RED) image
  • It will ask for "Autocorrelate? (y/n)". If the two a far apart (more than a few pixels), then autocorrelate won't work, but you can try. Typically, say "n". If you try to autocorrelate, and it doesn't work, you will just continue as if you said 'n'
  • If it didn't autocorrelate or you selected "n", then you give it a delta X, delta Y to shift the image.
    • This is where SPC Web Tools is very useful. You click on the same feature in both the left hand side (image) and the right hand side (model) and it will tell you the values to enter in.
  • Once you have it well aligned at the current resolution, increase the resolution (we started with .1, then try: .01, .005, .002)
    • Select menu: 1. Change scale
    • Type in the next scale value (such as .01).
  • Now, you can go back to evaluating if the image and model is align and adjusting like you did. Continue until you have the model as aligned as possible. In this case, keep improving the alignment and resolution until it is good at .002 km)
  • In order to save what you have done, select: 0. Quit
    • It will ask you: Accept shift? (y/n) -- type "y"
    • It will ask you: Update/Create rotation history file? (y/n) -- type "n"
    • It will ask you: Update nominal file? (y/n) -- type "n"

• register has a lot of options to support a variety of difficult and complex problems. There are options to let it automatically register when you have a partial object in the field of view, when you are zoomed in very close. You can align to other images and even bigmaps.

Now register the images that are listed in the toDoB.

Autoregister

Build a shape model

• ICQ format
• DENSIFY.TXT

• densify

• dumber

• buildShape.sh - runs all of the commands as needed

Create a higher resolution

Block 6B - Bigmap Tiling v2

• Tile 80cm
  • Setup

    • ln -s

    • make_scriptT

  • Running
    • sh run_script.b

    • find_nofitT

Blocks

We have provided some procedural blocks for different tasks. They include some additional log and configuration that will be different from you, but the main SPC steps are the same. Additionally, Bob has some "PROCEDURES" directory with his source code. There are instructions there and copy/paste commands.

SPC Blocks, V2

Batch Processing

Register

• register

  • Practice
  • Batch

    • Clear out old TESTFILES and TESTFILES1

      • rm TESTFILES*/*

    • make_script.in

      • This must be created from the new list of images
      • Make a file of new image names and write them into make_script.in (with a proceeding space and the last line "END")
    • ln -s scripts/make_scriptR_01.seed make_scriptR.seed

    • make_scriptR

    • sh run_script.b
    • find_nofit

Autoregister

• Batch associate existing landmarks to new images

  • When you bring in new images, they have no landmarks. While you can add them into landmarks using lithos, that is not very productive. Thus, we use a program autoregister that loads all of the landmarks into the image you are working with.

  • autoregister

    • make_lmrklistX

    • Practice -
  • Batch

    • make_script.in -- This must be created from the new list of images

    • ln -s scripts/make_scriptA.seed make_scriptA.seed

    • make_scriptA

    • sh run_script.b

Tiling by BIGMAP

Block 6B - Bigmap Tiling v2

• Making a bigmap

  • Criteria for size, scale
    • Lat: -10
    • Lon: 350
    • Q size: 500
    • GSD: 25

  • Create support/RPI001.in

  Note - we use XXXXXX.MAP to allow for scripting. I use symbolic links to aid in keeping track of progress. By running showmap after building and linking the bigmap, you can prove that you have the correct map.

  bigmap < support/BIG001.in
  cd MAPFILES
  ln -s RPI001.MAP XXXXXX.MAP
  cd ..
  echo XXXXXX | SHOWMAP
  convert XXXXXX.pgm lowMap.jpg

Checking for existing maplets

  map_coverage
  XXXXXX
  0 .0005
  convert coverage_m.pgm preCoverage.jpg

Building the make_scriptT.in file

• Build the make_scriptT.in file

  • This includes some shell commands to make something you can copy/paste. Or you can use a text editor to build the make_scriptT.in, just make sure it follows the correct format for map-based referencing.

  echo N | make_tilefile > tmp
  echo XXXXXX > support/bigmap_tile.in
  echo scripts/XXX035.seed >> support/bigmap_tile.in
  sed 1d tmp >> support/bigmap_tile.in
  ln -s support/bigmap_tile.in make_scriptT.in

Run Tiling

• Run the tiling script. Once you make the INN files, you can track the progress *.OOT files.

    make_scriptT
    sh run_script.b
    find_nofitT

Support Files

• Review metadata and data files

  • IMAGEFILES

    • format -- 8bit, 16bit, 32bit

  • NOMINALS

  • SUMFILES

  • make_sumfiles.txt

  • INIT_LITHOS.TXT

  • LMKFILES

Evaluation of the bigmap

Now that it is done, you can rebuild the bigmap and see the difference

  bigmap < support/BIG001.in
  echo BIG001 | SHOWMAP
  convert BIG001.pgm highMap.jpg

map_coverage
BIG001
0 .0005
convert coverage_m.pgm postCoverage.jpg

• SIGMAS.TXT

• SIGMAS.pgm

System Evaluation

• RESIDUALS.TXT

• MAPINFO.TXT

• PICINFO.TXT

Iterate over region

This uses the fact that you recently made a bigmap as inputs for running a region-specific iteration. Remember, duplicates takes whatever list of landmarks in LIST.TXT, sorts them, removes duplicates and replaced make_script.in. USED_MAPS.TXT is created by bigmap, along with USED_PICTS.TXT. This contains all of the mapfiles that were used in building the bigmap.

  make_lmrklistX
  rm TESTFILES*/*
  cp USED_MAPS.TXT LIST.TXT
  duplicates
  make_scriptP > run.sh
  sh run.sh
  find_nofitP

Questions

• How are shape models used in navigation?

See

• Notes MMX-D1

• Notes MMX D2

• Notes MMX D3