Block 7 - Clean

-- version 0

need to fix links to coverage, overlaps, and Zmaps.

Description

This block is used to complete all of the stages but it does not follow a set of steps. Instead, to complete Block 7, you need to assure that the DTM has achieved certain states. To achieve these states, you may need to run additional iterations, return to earlier Blocks, or make other adjustments. Errors and problems in the data will determine the steps and processes you need to follow.

Assumptions

This assumes that you are working with a bigmap region. If you have recently iterated, you can skip some steps (if you know what to do).

Standard Process of States

The following table lists the states that a DTM goes through on its way to being perfected for a specific resolution. Each state criterion defines the quality of processing necessary to complete that state. The state description gives general guidance for achieving the state. However, it does not include all the steps that you may need to take to achieve the criterion. You may need to do many steps multiple times (such as building and loading an updated shape model). Because these steps can affect other states, you may then need to do more work to meet those criteria again.

State	Criterion	Description
Tiled	The DTM has undergone the basic stages of Lat/Lon Tiling (Block 6A) or Bigmap Tiling (Block 6B).	This state does not require any additional processing, just the batch mode.
Correlated	All the landmarks indicate a correlation score above 0.0000.	When you run find_nofitP, none of the landmarks throw the "*" error.
Imaged	There are at least 5 images for every landmark.	Five images are needed to have a robust DTM that accounts for error. If there are not 5, then 4 or 3 are acceptable; however, the number of images can never be less than 3.
Fitted	There are no maplets with more than 10 "no fit errors".	No Fit errors are thrown when overlaps is unable to match two maplets that should overlap. This error indicates that there are too many differences between the two maplets for SPC to find common ground. It is common for a few maplets to be missing one or two of their connections with other maplets. However, no maplet may have more than 10 no fit errors.
Geometry	You have run the program geometry with option 2 to update the spacecraft position and camera vector.	You should also iterate at least once after you run geometry. Do we need to explain a bit more here? Or is it just a stipulation to do geometry and then iterate??
Residualed	All errors thrown by residuals and marked by ">>" have been corrected.	Typically, you will run geometry with option 2 before running residuals. The standard input for this level is: `5 5xaRMS bestResImg` . These inputs refer to: 5 image (or maplet) pixels (how close they are), 5 times the acceptable RMS, and the best image resolution. However, you may choose looser values to get a smaller list first. I still need help here to get this clear.
Re-tiled	Any holes in the DTM have been filled in.	For the global scale, use coverage and look for where the missing maplets are. You can also use ZMaps if it's available. For the local scale, build a bigmap and look for regions where the maplets were deleted. You can then build the missing ones by hand, or you can run the Bigmap Tiling (Block 6B) procedure.
Sigma'ed	A DTM created by densify or bigmap has sigma values showing all peaks reduced to < 2xGSD and the average is <GSD of the smallest maplet.	These values are shown in SIGMAS.TXT. Will those values make more sense if we write them out in words, as on residualed?

Advanced States

In some instances, you need to assure that the DTM achieves these additional, advanced state.

State	Criterion	Description
Re-imaged	You have added images to get as many as possible, doing the processing by hand as needed.	Adding more images reduces the amount of error in the model, so adding more images will make the model better. Do we want to add something about how to know when to stop??
Image geometry	You have completed an evaluation of all images such that each one has at least 4 landmarks defined to set the spacecraft position/pointing.	we need some description here. Maybe what I'd use to do the evaluation? how I could get at least 4?
Re-Residualed	You have used tighter limits for anything that throws an error.	Typically, you do this using these inputs: `2 2xGSD GSD` . This sets the image pixels to 2, and uses 2 times the ground sample distance (resolution of the maplets).

Setup

You need a list of landmarks that cover your region. To get a full list of landmarks, run bigmap in order to generate the file USED_MAPS, which is a list of all the maplets (landmarks) that is included in the boundaries of your bigmap.

bigmapName=<name>
GSD=.00005
bigmap < $bigmapName
cp USED_MAPS workingList.txt

-- --

Correlated

You can use support/evalLog.sh to get this information, which is much faster, but it requires that you have done an iteration and you are parsing the logs. If you don't have current *.OOT files, then run batchCheck. It will generate tmpResults.txt that show you the worst correlated image and how many excluded images (stars) it has.

support/batchCheck workingList
grep -v "+++" tmpResults.txt

Imaged

Run residuals to get the statistics needed for this one. Then evaluate the 7th column (6th if you don't count the "m") for number of images in a landmark.

echo 5 .005 .0002 | residuals
sort -n -k 7 MAPINFO.TXT > tmpWorking

Review the beginning of the file. If there are landmarks with fewer than 3 images, try to add them using lithos. If that doesn't work, delete them.

Fitted

This is best done from an iteration (in fact, I can't think of a non-iteration way to do it without building a script that does just that).

Assuming you've done an iteration -- Block 5 - Iterate, then do

support/evalLog.sh > tmpWorking

Any landmarks that carry more than 10 Nofit errors need to be fixed. If you have a few between 5 and 10, take a look at them to see if you can fix things. To fix them, use lithos and rebuild topography and run overlaps/limbs. You may find it necessary to rebuild the topography for the neighboring landmarks that are failing also.

Geometry

Residuals

Execute Block 8 - Geometry/Residuals

Re-tiled

Here we need to see if there are any holes in the bigmap due to deleted landmarks or shifted landmarks that have left a small hole.

map_coverage
$bigmapName
0 $GSD
convert coverage_m.pgm ~/send/$bigmapName-cov.jpg

If there is a BARE POINT IN $bigmapName, then you have a location with no maplet at all, let alone a lack of overlaps. Review the ~bigmapName-cov.jpg to see how the distribution of landmarks are. If more are needed, get the pixel/line location and use the following to create new landmarks.

c
a
m
$bigmapName
<line> <sample>
$GSD 49
g
i
a
y
.5
n
u
1
.......

Sigma'ed

Here we evaluate the max deviation from SIGMA.pgm and SIGMA.TXT to identify if there are too large of problems.

convert SIGMA.pgm ~/send/$bigmap-sig.jpg
tail -1 SIGMA.TXT

The values from SIGMA.TXT are the maximum deviation (what DN 255 equates to in SIGMA.pgm) and the average standard deviation of all the pixels in the bigmap.

The goal is for a image that has very little squares in it, but show rims of craters and boulders as the worst deviations. If the image has clear squares that are brighter, then that associated landmark needs to be fixed.

Get the pixel/line location of the center of the square.
Run lithos to get the list of landmarks, so you know the ones that need to be fixed.

lithos
f
m
$bigmapName
<x-pos> <y-pos>

This will give you a list of all the landmarks that intersect that location. The problem landmark will have intersection coordinates near 50/50 and a width that is close to the bright box in the SIGMA.pgm. Run lithos on this landmark and improve it as much as you can. Rerun bigmap to see if you fixed the problem.

Re-imaged

We want to be sure we have sufficient landmarks in all the images. As such, we can run autoregister over the entire suite of images, or at least just the next images. With the improvement of camera pointing and spacecraft position, this should draw in all the landmarks that can be made available. Note -- We really should build a tool to see how many images get deleted using the o 0.5 option.

imageList=PICTLIST.TXT     # or some other list
ln -s script/make_scriptA_v0.seed make_scriptA.seed
cp -f imageList make_script.in
make_scriptA
sh run_script.b

Image Geometry

Here we are focusing on cleaning up the images to ensure there is sufficient information (at least 3 landmarks per image).

Step 1 - dealing with stars

You've already run register, so we will just continue with that file. If there has been substantial changed, rerun register.

grep -v "*" PICINFO.TXT | sort -n -k 8 > workingList

/usr/local/bin/Display 
<imageNm>
n
n

Run Display for all images that have some landmarks. First, see if the image is usable. There needs to be enough visible and illuminated surface to calculate a good position/pointing for the spacecraft. The bright squares will show you where the center of the landmarks are. If they are all clustered in one side of the image, that is bad. If the image really is unusable, then it should be tucked.
Run [[autoregister] by hand to evaluate the quality of the landmarks and the likely hood of getting enough to fix the position of the spacecraft/pointing. There can be times where, due to a poor DTM, images are eliminated during the correlation test. If there are a lot of landmarks that *should* be there, but are not auto-aligning, you may need to do additional work to build topography for them to get this image to work.
For whatever images you cannot get more than 3 landmarks, tuck them away. Under no circumstances can we have images with weak landmarks that are being used for limbs

Step 2 - dealing with chevrons

The next error is based on residuals being to high. It shows up in a similar column a the stars.

grep ">" PICINFO.TXT | sort -n -r -k 11 > workingList

Review the workingList file and start with the largest problem images (at the top of the list). Clean as well as possible. Remember, when you accept the auto-alignment, it will update the spacecraft position and position (like a single image geometry).

Tuck an image

lithos
Main Menu: Q 0123O IUCRSG NVAMBX DEPL FZH?  >p                                                                               
p

 Input 12-character picture name.
<PICTNM>


 0. No change
 1. Tuck it away
 2. Ingore image in Vlm
1

(Compiled by EP)

CategoryHowTos