• Normalized Cross-Correlation Evaluation

Normalized Cross-Correlation Evaluation

This page describes the methodology for using normalized cross-correlation to evaluate a region of the model.

Cross-correlation is performed on the following images:

• Model: Image of bigmap
• Truth: Image of Zmaps

Model (left), Truth (right), S/C Azimuth 0, Zenith 30deg, Sun Azimuth 90deg, Zenith 45deg:

Images are generated using the following set of dirty sumfiles at the required resolution to include:

• low and high phase
• low and moderate incidence
• full set of s/c azimuth angles (observe region from all directions).

Image resolution is chosen from the following based on the state of the model:

• Resolution: 120cm, 75cm, 35cm, 18cm, 9cm, 5cm.

The following regions have been identified for evaluation:

Where are the Dirty Sumfiles and Truth Images?

SUMFILES: ormacsrv1:/opt/local/spc/shape4/imageGen/DIRTYSUMFILES/
Truth .pgm files: ormacsrv1:/opt/local/spc/shape4/imageGen/PGMFILES/

Follow this link for details of the image file and bigmap naming conventions: Normalized Cross-Correlation Evaluation - Naming Conventions

Obtain Model Bigmap

Obtain the required bigmap/s from the model working directory (or back-up). A full set of bigmap .in files is stored here:

/opt/local/spc/shape4/support4/

MR1051_model_REG1_GSD5cm_width100m.in
MR1052_model_REG1_GSD5cm_width20m.in
MR1055_model_REG1_GSD5cm_width50m.in
MR1101_model_REG1_GSD10cm_width100m.in
MR1102_model_REG1_GSD10cm_width20m.in
MR1105_model_REG1_GSD10cm_width50m.in
MR1351_model_REG1_GSD35cm_width100m.in
MR1352_model_REG1_GSD35cm_width20m.in
MR1355_model_REG1_GSD35cm_width50m.in
MTG051_model_TAG1_GSD5cm_width100m.in
MTG052_model_TAG1_GSD5cm_width20m.in
MTG055_model_TAG1_GSD5cm_width50m.in
MTG101_model_TAG1_GSD10cm_width100m.in
MTG102_model_TAG1_GSD10cm_width20m.in
MTG105_model_TAG1_GSD10cm_width50m.in
MTG351_model_TAG1_GSD35cm_width100m.in
MTG352_model_TAG1_GSD35cm_width20m.in
MTG355_model_TAG1_GSD35cm_width50m.in

Populate Evaluation Directory

The F6 evaluation directory is here:

/Volumes/Data1/spc-test/F6-evaluation/

drwxrwxr-x  227 dlambert  OREX\spc  7718 Jul 22 15:09 DIRTYNOMINALS
drwxrwxr-x  227 dlambert  OREX\spc  7718 Jul 22 15:09 DIRTYSUMFILES
drwxrwxr-x  226 dlambert  OREX\spc  7684 Jul 21 16:01 SUMFILES
drwxrwx--x   13 jweirich  OREX\spc   442 Jul 21 14:38 eval_F6_BBD1_TG
drwxrwx--x   15 jweirich  OREX\spc   510 Jul 21 14:34 eval_F6_EndApp_TG
drwxrwxr-x   14 dlambert  OREX\spc   476 Jul 20 16:36 eval_F6_start
drwxr-xr-x   13 dlambert  OREX\spc   442 Jul 22 15:17 eval_F6_template
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 22 15:42 listSUM_R1_120cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 22 15:42 listSUM_R1_18cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 22 15:42 listSUM_R1_35cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 22 15:42 listSUM_R1_5cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 22 15:42 listSUM_R1_75cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 22 15:42 listSUM_R1_9cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 20 16:21 listSUM_TG_120cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 20 16:22 listSUM_TG_18cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 20 16:21 listSUM_TG_35cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 20 16:22 listSUM_TG_5cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 20 16:21 listSUM_TG_75cm
-rw-r--r--    1 dlambert  OREX\spc   117 Jul 20 16:22 listSUM_TG_9cm
drwxrwxr-x    4 dlambert  OREX\spc   136 Jun 24 16:16 lsupport
-rw-r--r--    1 jweirich  OREX\spc   587 Jul 21 14:49 parseXcorr.sh
-rw-rw-r--    1 dlambert  OREX\spc  1927 Jun 23 15:12 processing.log
lrwxr-xr-x    1 dlambert  OREX\spc    30 Jun 24 12:27 support -> /opt/local/spc/shape4/support4
lrwxr-xr-x    1 dlambert  OREX\spc    39 Jun 24 16:54 truth_PGMFILES -> /opt/local/spc/shape4/imageGen/PGMFILES

Make a subdirectory and populate as follows:

• listSUM -> ../listSUM_35cm - symbolic link to list of dirty sumfiles;

• lsupport -> ../lsupport/ - symbolic link to main directory local support directory;

• MAPFILES/ - directory containing model bigmap/s;

• MAPLIST.TXT - file listing the bigmap to image, appended with 'END';

• PGMFILES/ - directory for storage of generated .pgm files;

• SUMFILES/ - directory for storage and retrieval of SUMFILES which will be generated for the evaluation;

• THUMBNAILS/ - directory for storage of generated .jpg files;

• imageXcorr_results/ - directory for storage of imXcor.py output files;

• truth_PGMFILES -> /opt/local/spc/shape4/imageGen/PGMFILES/ - symbolic link to truth images.

dirID=eval_F6_start        # Update this ID!!

cd /Volumes/Data1/spc-test/F6-evaluation/
mkdir $dirID
cd /Volumes/Data1/spc-test/F6-evaluation/$dirID
mkdir MAPFILES PGMFILES SUMFILES THUMBNAILS imageXcorr_results
ln -s ../lsupport .
ln -s /opt/local/spc/shape4/imageGen/PGMFILES truth_PGMFILES
ln -s ../listSUM_TG_35cm listSUM
printf "%s\n%s\n" "MTG105" "END" > MAPLIST.TXT 

Make SUMFILES which will have a unique name for the evaluated model state:

modelStateID=AA             # Update this ID!!
while read name; do
  echo $name
  newNmPart1=`echo $name | cut -c1-8`
  newNmPart2=$modelStateID
  newNmPart3=`echo $name | cut -c11-12`
  newNm=`echo ${newNmPart1}${newNmPart2}${newNmPart3}`
  sed 's/'$name'/'$newNm'/g' ../SUMFILES/${name}.SUM > SUMFILES/${newNm}.SUM  
done < listSUM

Generate Model Images

The following script has been written to automate the generation of images of a bigmap using imager_MG. It expects the user to identify an existing list of dirty sumfiles as a common line item, e.g.:

sh lsupport/makeImages_imagerMG.sh listSUM

where

cat listSUM

PBF6TG3E0001
PBF6TG3E0002
PBF6TG3E0003
PBF6TG3E0004
PBF6TG3E0005
PBF6TG3E0006
PBF6TG3E0007
PBF6TG3E0008
PBF6TG3E0009

Generate thumbnails for visual inspection of the bigmap images:

while read name; do
  echo $name
  /opt/local/bin/convert PGMFILES/${name}_MTG105.pgm THUMBNAILS/${name}_MTG105.jpg
done < listSUM 

Conduct Normalized Cross-Correlation

Conduct cross-correlation using Tanner's script, storing the output in the imageXcorr_results directory:

Mapname=MTG105
while read name; do
  echo $name
  truthNM=`echo $name | sed 's/PB/P2/g'`
  python /opt/local/spc/bin/imXcorr.py PGMFILES/${name}_${Mapname}.pgm truth_PGMFILES/${truthNM}.pgm | tee imageXcorr_results/imXcor_${name}_${Mapname}.out
done < listSUM

Algorithm Source: J.P. Lewis

• Fast_normalized_cross-correlation.pdf

• vi95_lewis.pdf