Differences between revisions 1 and 23 (spanning 22 versions)
Revision 1 as of 2016-01-15 14:07:54
Size: 528
Comment:
Revision 23 as of 2017-12-26 10:25:25
Size: 4539
Editor: EricPalmer
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= process_fits = = PROCESS_FITS =
||Category B ||Version 3.0.1 ||
Line 3: Line 4:
== Descriptions ==
Line 5: Line 5:
This routine reads a FITS image file, extracts relevant information from the header, adds a record to the [[make_sumfiles.in]] file, and adds a raw 8- or 16- bit image to the [[IMAGEFILES]] directory. == Description ==
Process_fits is the main ingestion program for SPC, specifically adjusted to support OREx's metadata and file format. It does the following:
Line 7: Line 8:
== Input Files ==
 * FITS image file, name provided in stdin
 * reads a FITS image file
 * extracts relevant information from the header
 * adds a record to the [[make_sumfiles.in]] file
 * adds a raw 16- bit unsigned integer image to the [[IMAGEFILES]] directory
Line 10: Line 13:
== Input stdin == [[SPC-OREx Naming Convention]] - FITS filenames are shortened to 12 characters. The 1st char and last two chars have embedded metadata in them. This explains how Bob encoded it. In general, it is camera symbol (P,M,N,T), the ET and then two char for filter number and processing status.
Line 12: Line 15:
 * FITS image file name ----

= Level 0 =
The data stored in each image file contained in the [[IMAGEFILES]] directory is a scaled unsigned integer (as 14 bits folded into 16 bits), with the exclusion of values less than 0. Acceptable values are from 0 to 16,383 (14 bit). The files will contain a bias -- i.e. an offset to ensure that the system can record no-photons detected without needing to use a negative value (in all flight conditions and temperatures)

== ECR Changes from 3.0A2 ==
 * Support OREx level 0 images
  * Trim the second channel from the image to make it 1024x1024
  * Remove dark/bias -- This comes from the shielded overscan region
  * Correct the flat field
  * Desmear the image

=== Required Files ===
 * FITS image file - You provide the name as part of the standard input.
 * DATA/[[DN2TEMP.TXT]]
 * DATA/[[FLATFILES.TXT]]
 

=== Outputs ===
 * a row for the image is appended to [[make_sumfiles.in]]
 * 16-bit unsigned integer image(s) are stored to [[IMAGEFILES]] directory. Stretched from 0 to 65536

= WARNING ==

 * The program [[display]] will put the 0,0 position of the image in the top left, but OREx wants 0,0 to be in the bottom left.

----
== Level 0 Processing Steps ==
The level 0 OCAMS and NavCam images need to have a basic level of calibration done to them, which occurs in process_fits. The details of the calibration comes from the following documents
||software version || Document ||
|| v3.0.1 and v3.0.2 || KinetX Interoffice Memorandum SNAFD.B / 023-16, Dated 25 Sept 2017 ||

=== OCAMS ===

Level 0 OCAMS images have the images stored as two FITS images. The first FITS image is the standard 1024x1024 FITS image that most of the science team will use (if they want level 0). The second FITS image array is ________x_______, which is the full sensor for OCAMS and contains the overscan region used for calibration. This format is fully described in the OCAMS SIS.


[[FITS Keywords]] The following keywords are read by SPC. This describes what is done with each.

 1) DN2TEMP


 2) FLATFILES.TXT - Scale and divide

 3) Darks - Subtract

 4) Reduce Size

=== TAGCAMS (NavCam / NFTCam) ===

[[FITS Keywords]] The following keywords are read by SPC. This describes what is done with each.

 1) DN2TEMP

 2) Reduce Size

 3) Darks
    {{{
    DN(i,j) -= DK(i,j)
    }}}



 4) FLATFILES.TXT
    {{{
    DN (i,j) *= FLAT(i,j)
    }}}





----

= Level 1 =
If the source is a real (or 32-bit float) value '''process_fits''' takes the original images' data float and calculates the maximum value. Then it scales the data to spread from 0 to 65536 to maximize the accuracy of the stored data.

[[FITS Keywords]] The following keywords are read by SPC. This describes what is done with each.


----
=== Required Files ===
 * FITS image file - You provide the name as part of the standard input.


=== Outputs ===
 * a row for the image is appended to [[make_sumfiles.in]]
 * 16-bit unsigned integer image(s) are stored to [[IMAGEFILES]] directory. Stretched from 0 to 65536

----------

== Using process_fits ==
process_fits can be run in two modes

 * Single file - just give the FITS filename at the prompt
 * Batch mode
  * If the file [[newpic.txt]] exists, then process_fits will search through ../OSIRIS_REX/NEW_IMAGES to find the image and process. [[NEW_IMAGES]] is configured as YYYY/DOY/<source images>

Here is a sample of the standard input:
Line 14: Line 116:
 Input filename
2019-01-20T22-47-05.541_PCAM_L0b_V004.fits
 OKAY
2019 JAN 20 22:47:05.050
EXPOSURE(ms) = 100.000
 MSB SIGNED
      0.00 0.00 0.00 0.00 0.00
   5.0000000000000000
P601296494J3
}}}
----------
Line 15: Line 128:
}}} Old - [[process_fits-3.0A2]]
Line 17: Line 130:
== Output ==
 * a row for the image entered into the routine is made in [[make_sumfiles.in]]
 * a raw 16- or 18- bit image(s) to [[IMAGEFILES]] directory
''(Compiled by EP)''

----
CategoryThreeOhOne

PROCESS_FITS

Category B

Version 3.0.1

Description

Process_fits is the main ingestion program for SPC, specifically adjusted to support OREx's metadata and file format. It does the following:

  • reads a FITS image file
  • extracts relevant information from the header
  • adds a record to the make_sumfiles.in file

  • adds a raw 16- bit unsigned integer image to the IMAGEFILES directory

SPC-OREx Naming Convention - FITS filenames are shortened to 12 characters. The 1st char and last two chars have embedded metadata in them. This explains how Bob encoded it. In general, it is camera symbol (P,M,N,T), the ET and then two char for filter number and processing status.


Level 0

The data stored in each image file contained in the IMAGEFILES directory is a scaled unsigned integer (as 14 bits folded into 16 bits), with the exclusion of values less than 0. Acceptable values are from 0 to 16,383 (14 bit). The files will contain a bias -- i.e. an offset to ensure that the system can record no-photons detected without needing to use a negative value (in all flight conditions and temperatures)

ECR Changes from 3.0A2

  • Support OREx level 0 images
    • Trim the second channel from the image to make it 1024x1024
    • Remove dark/bias -- This comes from the shielded overscan region
    • Correct the flat field
    • Desmear the image

Required Files

Outputs

  • a row for the image is appended to make_sumfiles.in

  • 16-bit unsigned integer image(s) are stored to IMAGEFILES directory. Stretched from 0 to 65536

= WARNING ==

  • The program display will put the 0,0 position of the image in the top left, but OREx wants 0,0 to be in the bottom left.


Level 0 Processing Steps

The level 0 OCAMS and NavCam images need to have a basic level of calibration done to them, which occurs in process_fits. The details of the calibration comes from the following documents

software version

Document

v3.0.1 and v3.0.2

KinetX Interoffice Memorandum SNAFD.B / 023-16, Dated 25 Sept 2017

OCAMS

Level 0 OCAMS images have the images stored as two FITS images. The first FITS image is the standard 1024x1024 FITS image that most of the science team will use (if they want level 0). The second FITS image array is x_, which is the full sensor for OCAMS and contains the overscan region used for calibration. This format is fully described in the OCAMS SIS.

FITS Keywords The following keywords are read by SPC. This describes what is done with each.

  • 1) DN2TEMP 2) FLATFILES.TXT - Scale and divide 3) Darks - Subtract 4) Reduce Size

TAGCAMS (NavCam / NFTCam)

FITS Keywords The following keywords are read by SPC. This describes what is done with each.

  • 1) DN2TEMP 2) Reduce Size 3) Darks
    •     DN(i,j) -= DK(i,j)
    4) FLATFILES.TXT
    •     DN (i,j) *= FLAT(i,j)


Level 1

If the source is a real (or 32-bit float) value process_fits takes the original images' data float and calculates the maximum value. Then it scales the data to spread from 0 to 65536 to maximize the accuracy of the stored data.

FITS Keywords The following keywords are read by SPC. This describes what is done with each.


Required Files

  • FITS image file - You provide the name as part of the standard input.

Outputs

  • a row for the image is appended to make_sumfiles.in

  • 16-bit unsigned integer image(s) are stored to IMAGEFILES directory. Stretched from 0 to 65536


Using process_fits

process_fits can be run in two modes

  • Single file - just give the FITS filename at the prompt
  • Batch mode
    • If the file newpic.txt exists, then process_fits will search through ../OSIRIS_REX/NEW_IMAGES to find the image and process. NEW_IMAGES is configured as YYYY/DOY/<source images>

Here is a sample of the standard input:

 Input filename
2019-01-20T22-47-05.541_PCAM_L0b_V004.fits
 OKAY
2019 JAN 20 22:47:05.050
EXPOSURE(ms) =    100.000
 MSB SIGNED
      0.00      0.00      0.00      0.00      0.00
   5.0000000000000000
P601296494J3


Old - process_fits-3.0A2

(Compiled by EP)


CategoryThreeOhOne

process_fits (last edited 2020-03-10 13:53:18 by EricPalmer)