IRAF help page for package noao.imred.ccdred, program ccdproc

• images
• ccdtype =
• max_cache = 0
• noproc = no

• fixpix = yes
• overscan = yes
• trim = yes
• zerocor = yes
• darkcor = yes
• flatcor = yes
• illumcor = no
• fringecor = no
• readcor = no
• scancor = no

• readaxis = line
• fixfile
• biassec
• trimsec
• zero =
• dark =
• flat =
• illum =
• fringe =
• minreplace = 1.
• scantype = shortscan
• shortscan
• longscan
• nscan

• interactive = no
• function = legendre
• order = 1
• sample = *
• naverage = 1
• niterate = 1
• low_reject = 3., high_reject = 3.
• grow = 0.

• 5 (1)
• (2)
• (3)
• (4)
• (5)
• (6)
• (7)
• (8)
• (9)
• (10)
• (11)
• (12)
• (13)
• (14)
• (15)
• (16)
• (1)
• (2)
• (3)

• CCDPROC: V2.10.3

ccdproc images

images

ccdtype =

max_cache = 0

noproc = no

overscan = yes

trim = yes

zerocor = yes

darkcor = yes

flatcor = yes

illumcor = no

fringecor = no

readcor = no

scancor = no

fixfile

biassec

trimsec

zero =

dark =

flat =

illum =

fringe =

minreplace = 1.

scantype = shortscan

longscan

nscan

function = legendre

order = 1

sample = *

naverage = 1

niterate = 1

low_reject = 3., high_reject = 3.

grow = 0.

Ccdproc processes CCD images to correct and calibrate for detector defects, readout bias, zero level bias, dark counts, response, illumination, and fringing. It also trims unwanted lines and columns and changes the pixel datatype. It is efficient and easy to use; all one has to do is set the parameters and then begin processing the images. The task takes care of most of the record keeping and automatically does the prerequisite processing of calibration images. Beneath this simplicity there is much that is going on. In this section a simple description of the usage is given. The following sections present more detailed discussions on the different operations performed and the order and logic of the processing steps. For a user's guide to the ccdred package see guide. Much of the ease of use derives from using information in the image header. If this information is missing see section 13.

One begins by setting the task parameters. There are many parameters but they may be easily reviewed and modified using the task eparam. The input CCD images to be processed are given as an image list. Previously processed images are ignored and calibration images are recognized, provided the CCD image types are in the image header (see instruments and ccdtypes). Therefore it is permissible to use simple image templates such as "*.imh". The ccdtype parameter may be used to select only certain types of CCD images to process (see ccdtypes).

The processing operations are selected by boolean (yes/no) parameters. Because calibration images are recognized and processed appropriately, the processing operations for object images should be set. Any combination of operations may be specified and the operations are performed simultaneously. While it is possible to do operations in separate steps this is much less efficient. Two of the operation parameters apply only to zero level and flat field images. These are used for certain types of CCDs and modes of operation.

The processing steps selected have related parameters which must be set. These are things like image sections defining the overscan and trim regions and calibration images. There are a number of parameters used for fitting the overscan or prescan bias section. These are parameters used by the standard IRAF curve fitting package icfit. The parameters are described in more detail in the following sections.

In addition to the task parameters there are package parameters which affect ccdproc. These include the instrument and subset files, the text and plot log files, the output pixel datatype, the amount of memory available for calibration image caching, the verbose parameter for logging to the terminal, and the backup prefix. These are described in ccdred.

Calibration images are specified by task parameters and/or in the input image list. If more than one calibration image is specified then the first one encountered is used and a warning is issued for the extra images. Calibration images specified by task parameters take precedence over calibration images in the input list. These images also need not have a CCD image type parameter since the task parameter identifies the type of calibration image. This method is best if there is only one calibration image for all images to be processed. This is almost always true for zero level and dark count images. If no calibration image is specified by task parameter then calibration images in the input image list are identified and used. This requires that the images have CCD image types recognized by the package. This method is useful if one may simply say "*.imh" as the image list to process all images or if the images are broken up into groups, in "@" files for example, each with their own calibration frames.

When an input image is processed the task first determines the processing parameters and calibration images. If a requested operation has been done it is skipped and if all requested operations have been completed then no processing takes place. When it determines that a calibration image is required it checks for the image from the task parameter and then for a calibration image of the proper type in the input list.

Having selected a calibration image it checks if it has been processed by looking for the image header flag CCDPROC. If it is not present then the calibration image is processed. When any image has been processed the CCDPROC flag is added. For images processed directly by ccdproc the individual processing flags are checked even if the CCDPROC flag is present. However, the automatic processing of the calibration images is only done if the CCDPROC flag is absent! This is to make the task more efficient by not having to check every flag for every calibration image for every input image. Thus, if additional processing steps are added after images have been partially reduced then input images will be processed for the new steps but calibration images will not be processed automatically.

After the calibration images have been identified, and processed if necessary, the images may be cached in memory. This is done when there are more than two input images (it is actually less efficient to cache the calibration images for one or two input images) and the parameter max_cache is greater than zero. When caching, as many calibration images as allowed by the specified memory are read into memory and kept there for all the input images. Cached images are, therefore, only read once from disk which reduces the amount of disk I/O. This makes a modest decrease in the execution time. It is not dramatic because the actual processing is fairly CPU intensive.

Once the processing parameters and calibration images have been determined the input image is processed for all the desired operations in one step; i.e. there are no intermediate results or images. This makes the task efficient. The corrected image is output as a temporary image until the entire image has been processed. When the image has been completely processed then the original image is deleted (or renamed using the specified backup prefix) and the corrected image replaces the original image. Using a temporary image protects the data in the event of an abort or computer failure. Keeping the original image name eliminates much of the record keeping and the need to generate new image names. .sh 1. Fixpix Regions of bad lines and columns may be replaced by linear interpolation from neighboring lines and columns when the parameter fixpix is set. This algorithm is the same as used in the task fixpix. The bad regions are specified in a bad pixel file. The file consists of lines with four fields, the starting and ending columns and the starting and ending lines. Any number of regions may be specified. Comment lines beginning with the character '#' may be included. If a comment line preceding the bad regions contains the word "untrimmed" then the coordinate system refers to the original CCD format of the images; i.e. before trimming. Note that if a subraster readout is used the coordinates MUST refer to the original CCD coordinates when the "untrimmed" mode is used. If an image has been trimmed previously then the trim region specified in the image header is used to convert the coordinates in the bad pixel file to those of the trimmed image. If the file does not contain the word "untrimmed" then the coordinate system must match that of the image being corrected; i.e. untrimmed coordinates if the image has not been trimmed and trimmed coordinates if the image has been trimmed. Standard bad pixel files should always be specified in terms of the original CCD detector coordinates.

The bad pixel file may be specified explicitly with the parameter fixfile or indirectly if the parameter has the value "image". In the latter case the instrument file must contain the name of the file. .sh 2. Overscan If an overscan or prescan correction is specified (overscan parameter) then the image section (biassec parameter) is averaged along the readout axis (readaxis parameter) to form a correction vector. A function is fit to this vector and for each readout line (image line or column) the function value for that line is subtracted from the image line. The fitting function is generally either a constant (polynomial of 1 term) or a high order function which fits the large scale shape of the overscan vector. Bad pixel rejection is also used to eliminate cosmic ray events. The function fitting may be done interactively using the standard icfit iteractive graphical curve fitting tool. Regardless of whether the fit is done interactively, the overscan vector and the fit may be recorded for later review in a metacode plot file named by the parameter ccdred.plotfile. The mean value of the bias function is also recorded in the image header and log file. .sh 3. Trim When the parameter trim is set the input image will be trimmed to the image section given by the parameter trimsec. This trim should, of course, be the same as that used for the calibration images. .sh 4. Zerocor After the readout bias is subtracted, as defined by the overscan or prescan region, there may still be a zero level bias. This level may be two dimensional or one dimensional (the same for every readout line). A zero level calibration is obtained by taking zero length exposures; generally many are taken and combined. To apply this zero level calibration the parameter zerocor is set. In addition if the zero level bias is only readout dependent then the parameter readcor is set to reduce two dimensional zero level images to one dimensional images. The zero level images may be specified by the parameter zero or given in the input image list (provided the CCD image type is defined).

When the zero level image is needed to correct an input image it is checked to see if it has been processed and, if not, it is processed automatically. Processing of zero level images consists of bad pixel replacement, overscan correction, trimming, and averaging to one dimension if the readout correction is specified. .sh 5. Darkcor Dark counts are subtracted by scaling a dark count calibration image to the same exposure time as the input image and subtracting. The exposure time used is the dark time which may be different than the actual integration or exposure time. A dark count calibration image is obtained by taking a very long exposure with the shutter closed; i.e. an exposure with no light reaching the detector. The dark count correction is selected with the parameter darkcor and the dark count calibration image is specified either with the parameter dark or as one of the input images. The dark count image is automatically processed as needed. Processing of dark count images consists of bad pixel replacement, overscan and zero level correction, and trimming. .sh 6. Flatcor The relative detector pixel response is calibrated by dividing by a scaled flat field calibration image. A flat field image is obtained by exposure to a spatially uniform source of light such as an lamp or twilight sky. Flat field images may be corrected for the spectral signature in spectroscopic images (see response and apnormalize), or for illumination effects (see mkillumflat or mkskyflat). For more on flat fields and illumination corrections see flatfields. The flat field response is dependent on the wavelength of light so if different filters or spectroscopic wavelength coverage are used a flat field calibration for each one is required. The different flat fields are automatically selected by a subset parameter (see subsets).

Flat field calibration is selected with the parameter flatcor and the flat field images are specified with the parameter flat or as part of the input image list. The appropriate subset is automatically selected for each input image processed. The flat field image is automatically processed as needed. Processing consists of bad pixel replacement, overscan subtraction, zero level subtraction, dark count subtraction, and trimming. Also if a scan mode is used and the parameter scancor is specified then a scan mode correction is applied (see below). The processing also computes the mean of the flat field image which is used later to scale the flat field before division into the input image. For scan mode flat fields the ramp part is included in computing the mean which will affect the level of images processed with this flat field. Note that there is no check for division by zero in the interest of efficiency. If division by zero does occur a fatal error will occur. The flat field can be fixed by replacing small values using a task such as imreplace or during processing using the minreplace parameter. Note that the minreplace parameter only applies to flat fields processed by ccdproc. .sh 7. Illumcor CCD images processed through the flat field calibration may not be completely flat (in the absence of objects). In particular, a blank sky image may still show gradients. This residual nonflatness is called the illumination pattern. It may be introduced even if the detector is uniformly illuminated by the sky because the flat field lamp illumination may be nonuniform. The illumination pattern is found from a blank sky, or even object image, by heavily smoothing and rejecting objects using sigma clipping. The illumination calibration image is divided into the data being processed to remove the illumination pattern. The illumination pattern is a function of the subset so there must be an illumination correction image for each subset to be processed. The tasks mkillumcor and mkskycor are used to create the illumination correction images. For more on illumination corrections see flatfields.

An alternative to treating the illumination correction as a separate operation is to combine the flat field and illumination correction into a corrected flat field image before processing the object images. This will save some processing time but does require creating the flat field first rather than correcting the images at the same time or later. There are two methods, removing the large scale shape of the flat field and combining a blank sky image illumination with the flat field. These methods are discussed further in the tasks which create them; mkillumcor and mkskycor. .sh 8. Fringecor There may be a fringe pattern in the images due to the night sky lines. To remove this fringe pattern a blank sky image is heavily smoothed to produce an illumination image which is then subtracted from the original sky image. The residual fringe pattern is scaled to the exposure time of the image to be fringe corrected and then subtracted. Because the intensity of the night sky lines varies with time an additional scaling factor may be given in the image header. The fringe pattern is a function of the subset so there must be a fringe correction image for each subset to be processed. The task mkfringecor is used to create the fringe correction images. .sh 9. Readcor If a zero level correction is desired (zerocor parameter) and the parameter readcor is yes then a single zero level correction vector is applied to each readout line or column. Use of a readout correction rather than a two dimensional zero level image depends on the nature of the detector or if the CCD is operated in longscan mode (see below). The readout correction is specified by a one dimensional image (zero parameter) and the readout axis (readaxis parameter). If the zero level image is two dimensional then it is automatically processed to a one dimensional image by averaging across the readout axis. Note that this modifies the zero level calibration image. .sh 10. Scancor CCD detectors may be operated in several modes in astronomical applications. The most common is as a direct imager where each pixel integrates one point in the sky or spectrum. However, the design of most CCD's allows the sky to be scanned across the CCD while shifting the accumulating signal at the same rate. Ccdproc provides for two scanning modes called "shortscan" and "longscan". The type of scan mode is set with the parameter scanmode.

In "shortscan" mode the detector is scanned over a specified number of lines (not necessarily at sideral rates). The lines that scroll off the detector during the integration are thrown away. At the end of the integration the detector is read out in the same way as an unscanned observation. The advantage of this mode is that the small scale, zero level, dark count and flat field responses are averaged in one dimension over the number of lines scanned. A zero level, dark count or flat field may be observed in the same way in which case there is no difference in the processing from unscanned imaging and the parameter scancor may be no. If it is yes, though, checking is done to insure that the calibration image used has the same number of scan lines as the object being processed. However, one obtains an increase in the statistical accuracy of if they are not scanned during the observation but digitally scanned during the processing. In shortscan mode with scancor set to yes, zero level, dark count and flat field images are digitally scanned, if needed, by the same number of scan lines as the object. The number of scan lines is determined from the object image header using the keyword nscanrow (or it's translation). If not found the object is assumed to have been scanned with the value given by the nscan parameter. Zero, dark and flat calibration images are assumed to be unscanned if the header keyword is not found.

If a scanned zero level, dark count or flat field image is not found matching the object then one may be created from the unscanned calibration image. The image will have the root name of the unscanned image with an extension of the number of scan rows; i.e. Flat1.32 is created from Flat1 with a digital scanning of 32 lines.

In "longscan" mode the detector is continuously read out to produce an arbitrarily long strip. Provided data which has not passed over the entire detector is thrown away, the zero level, dark count, and flat field corrections will be one dimensional. If scancor is specified and the scan mode is "longscan" then a one dimensional zero level, dark count, and flat field correction will be applied. .sh 11. Processing Steps The following describes the steps taken by the task. This detailed outline provides the most detailed specification of the task.

5 (1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

(10)

(11)

(12)

(13)

(14)

(15)

(16)

.sh 12. Processing Arithmetic The ccdproc task has two data paths, one for real image pixel datatypes and one for short integer pixel datatype. In addition internal arithmetic is based on the rules of FORTRAN. For efficiency there is no checking for division by zero in the flat field calibration. The following rules describe the processing arithmetic and data paths.

(1)

(2)

(3)

.sh 13. In the Absence of Image Header Information The tasks in the ccdred package are most convenient to use when the CCD image type, subset, and exposure time are contained in the image header. The ability to redefine which header parameters contain this information makes it possible to use the package at many different observatories (see instruments). However, in the absence of any image header information the tasks may still be used effectively. There are two ways to proceed. One way is to use ccdhedit to place the information in the image header.

The second way is to specify the processing operations more explicitly than is needed when the header information is present. The parameter ccdtype is set to "" or to "none". The calibration images are specified explicitly by task parameter since they cannot be recognized in the input list. Only one subset at a time may be processed.

If dark count and fringe corrections are to be applied the exposure times must be added to all the images. Alternatively, the dark count and fringe images may be scaled explicitly for each input image. This works because the exposure times default to 1 if they are not given in the image header.

The user's guide presents a tutorial in the use of this task.

1. In general all that needs to be done is to set the task parameters and enter

cl> ccdproc *.imh &

This will run in the background and process all images which have not been processed previously.

o SUN-3, 15 MHz 68020 with 68881 floating point hardware (no FPA)
o 8 Mb RAM, 2 Fuji Eagle disks.
o Input images = 544 x 512 short
o Output image = 500 x 500 real
o Operations are overscan subtraction (O), trimming to 500x500 (T),
  zero level subtraction (Z), dark count scaling and subtraction (D),
  and flat field scaling and subtraction (F).
o UNIX statistics
  (user, system, and clock time, and misc. memory and i/o statistics):
[OTF] One calibration image and 9 object images:
No caching:  110.6u 25.5s 3:18 68% 28+ 40K 3093+1645io   9pf+0w
Caching:     111.2u 23.0s 2:59 74% 28+105K 2043+1618io   9pf+0w
[OTZF] Two calibration images and 9 object images:
No caching:  119.2u 29.0s 3:45 65% 28+ 50K 4310+1660io   9pf+0w
Caching:     119.3u 23.0s 3:07 75% 28+124K 2179+1601io   9pf+0w
[OTZDF] Three calibration images and 9 object images:
No caching:  149.4u 31.6s 4:41 64% 28+ 59K 5501+1680io  19pf+0w
Caching:     151.5u 29.0s 4:14 70% 27+227K 2346+1637io 148pf+0w
[OTZF] 2 calibration images and 20 images processed:
No caching:  272.7u 63.8u 8:47 63% 28+ 50K 9598+3713io  12pf+0w
Caching:     271.2u 50.9s 7:00 76% 28+173K 4487+3613io  51pf+0w

REVISIONS

CCDPROC: V2.10.3

For short scan data the task now looks for the number of scan lines in the image header. Also when a calibration image is software scanned a new image is created. This allows processing objects with different numbers of scan lines and preserving the unscanned calibration image.

It is an error if no biassec is specified rather than defaulting to the whole image.

The time, in a internal format, when the CCDMEAN value is calculated is stored in the CCDMEANT keyword. The time is checked against the image modify time to determine if the value is valid or needs to be recomputed.

instruments, 
ccdtypes, 
flatfields, 
icfit, 
ccdred, 
guide, 
mkillumcor, 
, 
mkskycor, 
mkfringecor, 

This page automatically generated from the iraf .hlp file. If you 
would like your local iraf package .hlp files converted into HTML 
please contact Dave Mills at NOAO.
dmills@noao.edu