6. INITIALISE
In this section you select the instrument, make some general adjustments, and lastly tell THELI about the whereabouts of the data.
6.1. Select instrument
At the right hand side you will find the instrument list, which is divided into 3 parts: a series of pre-defined cameras at various observatories, a list of commercially available cameras which you can use as a reference when defining your own setup with one of those cameras, and lastly the list of user-defined instruments.
6.2. General settings
6.2.1. The Project name
All settings, parameter choices and tasks executed are stored in a logfile (hereafter: LOG). This is the memory of THELI. Whenever you close THELI and then open it again, it will read the last state from the LOG and open again in the exactly the same state you left it. Therefore, it is easy resume data reduction at a later time. You can even switch between different data reduction tasks without confusion. Therefore it is important that you give your project a meaningful name, for example NGC1234_SuprimeCam_rband. This allows you to recognise the target, the instrument and e.g. possibly the filter or another observational detail you deem important. The LOGs are not overwritten or erased if you install a new GUI version, as they are kept in a separate place.
Starting with a new Project
Just enter a new name for the project and then click on Reset to initialise
it to default values. If you do not do the reset, all parameter values
from the previous run still present in the GUI will be copied, which is
what you want if you reduce similar data sets.
The Reset button is also useful in case you get totally lost
with the various parameter settings. It will re-initialise all
parameters in the GUI to their default values, which should work
for most cases.
Other settings
- Data diskspace warning: If the available space on the hard disk containing the data drops below this value, a warning message will be shown. The data disk can be identical to the home disk containing your /home directory.
- Home diskspace warning: THELI writes some temporary data onto your home disk, which might fill up quickly in particular if your SysOp imposed disk quotas for users. See the whereabouts for details.
- Number of CPUs: see parallelisation
- GUI style: Allows you to change the look and feel of the GUI depending on the styles your desktop allows.
NFRAMES: This is the number of chips that can be processed simultaneously in memory without causing swapping. This number is automatically and conservatively estimated based on the physically available RAM, the number of CPUs selected, and the image size. Should your computer become sticky, reduce this parameter, or stop other memory-intense applications.
You cannot select a value larger than the one determined automatically.
6.2.2. Parallelisation
THELI offers a convenient parallelisation scheme for multi-core CPUs: simply choose the number of CPUs you want. It can be changed any time while THELI is not reducing data. Each CPU will be assigned one or several chips. For example, if your camera has 16 detectors and you work with 4 cores, the detectors will be distributed by THELI’s parallel manager in the following way:
| #CPU | works on chips |
|---|---|
| 1 | 1 5 9 13 |
| 2 | 2 6 10 14 |
| 3 | 3 7 11 15 |
| 4 | 4 8 12 16 |
Ideally, the number of detectors should be an integer multiple of the number of chips. In the previous example there would be no gain in speed if you used 5 CPUs instead of 4, as 1 out of the 5 CPUs would still have to process data from 4 chips. The other CPUs will finish sooner, but you have to wait until CPU #1 has finished too:
| #CPU | works on chips |
|---|---|
| 1 | 1 6 11 16 |
| 2 | 2 7 12 |
| 3 | 3 8 13 |
| 4 | 4 9 14 |
| 5 | 5 10 15 |
If you reduce single-chip data, you can still select more than one CPU. You will in general not gain speed as the additional CPUs will simply be idle. The only three exceptions are astrometry using Scamp, sky subtraction, and the final coaddition step with Swarp, as these are multi-threaded.
Note
If all your data of a multi-chip camera is on the same disk, it does not make much sense to use more than 4 CPUs at the same time. The I/O load on that disk becomes very large when all 4 processes read and write to it, thus the gain in speed is negligible (alternative: linked file systems)
Exception (1): You run a suitable RAID configuration that allows for high performance I/O, and/or have a fast SSD.
Exception (2): Processes that have low I/O rates or high CPU demand can efficiently make use of all CPUs available. These are:
- Catalog creation
- Astrometry (scamp)
- Sky subtraction
- Coaddition
6.3. Data directories
It is the user’s responsibility to sort data into calibration files and target observations. THELI will not check if e.g. a flat field is actually a flat field.
THELI does not rely on the information given in the FITS headers of raw data. Even though FITS is supposed to be a standard, not everyone sticks to it. In many cases FITS headers are essentially incomplete, keywords contradict each other, or information in them is wrong. Therefore you have to manually sort your exposures into a directory tree which you then declare in THELI. If you KNOW that your headers are reliable, THELI offers you a little support in the preparation section.
6.3.1. Directory tree
This is where you tell THELI where the data are. THELI requires all data to lie in some main directory, being distributed in individual sub-directories for biases, darks, flats and the exposures of your target (hereafter: science). Standard star exposures and observations of a blank sky field must be copied to separate sub-directories as well.
Example 1: You observed NGC 1234 in a single filter, then your data directory tree should look like:
/thats/the/mainpath/
/thats/the/mainpath/BIAS
/thats/the/mainpath/FLAT
/thats/the/mainpath/SCIENCE
and you would enter in THELI:
Main path: /thats/the/mainpath
Bias: BIAS
Flat: FLAT
Science: SCIENCE
Example 2: You observed NGC 1234 and a standard star in filters B, V and R:
/thats/the/mainpath/
/thats/the/mainpath/BIAS
/thats/the/mainpath/FLAT_B
/thats/the/mainpath/FLAT_V
/thats/the/mainpath/FLAT_R
/thats/the/mainpath/NGC1234_B
/thats/the/mainpath/NGC1234_V
/thats/the/mainpath/NGC1234_R
/thats/the/mainpath/STD_B
/thats/the/mainpath/STD_V
/thats/the/mainpath/STD_R
In THELI you would start with
Main path: /thats/the/mainpath
Bias: BIAS
Flat: FLAT_B
Science: NGC1234_B
Standard: STD_B
Once finished, switch to the next filter. See the Section about multi-color data processing for more details.
Warning
When clicking upon the Restore ORIG button, all data in the corresponding sub-directory will be deleted, and the original raw images will be restored.
6.3.2. Fully process SKY data
In case of extended targets one often needs observations of a blank sky field (hereafter: SKY) to perform proper background correction. THELI will process the SKY data automatically when present up to the point when it is no longer needed, i.e. once the correction images calculated from it were applied.
However, in particular in the near-IR the total exposure time for the SKY field is often equal to that of the primary target. In this case it makes sense to have THELI continue processing the SKY up to and including the coaddition. The extra field may either double your field of view, provide some important calibration statistics, or just be useful for one of your colleagues. If you want this full scale processing, then activate this setting.