Calibrate an exposure: measure PSF, subtract background, measure astrometry and photometry. More...

Inheritance diagram for lsst.pipe.tasks.calibrate.CalibrateTask:

Public Member Functions
def	__init__
	Create the calibration task. More...

def	getCalibKeys
	Return a sequence of schema keys that represent fields that should be propagated from icSrc to src by ProcessCcdTask. More...

def	run
	Run the calibration task on an exposure. More...

def	installInitialPsf
	Initialise the calibration procedure by setting the PSF to a configuration-defined guess. More...

Public Attributes
	schema1

	algMetadata

	schemaMapper

	schema

Static Public Attributes
	ConfigClass = CalibrateConfig

Static Private Attributes
string	_DefaultName = "calibrate"

Detailed Description

Calibrate an exposure: measure PSF, subtract background, measure astrometry and photometry.

Description

CalibrateTask

Calculate an Exposure's zero-point given a set of flux measurements of stars matched to an input catalogue. The type of flux to use is specified by CalibrateConfig.fluxField.

The algorithm clips outliers iteratively, with parameters set in the configuration.

Note: This task can adds fields to the schema, so any code calling this task must ensure that these columns are indeed present in the input match list; see A complete example of using CalibrateTask

Task initialisation

Create the calibration task.

Parameters

**kwargs keyword arguments to be passed to lsst.pipe.base.task.Task.__init__

CalibrateTask delegates most of its work to a set of sub-Tasks:

repair RepairTask: Interpolate over defects such as bad columns and cosmic rays. This task is called twice; once before the measurePsf step and again after the PSF has been measured.
detection SourceDetectionTask: Initial (high-threshold) detection phase for calibration
initialMeasurement SingleFrameMeasurementTask: Make the initial measurements used to feed PSF determination and aperture correction determination
astrometry AstrometryTask: Solve the astrometry. May be disabled by setting CalibrateTaskConfig.doAstrometry to be False. This task is called twice; once before the measurePsf step and again after the PSF has been measured.
measurePsf MeasurePsfTask: Estimate the PSF. May be disabled by setting CalibrateTaskConfig.doPsf to be False. If requested the astrometry is solved before this is called, so if you disable the astrometry the measurePsf task won't have access to objects positions.
measurement SingleFrameMeasurementTask: Post-PSF-determination measurements used to feed other calibrations
photocal PhotoCalTask: Solve for the photometric zeropoint. May be disabled by setting CalibrateTaskConfig.doPhotoCal to be False. N.b. Requires that astrometry was successfully run.

You can replace any of these subtasks if you wish, see Using a Custom Astrometry Task.

Note: These task APIs are not well controlled, so replacing a task is a matter of matching a poorly specified interface. We will be working on this over the first year of construction.

Invoking the Task

Run the calibration task on an exposure.

Parameters

[in,out]	exposure	Exposure to calibrate; measured PSF will be installed there as well
[in]	defects	List of defects on exposure
[in]	idFactory	afw.table.IdFactory to use for source catalog.

Returns

a pipeBase.Struct with fields:

exposure: Repaired exposure
backgrounds: A list of background models applied in the calibration phase
psf: Point spread function
sources: Sources used in calibration
matches: A list of reference object/source matches (an lsst.afw.table.ReferenceMatchVector)
matchMeta: Metadata about the field (an lsst.daf.base.PropertyList)
photocal: Output of photocal subtask

It is moderately important to provide a decent initial guess for the seeing if you want to deal with cosmic rays. If there's a PSF in the exposure it'll be used; failing that the CalibrateConfig.initialPsf is consulted (although the pixel scale will be taken from the WCS if available).

If the exposure contains an lsst.afw.image.Calib object with the exposure time set, MAGZERO will be set in the task metadata.

Configuration parameters

See CalibrateConfig

#-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Quantities set in Metadata

Task metadata

MAGZERO: Measured zeropoint (DN per second)

Exposure metadata

MAGZERO_RMS: MAGZERO's RMS == return.sigma
MAGZERO_NOBJ: Number of stars used == return.ngood
COLORTERM1: ?? (always 0.0)
COLORTERM2: ?? (always 0.0)
COLORTERM3: ?? (always 0.0)

#-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Debug variables

The command line task interface supports a flag -d to import debug.py from your PYTHONPATH; see Using lsstDebug to control debugging output for more about debug.py files.

The calibrate task has a debug dictionary with keys which correspond to stages of the CalibrationTask processing:

repair

Fixed defects and masked cosmic rays with a guessed PSF. Action: show the exposure.

background

Subtracted background (no sources masked). Action: show the exposure

PSF_repair

Fixed defects and removed cosmic rays with an estimated PSF. Action: show the exposure

PSF_background

Subtracted background (calibration sources masked). Action: show the exposure

calibrate

Just before astro/photo calibration. Action: show the exposure, and

sources as smallish green o
matches (if exposure has a Wcs).
- catalog position as a largish yellow +
- source position as a largish red x

The values are the ds9 frame to display in (if >= 1); if <= 0, nothing's displayed. There's an example here.

Some subtasks may also have their own debug information; see individual Task documentation.

A complete example of using CalibrateTask

This code is in calibrateTask.py in the examples directory, and can be run as e.g.

1 examples/calibrateTask.py --ds9

Import the task (there are some other standard imports; read the file if you're curious)

1 from lsst.pipe.tasks.calibrate import CalibrateTask

Create the detection task

     config = CalibrateTask.ConfigClass()
     config.initialPsf.pixelScale = 0.185 # arcsec per pixel
     config.initialPsf.fwhm = 1.0
     config.astrometry.retarget(MyAstrometryTask)
     calibrateTask = CalibrateTask(config=config)

Note that we're using a custom AstrometryTask (because we don't have a valid astrometric catalogue handy); see Using a Custom Astrometry Task.

We're now ready to process the data (we could loop over multiple exposures/catalogues using the same task objects) and unpack the results

     exposure = loadData(config.initialPsf.pixelScale)
     result = calibrateTask.run(exposure)
 
     exposure0, exposure = exposure, result.exposure
     sources = result.sources

We then might plot the results (e.g. if you set –ds9 on the command line)

     if display:                         # display on ds9 (see also --debug argparse option)
         frame = 1
         ds9.mtv(exposure, frame=frame)
 
         with ds9.Buffering():
             for s in sources:
                 xy = s.getCentroid()
                 ds9.dot('+', *xy, ctype=ds9.CYAN if s.get("flags.negative") else ds9.GREEN, frame=frame)

Using a Custom Astrometry Task

The first thing to do is define my own task:

     # and a trivial WCS (needed by MyAstrometryTask)
     pixelScale /= 3600.0                # degrees per pixel
     wcs = afwImage.makeWcs(afwCoord.Coord(afwGeom.PointD(15, 1)), afwGeom.PointD(0, 0),
                            pixelScale, 0.0, 0.0, pixelScale)
     exposure.setWcs(wcs)
 
     return exposure
 
 class MyAstrometryTask(AstrometryTask):
     """An override for CalibrateTask's astrometry task"""
     def __init__(self, *args, **kwargs):
         super(MyAstrometryTask, self).__init__(*args, **kwargs)

Then we need our own run method. First unpack the filtername and wcs

     def run(self, exposure, sourceCat):
         """My run method that totally fakes the astrometric solution"""
 
         filterName = exposure.getFilter().getName()
         wcs = exposure.getWcs()

Then build a "reference catalog" by shamelessly copying the catalog of detected sources

         schema = afwTable.SimpleTable.makeMinimalSchema()
         schema.addField(afwTable.Field[float](filterName, "Flux in appropriate band"))
         schema.addField(afwTable.Field[float]("flux", "Reference flux"))
         schema.addField(afwTable.Field[float]("photometric", "I am a reference star"))
         refCat = afwTable.SimpleCatalog(schema)
 
         for s in sourceCat:
             m = refCat.addNew()
             flux = 1e-3*s.getPsfFlux()*np.random.normal(1.0, 2e-2)
             m.set(filterName, flux)
             m.set("flux", flux)
             m.set("coord", wcs.pixelToSky(s.getCentroid()))
 
         refCat.get("photometric")[:] = True

(you need to set "flux" as well as filterName due to a bug in the photometric calibration code; DM-933).

Then "match" by zipping up the two catalogs,

         matches = []
         md = dafBase.PropertyList()
         for m, s in zip(refCat, sourceCat):
             matches.append(afwTable.ReferenceMatch(m, s, 0.0))

and finally return the desired results.

         return pipeBase.Struct(
             matches=matches,
             matchMeta=md
             )

To investigate the Debug variables, put something like

 import lsstDebug
 def DebugInfo(name):
     di = lsstDebug.getInfo(name)        # N.b. lsstDebug.Info(name) would call us recursively
     if name == "lsst.pipe.tasks.calibrate":
         di.display = dict(
             repair = 1,
             calibrate = 2,
         )
 
     return di
 
 lsstDebug.Info = DebugInfo

into your debug.py file and run calibrateTask.py with the –debug flag.

Definition at line 155 of file calibrate.py.

Constructor & Destructor Documentation

def lsst.pipe.tasks.calibrate.CalibrateTask.__init__	(	self,
		kwargs
	)

Create the calibration task.

Parameters

**kwargs keyword arguments to be passed to lsst.pipe.base.task.Task.__init__

Definition at line 347 of file calibrate.py.

 
     def __init__(self, **kwargs):
         """!
         Create the calibration task
 
         \param **kwargs keyword arguments to be passed to lsst.pipe.base.task.Task.__init__
         """
         pipeBase.Task.__init__(self, **kwargs)
 
         # the calibrate Source Catalog is divided into two catalogs to allow measurement to be run twice
         # schema1 contains everything except what is added by the second measurement task.
         # Before the second measurement task is run, self.schemaMapper transforms the sources into
         # the final output schema, at the same time renaming the measurement fields to "initial_" 
         self.schema1 = afwTable.SourceTable.makeMinimalSchema()
         self.algMetadata = dafBase.PropertyList()
         self.makeSubtask("repair")
         self.makeSubtask("detection", schema=self.schema1)
         beginInitial = self.schema1.getFieldCount()
         self.makeSubtask("initialMeasurement", schema=self.schema1, algMetadata=self.algMetadata)
         endInitial = self.schema1.getFieldCount()
 
         # create subtasks that are run with schema1 (and possibly also the final schema)
         self.makeSubtask("measurePsf", schema=self.schema1)
         self.makeSubtask("astrometry", schema=self.schema1)
 
         # create a schemaMapper to map schema1 into the final schema
         self.schemaMapper = afwTable.SchemaMapper(self.schema1)
         separator =  "_"
         count = 0
         for item in self.schema1:
             count = count + 1
             field = item.getField()
             name = field.getName()
             if count > beginInitial and count <= endInitial:
                 name = "initial" + separator + name
             self.schemaMapper.addMapping(item.key, name)
 
         # create subtasks that are run only with the final schema
         schema = self.schemaMapper.editOutputSchema()
         self.makeSubtask("measurement", schema=schema, algMetadata=self.algMetadata)
         self.makeSubtask("measureApCorr", schema=schema)
         self.makeSubtask("photocal", schema=schema)
 
         # the final schema is the same as the schemaMapper output
         self.schema = self.schemaMapper.getOutputSchema()