Inheritance diagram for lsst.pipe.tasks.propagateVisitFlags.PropagateVisitFlagsTask:

Public Member Functions
	__init__ (self, schema, **kwargs)

	run (self, butler, coaddSources, ccdInputs, coaddWcs, visitCatalogs=None, wcsUpdates=None)

Static Public Member Functions
	getCcdInputs (coaddExposure)

Public Attributes
	schema

Static Public Attributes
	ConfigClass = PropagateVisitFlagsConfig

Protected Attributes
	_keys

Detailed Description

Task to propagate flags from single-frame measurements to coadd measurements.

Parameters
----------
schema : `lsst.afw.table.Schema`
The input schema for the reference source catalog, used to initialize
the output schema.
**kwargs
Additional keyword arguments.

Notes
-----
We want to be able to set a flag for sources on the coadds using flags
that were determined from the individual visits. A common example is sources
that were used for PSF determination, since we do not do any PSF determination
on the coadd but use the individual visits. This requires matching the coadd
source catalog to each of the catalogs from the inputs (see
PropagateVisitFlagsConfig.matchRadius), and thresholding on the number of
times a source is flagged on the input catalog.

An important consideration in this is that the flagging of sources in the
individual visits can be somewhat stochastic, e.g., the same stars may not
always be used for PSF determination because the field of view moves slightly
between visits, or the seeing changed. We there threshold on the relative
occurrence of the flag in the visits (see PropagateVisitFlagsConfig.flags).
Flagging a source that is always flagged in inputs corresponds to a threshold
of 1, while flagging a source that is flagged in any of the input corresponds
to a threshold of 0. But neither of these extrema are really useful in
practise.

Setting the threshold too high means that sources that are not consistently
flagged (e.g., due to chip gaps) will not have the flag propagated. Setting
that threshold too low means that random sources which are falsely flagged in
the inputs will start to dominate. If in doubt, we suggest making this
threshold relatively low, but not zero (e.g., 0.1 to 0.2 or so). The more
confidence in the quality of the flagging, the lower the threshold can be.
The relative occurrence accounts for the edge of the field-of-view of the
camera, but does not include chip gaps, bad or saturated pixels, etc.

Initialization

Beyond the usual Task initialization, PropagateVisitFlagsTask also requires
a schema for the catalog that is being constructed.

The 'run' method (described below) is the entry-point for operations. The
'getCcdInputs' staticmethod is provided as a convenience for retrieving the
'ccdInputs' (CCD inputs table) from an Exposure.

.. code-block :: none

# Requires:
# * butler: data butler, for retrieving the CCD catalogs
# * coaddCatalog: catalog of source measurements on the coadd (lsst.afw.table.SourceCatalog)
# * coaddExposure: coadd (lsst.afw.image.Exposure)
from lsst.pipe.tasks.propagateVisitFlags import PropagateVisitFlagsTask, PropagateVisitFlagsConfig
config = PropagateVisitFlagsConfig()
config.flags["calib_psf_used"] = 0.3 # Relative threshold for this flag
config.matchRadius = 0.5 # Matching radius in arcsec
task = PropagateVisitFlagsTask(coaddCatalog.schema, config=config)
ccdInputs = task.getCcdInputs(coaddExposure)
task.run(butler, coaddCatalog, ccdInputs, coaddExposure.getWcs())

Definition at line 48 of file propagateVisitFlags.py.

Constructor & Destructor Documentation

◆ init()

lsst.pipe.tasks.propagateVisitFlags.PropagateVisitFlagsTask.__init__	(		self,
			schema,
		**	kwargs )

Definition at line 114 of file propagateVisitFlags.py.

    def __init__(self, schema, **kwargs):
        Task.__init__(self, **kwargs)
        self.schema = schema
        self._keys = dict((f, self.schema.addField(f, type="Flag", doc="Propagated from visits")) for
                          f in self.config.flags)
 

Member Function Documentation

◆ getCcdInputs()

lsst.pipe.tasks.propagateVisitFlags.PropagateVisitFlagsTask.getCcdInputs ( coaddExposure )

static

Convenience method to retrieve the CCD inputs table from a coadd exposure.

Parameters
----------
coaddExposure : `lsst.afw.image.Exposure`
    The exposure we need to retrieve the CCD inputs table from.

Returns
-------
ccdInputs : ``
    CCD inputs table from a coadd exposure.

Definition at line 121 of file propagateVisitFlags.py.

    def getCcdInputs(coaddExposure):
        """Convenience method to retrieve the CCD inputs table from a coadd exposure.
 
        Parameters
        ----------
        coaddExposure : `lsst.afw.image.Exposure`
            The exposure we need to retrieve the CCD inputs table from.
 
        Returns
        -------
        ccdInputs : ``
            CCD inputs table from a coadd exposure.
        """
        return coaddExposure.getInfo().getCoaddInputs().ccds
 

◆ run()

lsst.pipe.tasks.propagateVisitFlags.PropagateVisitFlagsTask.run	(	self,
		butler,
		coaddSources,
		ccdInputs,
		coaddWcs,
		visitCatalogs = None,
		wcsUpdates = None )

Propagate flags from individual visit measurements to coadd.

This requires matching the coadd source catalog to each of the catalogs
from the inputs, and thresholding on the number of times a source is
flagged on the input catalog.  The threshold is made on the relative
occurrence of the flag in each source.  Flagging a source that is always
flagged in inputs corresponds to a threshold of 1, while flagging a
source that is flagged in any of the input corresponds to a threshold of
0.  But neither of these extrema are really useful in practise.

Setting the threshold too high means that sources that are not consistently
flagged (e.g., due to chip gaps) will not have the flag propagated.  Setting
that threshold too low means that random sources which are falsely flagged in
the inputs will start to dominate.  If in doubt, we suggest making this threshold
relatively low, but not zero (e.g., 0.1 to 0.2 or so).  The more confidence in
the quality of the flagging, the lower the threshold can be.

The relative occurrence accounts for the edge of the field-of-view of
the camera, but does not include chip gaps, bad or saturated pixels, etc.

Parameters
----------
butler : `Unknown`
    Data butler, for retrieving the input source catalogs.
coaddSources : `lsst.afw.image.SourceCatalog`
    Source catalog from the coadd.
ccdInputs : `lsst.afw.table.ExposureCatalog`
    Table of CCDs that contribute to the coadd.
coaddWcs : `lsst.afw.geom.SkyWcs`
    Wcs for coadd.
visitCatalogs : `list` of `lsst.afw.image.SourceCatalog`, optional
    List of loaded source catalogs for each input ccd in
    the coadd. If provided this is used instead of this
    method loading in the catalogs itself.
wcsUpdates : `list` of `lsst.afw.geom.SkyWcs`, optional
    If visitCatalogs is a list of ccd catalogs, this
    should be a list of updated wcs to apply.

Raises
------
ValueError
    Raised if any of the following occur:
    - A list of wcs updates for each catalog is not supplied in the wcsUpdates parameter
    and ccdInputs is a list of src catalogs.
    - The visitCatalogs and ccdInput parameters are both `None`.

Definition at line 136 of file propagateVisitFlags.py.

    def run(self, butler, coaddSources, ccdInputs, coaddWcs, visitCatalogs=None, wcsUpdates=None):
        """Propagate flags from individual visit measurements to coadd.
 
        This requires matching the coadd source catalog to each of the catalogs
        from the inputs, and thresholding on the number of times a source is
        flagged on the input catalog.  The threshold is made on the relative
        occurrence of the flag in each source.  Flagging a source that is always
        flagged in inputs corresponds to a threshold of 1, while flagging a
        source that is flagged in any of the input corresponds to a threshold of
        0.  But neither of these extrema are really useful in practise.
 
        Setting the threshold too high means that sources that are not consistently
        flagged (e.g., due to chip gaps) will not have the flag propagated.  Setting
        that threshold too low means that random sources which are falsely flagged in
        the inputs will start to dominate.  If in doubt, we suggest making this threshold
        relatively low, but not zero (e.g., 0.1 to 0.2 or so).  The more confidence in
        the quality of the flagging, the lower the threshold can be.
 
        The relative occurrence accounts for the edge of the field-of-view of
        the camera, but does not include chip gaps, bad or saturated pixels, etc.
 
        Parameters
        ----------
        butler : `Unknown`
            Data butler, for retrieving the input source catalogs.
        coaddSources : `lsst.afw.image.SourceCatalog`
            Source catalog from the coadd.
        ccdInputs : `lsst.afw.table.ExposureCatalog`
            Table of CCDs that contribute to the coadd.
        coaddWcs : `lsst.afw.geom.SkyWcs`
            Wcs for coadd.
        visitCatalogs : `list` of `lsst.afw.image.SourceCatalog`, optional
            List of loaded source catalogs for each input ccd in
            the coadd. If provided this is used instead of this
            method loading in the catalogs itself.
        wcsUpdates : `list` of `lsst.afw.geom.SkyWcs`, optional
            If visitCatalogs is a list of ccd catalogs, this
            should be a list of updated wcs to apply.
 
        Raises
        ------
        ValueError
            Raised if any of the following occur:
            - A list of wcs updates for each catalog is not supplied in the wcsUpdates parameter
            and ccdInputs is a list of src catalogs.
            - The visitCatalogs and ccdInput parameters are both `None`.
        """
        if len(self.config.flags) == 0:
            return
 
        flags = self._keys.keys()
        counts = dict((f, numpy.zeros(len(coaddSources), dtype=int)) for f in flags)
        indices = numpy.array([s.getId() for s in coaddSources])  # Allowing for non-contiguous data
        radius = self.config.matchRadius*geom.arcseconds
 
        def processCcd(ccdSources, wcsUpdate):
            for sourceRecord in ccdSources:
                sourceRecord.updateCoord(wcsUpdate)
            for flag in flags:
                # We assume that the flags will be relatively rare, so it is more efficient to match
                # against a subset of the input catalog for each flag than it is to match once against
                # the entire catalog.  It would be best to have built a kd-tree on coaddSources and
                # keep reusing that for the matching, but we don't have a suitable implementation.
                mc = afwTable.MatchControl()
                mc.findOnlyClosest = False
                matches = afwTable.matchRaDec(coaddSources, ccdSources[ccdSources.get(flag)], radius, mc)
                for m in matches:
                    index = (numpy.where(indices == m.first.getId()))[0][0]
                    counts[flag][index] += 1
 
        if visitCatalogs is not None:
            if wcsUpdates is None:
                raise pexExceptions.ValueError("If ccdInputs is a list of src catalogs, a list of wcs"
                                               " updates for each catalog must be supplied in the "
                                               "wcsUpdates parameter")
            for i, ccdSource in enumerate(visitCatalogs):
                processCcd(ccdSource, wcsUpdates[i])
        else:
            if ccdInputs is None:
                raise pexExceptions.ValueError("The visitCatalogs and ccdInput parameters can't both be None")
            visitKey = ccdInputs.schema.find("visit").key
            ccdKey = ccdInputs.schema.find("ccd").key
 
            self.log.info("Propagating flags %s from inputs", flags)
 
            # Accumulate counts of flags being set
            for ccdRecord in ccdInputs:
                v = ccdRecord.get(visitKey)
                c = ccdRecord.get(ccdKey)
                dataId = {"visit": int(v), self.config.ccdName: int(c)}
                ccdSources = butler.get("src", dataId=dataId)
                processCcd(ccdSources, ccdRecord.getWcs())
 
        # Apply threshold
        for f in flags:
            key = self._keys[f]
            for s, num in zip(coaddSources, counts[f]):
                numOverlaps = len(ccdInputs.subsetContaining(s.getCentroid(), coaddWcs, True))
                s.setFlag(key, bool(num > numOverlaps*self.config.flags[f]))
            self.log.info("Propagated %d sources with flag %s", sum(s.get(key) for s in coaddSources), f)

Member Data Documentation

◆ _keys

lsst.pipe.tasks.propagateVisitFlags.PropagateVisitFlagsTask._keys

protected

Definition at line 117 of file propagateVisitFlags.py.

◆ ConfigClass

lsst.pipe.tasks.propagateVisitFlags.PropagateVisitFlagsTask.ConfigClass = PropagateVisitFlagsConfig

static

Definition at line 112 of file propagateVisitFlags.py.

◆ schema

lsst.pipe.tasks.propagateVisitFlags.PropagateVisitFlagsTask.schema

Definition at line 116 of file propagateVisitFlags.py.

The documentation for this class was generated from the following file:

/j/snowflake/release/lsstsw/stack/lsst-scipipe-8.0.0/Linux64/pipe_tasks/g06b2ea86fd+734f9505a2/python/lsst/pipe/tasks/propagateVisitFlags.py

Public Member Functions

Static Public Member Functions

Public Attributes

Static Public Attributes

Protected Attributes