Inheritance diagram for lsst.pipe.tasks.imageDifference.ImageDifferenceTask:

Public Member Functions
def	__init__ (self, butler=None, kwargs)
	Construct an ImageDifference Task. More...

def	runDataRef (self, sensorRef, templateIdList=None)

def	run (self, exposure=None, selectSources=None, templateExposure=None, templateSources=None, idFactory=None, calexpBackgroundExposure=None, subtractedExposure=None)

def	fitAstrometry (self, templateSources, templateExposure, selectSources)

def	runDebug (self, exposure, subtractRes, selectSources, kernelSources, diaSources)

def	getSchemaCatalogs (self)

Static Public Member Functions
def	makeIdFactory (expId, expBits)

Public Attributes
	schema

	algMetadata

Static Public Attributes
	ConfigClass

	RunnerClass

Detailed Description

Subtract an image from a template and measure the result

Definition at line 253 of file imageDifference.py.

Constructor & Destructor Documentation

◆ init()

def lsst.pipe.tasks.imageDifference.ImageDifferenceTask.__init__	(	self,
		butler = `None`,
		kwargs
	)

Construct an ImageDifference Task.

Parameters

[in] butler Butler object to use in constructing reference object loaders

Definition at line 260 of file imageDifference.py.

     def __init__(self, butler=None, **kwargs):
         """!Construct an ImageDifference Task
 
         @param[in] butler  Butler object to use in constructing reference object loaders
         """
         pipeBase.CmdLineTask.__init__(self, **kwargs)
         self.makeSubtask("getTemplate")
 
         self.makeSubtask("subtract")
 
         if self.config.subtract.name == 'al' and self.config.doDecorrelation:
             self.makeSubtask("decorrelate")
 
         if self.config.doScaleTemplateVariance:
             self.makeSubtask("scaleVariance")
 
         if self.config.doUseRegister:
             self.makeSubtask("register")
         self.schema = afwTable.SourceTable.makeMinimalSchema()
 
         if self.config.doSelectSources:
             self.makeSubtask("sourceSelector")
             if self.config.kernelSourcesFromRef:
                 self.makeSubtask('refObjLoader', butler=butler)
                 self.makeSubtask("astrometer", refObjLoader=self.refObjLoader)
 
         self.algMetadata = dafBase.PropertyList()
         if self.config.doDetection:
             self.makeSubtask("detection", schema=self.schema)
         if self.config.doMeasurement:
             self.makeSubtask("measurement", schema=self.schema,
                              algMetadata=self.algMetadata)
         if self.config.doForcedMeasurement:
             self.schema.addField(
                 "ip_diffim_forced_PsfFlux_instFlux", "D",
                 "Forced PSF flux measured on the direct image.")
             self.schema.addField(
                 "ip_diffim_forced_PsfFlux_instFluxErr", "D",
                 "Forced PSF flux error measured on the direct image.")
             self.schema.addField(
                 "ip_diffim_forced_PsfFlux_area", "F",
                 "Forced PSF flux effective area of PSF.",
                 units="pixel")
             self.schema.addField(
                 "ip_diffim_forced_PsfFlux_flag", "Flag",
                 "Forced PSF flux general failure flag.")
             self.schema.addField(
                 "ip_diffim_forced_PsfFlux_flag_noGoodPixels", "Flag",
                 "Forced PSF flux not enough non-rejected pixels in data to attempt the fit.")
             self.schema.addField(
                 "ip_diffim_forced_PsfFlux_flag_edge", "Flag",
                 "Forced PSF flux object was too close to the edge of the image to use the full PSF model.")
             self.makeSubtask("forcedMeasurement", refSchema=self.schema)
         if self.config.doMatchSources:
             self.schema.addField("refMatchId", "L", "unique id of reference catalog match")
             self.schema.addField("srcMatchId", "L", "unique id of source match")
 

Member Function Documentation

◆ fitAstrometry()

def lsst.pipe.tasks.imageDifference.ImageDifferenceTask.fitAstrometry	(	self,
		templateSources,
		templateExposure,
		selectSources
	)

Fit the relative astrometry between templateSources and selectSources

Todo
----

Remove this method. It originally fit a new WCS to the template before calling register.run
because our TAN-SIP fitter behaved badly for points far from CRPIX, but that's been fixed.
It remains because a subtask overrides it.

Definition at line 885 of file imageDifference.py.

     def fitAstrometry(self, templateSources, templateExposure, selectSources):
         """Fit the relative astrometry between templateSources and selectSources
 
         Todo
         ----
 
         Remove this method. It originally fit a new WCS to the template before calling register.run
         because our TAN-SIP fitter behaved badly for points far from CRPIX, but that's been fixed.
         It remains because a subtask overrides it.
         """
         results = self.register.run(templateSources, templateExposure.getWcs(),
                                     templateExposure.getBBox(), selectSources)
         return results
 

◆ getSchemaCatalogs()

def lsst.pipe.tasks.imageDifference.ImageDifferenceTask.getSchemaCatalogs ( self )

Return a dict of empty catalogs for each catalog dataset produced by this task.

Definition at line 987 of file imageDifference.py.

     def getSchemaCatalogs(self):
         """Return a dict of empty catalogs for each catalog dataset produced by this task."""
         diaSrc = afwTable.SourceCatalog(self.schema)
         diaSrc.getTable().setMetadata(self.algMetadata)
         return {self.config.coaddName + "Diff_diaSrc": diaSrc}
 

◆ makeIdFactory()

def lsst.pipe.tasks.imageDifference.ImageDifferenceTask.makeIdFactory	(	expId,
		expBits
	)

static

Create IdFactory instance for unique 64 bit diaSource id-s.

Parameters
----------
expId : `int`
    Exposure id.

expBits: `int`
    Number of used bits in ``expId``.

Note
----
The diasource id-s consists of the ``expId`` stored fixed in the highest value
``expBits`` of the 64-bit integer plus (bitwise or) a generated sequence number in the
low value end of the integer.

Returns
-------
idFactory: `lsst.afw.table.IdFactory`

Definition at line 318 of file imageDifference.py.

     def makeIdFactory(expId, expBits):
         """Create IdFactory instance for unique 64 bit diaSource id-s.
 
         Parameters
         ----------
         expId : `int`
             Exposure id.
 
         expBits: `int`
             Number of used bits in ``expId``.
 
         Note
         ----
         The diasource id-s consists of the ``expId`` stored fixed in the highest value
         ``expBits`` of the 64-bit integer plus (bitwise or) a generated sequence number in the
         low value end of the integer.
 
         Returns
         -------
         idFactory: `lsst.afw.table.IdFactory`
         """
         return afwTable.IdFactory.makeSource(expId, 64 - expBits)
 

◆ run()

def lsst.pipe.tasks.imageDifference.ImageDifferenceTask.run	(	self,
		exposure = `None`,
		selectSources = `None`,
		templateExposure = `None`,
		templateSources = `None`,
		idFactory = `None`,
		calexpBackgroundExposure = `None`,
		subtractedExposure = `None`
	)

PSF matches, subtract two images and perform detection on the difference image.

Parameters
----------
exposure : `lsst.afw.image.ExposureF`, optional
    The science exposure, the minuend in the image subtraction.
    Can be None only if ``config.doSubtract==False``.
selectSources : `lsst.afw.table.SourceCatalog`, optional
    Identified sources on the science exposure. This catalog is used to
    select sources in order to perform the AL PSF matching on stamp images
    around them. The selection steps depend on config options and whether
    ``templateSources`` and ``matchingSources`` specified.
templateExposure : `lsst.afw.image.ExposureF`, optional
    The template to be subtracted from ``exposure`` in the image subtraction.
    The template exposure should cover the same sky area as the science exposure.
    It is either a stich of patches of a coadd skymap image or a calexp
    of the same pointing as the science exposure. Can be None only
    if ``config.doSubtract==False`` and ``subtractedExposure`` is not None.
templateSources : `lsst.afw.table.SourceCatalog`, optional
    Identified sources on the template exposure.
idFactory : `lsst.afw.table.IdFactory`
    Generator object to assign ids to detected sources in the difference image.
calexpBackgroundExposure : `lsst.afw.image.ExposureF`, optional
    Background exposure to be added back to the science exposure
    if ``config.doAddCalexpBackground==True``
subtractedExposure : `lsst.afw.image.ExposureF`, optional
    If ``config.doSubtract==False`` and ``config.doDetection==True``,
    performs the post subtraction source detection only on this exposure.
    Otherwise should be None.

Returns
-------
results : `lsst.pipe.base.Struct`
    ``subtractedExposure`` : `lsst.afw.image.ExposureF`
Difference image.
    ``matchedExposure`` : `lsst.afw.image.ExposureF`
The matched PSF exposure.
    ``subtractRes`` : `lsst.pipe.base.Struct`
The returned result structure of the ImagePsfMatchTask subtask.
    ``diaSources``  : `lsst.afw.table.SourceCatalog`
The catalog of detected sources.
    ``selectSources`` : `lsst.afw.table.SourceCatalog`
The input source catalog with optionally added Qa information.

Notes
-----
The following major steps are included:

- warp template coadd to match WCS of image
- PSF match image to warped template
- subtract image from PSF-matched, warped template
- detect sources
- measure sources

For details about the image subtraction configuration modes
see `lsst.ip.diffim`.

Definition at line 419 of file imageDifference.py.

             idFactory=None, calexpBackgroundExposure=None, subtractedExposure=None):
         """PSF matches, subtract two images and perform detection on the difference image.
 
         Parameters
         ----------
         exposure : `lsst.afw.image.ExposureF`, optional
             The science exposure, the minuend in the image subtraction.
             Can be None only if ``config.doSubtract==False``.
         selectSources : `lsst.afw.table.SourceCatalog`, optional
             Identified sources on the science exposure. This catalog is used to
             select sources in order to perform the AL PSF matching on stamp images
             around them. The selection steps depend on config options and whether
             ``templateSources`` and ``matchingSources`` specified.
         templateExposure : `lsst.afw.image.ExposureF`, optional
             The template to be subtracted from ``exposure`` in the image subtraction.
             The template exposure should cover the same sky area as the science exposure.
             It is either a stich of patches of a coadd skymap image or a calexp
             of the same pointing as the science exposure. Can be None only
             if ``config.doSubtract==False`` and ``subtractedExposure`` is not None.
         templateSources : `lsst.afw.table.SourceCatalog`, optional
             Identified sources on the template exposure.
         idFactory : `lsst.afw.table.IdFactory`
             Generator object to assign ids to detected sources in the difference image.
         calexpBackgroundExposure : `lsst.afw.image.ExposureF`, optional
             Background exposure to be added back to the science exposure
             if ``config.doAddCalexpBackground==True``
         subtractedExposure : `lsst.afw.image.ExposureF`, optional
             If ``config.doSubtract==False`` and ``config.doDetection==True``,
             performs the post subtraction source detection only on this exposure.
             Otherwise should be None.
 
         Returns
         -------
         results : `lsst.pipe.base.Struct`
             ``subtractedExposure`` : `lsst.afw.image.ExposureF`
                 Difference image.
             ``matchedExposure`` : `lsst.afw.image.ExposureF`
                 The matched PSF exposure.
             ``subtractRes`` : `lsst.pipe.base.Struct`
                 The returned result structure of the ImagePsfMatchTask subtask.
             ``diaSources``  : `lsst.afw.table.SourceCatalog`
                 The catalog of detected sources.
             ``selectSources`` : `lsst.afw.table.SourceCatalog`
                 The input source catalog with optionally added Qa information.
 
         Notes
         -----
         The following major steps are included:
 
         - warp template coadd to match WCS of image
         - PSF match image to warped template
         - subtract image from PSF-matched, warped template
         - detect sources
         - measure sources
 
         For details about the image subtraction configuration modes
         see `lsst.ip.diffim`.
         """
         subtractRes = None
         controlSources = None
         diaSources = None
         kernelSources = None
 
         if self.config.doAddCalexpBackground:
             mi = exposure.getMaskedImage()
             mi += calexpBackgroundExposure.getImage()
 
         if not exposure.hasPsf():
             raise pipeBase.TaskError("Exposure has no psf")
         sciencePsf = exposure.getPsf()
 
         if self.config.doSubtract:
             if self.config.doScaleTemplateVariance:
                 templateVarFactor = self.scaleVariance.run(
                     templateExposure.getMaskedImage())
                 self.metadata.add("scaleTemplateVarianceFactor", templateVarFactor)
 
             if self.config.subtract.name == 'zogy':
                 subtractRes = self.subtract.subtractExposures(templateExposure, exposure,
                                                               doWarping=True,
                                                               spatiallyVarying=self.config.doSpatiallyVarying,
                                                               doPreConvolve=self.config.doPreConvolve)
                 subtractedExposure = subtractRes.subtractedExposure
 
             elif self.config.subtract.name == 'al':
                 # compute scienceSigmaOrig: sigma of PSF of science image before pre-convolution
                 scienceSigmaOrig = sciencePsf.computeShape().getDeterminantRadius()
                 templateSigma = templateExposure.getPsf().computeShape().getDeterminantRadius()
 
                 # if requested, convolve the science exposure with its PSF
                 # (properly, this should be a cross-correlation, but our code does not yet support that)
                 # compute scienceSigmaPost: sigma of science exposure with pre-convolution, if done,
                 # else sigma of original science exposure
                 # TODO: DM-22762 This functional block should be moved into its own method
                 preConvPsf = None
                 if self.config.doPreConvolve:
                     convControl = afwMath.ConvolutionControl()
                     # cannot convolve in place, so make a new MI to receive convolved image
                     srcMI = exposure.getMaskedImage()
                     destMI = srcMI.Factory(srcMI.getDimensions())
                     srcPsf = sciencePsf
                     if self.config.useGaussianForPreConvolution:
                         # convolve with a simplified PSF model: a double Gaussian
                         kWidth, kHeight = sciencePsf.getLocalKernel().getDimensions()
                         preConvPsf = SingleGaussianPsf(kWidth, kHeight, scienceSigmaOrig)
                     else:
                         # convolve with science exposure's PSF model
                         preConvPsf = srcPsf
                     afwMath.convolve(destMI, srcMI, preConvPsf.getLocalKernel(), convControl)
                     exposure.setMaskedImage(destMI)
                     scienceSigmaPost = scienceSigmaOrig*math.sqrt(2)
                 else:
                     scienceSigmaPost = scienceSigmaOrig
 
                 # If requested, find and select sources from the image
                 # else, AL subtraction will do its own source detection
                 # TODO: DM-22762 This functional block should be moved into its own method
                 if self.config.doSelectSources:
                     if selectSources is None:
                         self.log.warn("Src product does not exist; running detection, measurement, selection")
                         # Run own detection and measurement; necessary in nightly processing
                         selectSources = self.subtract.getSelectSources(
                             exposure,
                             sigma=scienceSigmaPost,
                             doSmooth=not self.doPreConvolve,
                             idFactory=idFactory,
                         )
 
                     if self.config.doAddMetrics:
                         # Number of basis functions
 
                         nparam = len(makeKernelBasisList(self.subtract.config.kernel.active,
                                                          referenceFwhmPix=scienceSigmaPost*FwhmPerSigma,
                                                          targetFwhmPix=templateSigma*FwhmPerSigma))
                         # Modify the schema of all Sources
                         # DEPRECATED: This is a data dependent (nparam) output product schema
                         # outside the task constructor.
                         # NOTE: The pre-determination of nparam at this point
                         # may be incorrect as the template psf is warped later in
                         # ImagePsfMatchTask.matchExposures()
                         kcQa = KernelCandidateQa(nparam)
                         selectSources = kcQa.addToSchema(selectSources)
                     if self.config.kernelSourcesFromRef:
                         # match exposure sources to reference catalog
                         astromRet = self.astrometer.loadAndMatch(exposure=exposure, sourceCat=selectSources)
                         matches = astromRet.matches
                     elif templateSources:
                         # match exposure sources to template sources
                         mc = afwTable.MatchControl()
                         mc.findOnlyClosest = False
                         matches = afwTable.matchRaDec(templateSources, selectSources, 1.0*geom.arcseconds,
                                                       mc)
                     else:
                         raise RuntimeError("doSelectSources=True and kernelSourcesFromRef=False,"
                                            "but template sources not available. Cannot match science "
                                            "sources with template sources. Run process* on data from "
                                            "which templates are built.")
 
                     kernelSources = self.sourceSelector.run(selectSources, exposure=exposure,
                                                             matches=matches).sourceCat
                     random.shuffle(kernelSources, random.random)
                     controlSources = kernelSources[::self.config.controlStepSize]
                     kernelSources = [k for i, k in enumerate(kernelSources)
                                      if i % self.config.controlStepSize]
 
                     if self.config.doSelectDcrCatalog:
                         redSelector = DiaCatalogSourceSelectorTask(
                             DiaCatalogSourceSelectorConfig(grMin=self.sourceSelector.config.grMax,
                                                            grMax=99.999))
                         redSources = redSelector.selectStars(exposure, selectSources, matches=matches).starCat
                         controlSources.extend(redSources)
 
                         blueSelector = DiaCatalogSourceSelectorTask(
                             DiaCatalogSourceSelectorConfig(grMin=-99.999,
                                                            grMax=self.sourceSelector.config.grMin))
                         blueSources = blueSelector.selectStars(exposure, selectSources,
                                                                matches=matches).starCat
                         controlSources.extend(blueSources)
 
                     if self.config.doSelectVariableCatalog:
                         varSelector = DiaCatalogSourceSelectorTask(
                             DiaCatalogSourceSelectorConfig(includeVariable=True))
                         varSources = varSelector.selectStars(exposure, selectSources, matches=matches).starCat
                         controlSources.extend(varSources)
 
                     self.log.info("Selected %d / %d sources for Psf matching (%d for control sample)"
                                   % (len(kernelSources), len(selectSources), len(controlSources)))
 
                 allresids = {}
                 # TODO: DM-22762 This functional block should be moved into its own method
                 if self.config.doUseRegister:
                     self.log.info("Registering images")
 
                     if templateSources is None:
                         # Run detection on the template, which is
                         # temporarily background-subtracted
                         # sigma of PSF of template image before warping
                         templateSigma = templateExposure.getPsf().computeShape().getDeterminantRadius()
                         templateSources = self.subtract.getSelectSources(
                             templateExposure,
                             sigma=templateSigma,
                             doSmooth=True,
                             idFactory=idFactory
                         )
 
                     # Third step: we need to fit the relative astrometry.
                     #
                     wcsResults = self.fitAstrometry(templateSources, templateExposure, selectSources)
                     warpedExp = self.register.warpExposure(templateExposure, wcsResults.wcs,
                                                            exposure.getWcs(), exposure.getBBox())
                     templateExposure = warpedExp
 
                     # Create debugging outputs on the astrometric
                     # residuals as a function of position.  Persistence
                     # not yet implemented; expected on (I believe) #2636.
                     if self.config.doDebugRegister:
                         # Grab matches to reference catalog
                         srcToMatch = {x.second.getId(): x.first for x in matches}
 
                         refCoordKey = wcsResults.matches[0].first.getTable().getCoordKey()
                         inCentroidKey = wcsResults.matches[0].second.getTable().getCentroidKey()
                         sids = [m.first.getId() for m in wcsResults.matches]
                         positions = [m.first.get(refCoordKey) for m in wcsResults.matches]
                         residuals = [m.first.get(refCoordKey).getOffsetFrom(wcsResults.wcs.pixelToSky(
                             m.second.get(inCentroidKey))) for m in wcsResults.matches]
                         allresids = dict(zip(sids, zip(positions, residuals)))
 
                         cresiduals = [m.first.get(refCoordKey).getTangentPlaneOffset(
                             wcsResults.wcs.pixelToSky(
                                 m.second.get(inCentroidKey))) for m in wcsResults.matches]
                         colors = numpy.array([-2.5*numpy.log10(srcToMatch[x].get("g")) +
                                               2.5*numpy.log10(srcToMatch[x].get("r"))
                                               for x in sids if x in srcToMatch.keys()])
                         dlong = numpy.array([r[0].asArcseconds() for s, r in zip(sids, cresiduals)
                                              if s in srcToMatch.keys()])
                         dlat = numpy.array([r[1].asArcseconds() for s, r in zip(sids, cresiduals)
                                             if s in srcToMatch.keys()])
                         idx1 = numpy.where(colors < self.sourceSelector.config.grMin)
                         idx2 = numpy.where((colors >= self.sourceSelector.config.grMin) &
                                            (colors <= self.sourceSelector.config.grMax))
                         idx3 = numpy.where(colors > self.sourceSelector.config.grMax)
                         rms1Long = IqrToSigma*(
                             (numpy.percentile(dlong[idx1], 75) - numpy.percentile(dlong[idx1], 25)))
                         rms1Lat = IqrToSigma*(numpy.percentile(dlat[idx1], 75) -
                                               numpy.percentile(dlat[idx1], 25))
                         rms2Long = IqrToSigma*(
                             (numpy.percentile(dlong[idx2], 75) - numpy.percentile(dlong[idx2], 25)))
                         rms2Lat = IqrToSigma*(numpy.percentile(dlat[idx2], 75) -
                                               numpy.percentile(dlat[idx2], 25))
                         rms3Long = IqrToSigma*(
                             (numpy.percentile(dlong[idx3], 75) - numpy.percentile(dlong[idx3], 25)))
                         rms3Lat = IqrToSigma*(numpy.percentile(dlat[idx3], 75) -
                                               numpy.percentile(dlat[idx3], 25))
                         self.log.info("Blue star offsets'': %.3f %.3f, %.3f %.3f" %
                                       (numpy.median(dlong[idx1]), rms1Long,
                                        numpy.median(dlat[idx1]), rms1Lat))
                         self.log.info("Green star offsets'': %.3f %.3f, %.3f %.3f" %
                                       (numpy.median(dlong[idx2]), rms2Long,
                                        numpy.median(dlat[idx2]), rms2Lat))
                         self.log.info("Red star offsets'': %.3f %.3f, %.3f %.3f" %
                                       (numpy.median(dlong[idx3]), rms3Long,
                                        numpy.median(dlat[idx3]), rms3Lat))
 
                         self.metadata.add("RegisterBlueLongOffsetMedian", numpy.median(dlong[idx1]))
                         self.metadata.add("RegisterGreenLongOffsetMedian", numpy.median(dlong[idx2]))
                         self.metadata.add("RegisterRedLongOffsetMedian", numpy.median(dlong[idx3]))
                         self.metadata.add("RegisterBlueLongOffsetStd", rms1Long)
                         self.metadata.add("RegisterGreenLongOffsetStd", rms2Long)
                         self.metadata.add("RegisterRedLongOffsetStd", rms3Long)
 
                         self.metadata.add("RegisterBlueLatOffsetMedian", numpy.median(dlat[idx1]))
                         self.metadata.add("RegisterGreenLatOffsetMedian", numpy.median(dlat[idx2]))
                         self.metadata.add("RegisterRedLatOffsetMedian", numpy.median(dlat[idx3]))
                         self.metadata.add("RegisterBlueLatOffsetStd", rms1Lat)
                         self.metadata.add("RegisterGreenLatOffsetStd", rms2Lat)
                         self.metadata.add("RegisterRedLatOffsetStd", rms3Lat)
 
                 # warp template exposure to match exposure,
                 # PSF match template exposure to exposure,
                 # then return the difference
 
                 # Return warped template...  Construct sourceKernelCand list after subtract
                 self.log.info("Subtracting images")
                 subtractRes = self.subtract.subtractExposures(
                     templateExposure=templateExposure,
                     scienceExposure=exposure,
                     candidateList=kernelSources,
                     convolveTemplate=self.config.convolveTemplate,
                     doWarping=not self.config.doUseRegister
                 )
                 subtractedExposure = subtractRes.subtractedExposure
 
                 if self.config.doDetection:
                     self.log.info("Computing diffim PSF")
 
                     # Get Psf from the appropriate input image if it doesn't exist
                     if not subtractedExposure.hasPsf():
                         if self.config.convolveTemplate:
                             subtractedExposure.setPsf(exposure.getPsf())
                         else:
                             subtractedExposure.setPsf(templateExposure.getPsf())
 
                 # If doSubtract is False, then subtractedExposure was fetched from disk (above),
                 # thus it may have already been decorrelated. Thus, we do not decorrelate if
                 # doSubtract is False.
 
                 # NOTE: At this point doSubtract == True
                 if self.config.doDecorrelation and self.config.doSubtract:
                     preConvKernel = None
                     if preConvPsf is not None:
                         preConvKernel = preConvPsf.getLocalKernel()
                     if self.config.convolveTemplate:
                         self.log.info("Decorrelation after template image convolution")
                         decorrResult = self.decorrelate.run(exposure, templateExposure,
                                                             subtractedExposure,
                                                             subtractRes.psfMatchingKernel,
                                                             spatiallyVarying=self.config.doSpatiallyVarying,
                                                             preConvKernel=preConvKernel)
                     else:
                         self.log.info("Decorrelation after science image convolution")
                         decorrResult = self.decorrelate.run(templateExposure, exposure,
                                                             subtractedExposure,
                                                             subtractRes.psfMatchingKernel,
                                                             spatiallyVarying=self.config.doSpatiallyVarying,
                                                             preConvKernel=preConvKernel)
                     subtractedExposure = decorrResult.correctedExposure
 
             # END (if subtractAlgorithm == 'AL')
         # END (if self.config.doSubtract)
         if self.config.doDetection:
             self.log.info("Running diaSource detection")
             # Erase existing detection mask planes
             mask = subtractedExposure.getMaskedImage().getMask()
             mask &= ~(mask.getPlaneBitMask("DETECTED") | mask.getPlaneBitMask("DETECTED_NEGATIVE"))
 
             table = afwTable.SourceTable.make(self.schema, idFactory)
             table.setMetadata(self.algMetadata)
             results = self.detection.run(
                 table=table,
                 exposure=subtractedExposure,
                 doSmooth=not self.config.doPreConvolve
             )
 
             if self.config.doMerge:
                 fpSet = results.fpSets.positive
                 fpSet.merge(results.fpSets.negative, self.config.growFootprint,
                             self.config.growFootprint, False)
                 diaSources = afwTable.SourceCatalog(table)
                 fpSet.makeSources(diaSources)
                 self.log.info("Merging detections into %d sources" % (len(diaSources)))
             else:
                 diaSources = results.sources
 
             if self.config.doMeasurement:
                 newDipoleFitting = self.config.doDipoleFitting
                 self.log.info("Running diaSource measurement: newDipoleFitting=%r", newDipoleFitting)
                 if not newDipoleFitting:
                     # Just fit dipole in diffim
                     self.measurement.run(diaSources, subtractedExposure)
                 else:
                     # Use (matched) template and science image (if avail.) to constrain dipole fitting
                     if self.config.doSubtract and 'matchedExposure' in subtractRes.getDict():
                         self.measurement.run(diaSources, subtractedExposure, exposure,
                                              subtractRes.matchedExposure)
                     else:
                         self.measurement.run(diaSources, subtractedExposure, exposure)
 
             if self.config.doForcedMeasurement:
                 # Run forced psf photometry on the PVI at the diaSource locations.
                 # Copy the measured flux and error into the diaSource.
                 forcedSources = self.forcedMeasurement.generateMeasCat(
                     exposure, diaSources, subtractedExposure.getWcs())
                 self.forcedMeasurement.run(forcedSources, exposure, diaSources, subtractedExposure.getWcs())
                 mapper = afwTable.SchemaMapper(forcedSources.schema, diaSources.schema)
                 mapper.addMapping(forcedSources.schema.find("base_PsfFlux_instFlux")[0],
                                   "ip_diffim_forced_PsfFlux_instFlux", True)
                 mapper.addMapping(forcedSources.schema.find("base_PsfFlux_instFluxErr")[0],
                                   "ip_diffim_forced_PsfFlux_instFluxErr", True)
                 mapper.addMapping(forcedSources.schema.find("base_PsfFlux_area")[0],
                                   "ip_diffim_forced_PsfFlux_area", True)
                 mapper.addMapping(forcedSources.schema.find("base_PsfFlux_flag")[0],
                                   "ip_diffim_forced_PsfFlux_flag", True)
                 mapper.addMapping(forcedSources.schema.find("base_PsfFlux_flag_noGoodPixels")[0],
                                   "ip_diffim_forced_PsfFlux_flag_noGoodPixels", True)
                 mapper.addMapping(forcedSources.schema.find("base_PsfFlux_flag_edge")[0],
                                   "ip_diffim_forced_PsfFlux_flag_edge", True)
                 for diaSource, forcedSource in zip(diaSources, forcedSources):
                     diaSource.assign(forcedSource, mapper)
 
             # Match with the calexp sources if possible
             if self.config.doMatchSources:
                 if selectSources is not None:
                     # Create key,val pair where key=diaSourceId and val=sourceId
                     matchRadAsec = self.config.diaSourceMatchRadius
                     matchRadPixel = matchRadAsec/exposure.getWcs().getPixelScale().asArcseconds()
 
                     srcMatches = afwTable.matchXy(selectSources, diaSources, matchRadPixel)
                     srcMatchDict = dict([(srcMatch.second.getId(), srcMatch.first.getId()) for
                                          srcMatch in srcMatches])
                     self.log.info("Matched %d / %d diaSources to sources" % (len(srcMatchDict),
                                                                              len(diaSources)))
                 else:
                     self.log.warn("Src product does not exist; cannot match with diaSources")
                     srcMatchDict = {}
 
                 # Create key,val pair where key=diaSourceId and val=refId
                 refAstromConfig = AstrometryConfig()
                 refAstromConfig.matcher.maxMatchDistArcSec = matchRadAsec
                 refAstrometer = AstrometryTask(refAstromConfig)
                 astromRet = refAstrometer.run(exposure=exposure, sourceCat=diaSources)
                 refMatches = astromRet.matches
                 if refMatches is None:
                     self.log.warn("No diaSource matches with reference catalog")
                     refMatchDict = {}
                 else:
                     self.log.info("Matched %d / %d diaSources to reference catalog" % (len(refMatches),
                                                                                        len(diaSources)))
                     refMatchDict = dict([(refMatch.second.getId(), refMatch.first.getId()) for
                                          refMatch in refMatches])
 
                 # Assign source Ids
                 for diaSource in diaSources:
                     sid = diaSource.getId()
                     if sid in srcMatchDict:
                         diaSource.set("srcMatchId", srcMatchDict[sid])
                     if sid in refMatchDict:
                         diaSource.set("refMatchId", refMatchDict[sid])
 
             if self.config.doAddMetrics and self.config.doSelectSources:
                 self.log.info("Evaluating metrics and control sample")
 
                 kernelCandList = []
                 for cell in subtractRes.kernelCellSet.getCellList():
                     for cand in cell.begin(False):  # include bad candidates
                         kernelCandList.append(cand)
 
                 # Get basis list to build control sample kernels
                 basisList = kernelCandList[0].getKernel(KernelCandidateF.ORIG).getKernelList()
                 nparam = len(kernelCandList[0].getKernel(KernelCandidateF.ORIG).getKernelParameters())
 
                 controlCandList = (
                     diffimTools.sourceTableToCandidateList(controlSources,
                                                            subtractRes.warpedExposure, exposure,
                                                            self.config.subtract.kernel.active,
                                                            self.config.subtract.kernel.active.detectionConfig,
                                                            self.log, doBuild=True, basisList=basisList))
 
                 KernelCandidateQa.apply(kernelCandList, subtractRes.psfMatchingKernel,
                                         subtractRes.backgroundModel, dof=nparam)
                 KernelCandidateQa.apply(controlCandList, subtractRes.psfMatchingKernel,
                                         subtractRes.backgroundModel)
 
                 if self.config.doDetection:
                     KernelCandidateQa.aggregate(selectSources, self.metadata, allresids, diaSources)
                 else:
                     KernelCandidateQa.aggregate(selectSources, self.metadata, allresids)
 
         self.runDebug(exposure, subtractRes, selectSources, kernelSources, diaSources)
         return pipeBase.Struct(
             subtractedExposure=subtractedExposure,
             matchedExposure=subtractRes.matchedExposure,
             subtractRes=subtractRes,
             diaSources=diaSources,
             selectSources=selectSources
         )
 

◆ runDataRef()

def lsst.pipe.tasks.imageDifference.ImageDifferenceTask.runDataRef	(	self,
		sensorRef,
		templateIdList = `None`
	)

Subtract an image from a template coadd and measure the result.

Data I/O wrapper around `run` using the butler in Gen2.

Parameters
----------
sensorRef : `lsst.daf.persistence.ButlerDataRef`
    Sensor-level butler data reference, used for the following data products:

    Input only:
    - calexp
    - psf
    - ccdExposureId
    - ccdExposureId_bits
    - self.config.coaddName + "Coadd_skyMap"
    - self.config.coaddName + "Coadd"
    Input or output, depending on config:
    - self.config.coaddName + "Diff_subtractedExp"
    Output, depending on config:
    - self.config.coaddName + "Diff_matchedExp"
    - self.config.coaddName + "Diff_src"

Returns
-------
results : `lsst.pipe.base.Struct`
    Returns the Struct by `run`.

Definition at line 342 of file imageDifference.py.

     def runDataRef(self, sensorRef, templateIdList=None):
         """Subtract an image from a template coadd and measure the result.
 
         Data I/O wrapper around `run` using the butler in Gen2.
 
         Parameters
         ----------
         sensorRef : `lsst.daf.persistence.ButlerDataRef`
             Sensor-level butler data reference, used for the following data products:
 
             Input only:
             - calexp
             - psf
             - ccdExposureId
             - ccdExposureId_bits
             - self.config.coaddName + "Coadd_skyMap"
             - self.config.coaddName + "Coadd"
             Input or output, depending on config:
             - self.config.coaddName + "Diff_subtractedExp"
             Output, depending on config:
             - self.config.coaddName + "Diff_matchedExp"
             - self.config.coaddName + "Diff_src"
 
         Returns
         -------
         results : `lsst.pipe.base.Struct`
             Returns the Struct by `run`.
         """
         subtractedExposureName = self.config.coaddName + "Diff_differenceExp"
         subtractedExposure = None
         selectSources = None
         calexpBackgroundExposure = None
         self.log.info("Processing %s" % (sensorRef.dataId))
 
         # We make one IdFactory that will be used by both icSrc and src datasets;
         # I don't know if this is the way we ultimately want to do things, but at least
         # this ensures the source IDs are fully unique.
         idFactory = self.makeIdFactory(expId=int(sensorRef.get("ccdExposureId")),
                                        expBits=sensorRef.get("ccdExposureId_bits"))
         if self.config.doAddCalexpBackground:
             calexpBackgroundExposure = sensorRef.get("calexpBackground")
 
         # Retrieve the science image we wish to analyze
         exposure = sensorRef.get("calexp", immediate=True)
 
         # Retrieve the template image
         template = self.getTemplate.run(exposure, sensorRef, templateIdList=templateIdList)
 
         if sensorRef.datasetExists("src"):
             self.log.info("Source selection via src product")
             # Sources already exist; for data release processing
             selectSources = sensorRef.get("src")
 
         if not self.config.doSubtract and self.config.doDetection:
             # If we don't do subtraction, we need the subtracted exposure from the repo
             subtractedExposure = sensorRef.get(subtractedExposureName)
         # Both doSubtract and doDetection cannot be False
 
         results = self.run(exposure=exposure,
                            selectSources=selectSources,
                            templateExposure=template.exposure,
                            templateSources=template.sources,
                            idFactory=idFactory,
                            calexpBackgroundExposure=calexpBackgroundExposure,
                            subtractedExposure=subtractedExposure)
 
         if self.config.doWriteSources and results.diaSources is not None:
             sensorRef.put(results.diaSources, self.config.coaddName + "Diff_diaSrc")
         if self.config.doWriteMatchedExp:
             sensorRef.put(results.matchedExposure, self.config.coaddName + "Diff_matchedExp")
         if self.config.doAddMetrics and self.config.doSelectSources:
             sensorRef.put(results.selectSources, self.config.coaddName + "Diff_kernelSrc")
         if self.config.doWriteSubtractedExp:
             sensorRef.put(results.subtractedExposure, subtractedExposureName)
         return results
 

◆ runDebug()

def lsst.pipe.tasks.imageDifference.ImageDifferenceTask.runDebug	(	self,
		exposure,
		subtractRes,
		selectSources,
		kernelSources,
		diaSources
	)

Make debug plots and displays.

Todo
----
Test and update for current debug display and slot names

Definition at line 899 of file imageDifference.py.

     def runDebug(self, exposure, subtractRes, selectSources, kernelSources, diaSources):
         """Make debug plots and displays.
 
         Todo
         ----
         Test and update for current debug display and slot names
         """
         import lsstDebug
         display = lsstDebug.Info(__name__).display
         showSubtracted = lsstDebug.Info(__name__).showSubtracted
         showPixelResiduals = lsstDebug.Info(__name__).showPixelResiduals
         showDiaSources = lsstDebug.Info(__name__).showDiaSources
         showDipoles = lsstDebug.Info(__name__).showDipoles
         maskTransparency = lsstDebug.Info(__name__).maskTransparency
         if display:
             disp = afwDisplay.getDisplay(frame=lsstDebug.frame)
             if not maskTransparency:
                 maskTransparency = 0
             disp.setMaskTransparency(maskTransparency)
 
         if display and showSubtracted:
             disp.mtv(subtractRes.subtractedExposure, title="Subtracted image")
             mi = subtractRes.subtractedExposure.getMaskedImage()
             x0, y0 = mi.getX0(), mi.getY0()
             with disp.Buffering():
                 for s in diaSources:
                     x, y = s.getX() - x0, s.getY() - y0
                     ctype = "red" if s.get("flags_negative") else "yellow"
                     if (s.get("base_PixelFlags_flag_interpolatedCenter") or
                             s.get("base_PixelFlags_flag_saturatedCenter") or
                             s.get("base_PixelFlags_flag_crCenter")):
                         ptype = "x"
                     elif (s.get("base_PixelFlags_flag_interpolated") or
                           s.get("base_PixelFlags_flag_saturated") or
                           s.get("base_PixelFlags_flag_cr")):
                         ptype = "+"
                     else:
                         ptype = "o"
                     disp.dot(ptype, x, y, size=4, ctype=ctype)
             lsstDebug.frame += 1
 
         if display and showPixelResiduals and selectSources:
             nonKernelSources = []
             for source in selectSources:
                 if source not in kernelSources:
                     nonKernelSources.append(source)
 
             diUtils.plotPixelResiduals(exposure,
                                        subtractRes.warpedExposure,
                                        subtractRes.subtractedExposure,
                                        subtractRes.kernelCellSet,
                                        subtractRes.psfMatchingKernel,
                                        subtractRes.backgroundModel,
                                        nonKernelSources,
                                        self.subtract.config.kernel.active.detectionConfig,
                                        origVariance=False)
             diUtils.plotPixelResiduals(exposure,
                                        subtractRes.warpedExposure,
                                        subtractRes.subtractedExposure,
                                        subtractRes.kernelCellSet,
                                        subtractRes.psfMatchingKernel,
                                        subtractRes.backgroundModel,
                                        nonKernelSources,
                                        self.subtract.config.kernel.active.detectionConfig,
                                        origVariance=True)
         if display and showDiaSources:
             flagChecker = SourceFlagChecker(diaSources)
             isFlagged = [flagChecker(x) for x in diaSources]
             isDipole = [x.get("ip_diffim_ClassificationDipole_value") for x in diaSources]
             diUtils.showDiaSources(diaSources, subtractRes.subtractedExposure, isFlagged, isDipole,
                                    frame=lsstDebug.frame)
             lsstDebug.frame += 1
 
         if display and showDipoles:
             DipoleAnalysis().displayDipoles(subtractRes.subtractedExposure, diaSources,
                                             frame=lsstDebug.frame)
             lsstDebug.frame += 1
 

Member Data Documentation

◆ algMetadata

lsst.pipe.tasks.imageDifference.ImageDifferenceTask.algMetadata

Definition at line 286 of file imageDifference.py.

◆ ConfigClass

lsst.pipe.tasks.imageDifference.ImageDifferenceTask.ConfigClass

static

Definition at line 256 of file imageDifference.py.

◆ RunnerClass

lsst.pipe.tasks.imageDifference.ImageDifferenceTask.RunnerClass

static

Definition at line 257 of file imageDifference.py.

◆ schema

lsst.pipe.tasks.imageDifference.ImageDifferenceTask.schema

Definition at line 278 of file imageDifference.py.

The documentation for this class was generated from the following file:

/j/snowflake/release/lsstsw/stack/Linux64/pipe_tasks/19.0.0-33-ge0e6817e/python/lsst/pipe/tasks/imageDifference.py

Public Member Functions

Static Public Member Functions

Public Attributes

Static Public Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ fitAstrometry()

◆ getSchemaCatalogs()

◆ makeIdFactory()

◆ run()

◆ runDataRef()

◆ runDebug()

Member Data Documentation

◆ algMetadata

◆ ConfigClass

◆ RunnerClass

◆ schema

◆ init()