LSSTApplications  17.0+124,17.0+14,17.0+73,18.0.0+37,18.0.0+80,18.0.0-4-g68ffd23+4,18.1.0-1-g0001055+12,18.1.0-1-g03d53ef+5,18.1.0-1-g1349e88+55,18.1.0-1-g2505f39+44,18.1.0-1-g5315e5e+4,18.1.0-1-g5e4b7ea+14,18.1.0-1-g7e8fceb+4,18.1.0-1-g85f8cd4+48,18.1.0-1-g8ff0b9f+4,18.1.0-1-ga2c679d+1,18.1.0-1-gd55f500+35,18.1.0-10-gb58edde+2,18.1.0-11-g0997b02+4,18.1.0-13-gfe4edf0b+12,18.1.0-14-g259bd21+21,18.1.0-19-gdb69f3f+2,18.1.0-2-g5f9922c+24,18.1.0-2-gd3b74e5+11,18.1.0-2-gfbf3545+32,18.1.0-26-g728bddb4+5,18.1.0-27-g6ff7ca9+2,18.1.0-3-g52aa583+25,18.1.0-3-g8ea57af+9,18.1.0-3-gb69f684+42,18.1.0-3-gfcaddf3+6,18.1.0-32-gd8786685a,18.1.0-4-gf3f9b77+6,18.1.0-5-g1dd662b+2,18.1.0-5-g6dbcb01+41,18.1.0-6-gae77429+3,18.1.0-7-g9d75d83+9,18.1.0-7-gae09a6d+30,18.1.0-9-gc381ef5+4,w.2019.45
LSSTDataManagementBasePackage
imagePsfMatch.py
Go to the documentation of this file.
1 # This file is part of ip_diffim.
2 #
3 # Developed for the LSST Data Management System.
4 # This product includes software developed by the LSST Project
5 # (https://www.lsst.org).
6 # See the COPYRIGHT file at the top-level directory of this distribution
7 # for details of code ownership.
8 #
9 # This program is free software: you can redistribute it and/or modify
10 # it under the terms of the GNU General Public License as published by
11 # the Free Software Foundation, either version 3 of the License, or
12 # (at your option) any later version.
13 #
14 # This program is distributed in the hope that it will be useful,
15 # but WITHOUT ANY WARRANTY; without even the implied warranty of
16 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
17 # GNU General Public License for more details.
18 #
19 # You should have received a copy of the GNU General Public License
20 # along with this program. If not, see <https://www.gnu.org/licenses/>.
21 
22 import numpy as np
23 
24 import lsst.daf.base as dafBase
25 import lsst.pex.config as pexConfig
26 import lsst.afw.detection as afwDetect
27 import lsst.afw.image as afwImage
28 import lsst.afw.math as afwMath
29 import lsst.afw.geom as afwGeom
30 import lsst.afw.table as afwTable
31 import lsst.geom as geom
32 import lsst.pipe.base as pipeBase
33 from lsst.meas.algorithms import SourceDetectionTask, SubtractBackgroundTask, WarpedPsf
34 from lsst.meas.base import SingleFrameMeasurementTask
35 from .makeKernelBasisList import makeKernelBasisList
36 from .psfMatch import PsfMatchTask, PsfMatchConfigDF, PsfMatchConfigAL
37 from . import utils as diffimUtils
38 from . import diffimLib
39 from . import diffimTools
40 import lsst.afw.display as afwDisplay
41 
42 __all__ = ["ImagePsfMatchConfig", "ImagePsfMatchTask", "subtractAlgorithmRegistry"]
43 
44 sigma2fwhm = 2.*np.sqrt(2.*np.log(2.))
45 
46 
47 class ImagePsfMatchConfig(pexConfig.Config):
48  """Configuration for image-to-image Psf matching.
49  """
50  kernel = pexConfig.ConfigChoiceField(
51  doc="kernel type",
52  typemap=dict(
53  AL=PsfMatchConfigAL,
54  DF=PsfMatchConfigDF
55  ),
56  default="AL",
57  )
58  selectDetection = pexConfig.ConfigurableField(
59  target=SourceDetectionTask,
60  doc="Initial detections used to feed stars to kernel fitting",
61  )
62  selectMeasurement = pexConfig.ConfigurableField(
63  target=SingleFrameMeasurementTask,
64  doc="Initial measurements used to feed stars to kernel fitting",
65  )
66 
67  def setDefaults(self):
68  # High sigma detections only
69  self.selectDetection.reEstimateBackground = False
70  self.selectDetection.thresholdValue = 10.0
71 
72  # Minimal set of measurments for star selection
73  self.selectMeasurement.algorithms.names.clear()
74  self.selectMeasurement.algorithms.names = ('base_SdssCentroid', 'base_PsfFlux', 'base_PixelFlags',
75  'base_SdssShape', 'base_GaussianFlux', 'base_SkyCoord')
76  self.selectMeasurement.slots.modelFlux = None
77  self.selectMeasurement.slots.apFlux = None
78  self.selectMeasurement.slots.calibFlux = None
79 
80 
82  """Psf-match two MaskedImages or Exposures using the sources in the images.
83 
84  Parameters
85  ----------
86  args :
87  Arguments to be passed to lsst.ip.diffim.PsfMatchTask.__init__
88  kwargs :
89  Keyword arguments to be passed to lsst.ip.diffim.PsfMatchTask.__init__
90 
91  Notes
92  -----
93  Upon initialization, the kernel configuration is defined by self.config.kernel.active.
94  The task creates an lsst.afw.math.Warper from the subConfig self.config.kernel.active.warpingConfig.
95  A schema for the selection and measurement of candidate lsst.ip.diffim.KernelCandidates is
96  defined, and used to initize subTasks selectDetection (for candidate detection) and selectMeasurement
97  (for candidate measurement).
98 
99  Description
100 
101  Build a Psf-matching kernel using two input images, either as MaskedImages (in which case they need
102  to be astrometrically aligned) or Exposures (in which case astrometric alignment will happen by
103  default but may be turned off). This requires a list of input Sources which may be provided
104  by the calling Task; if not, the Task will perform a coarse source detection
105  and selection for this purpose. Sources are vetted for signal-to-noise and masked pixels
106  (in both the template and science image), and substamps around each acceptable
107  source are extracted and used to create an instance of KernelCandidate.
108  Each KernelCandidate is then placed within a lsst.afw.math.SpatialCellSet, which is used by an ensemble of
109  lsst.afw.math.CandidateVisitor instances to build the Psf-matching kernel. These visitors include, in
110  the order that they are called: BuildSingleKernelVisitor, KernelSumVisitor, BuildSpatialKernelVisitor,
111  and AssessSpatialKernelVisitor.
112 
113  Sigma clipping of KernelCandidates is performed as follows:
114 
115  - BuildSingleKernelVisitor, using the substamp diffim residuals from the per-source kernel fit,
116  if PsfMatchConfig.singleKernelClipping is True
117  - KernelSumVisitor, using the mean and standard deviation of the kernel sum from all candidates,
118  if PsfMatchConfig.kernelSumClipping is True
119  - AssessSpatialKernelVisitor, using the substamp diffim ressiduals from the spatial kernel fit,
120  if PsfMatchConfig.spatialKernelClipping is True
121 
122  The actual solving for the kernel (and differential background model) happens in
123  lsst.ip.diffim.PsfMatchTask._solve. This involves a loop over the SpatialCellSet that first builds the
124  per-candidate matching kernel for the requested number of KernelCandidates per cell
125  (PsfMatchConfig.nStarPerCell). The quality of this initial per-candidate difference image is examined,
126  using moments of the pixel residuals in the difference image normalized by the square root of the variance
127  (i.e. sigma); ideally this should follow a normal (0, 1) distribution,
128  but the rejection thresholds are set
129  by the config (PsfMatchConfig.candidateResidualMeanMax and PsfMatchConfig.candidateResidualStdMax).
130  All candidates that pass this initial build are then examined en masse to find the
131  mean/stdev of the kernel sums across all candidates.
132  Objects that are significantly above or below the mean,
133  typically due to variability or sources that are saturated in one image but not the other,
134  are also rejected.This threshold is defined by PsfMatchConfig.maxKsumSigma.
135  Finally, a spatial model is built using all currently-acceptable candidates,
136  and the spatial model used to derive a second set of (spatial) residuals
137  which are again used to reject bad candidates, using the same thresholds as above.
138 
139  Invoking the Task
140 
141  There is no run() method for this Task. Instead there are 4 methods that
142  may be used to invoke the Psf-matching. These are
143  `~lsst.ip.diffim.imagePsfMatch.ImagePsfMatchTask.matchMaskedImages`,
144  `~lsst.ip.diffim.imagePsfMatch.ImagePsfMatchTask.subtractMaskedImages`,
145  `~lsst.ip.diffim.imagePsfMatch.ImagePsfMatchTask.matchExposures`, and
146  `~lsst.ip.diffim.imagePsfMatch.ImagePsfMatchTask.subtractExposures`.
147 
148  The methods that operate on lsst.afw.image.MaskedImage require that the images already be astrometrically
149  aligned, and are the same shape. The methods that operate on lsst.afw.image.Exposure allow for the
150  input images to be misregistered and potentially be different sizes; by default a
151  lsst.afw.math.LanczosWarpingKernel is used to perform the astrometric alignment. The methods
152  that "match" images return a Psf-matched image, while the methods that "subtract" images
153  return a Psf-matched and template subtracted image.
154 
155  See each method's returned lsst.pipe.base.Struct for more details.
156 
157  Debug variables
158 
159  The lsst.pipe.base.cmdLineTask.CmdLineTask command line task interface supports a
160  flag -d/--debug to import debug.py from your PYTHONPATH. The relevant contents of debug.py
161  for this Task include:
162 
163  .. code-block:: py
164 
165  import sys
166  import lsstDebug
167  def DebugInfo(name):
168  di = lsstDebug.getInfo(name)
169  if name == "lsst.ip.diffim.psfMatch":
170  di.display = True # enable debug output
171  di.maskTransparency = 80 # display mask transparency
172  di.displayCandidates = True # show all the candidates and residuals
173  di.displayKernelBasis = False # show kernel basis functions
174  di.displayKernelMosaic = True # show kernel realized across the image
175  di.plotKernelSpatialModel = False # show coefficients of spatial model
176  di.showBadCandidates = True # show the bad candidates (red) along with good (green)
177  elif name == "lsst.ip.diffim.imagePsfMatch":
178  di.display = True # enable debug output
179  di.maskTransparency = 30 # display mask transparency
180  di.displayTemplate = True # show full (remapped) template
181  di.displaySciIm = True # show science image to match to
182  di.displaySpatialCells = True # show spatial cells
183  di.displayDiffIm = True # show difference image
184  di.showBadCandidates = True # show the bad candidates (red) along with good (green)
185  elif name == "lsst.ip.diffim.diaCatalogSourceSelector":
186  di.display = False # enable debug output
187  di.maskTransparency = 30 # display mask transparency
188  di.displayExposure = True # show exposure with candidates indicated
189  di.pauseAtEnd = False # pause when done
190  return di
191  lsstDebug.Info = DebugInfo
192  lsstDebug.frame = 1
193 
194  Note that if you want addional logging info, you may add to your scripts:
195 
196  .. code-block:: py
197 
198  import lsst.log.utils as logUtils
199  logUtils.traceSetAt("ip.diffim", 4)
200 
201  Examples
202  --------
203  A complete example of using ImagePsfMatchTask
204 
205  This code is imagePsfMatchTask.py in the examples directory, and can be run as e.g.
206 
207  .. code-block:: none
208 
209  examples/imagePsfMatchTask.py --debug
210  examples/imagePsfMatchTask.py --debug --mode="matchExposures"
211  examples/imagePsfMatchTask.py --debug --template /path/to/templateExp.fits
212  --science /path/to/scienceExp.fits
213 
214  Create a subclass of ImagePsfMatchTask that allows us to either match exposures, or subtract exposures:
215 
216  .. code-block:: none
217 
218  class MyImagePsfMatchTask(ImagePsfMatchTask):
219 
220  def __init__(self, args, kwargs):
221  ImagePsfMatchTask.__init__(self, args, kwargs)
222 
223  def run(self, templateExp, scienceExp, mode):
224  if mode == "matchExposures":
225  return self.matchExposures(templateExp, scienceExp)
226  elif mode == "subtractExposures":
227  return self.subtractExposures(templateExp, scienceExp)
228 
229  And allow the user the freedom to either run the script in default mode,
230  or point to their own images on disk.
231  Note that these images must be readable as an lsst.afw.image.Exposure.
232 
233  We have enabled some minor display debugging in this script via the --debug option. However, if you
234  have an lsstDebug debug.py in your PYTHONPATH you will get additional debugging displays. The following
235  block checks for this script:
236 
237  .. code-block:: py
238 
239  if args.debug:
240  try:
241  import debug
242  # Since I am displaying 2 images here, set the starting frame number for the LSST debug LSST
243  debug.lsstDebug.frame = 3
244  except ImportError as e:
245  print(e, file=sys.stderr)
246 
247  Finally, we call a run method that we define below.
248  First set up a Config and modify some of the parameters.
249  E.g. use an "Alard-Lupton" sum-of-Gaussian basis,
250  fit for a differential background, and use low order spatial
251  variation in the kernel and background:
252 
253  .. code-block:: py
254 
255  def run(args):
256  #
257  # Create the Config and use sum of gaussian basis
258  #
259  config = ImagePsfMatchTask.ConfigClass()
260  config.kernel.name = "AL"
261  config.kernel.active.fitForBackground = True
262  config.kernel.active.spatialKernelOrder = 1
263  config.kernel.active.spatialBgOrder = 0
264 
265  Make sure the images (if any) that were sent to the script exist on disk and are readable. If no images
266  are sent, make some fake data up for the sake of this example script (have a look at the code if you want
267  more details on generateFakeImages):
268 
269  .. code-block:: py
270 
271  # Run the requested method of the Task
272  if args.template is not None and args.science is not None:
273  if not os.path.isfile(args.template):
274  raise Exception("Template image %s does not exist" % (args.template))
275  if not os.path.isfile(args.science):
276  raise Exception("Science image %s does not exist" % (args.science))
277  try:
278  templateExp = afwImage.ExposureF(args.template)
279  except Exception as e:
280  raise Exception("Cannot read template image %s" % (args.template))
281  try:
282  scienceExp = afwImage.ExposureF(args.science)
283  except Exception as e:
284  raise Exception("Cannot read science image %s" % (args.science))
285  else:
286  templateExp, scienceExp = generateFakeImages()
287  config.kernel.active.sizeCellX = 128
288  config.kernel.active.sizeCellY = 128
289 
290  Create and run the Task:
291 
292  .. code-block:: py
293 
294  # Create the Task
295  psfMatchTask = MyImagePsfMatchTask(config=config)
296  # Run the Task
297  result = psfMatchTask.run(templateExp, scienceExp, args.mode)
298 
299  And finally provide some optional debugging displays:
300 
301  .. code-block:: py
302 
303  if args.debug:
304  # See if the LSST debug has incremented the frame number; if not start with frame 3
305  try:
306  frame = debug.lsstDebug.frame + 1
307  except Exception:
308  frame = 3
309  afwDisplay.Display(frame=frame).mtv(result.matchedExposure,
310  title="Example script: Matched Template Image")
311  if "subtractedExposure" in result.getDict():
312  afwDisplay.Display(frame=frame + 1).mtv(result.subtractedExposure,
313  title="Example script: Subtracted Image")
314  """
315 
316  ConfigClass = ImagePsfMatchConfig
317 
318  def __init__(self, *args, **kwargs):
319  """Create the ImagePsfMatchTask.
320  """
321  PsfMatchTask.__init__(self, *args, **kwargs)
322  self.kConfig = self.config.kernel.active
323  self._warper = afwMath.Warper.fromConfig(self.kConfig.warpingConfig)
324  # the background subtraction task uses a config from an unusual location,
325  # so cannot easily be constructed with makeSubtask
326  self.background = SubtractBackgroundTask(config=self.kConfig.afwBackgroundConfig, name="background",
327  parentTask=self)
328  self.selectSchema = afwTable.SourceTable.makeMinimalSchema()
330  self.makeSubtask("selectDetection", schema=self.selectSchema)
331  self.makeSubtask("selectMeasurement", schema=self.selectSchema, algMetadata=self.selectAlgMetadata)
332 
333  def getFwhmPix(self, psf):
334  """Return the FWHM in pixels of a Psf.
335  """
336  sigPix = psf.computeShape().getDeterminantRadius()
337  return sigPix*sigma2fwhm
338 
339  @pipeBase.timeMethod
340  def matchExposures(self, templateExposure, scienceExposure,
341  templateFwhmPix=None, scienceFwhmPix=None,
342  candidateList=None, doWarping=True, convolveTemplate=True):
343  """Warp and PSF-match an exposure to the reference.
344 
345  Do the following, in order:
346 
347  - Warp templateExposure to match scienceExposure,
348  if doWarping True and their WCSs do not already match
349  - Determine a PSF matching kernel and differential background model
350  that matches templateExposure to scienceExposure
351  - Convolve templateExposure by PSF matching kernel
352 
353  Parameters
354  ----------
355  templateExposure : `lsst.afw.image.Exposure`
356  Exposure to warp and PSF-match to the reference masked image
357  scienceExposure : `lsst.afw.image.Exposure`
358  Exposure whose WCS and PSF are to be matched to
359  templateFwhmPix :`float`
360  FWHM (in pixels) of the Psf in the template image (image to convolve)
361  scienceFwhmPix : `float`
362  FWHM (in pixels) of the Psf in the science image
363  candidateList : `list`, optional
364  a list of footprints/maskedImages for kernel candidates;
365  if `None` then source detection is run.
366 
367  - Currently supported: list of Footprints or measAlg.PsfCandidateF
368 
369  doWarping : `bool`
370  what to do if ``templateExposure`` and ``scienceExposure`` WCSs do not match:
371 
372  - if `True` then warp ``templateExposure`` to match ``scienceExposure``
373  - if `False` then raise an Exception
374 
375  convolveTemplate : `bool`
376  Whether to convolve the template image or the science image:
377 
378  - if `True`, ``templateExposure`` is warped if doWarping,
379  ``templateExposure`` is convolved
380  - if `False`, ``templateExposure`` is warped if doWarping,
381  ``scienceExposure`` is convolved
382 
383  Returns
384  -------
385  results : `lsst.pipe.base.Struct`
386  An `lsst.pipe.base.Struct` containing these fields:
387 
388  - ``matchedImage`` : the PSF-matched exposure =
389  Warped ``templateExposure`` convolved by psfMatchingKernel. This has:
390 
391  - the same parent bbox, Wcs and PhotoCalib as scienceExposure
392  - the same filter as templateExposure
393  - no Psf (because the PSF-matching process does not compute one)
394 
395  - ``psfMatchingKernel`` : the PSF matching kernel
396  - ``backgroundModel`` : differential background model
397  - ``kernelCellSet`` : SpatialCellSet used to solve for the PSF matching kernel
398 
399  Raises
400  ------
401  RuntimeError
402  Raised if doWarping is False and ``templateExposure`` and
403  ``scienceExposure`` WCSs do not match
404  """
405  if not self._validateWcs(templateExposure, scienceExposure):
406  if doWarping:
407  self.log.info("Astrometrically registering template to science image")
408  templatePsf = templateExposure.getPsf()
409  # Warp PSF before overwriting exposure
410  xyTransform = afwGeom.makeWcsPairTransform(templateExposure.getWcs(),
411  scienceExposure.getWcs())
412  psfWarped = WarpedPsf(templatePsf, xyTransform)
413  templateExposure = self._warper.warpExposure(scienceExposure.getWcs(),
414  templateExposure,
415  destBBox=scienceExposure.getBBox())
416  templateExposure.setPsf(psfWarped)
417  else:
418  self.log.error("ERROR: Input images not registered")
419  raise RuntimeError("Input images not registered")
420 
421  if templateFwhmPix is None:
422  if not templateExposure.hasPsf():
423  self.log.warn("No estimate of Psf FWHM for template image")
424  else:
425  templateFwhmPix = self.getFwhmPix(templateExposure.getPsf())
426  self.log.info("templateFwhmPix: {}".format(templateFwhmPix))
427 
428  if scienceFwhmPix is None:
429  if not scienceExposure.hasPsf():
430  self.log.warn("No estimate of Psf FWHM for science image")
431  else:
432  scienceFwhmPix = self.getFwhmPix(scienceExposure.getPsf())
433  self.log.info("scienceFwhmPix: {}".format(scienceFwhmPix))
434 
435  if convolveTemplate:
436  kernelSize = makeKernelBasisList(self.kConfig, templateFwhmPix, scienceFwhmPix)[0].getWidth()
437  candidateList = self.makeCandidateList(
438  templateExposure, scienceExposure, kernelSize, candidateList)
439  results = self.matchMaskedImages(
440  templateExposure.getMaskedImage(), scienceExposure.getMaskedImage(), candidateList,
441  templateFwhmPix=templateFwhmPix, scienceFwhmPix=scienceFwhmPix)
442  else:
443  kernelSize = makeKernelBasisList(self.kConfig, scienceFwhmPix, templateFwhmPix)[0].getWidth()
444  candidateList = self.makeCandidateList(
445  templateExposure, scienceExposure, kernelSize, candidateList)
446  results = self.matchMaskedImages(
447  scienceExposure.getMaskedImage(), templateExposure.getMaskedImage(), candidateList,
448  templateFwhmPix=scienceFwhmPix, scienceFwhmPix=templateFwhmPix)
449 
450  psfMatchedExposure = afwImage.makeExposure(results.matchedImage, scienceExposure.getWcs())
451  psfMatchedExposure.setFilter(templateExposure.getFilter())
452  psfMatchedExposure.setPhotoCalib(scienceExposure.getPhotoCalib())
453  results.warpedExposure = templateExposure
454  results.matchedExposure = psfMatchedExposure
455  return results
456 
457  @pipeBase.timeMethod
458  def matchMaskedImages(self, templateMaskedImage, scienceMaskedImage, candidateList,
459  templateFwhmPix=None, scienceFwhmPix=None):
460  """PSF-match a MaskedImage (templateMaskedImage) to a reference MaskedImage (scienceMaskedImage).
461 
462  Do the following, in order:
463 
464  - Determine a PSF matching kernel and differential background model
465  that matches templateMaskedImage to scienceMaskedImage
466  - Convolve templateMaskedImage by the PSF matching kernel
467 
468  Parameters
469  ----------
470  templateMaskedImage : `lsst.afw.image.MaskedImage`
471  masked image to PSF-match to the reference masked image;
472  must be warped to match the reference masked image
473  scienceMaskedImage : `lsst.afw.image.MaskedImage`
474  maskedImage whose PSF is to be matched to
475  templateFwhmPix : `float`
476  FWHM (in pixels) of the Psf in the template image (image to convolve)
477  scienceFwhmPix : `float`
478  FWHM (in pixels) of the Psf in the science image
479  candidateList : `list`, optional
480  A list of footprints/maskedImages for kernel candidates;
481  if `None` then source detection is run.
482 
483  - Currently supported: list of Footprints or measAlg.PsfCandidateF
484 
485  Returns
486  -------
487  result : `callable`
488  An `lsst.pipe.base.Struct` containing these fields:
489 
490  - psfMatchedMaskedImage: the PSF-matched masked image =
491  ``templateMaskedImage`` convolved with psfMatchingKernel.
492  This has the same xy0, dimensions and wcs as ``scienceMaskedImage``.
493  - psfMatchingKernel: the PSF matching kernel
494  - backgroundModel: differential background model
495  - kernelCellSet: SpatialCellSet used to solve for the PSF matching kernel
496 
497  Raises
498  ------
499  RuntimeError
500  Raised if input images have different dimensions
501  """
502  import lsstDebug
503  display = lsstDebug.Info(__name__).display
504  displayTemplate = lsstDebug.Info(__name__).displayTemplate
505  displaySciIm = lsstDebug.Info(__name__).displaySciIm
506  displaySpatialCells = lsstDebug.Info(__name__).displaySpatialCells
507  maskTransparency = lsstDebug.Info(__name__).maskTransparency
508  if not maskTransparency:
509  maskTransparency = 0
510  if display:
511  afwDisplay.setDefaultMaskTransparency(maskTransparency)
512 
513  if not candidateList:
514  raise RuntimeError("Candidate list must be populated by makeCandidateList")
515 
516  if not self._validateSize(templateMaskedImage, scienceMaskedImage):
517  self.log.error("ERROR: Input images different size")
518  raise RuntimeError("Input images different size")
519 
520  if display and displayTemplate:
521  disp = afwDisplay.Display(frame=lsstDebug.frame)
522  disp.mtv(templateMaskedImage, title="Image to convolve")
523  lsstDebug.frame += 1
524 
525  if display and displaySciIm:
526  disp = afwDisplay.Display(frame=lsstDebug.frame)
527  disp.mtv(scienceMaskedImage, title="Image to not convolve")
528  lsstDebug.frame += 1
529 
530  kernelCellSet = self._buildCellSet(templateMaskedImage,
531  scienceMaskedImage,
532  candidateList)
533 
534  if display and displaySpatialCells:
535  diffimUtils.showKernelSpatialCells(scienceMaskedImage, kernelCellSet,
536  symb="o", ctype=afwDisplay.CYAN, ctypeUnused=afwDisplay.YELLOW,
537  ctypeBad=afwDisplay.RED, size=4, frame=lsstDebug.frame,
538  title="Image to not convolve")
539  lsstDebug.frame += 1
540 
541  if templateFwhmPix and scienceFwhmPix:
542  self.log.info("Matching Psf FWHM %.2f -> %.2f pix", templateFwhmPix, scienceFwhmPix)
543 
544  if self.kConfig.useBicForKernelBasis:
545  tmpKernelCellSet = self._buildCellSet(templateMaskedImage,
546  scienceMaskedImage,
547  candidateList)
548  nbe = diffimTools.NbasisEvaluator(self.kConfig, templateFwhmPix, scienceFwhmPix)
549  bicDegrees = nbe(tmpKernelCellSet, self.log)
550  basisList = makeKernelBasisList(self.kConfig, templateFwhmPix, scienceFwhmPix,
551  alardDegGauss=bicDegrees[0], metadata=self.metadata)
552  del tmpKernelCellSet
553  else:
554  basisList = makeKernelBasisList(self.kConfig, templateFwhmPix, scienceFwhmPix,
555  metadata=self.metadata)
556 
557  spatialSolution, psfMatchingKernel, backgroundModel = self._solve(kernelCellSet, basisList)
558 
559  psfMatchedMaskedImage = afwImage.MaskedImageF(templateMaskedImage.getBBox())
560  doNormalize = False
561  afwMath.convolve(psfMatchedMaskedImage, templateMaskedImage, psfMatchingKernel, doNormalize)
562  return pipeBase.Struct(
563  matchedImage=psfMatchedMaskedImage,
564  psfMatchingKernel=psfMatchingKernel,
565  backgroundModel=backgroundModel,
566  kernelCellSet=kernelCellSet,
567  )
568 
569  @pipeBase.timeMethod
570  def subtractExposures(self, templateExposure, scienceExposure,
571  templateFwhmPix=None, scienceFwhmPix=None,
572  candidateList=None, doWarping=True, convolveTemplate=True):
573  """Register, Psf-match and subtract two Exposures.
574 
575  Do the following, in order:
576 
577  - Warp templateExposure to match scienceExposure, if their WCSs do not already match
578  - Determine a PSF matching kernel and differential background model
579  that matches templateExposure to scienceExposure
580  - PSF-match templateExposure to scienceExposure
581  - Compute subtracted exposure (see return values for equation).
582 
583  Parameters
584  ----------
585  templateExposure : `lsst.afw.image.Exposure`
586  Exposure to PSF-match to scienceExposure
587  scienceExposure : `lsst.afw.image.Exposure`
588  Reference Exposure
589  templateFwhmPix : `float`
590  FWHM (in pixels) of the Psf in the template image (image to convolve)
591  scienceFwhmPix : `float`
592  FWHM (in pixels) of the Psf in the science image
593  candidateList : `list`, optional
594  A list of footprints/maskedImages for kernel candidates;
595  if `None` then source detection is run.
596 
597  - Currently supported: list of Footprints or measAlg.PsfCandidateF
598 
599  doWarping : `bool`
600  What to do if ``templateExposure``` and ``scienceExposure`` WCSs do
601  not match:
602 
603  - if `True` then warp ``templateExposure`` to match ``scienceExposure``
604  - if `False` then raise an Exception
605 
606  convolveTemplate : `bool`
607  Convolve the template image or the science image
608 
609  - if `True`, ``templateExposure`` is warped if doWarping,
610  ``templateExposure`` is convolved
611  - if `False`, ``templateExposure`` is warped if doWarping,
612  ``scienceExposure is`` convolved
613 
614  Returns
615  -------
616  result : `lsst.pipe.base.Struct`
617  An `lsst.pipe.base.Struct` containing these fields:
618 
619  - ``subtractedExposure`` : subtracted Exposure
620  scienceExposure - (matchedImage + backgroundModel)
621  - ``matchedImage`` : ``templateExposure`` after warping to match
622  ``templateExposure`` (if doWarping true),
623  and convolving with psfMatchingKernel
624  - ``psfMatchingKernel`` : PSF matching kernel
625  - ``backgroundModel`` : differential background model
626  - ``kernelCellSet`` : SpatialCellSet used to determine PSF matching kernel
627  """
628  results = self.matchExposures(
629  templateExposure=templateExposure,
630  scienceExposure=scienceExposure,
631  templateFwhmPix=templateFwhmPix,
632  scienceFwhmPix=scienceFwhmPix,
633  candidateList=candidateList,
634  doWarping=doWarping,
635  convolveTemplate=convolveTemplate
636  )
637 
638  subtractedExposure = afwImage.ExposureF(scienceExposure, True)
639  if convolveTemplate:
640  subtractedMaskedImage = subtractedExposure.getMaskedImage()
641  subtractedMaskedImage -= results.matchedExposure.getMaskedImage()
642  subtractedMaskedImage -= results.backgroundModel
643  else:
644  subtractedExposure.setMaskedImage(results.warpedExposure.getMaskedImage())
645  subtractedMaskedImage = subtractedExposure.getMaskedImage()
646  subtractedMaskedImage -= results.matchedExposure.getMaskedImage()
647  subtractedMaskedImage -= results.backgroundModel
648 
649  # Preserve polarity of differences
650  subtractedMaskedImage *= -1
651 
652  # Place back on native photometric scale
653  subtractedMaskedImage /= results.psfMatchingKernel.computeImage(
654  afwImage.ImageD(results.psfMatchingKernel.getDimensions()), False)
655 
656  import lsstDebug
657  display = lsstDebug.Info(__name__).display
658  displayDiffIm = lsstDebug.Info(__name__).displayDiffIm
659  maskTransparency = lsstDebug.Info(__name__).maskTransparency
660  if not maskTransparency:
661  maskTransparency = 0
662  if display:
663  afwDisplay.setDefaultMaskTransparency(maskTransparency)
664  if display and displayDiffIm:
665  disp = afwDisplay.Display(frame=lsstDebug.frame)
666  disp.mtv(templateExposure, title="Template")
667  lsstDebug.frame += 1
668  disp = afwDisplay.Display(frame=lsstDebug.frame)
669  disp.mtv(results.matchedExposure, title="Matched template")
670  lsstDebug.frame += 1
671  disp = afwDisplay.Display(frame=lsstDebug.frame)
672  disp.mtv(scienceExposure, title="Science Image")
673  lsstDebug.frame += 1
674  disp = afwDisplay.Display(frame=lsstDebug.frame)
675  disp.mtv(subtractedExposure, title="Difference Image")
676  lsstDebug.frame += 1
677 
678  results.subtractedExposure = subtractedExposure
679  return results
680 
681  @pipeBase.timeMethod
682  def subtractMaskedImages(self, templateMaskedImage, scienceMaskedImage, candidateList,
683  templateFwhmPix=None, scienceFwhmPix=None):
684  """Psf-match and subtract two MaskedImages.
685 
686  Do the following, in order:
687 
688  - PSF-match templateMaskedImage to scienceMaskedImage
689  - Determine the differential background
690  - Return the difference: scienceMaskedImage
691  ((warped templateMaskedImage convolved with psfMatchingKernel) + backgroundModel)
692 
693  Parameters
694  ----------
695  templateMaskedImage : `lsst.afw.image.MaskedImage`
696  MaskedImage to PSF-match to ``scienceMaskedImage``
697  scienceMaskedImage : `lsst.afw.image.MaskedImage`
698  Reference MaskedImage
699  templateFwhmPix : `float`
700  FWHM (in pixels) of the Psf in the template image (image to convolve)
701  scienceFwhmPix : `float`
702  FWHM (in pixels) of the Psf in the science image
703  candidateList : `list`, optional
704  A list of footprints/maskedImages for kernel candidates;
705  if `None` then source detection is run.
706 
707  - Currently supported: list of Footprints or measAlg.PsfCandidateF
708 
709  Returns
710  -------
711  results : `lsst.pipe.base.Struct`
712  An `lsst.pipe.base.Struct` containing these fields:
713 
714  - ``subtractedMaskedImage`` : ``scienceMaskedImage`` - (matchedImage + backgroundModel)
715  - ``matchedImage`` : templateMaskedImage convolved with psfMatchingKernel
716  - `psfMatchingKernel`` : PSF matching kernel
717  - ``backgroundModel`` : differential background model
718  - ``kernelCellSet`` : SpatialCellSet used to determine PSF matching kernel
719 
720  """
721  if not candidateList:
722  raise RuntimeError("Candidate list must be populated by makeCandidateList")
723 
724  results = self.matchMaskedImages(
725  templateMaskedImage=templateMaskedImage,
726  scienceMaskedImage=scienceMaskedImage,
727  candidateList=candidateList,
728  templateFwhmPix=templateFwhmPix,
729  scienceFwhmPix=scienceFwhmPix,
730  )
731 
732  subtractedMaskedImage = afwImage.MaskedImageF(scienceMaskedImage, True)
733  subtractedMaskedImage -= results.matchedImage
734  subtractedMaskedImage -= results.backgroundModel
735  results.subtractedMaskedImage = subtractedMaskedImage
736 
737  import lsstDebug
738  display = lsstDebug.Info(__name__).display
739  displayDiffIm = lsstDebug.Info(__name__).displayDiffIm
740  maskTransparency = lsstDebug.Info(__name__).maskTransparency
741  if not maskTransparency:
742  maskTransparency = 0
743  if display:
744  afwDisplay.setDefaultMaskTransparency(maskTransparency)
745  if display and displayDiffIm:
746  disp = afwDisplay.Display(frame=lsstDebug.frame)
747  disp.mtv(subtractedMaskedImage, title="Subtracted masked image")
748  lsstDebug.frame += 1
749 
750  return results
751 
752  def getSelectSources(self, exposure, sigma=None, doSmooth=True, idFactory=None):
753  """Get sources to use for Psf-matching.
754 
755  This method runs detection and measurement on an exposure.
756  The returned set of sources will be used as candidates for
757  Psf-matching.
758 
759  Parameters
760  ----------
761  exposure : `lsst.afw.image.Exposure`
762  Exposure on which to run detection/measurement
763  sigma : `float`
764  Detection threshold
765  doSmooth : `bool`
766  Whether or not to smooth the Exposure with Psf before detection
767  idFactory :
768  Factory for the generation of Source ids
769 
770  Returns
771  -------
772  selectSources :
773  source catalog containing candidates for the Psf-matching
774  """
775  if idFactory:
776  table = afwTable.SourceTable.make(self.selectSchema, idFactory)
777  else:
778  table = afwTable.SourceTable.make(self.selectSchema)
779  mi = exposure.getMaskedImage()
780 
781  imArr = mi.getImage().getArray()
782  maskArr = mi.getMask().getArray()
783  miArr = np.ma.masked_array(imArr, mask=maskArr)
784  try:
785  bkgd = self.background.fitBackground(mi).getImageF()
786  except Exception:
787  self.log.warn("Failed to get background model. Falling back to median background estimation")
788  bkgd = np.ma.extras.median(miArr)
789 
790  # Take off background for detection
791  mi -= bkgd
792  try:
793  table.setMetadata(self.selectAlgMetadata)
794  detRet = self.selectDetection.makeSourceCatalog(
795  table=table,
796  exposure=exposure,
797  sigma=sigma,
798  doSmooth=doSmooth
799  )
800  selectSources = detRet.sources
801  self.selectMeasurement.run(measCat=selectSources, exposure=exposure)
802  finally:
803  # Put back on the background in case it is needed down stream
804  mi += bkgd
805  del bkgd
806  return selectSources
807 
808  def makeCandidateList(self, templateExposure, scienceExposure, kernelSize, candidateList=None):
809  """Make a list of acceptable KernelCandidates.
810 
811  Accept or generate a list of candidate sources for
812  Psf-matching, and examine the Mask planes in both of the
813  images for indications of bad pixels
814 
815  Parameters
816  ----------
817  templateExposure : `lsst.afw.image.Exposure`
818  Exposure that will be convolved
819  scienceExposure : `lsst.afw.image.Exposure`
820  Exposure that will be matched-to
821  kernelSize : `float`
822  Dimensions of the Psf-matching Kernel, used to grow detection footprints
823  candidateList : `list`, optional
824  List of Sources to examine. Elements must be of type afw.table.Source
825  or a type that wraps a Source and has a getSource() method, such as
826  meas.algorithms.PsfCandidateF.
827 
828  Returns
829  -------
830  candidateList : `list` of `dict`
831  A list of dicts having a "source" and "footprint"
832  field for the Sources deemed to be appropriate for Psf
833  matching
834  """
835  if candidateList is None:
836  candidateList = self.getSelectSources(scienceExposure)
837 
838  if len(candidateList) < 1:
839  raise RuntimeError("No candidates in candidateList")
840 
841  listTypes = set(type(x) for x in candidateList)
842  if len(listTypes) > 1:
843  raise RuntimeError("Candidate list contains mixed types: %s" % [l for l in listTypes])
844 
845  if not isinstance(candidateList[0], afwTable.SourceRecord):
846  try:
847  candidateList[0].getSource()
848  except Exception as e:
849  raise RuntimeError("Candidate List is of type: %s. " % (type(candidateList[0])) +
850  "Can only make candidate list from list of afwTable.SourceRecords, " +
851  "measAlg.PsfCandidateF or other type with a getSource() method: %s" % (e))
852  candidateList = [c.getSource() for c in candidateList]
853 
854  candidateList = diffimTools.sourceToFootprintList(candidateList,
855  templateExposure, scienceExposure,
856  kernelSize,
857  self.kConfig.detectionConfig,
858  self.log)
859  if len(candidateList) == 0:
860  raise RuntimeError("Cannot find any objects suitable for KernelCandidacy")
861 
862  return candidateList
863 
864  def _adaptCellSize(self, candidateList):
865  """NOT IMPLEMENTED YET.
866  """
867  return self.kConfig.sizeCellX, self.kConfig.sizeCellY
868 
869  def _buildCellSet(self, templateMaskedImage, scienceMaskedImage, candidateList):
870  """Build a SpatialCellSet for use with the solve method.
871 
872  Parameters
873  ----------
874  templateMaskedImage : `lsst.afw.image.MaskedImage`
875  MaskedImage to PSF-matched to scienceMaskedImage
876  scienceMaskedImage : `lsst.afw.image.MaskedImage`
877  Reference MaskedImage
878  candidateList : `list`
879  A list of footprints/maskedImages for kernel candidates;
880 
881  - Currently supported: list of Footprints or measAlg.PsfCandidateF
882 
883  Returns
884  -------
885  kernelCellSet : `lsst.afw.math.SpatialCellSet`
886  a SpatialCellSet for use with self._solve
887  """
888  if not candidateList:
889  raise RuntimeError("Candidate list must be populated by makeCandidateList")
890 
891  sizeCellX, sizeCellY = self._adaptCellSize(candidateList)
892 
893  # Object to store the KernelCandidates for spatial modeling
894  kernelCellSet = afwMath.SpatialCellSet(templateMaskedImage.getBBox(),
895  sizeCellX, sizeCellY)
896 
897  ps = pexConfig.makePropertySet(self.kConfig)
898  # Place candidates within the spatial grid
899  for cand in candidateList:
900  if isinstance(cand, afwDetect.Footprint):
901  bbox = cand.getBBox()
902  else:
903  bbox = cand['footprint'].getBBox()
904  tmi = afwImage.MaskedImageF(templateMaskedImage, bbox)
905  smi = afwImage.MaskedImageF(scienceMaskedImage, bbox)
906 
907  if not isinstance(cand, afwDetect.Footprint):
908  if 'source' in cand:
909  cand = cand['source']
910  xPos = cand.getCentroid()[0]
911  yPos = cand.getCentroid()[1]
912  cand = diffimLib.makeKernelCandidate(xPos, yPos, tmi, smi, ps)
913 
914  self.log.debug("Candidate %d at %f, %f", cand.getId(), cand.getXCenter(), cand.getYCenter())
915  kernelCellSet.insertCandidate(cand)
916 
917  return kernelCellSet
918 
919  def _validateSize(self, templateMaskedImage, scienceMaskedImage):
920  """Return True if two image-like objects are the same size.
921  """
922  return templateMaskedImage.getDimensions() == scienceMaskedImage.getDimensions()
923 
924  def _validateWcs(self, templateExposure, scienceExposure):
925  """Return True if the WCS of the two Exposures have the same origin and extent.
926  """
927  templateWcs = templateExposure.getWcs()
928  scienceWcs = scienceExposure.getWcs()
929  templateBBox = templateExposure.getBBox()
930  scienceBBox = scienceExposure.getBBox()
931 
932  # LLC
933  templateOrigin = templateWcs.pixelToSky(geom.Point2D(templateBBox.getBegin()))
934  scienceOrigin = scienceWcs.pixelToSky(geom.Point2D(scienceBBox.getBegin()))
935 
936  # URC
937  templateLimit = templateWcs.pixelToSky(geom.Point2D(templateBBox.getEnd()))
938  scienceLimit = scienceWcs.pixelToSky(geom.Point2D(scienceBBox.getEnd()))
939 
940  self.log.info("Template Wcs : %f,%f -> %f,%f",
941  templateOrigin[0], templateOrigin[1],
942  templateLimit[0], templateLimit[1])
943  self.log.info("Science Wcs : %f,%f -> %f,%f",
944  scienceOrigin[0], scienceOrigin[1],
945  scienceLimit[0], scienceLimit[1])
946 
947  templateBBox = geom.Box2D(templateOrigin.getPosition(geom.degrees),
948  templateLimit.getPosition(geom.degrees))
949  scienceBBox = geom.Box2D(scienceOrigin.getPosition(geom.degrees),
950  scienceLimit.getPosition(geom.degrees))
951  if not (templateBBox.overlaps(scienceBBox)):
952  raise RuntimeError("Input images do not overlap at all")
953 
954  if ((templateOrigin != scienceOrigin) or
955  (templateLimit != scienceLimit) or
956  (templateExposure.getDimensions() != scienceExposure.getDimensions())):
957  return False
958  return True
959 
960 
961 subtractAlgorithmRegistry = pexConfig.makeRegistry(
962  doc="A registry of subtraction algorithms for use as a subtask in imageDifference",
963 )
964 
965 subtractAlgorithmRegistry.register('al', ImagePsfMatchTask)
def format(config, name=None, writeSourceLine=True, prefix="", verbose=False)
Definition: history.py:174
A floating-point coordinate rectangle geometry.
Definition: Box.h:413
Class for storing ordered metadata with comments.
Definition: PropertyList.h:68
def makeKernelBasisList(config, targetFwhmPix=None, referenceFwhmPix=None, basisDegGauss=None, metadata=None)
def getSelectSources(self, exposure, sigma=None, doSmooth=True, idFactory=None)
def _validateSize(self, templateMaskedImage, scienceMaskedImage)
Fit spatial kernel using approximate fluxes for candidates, and solving a linear system of equations...
daf::base::PropertySet * set
Definition: fits.cc:902
def _buildCellSet(self, templateMaskedImage, scienceMaskedImage, candidateList)
std::shared_ptr< Exposure< ImagePixelT, MaskPixelT, VariancePixelT > > makeExposure(MaskedImage< ImagePixelT, MaskPixelT, VariancePixelT > &mimage, std::shared_ptr< geom::SkyWcs const > wcs=std::shared_ptr< geom::SkyWcs const >())
A function to return an Exposure of the correct type (cf.
Definition: Exposure.h:442
A collection of SpatialCells covering an entire image.
Definition: SpatialCell.h:387
def matchMaskedImages(self, templateMaskedImage, scienceMaskedImage, candidateList, templateFwhmPix=None, scienceFwhmPix=None)
table::Key< int > type
Definition: Detector.cc:163
def _solve(self, kernelCellSet, basisList, returnOnExcept=False)
Definition: psfMatch.py:880
def matchExposures(self, templateExposure, scienceExposure, templateFwhmPix=None, scienceFwhmPix=None, candidateList=None, doWarping=True, convolveTemplate=True)
def makeCandidateList(self, templateExposure, scienceExposure, kernelSize, candidateList=None)
Class to describe the properties of a detected object from an image.
Definition: Footprint.h:62
std::shared_ptr< TransformPoint2ToPoint2 > makeWcsPairTransform(SkyWcs const &src, SkyWcs const &dst)
A Transform obtained by putting two SkyWcs objects "back to back".
Definition: SkyWcs.cc:151
def run(self, skyInfo, tempExpRefList, imageScalerList, weightList, altMaskList=None, mask=None, supplementaryData=None)
def subtractMaskedImages(self, templateMaskedImage, scienceMaskedImage, candidateList, templateFwhmPix=None, scienceFwhmPix=None)
Record class that contains measurements made on a single exposure.
Definition: Source.h:82
void convolve(OutImageT &convolvedImage, InImageT const &inImage, KernelT const &kernel, bool doNormalize, bool doCopyEdge=false)
Old, deprecated version of convolve.
Backwards-compatibility support for depersisting the old Calib (FluxMag0/FluxMag0Err) objects...
def subtractExposures(self, templateExposure, scienceExposure, templateFwhmPix=None, scienceFwhmPix=None, candidateList=None, doWarping=True, convolveTemplate=True)
def _validateWcs(self, templateExposure, scienceExposure)
A Psf class that maps an arbitrary Psf through a coordinate transformation.
Definition: WarpedPsf.h:50