LSST Applications  21.0.0+04719a4bac,21.0.0-1-ga51b5d4+f5e6047307,21.0.0-11-g2b59f77+a9c1acf22d,21.0.0-11-ga42c5b2+86977b0b17,21.0.0-12-gf4ce030+76814010d2,21.0.0-13-g1721dae+760e7a6536,21.0.0-13-g3a573fe+768d78a30a,21.0.0-15-g5a7caf0+f21cbc5713,21.0.0-16-g0fb55c1+b60e2d390c,21.0.0-19-g4cded4ca+71a93a33c0,21.0.0-2-g103fe59+bb20972958,21.0.0-2-g45278ab+04719a4bac,21.0.0-2-g5242d73+3ad5d60fb1,21.0.0-2-g7f82c8f+8babb168e8,21.0.0-2-g8f08a60+06509c8b61,21.0.0-2-g8faa9b5+616205b9df,21.0.0-2-ga326454+8babb168e8,21.0.0-2-gde069b7+5e4aea9c2f,21.0.0-2-gecfae73+1d3a86e577,21.0.0-2-gfc62afb+3ad5d60fb1,21.0.0-25-g1d57be3cd+e73869a214,21.0.0-3-g357aad2+ed88757d29,21.0.0-3-g4a4ce7f+3ad5d60fb1,21.0.0-3-g4be5c26+3ad5d60fb1,21.0.0-3-g65f322c+e0b24896a3,21.0.0-3-g7d9da8d+616205b9df,21.0.0-3-ge02ed75+a9c1acf22d,21.0.0-4-g591bb35+a9c1acf22d,21.0.0-4-g65b4814+b60e2d390c,21.0.0-4-gccdca77+0de219a2bc,21.0.0-4-ge8a399c+6c55c39e83,21.0.0-5-gd00fb1e+05fce91b99,21.0.0-6-gc675373+3ad5d60fb1,21.0.0-64-g1122c245+4fb2b8f86e,21.0.0-7-g04766d7+cd19d05db2,21.0.0-7-gdf92d54+04719a4bac,21.0.0-8-g5674e7b+d1bd76f71f,master-gac4afde19b+a9c1acf22d,w.2021.13
LSST Data Management Base Package
repair.py
Go to the documentation of this file.
1 # This file is part of pipe_tasks.
2 #
3 # Developed for the LSST Data Management System.
4 # This product includes software developed by the LSST Project
5 # (https://www.lsst.org).
6 # See the COPYRIGHT file at the top-level directory of this distribution
7 # for details of code ownership.
8 #
9 # This program is free software: you can redistribute it and/or modify
10 # it under the terms of the GNU General Public License as published by
11 # the Free Software Foundation, either version 3 of the License, or
12 # (at your option) any later version.
13 #
14 # This program is distributed in the hope that it will be useful,
15 # but WITHOUT ANY WARRANTY; without even the implied warranty of
16 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
17 # GNU General Public License for more details.
18 #
19 # You should have received a copy of the GNU General Public License
20 # along with this program. If not, see <https://www.gnu.org/licenses/>.
21 
22 import lsst.pex.config as pexConfig
23 import lsst.afw.math as afwMath
24 import lsst.afw.detection as afwDet
25 import lsst.meas.algorithms as measAlg
26 import lsst.pipe.base as pipeBase
27 from lsstDebug import getDebugFrame
28 import lsst.afw.display as afwDisplay
29 from lsst.pipe.tasks.interpImage import InterpImageTask
30 
31 
32 class RepairConfig(pexConfig.Config):
33  doInterpolate = pexConfig.Field(
34  dtype=bool,
35  doc="Interpolate over defects? (ignored unless you provide a list of defects)",
36  default=True,
37  )
38  doCosmicRay = pexConfig.Field(
39  dtype=bool,
40  doc="Find and mask out cosmic rays?",
41  default=True,
42  )
43  cosmicray = pexConfig.ConfigField(
44  dtype=measAlg.FindCosmicRaysConfig,
45  doc="Options for finding and masking cosmic rays",
46  )
47  interp = pexConfig.ConfigurableField(
48  target=InterpImageTask,
49  doc="Interpolate over bad image pixels",
50  )
51 
52  def setDefaults(self):
53  self.interpinterp.useFallbackValueAtEdge = True
54  self.interpinterp.fallbackValueType = "MEANCLIP"
55  self.interpinterp.negativeFallbackAllowed = True
56 
57 
63 
64 
65 class RepairTask(pipeBase.Task):
66  r"""!
67  @anchor RepairTask_
68 
69  @brief Interpolate over defects in an exposure and handle cosmic rays
70 
71  @section pipe_tasks_repair_Contents Contents
72 
73  - @ref pipe_tasks_repair_Purpose
74  - @ref pipe_tasks_repair_Initialize
75  - @ref pipe_tasks_repair_IO
76  - @ref pipe_tasks_repair_Config
77  - @ref pipe_tasks_repair_Debug
78  - @ref pipe_tasks_repair_Example
79 
80  @section pipe_tasks_repair_Purpose Description
81 
82  @copybrief RepairTask
83 
84  This task operates on an lsst.afw.image.Exposure in place to interpolate over a set of
85  lsst.meas.algorithms.Defect objects.
86  It will also, optionally, find and interpolate any cosmic rays in the lsst.afw.image.Exposure.
87 
88  @section pipe_tasks_repair_Initialize Task initialization
89 
90  See: lsst.pipe.base.task.Task.__init__
91 
92  @section pipe_tasks_repair_IO Inputs/Outputs to the run method
93 
94  @copydoc run
95 
96  @section pipe_tasks_repair_Config Configuration parameters
97 
98  See @ref RepairConfig
99 
100  @section pipe_tasks_repair_Debug Debug variables
101 
102  The @link lsst.pipe.base.cmdLineTask.CmdLineTask command line task@endlink interface supports a
103  flag @c -d to import @b debug.py from your @c PYTHONPATH; see <a
104  href="https://developer.lsst.io/stack/debug.html">Debugging Tasks with lsstDebug</a> for more
105  about @b debug.py files.
106 
107  The available variables in RepairTask are:
108  <DL>
109  <DT> @c display
110  <DD> A dictionary containing debug point names as keys with frame number as value. Valid keys are:
111  <DL>
112  <DT> repair.before
113  <DD> display image before any repair is done
114  <DT> repair.after
115  <DD> display image after cosmic ray and defect correction
116  </DL>
117  <DT> @c displayCR
118  <DD> If True, display the exposure on display's frame 1 and overlay bounding boxes around detects CRs.
119  </DL>
120  @section pipe_tasks_repair_Example A complete example of using RepairTask
121 
122  This code is in runRepair.py in the examples directory, and can be run as @em e.g.
123  @code
124  examples/runRepair.py
125  @endcode
126  @dontinclude runRepair.py
127  Import the task. There are other imports. Read the source file for more info.
128  @skipline RepairTask
129 
130  For this example, we manufacture a test image to run on.
131 
132  First, create a pure Poisson noise image and a Psf to go with it. The mask plane
133  and variance can be constructed at the same time.
134  @skip poisson
135  @until mask
136 
137  Inject some cosmic rays and generate the Exposure. Exposures are MaskedImages (image + mask + variance)
138  with other metadata (e.g. Psf and Wcs objects).
139  @skip some CRs
140  @until setPsf
141 
142  Defects are represented as bad columns of random lengths. A defect list must be constructed to pass
143  on to the RepairTask.
144  @bug This is addressed in <a href="https://jira.lsstcorp.org/browse/DM-963"> DM-963</a>
145 
146  @skip addDefects
147  @until push_back
148 
149  Finally, the exposure can be repaired. Create an instance of the task and run it. The exposure
150  is modified in place.
151  @skip RepairTask
152  @until repair.run
153 
154  <HR>
155  To investigate the @ref pipe_tasks_repair_Debug, put something like
156  @code{.py}
157  import lsstDebug
158  def DebugInfo(name):
159  di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively
160  if name == "lsst.pipe.tasks.repair":
161  di.display = {'repair.before':2, 'repair.after':3}
162  di.displayCR = True
163  return di
164 
165  lsstDebug.Info = DebugInfo
166  @endcode
167  into your debug.py file and run runRepair.py with the @c --debug flag.
168 
169 
170  Conversion notes:
171  Display code should be updated once we settle on a standard way of controlling what is displayed.
172  """
173  ConfigClass = RepairConfig
174  _DefaultName = "repair"
175 
176  def __init__(self, **kwargs):
177  pipeBase.Task.__init__(self, **kwargs)
178  if self.config.doInterpolate:
179  self.makeSubtask("interp")
180 
181  @pipeBase.timeMethod
182  def run(self, exposure, defects=None, keepCRs=None):
183  """!Repair an Exposure's defects and cosmic rays
184 
185  @param[in, out] exposure lsst.afw.image.Exposure to process. Exposure must have a valid Psf.
186  Modified in place.
187  @param[in] defects an lsst.meas.algorithms.DefectListT object. If None, do no
188  defect correction.
189  @param[in] keepCRs don't interpolate over the CR pixels (defer to RepairConfig if None)
190 
191  @throws AssertionError with the following strings:
192 
193  <DL>
194  <DT> No exposure provided
195  <DD> The object provided as exposure evaluates to False
196  <DT> No PSF provided
197  <DD> The Exposure has no associated Psf
198  </DL>
199  """
200  assert exposure, "No exposure provided"
201  psf = exposure.getPsf()
202  assert psf, "No PSF provided"
203 
204  frame = getDebugFrame(self._display, "repair.before")
205  if frame:
206  afwDisplay.Display(frame).mtv(exposure)
207 
208  if defects is not None and self.config.doInterpolate:
209  self.interp.run(exposure, defects=defects)
210 
211  if self.config.doCosmicRay:
212  self.cosmicRaycosmicRay(exposure, keepCRs=keepCRs)
213 
214  frame = getDebugFrame(self._display, "repair.after")
215  if frame:
216  afwDisplay.Display(frame).mtv(exposure)
217 
218  def cosmicRay(self, exposure, keepCRs=None):
219  """Mask cosmic rays
220 
221  @param[in,out] exposure Exposure to process
222  @param[in] keepCRs Don't interpolate over the CR pixels (defer to pex_config if None)
223  """
224  import lsstDebug
225  display = lsstDebug.Info(__name__).display
226  displayCR = lsstDebug.Info(__name__).displayCR
227 
228  assert exposure, "No exposure provided"
229  psf = exposure.getPsf()
230  assert psf, "No psf provided"
231 
232  # Blow away old mask
233  try:
234  mask = exposure.getMaskedImage().getMask()
235  crBit = mask.getMaskPlane("CR")
236  mask.clearMaskPlane(crBit)
237  except Exception:
238  pass
239 
240  exposure0 = exposure # initial value of exposure
241  binSize = self.config.cosmicray.background.binSize
242  nx, ny = exposure.getWidth()/binSize, exposure.getHeight()/binSize
243  # Treat constant background as a special case to avoid the extra complexity in calling
244  # measAlg.SubtractBackgroundTask().
245  if nx*ny <= 1:
246  medianBg = afwMath.makeStatistics(exposure.getMaskedImage(), afwMath.MEDIAN).getValue()
247  modelBg = None
248  else:
249  # make a deep copy of the exposure before subtracting its background,
250  # because this routine is only allowed to modify the exposure by setting mask planes
251  # and interpolating over defects, not changing the background level
252  exposure = exposure.Factory(exposure, True)
253  subtractBackgroundTask = measAlg.SubtractBackgroundTask(config=self.config.cosmicray.background)
254  modelBg = subtractBackgroundTask.run(exposure).background
255  medianBg = 0.0
256 
257  if keepCRs is None:
258  keepCRs = self.config.cosmicray.keepCRs
259  try:
260  crs = measAlg.findCosmicRays(exposure.getMaskedImage(), psf, medianBg,
261  pexConfig.makePropertySet(self.config.cosmicray), keepCRs)
262  if modelBg:
263  # Add back background image
264  img = exposure.getMaskedImage()
265  img += modelBg.getImageF()
266  del img
267  # Replace original image with CR subtracted image
268  exposure0.setMaskedImage(exposure.getMaskedImage())
269 
270  except Exception:
271  if display:
272  afwDisplay.Display().mtv(exposure0, title="Failed CR")
273  raise
274 
275  num = 0
276  if crs is not None:
277  mask = exposure0.getMaskedImage().getMask()
278  crBit = mask.getPlaneBitMask("CR")
279  afwDet.setMaskFromFootprintList(mask, crs, crBit)
280  num = len(crs)
281 
282  if display and displayCR:
283  disp = afwDisplay.Display()
284  disp.incrDefaultFrame()
285  disp.mtv(exposure0, title="Post-CR")
286 
287  with disp.Buffering():
288  for cr in crs:
289  afwDisplay.utils.drawBBox(cr.getBBox(), borderWidth=0.55)
290 
291  self.log.info("Identified %s cosmic rays." % (num,))
def __init__(self, **kwargs)
Definition: repair.py:176
def run(self, exposure, defects=None, keepCRs=None)
Repair an Exposure's defects and cosmic rays.
Definition: repair.py:182
def cosmicRay(self, exposure, keepCRs=None)
Definition: repair.py:218
def mtv(data, frame=None, title="", wcs=None, *args, **kwargs)
Definition: ds9.py:92
Statistics makeStatistics(lsst::afw::image::Image< Pixel > const &img, lsst::afw::image::Mask< image::MaskPixel > const &msk, int const flags, StatisticsControl const &sctrl=StatisticsControl())
Handle a watered-down front-end to the constructor (no variance)
Definition: Statistics.h:354
Fit spatial kernel using approximate fluxes for candidates, and solving a linear system of equations.
def getDebugFrame(debugDisplay, name)
Definition: lsstDebug.py:95