LSSTApplications  10.0-2-g4f67435,11.0.rc2+1,11.0.rc2+12,11.0.rc2+3,11.0.rc2+4,11.0.rc2+5,11.0.rc2+6,11.0.rc2+7,11.0.rc2+8
LSSTDataManagementBasePackage
repair.py
Go to the documentation of this file.
1 #
2 # LSST Data Management System
3 # Copyright 2008, 2009, 2010, 2011 LSST Corporation.
4 #
5 # This product includes software developed by the
6 # LSST Project (http://www.lsst.org/).
7 #
8 # This program is free software: you can redistribute it and/or modify
9 # it under the terms of the GNU General Public License as published by
10 # the Free Software Foundation, either version 3 of the License, or
11 # (at your option) any later version.
12 #
13 # This program is distributed in the hope that it will be useful,
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 # GNU General Public License for more details.
17 #
18 # You should have received a copy of the LSST License Statement and
19 # the GNU General Public License along with this program. If not,
20 # see <http://www.lsstcorp.org/LegalNotices/>.
21 #
22 import lsst.pex.config as pexConfig
23 import lsst.afw.math as afwMath
24 import lsst.afw.detection as afwDet
25 import lsst.meas.algorithms as measAlg
26 import lsst.pipe.base as pipeBase
27 
28 import lsstDebug
29 
30 class RepairConfig(pexConfig.Config):
31  doInterpolate = pexConfig.Field(
32  dtype = bool,
33  doc = "Interpolate over defects? (ignored unless you provide a list of defects)",
34  default = True,
35  )
36  doCosmicRay = pexConfig.Field(
37  dtype = bool,
38  doc = "Find and mask out cosmic rays?",
39  default = True,
40  )
41  cosmicray = pexConfig.ConfigField(
42  dtype = measAlg.FindCosmicRaysConfig,
43  doc = "Options for finding and masking cosmic rays",
44  )
45 
46 ## \addtogroup LSST_task_documentation
47 ## \{
48 ## \page RepairTask
49 ## \ref RepairTask_ "RepairTask"
50 ## \copybrief RepairTask
51 ## \}
52 
53 class RepairTask(pipeBase.Task):
54  """!
55  \anchor RepairTask_
56 
57  \brief Interpolate over defects in an exposure and handle cosmic rays
58 
59  \section pipe_tasks_repair_Contents Contents
60 
61  - \ref pipe_tasks_repair_Purpose
62  - \ref pipe_tasks_repair_Initialize
63  - \ref pipe_tasks_repair_IO
64  - \ref pipe_tasks_repair_Config
65  - \ref pipe_tasks_repair_Debug
66  - \ref pipe_tasks_repair_Example
67 
68  \section pipe_tasks_repair_Purpose Description
69 
70  \copybrief RepairTask
71 
72  This task operates on an lsst.afw.image.Exposure in place to interpolate over a set of
73  lsst.meas.algorithms.Defect objects.
74  It will also, optionally, find and interpolate any cosmic rays in the lsst.afw.image.Exposure.
75 
76  \section pipe_tasks_repair_Initialize Task initialization
77 
78  See: lsst.pipe.base.task.Task.__init__
79 
80  \section pipe_tasks_repair_IO Inputs/Outputs to the run method
81 
82  \copydoc run
83 
84  \section pipe_tasks_repair_Config Configuration parameters
85 
86  See \ref RepairConfig
87 
88  \section pipe_tasks_repair_Debug Debug variables
89 
90  The \link lsst.pipe.base.cmdLineTask.CmdLineTask command line task\endlink interface supports a
91  flag \c -d to import \b debug.py from your \c PYTHONPATH; see <a
92  href="http://lsst-web.ncsa.illinois.edu/~buildbot/doxygen/x_masterDoxyDoc/base_debug.html">
93  Using lsstDebug to control debugging output</a> for more about \b debug.py files.
94 
95  The available variables in RepairTask are:
96  <DL>
97  <DT> \c display
98  <DD> A dictionary containing debug point names as keys with frame number as value. Valid keys are:
99  <DL>
100  <DT> repair.before
101  <DD> display image before any repair is done
102  <DT> repair.after
103  <DD> display image after cosmic ray and defect correction
104  </DL>
105  <DT> \c displayCR
106  <DD> If True, display the exposure on ds9's frame 1 and overlay bounding boxes around detects CRs.
107  </DL>
108  \section pipe_tasks_repair_Example A complete example of using RepairTask
109 
110  This code is in runRepair.py in the examples directory, and can be run as \em e.g.
111  \code
112  examples/runRepair.py
113  \endcode
114  \dontinclude runRepair.py
115  Import the task. There are other imports. Read the source file for more info.
116  \skipline RepairTask
117 
118  For this example, we manufacture a test image to run on.
119 
120  First, create a pure Poisson noise image and a Psf to go with it. The mask plane
121  and variance can be constructed at the same time.
122  \skip poisson
123  \until mask
124 
125  Inject some cosmic rays and generate the Exposure. Exposures are MaskedImages (image + mask + variance)
126  with other metadata (e.g. Psf and Wcs objects).
127  \skip some CRs
128  \until setPsf
129 
130  Defects are represented as bad columns of random lengths. A defect list must be constructed to pass
131  on to the RepairTask.
132  \bug This is addressed in <a href="https://jira.lsstcorp.org/browse/DM-963"> DM-963</a>
133 
134  \skip addDefects
135  \until push_back
136 
137  Finally, the exposure can be repaired. Create an instance of the task and run it. The exposure is modified in place.
138  \skip RepairTask
139  \until repair.run
140 
141  <HR>
142  To investigate the \ref pipe_tasks_repair_Debug, put something like
143  \code{.py}
144  import lsstDebug
145  def DebugInfo(name):
146  di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively
147  if name == "lsst.pipe.tasks.repair":
148  di.display = {'repair.before':2, 'repair.after':3}
149  di.displayCR = True
150  return di
151 
152  lsstDebug.Info = DebugInfo
153  \endcode
154  into your debug.py file and run runRepair.py with the \c --debug flag.
155 
156 
157  Conversion notes:
158  Display code should be updated once we settle on a standard way of controlling what is displayed.
159  """
160  ConfigClass = RepairConfig
161 
162  @pipeBase.timeMethod
163  def run(self, exposure, defects=None, keepCRs=None):
164  """!Repair an Exposure's defects and cosmic rays
165 
166  \param[in, out] exposure lsst.afw.image.Exposure to process. Exposure must have a valid Psf. Modified in place.
167  \param[in] defects an lsst.meas.algorithms.DefectListT object. If None, do no defect correction.
168  \param[in] keepCRs don't interpolate over the CR pixels (defer to RepairConfig if None)
169 
170  \throws AssertionError with the following strings:
171 
172  <DL>
173  <DT> No exposure provided
174  <DD> The object provided as exposure evaluates to False
175  <DT> No PSF provided
176  <DD> The Exposure has no associated Psf
177  </DL>
178  """
179  assert exposure, "No exposure provided"
180  psf = exposure.getPsf()
181  assert psf, "No PSF provided"
182 
183  self.display('repair.before', exposure=exposure)
184  if defects is not None and self.config.doInterpolate:
185  self.interpolate(exposure, defects)
186 
187  if self.config.doCosmicRay:
188  self.cosmicRay(exposure, keepCRs=keepCRs)
189 
190  self.display('repair.after', exposure=exposure)
191 
192  def interpolate(self, exposure, defects):
193  """Interpolate over defects
194 
195  @param[in,out] exposure Exposure to process
196  @param defects Defect list
197  """
198  assert exposure, "No exposure provided"
199  assert defects is not None, "No defects provided"
200  psf = exposure.getPsf()
201  assert psf, "No psf provided"
202 
203  mi = exposure.getMaskedImage()
204  fallbackValue = afwMath.makeStatistics(mi, afwMath.MEANCLIP).getValue()
205  measAlg.interpolateOverDefects(mi, psf, defects, fallbackValue)
206  self.log.info("Interpolated over %d defects." % len(defects))
207 
208  def cosmicRay(self, exposure, keepCRs=None):
209  """Mask cosmic rays
210 
211  @param[in,out] exposure Exposure to process
212  @param keepCRs Don't interpolate over the CR pixels (defer to pex_config if None)
213  """
214  import lsstDebug
215  display = lsstDebug.Info(__name__).display
216  displayCR = lsstDebug.Info(__name__).displayCR
217 
218  assert exposure, "No exposure provided"
219  psf = exposure.getPsf()
220  assert psf, "No psf provided"
221 
222  # Blow away old mask
223  try:
224  mask = exposure.getMaskedImage().getMask()
225  crBit = mask.getMaskPlane("CR")
226  mask.clearMaskPlane(crBit)
227  except Exception:
228  pass
229 
230  mi = exposure.getMaskedImage()
231  bg = afwMath.makeStatistics(mi, afwMath.MEDIAN).getValue()
232 
233  if keepCRs is None:
234  keepCRs = self.config.cosmicray.keepCRs
235  try:
236  crs = measAlg.findCosmicRays(mi, psf, bg, pexConfig.makePolicy(self.config.cosmicray), keepCRs)
237  except Exception, e:
238  if display:
239  import lsst.afw.display.ds9 as ds9
240  ds9.mtv(exposure, title="Failed CR")
241  raise
242 
243  num = 0
244  if crs is not None:
245  mask = mi.getMask()
246  crBit = mask.getPlaneBitMask("CR")
247  afwDet.setMaskFromFootprintList(mask, crs, crBit)
248  num = len(crs)
249 
250  if display and displayCR:
251  import lsst.afw.display.ds9 as ds9
252  import lsst.afw.display.utils as displayUtils
253 
254  ds9.incrDefaultFrame()
255  ds9.mtv(exposure, title="Post-CR")
256 
257  with ds9.Buffering():
258  for cr in crs:
259  displayUtils.drawBBox(cr.getBBox(), borderWidth=0.55)
260 
261  self.log.info("Identified %s cosmic rays." % (num,))
262 
void interpolateOverDefects(MaskedImageT &image, lsst::afw::detection::Psf const &psf, std::vector< Defect::Ptr > &badList, double fallbackValue=0.0, bool useFallbackValueAtEdge=false)
Process a set of known bad pixels in an image.
Definition: Interp.cc:2058
def run
Repair an Exposure&#39;s defects and cosmic rays.
Definition: repair.py:163
std::vector< detection::Footprint::Ptr > findCosmicRays(MaskedImageT &mimage, detection::Psf const &psf, double const bkgd, lsst::pex::policy::Policy const &policy, bool const keep)
Find cosmic rays in an Image, and mask and remove them.
Definition: CR.cc:341
Statistics makeStatistics(afwImage::Mask< afwImage::MaskPixel > const &msk, int const flags, StatisticsControl const &sctrl)
Specialization to handle Masks.
Definition: Statistics.cc:1082
MaskT setMaskFromFootprintList(lsst::afw::image::Mask< MaskT > *mask, boost::shared_ptr< std::vector< boost::shared_ptr< Footprint >> const > const &footprints, MaskT const bitmask)
OR bitmask into all the Mask&#39;s pixels which are in the set of Footprints.
Interpolate over defects in an exposure and handle cosmic rays.
Definition: repair.py:53