LSSTApplications  10.0+286,10.0+36,10.0+46,10.0-2-g4f67435,10.1+152,10.1+37,11.0,11.0+1,11.0-1-g47edd16,11.0-1-g60db491,11.0-1-g7418c06,11.0-2-g04d2804,11.0-2-g68503cd,11.0-2-g818369d,11.0-2-gb8b8ce7
LSSTDataManagementBasePackage
Public Member Functions | Public Attributes | Static Public Attributes | Private Member Functions | Static Private Attributes | List of all members
lsst.meas.deblender.deblend.SourceDeblendTask Class Reference

Split blended sources into individual sources. More...

Inheritance diagram for lsst.meas.deblender.deblend.SourceDeblendTask:

Public Member Functions

def __init__
 Create the task, adding necessary fields to the given schema. More...
 
def addSchemaKeys
 
def run
 Run deblend(). More...
 
def deblend
 Deblend. More...
 
def preSingleDeblendHook
 
def postSingleDeblendHook
 
def isLargeFootprint
 
def isMasked
 
def skipParent
 
def calculateBlendedness
 

Public Attributes

 peakSchemaMapper
 
 nChildKey
 
 psfKey
 
 psfCenterKey
 
 psfFluxKey
 
 tooManyPeaksKey
 
 tooBigKey
 
 maskedKey
 
 deblendFailedKey
 
 deblendSkippedKey
 
 deblendRampedTemplateKey
 
 deblendPatchedTemplateKey
 
 hasStrayFluxKey
 
 blendednessKey
 

Static Public Attributes

 ConfigClass = SourceDeblendConfig
 

Private Member Functions

def _getPsfFwhm
 

Static Private Attributes

string _DefaultName = "sourceDeblend"
 

Detailed Description

Split blended sources into individual sources.

This task has no return value; it only modifies the SourceCatalog in-place.

Definition at line 132 of file deblend.py.

Constructor & Destructor Documentation

def lsst.meas.deblender.deblend.SourceDeblendTask.__init__ (   self,
  schema,
  peakSchema = None,
  kwargs 
)

Create the task, adding necessary fields to the given schema.

Parameters
[in,out]schemaSchema object for measurement fields; will be modified in-place.
[in]peakSchemaSchema of Footprint Peaks that will be passed to the deblender. Any fields beyond the PeakTable minimal schema will be transferred to the main source Schema. If None, no fields will be transferred from the Peaks.
[in]**kwargsPassed to Task.__init__.

Definition at line 143 of file deblend.py.

144  def __init__(self, schema, peakSchema=None, **kwargs):
145  """!
146  Create the task, adding necessary fields to the given schema.
147 
148  @param[in,out] schema Schema object for measurement fields; will be modified in-place.
149  @param[in] peakSchema Schema of Footprint Peaks that will be passed to the deblender.
150  Any fields beyond the PeakTable minimal schema will be transferred
151  to the main source Schema. If None, no fields will be transferred
152  from the Peaks.
153  @param[in] **kwargs Passed to Task.__init__.
154  """
155  pipeBase.Task.__init__(self, **kwargs)
156  peakMinimalSchema = afwDet.PeakTable.makeMinimalSchema()
157  if peakSchema is None:
158  # In this case, the peakSchemaMapper will transfer nothing, but we'll still have one
159  # to simplify downstream code
160  self.peakSchemaMapper = afwTable.SchemaMapper(peakMinimalSchema, schema)
161  else:
162  self.peakSchemaMapper = afwTable.SchemaMapper(peakSchema, schema)
163  for item in peakSchema:
164  if item.key not in peakMinimalSchema:
165  self.peakSchemaMapper.addMapping(item.key, item.field)
166  # Because SchemaMapper makes a copy of the output schema you give its ctor, it isn't
167  # updating this Schema in place. That's probably a design flaw, but in the meantime,
168  # we'll keep that schema in sync with the peakSchemaMapper.getOutputSchema() manually,
169  # by adding the same fields to both.
170  schema.addField(item.field)
171  assert schema == self.peakSchemaMapper.getOutputSchema(), "Logic bug mapping schemas"
172  self.addSchemaKeys(schema)
A mapping between the keys of two Schemas, used to copy data between them.
Definition: SchemaMapper.h:19
def __init__
Create the task, adding necessary fields to the given schema.
Definition: deblend.py:143

Member Function Documentation

def lsst.meas.deblender.deblend.SourceDeblendTask._getPsfFwhm (   self,
  psf,
  bbox 
)
private

Definition at line 234 of file deblend.py.

235  def _getPsfFwhm(self, psf, bbox):
236  # It should be easier to get a PSF's fwhm;
237  # https://dev.lsstcorp.org/trac/ticket/3030
238  return psf.computeShape().getDeterminantRadius() * 2.35
def lsst.meas.deblender.deblend.SourceDeblendTask.addSchemaKeys (   self,
  schema 
)

Definition at line 173 of file deblend.py.

174  def addSchemaKeys(self, schema):
175  self.nChildKey = schema.addField('deblend_nChild', type=int,
176  doc='Number of children this object has (defaults to 0)')
177  self.psfKey = schema.addField('deblend_deblendedAsPsf', type='Flag',
178  doc='Deblender thought this source looked like a PSF')
179  self.psfCenterKey = afwTable.Point2DKey.addFields(schema, 'deblend_psfCenter',
180  'If deblended-as-psf, the PSF centroid', "pixels")
181  self.psfFluxKey = schema.addField('deblend_psfFlux', type='D',
182  doc='If deblended-as-psf, the PSF flux')
183  self.tooManyPeaksKey = schema.addField('deblend_tooManyPeaks', type='Flag',
184  doc='Source had too many peaks; '
185  'only the brightest were included')
186  self.tooBigKey = schema.addField('deblend_parentTooBig', type='Flag',
187  doc='Parent footprint covered too many pixels')
188  self.maskedKey = schema.addField('deblend.masked', type='Flag',
189  doc='Parent footprint was predominantly masked')
190 
191  if self.config.catchFailures:
192  self.deblendFailedKey = schema.addField('deblend_failed', type='Flag',
193  doc="Deblending failed on source")
195  self.deblendSkippedKey = schema.addField('deblend_skipped', type='Flag',
196  doc="Deblender skipped this source")
198  self.deblendRampedTemplateKey = schema.addField(
199  'deblend_rampedTemplate', type='Flag',
200  doc=('This source was near an image edge and the deblender used '
201  '"ramp" edge-handling.'))
203  self.deblendPatchedTemplateKey = schema.addField(
204  'deblend_patchedTemplate', type='Flag',
205  doc=('This source was near an image edge and the deblender used '
206  '"patched" edge-handling.'))
208  self.hasStrayFluxKey = schema.addField(
209  'deblend_hasStrayFlux', type='Flag',
210  doc=('This source was assigned some stray flux'))
212  self.blendednessKey = schema.addField(
213  'deblend.blendedness', type=float,
214  doc=("A measure of how blended the source is. This is the sum of dot products between the source "
215  "and all of its deblended siblings, divided by the dot product of the deblended source with "
216  "itself"))
217 
218  self.log.logdebug('Added keys to schema: %s' % ", ".join(str(x) for x in (
219  self.nChildKey, self.psfKey, self.psfCenterKey, self.psfFluxKey,
220  self.tooManyPeaksKey, self.tooBigKey)))
def lsst.meas.deblender.deblend.SourceDeblendTask.calculateBlendedness (   self,
  maskedImage,
  parent,
  kids 
)
Calculate the blendedness values for a blend family

The blendedness is defined as:

    [heavy[i].dot(heavy[j]) for j in neighbors].sum() / heavy[i].dot(heavy[i])

where 'heavy' is the heavy footprint representation of the flux.

Definition at line 456 of file deblend.py.

457  def calculateBlendedness(self, maskedImage, parent, kids):
458  """Calculate the blendedness values for a blend family
459 
460  The blendedness is defined as:
461 
462  [heavy[i].dot(heavy[j]) for j in neighbors].sum() / heavy[i].dot(heavy[i])
463 
464  where 'heavy' is the heavy footprint representation of the flux.
465  """
466  bbox = parent.getFootprint().getBBox()
467  if not kids or bbox.isEmpty():
468  parent.set(self.blendednessKey, 0.0)
469  return
470 
471  def getHeavyFootprint(src):
472  """Provide the HeavyFootprint for a source"""
473  fp = src.getFootprint()
474  return (afwDet.HeavyFootprintF.cast(fp) if fp.isHeavy() else
475  afwDet.makeHeavyFootprint(fp, maskedImage))
476 
477  parentFoot = getHeavyFootprint(parent)
478  kidFeet = [getHeavyFootprint(src) for src in kids]
479 
480  def setBlendedness(src, foot):
481  """Calculate and set the blendedness value for a source, given its image"""
482  if foot.getBBox().isEmpty() or numpy.all(foot.getImageArray() == 0.0):
483  src.set(self.blendednessKey, 0.0)
484  return
485  srcId = src.getId()
486  numerator = sum(foot.dot(f) for k, f in zip(kids, kidFeet) if k.getId() != srcId)
487  denominator = foot.dot(foot)
488  src.set(self.blendednessKey, numerator/denominator)
489 
490  setBlendedness(parent, parentFoot)
491  for k, f in zip(kids, kidFeet):
492  setBlendedness(k, f)
boost::enable_if< typename ExpressionTraits< Scalar >::IsScalar, Scalar >::type sum(Scalar const &scalar)
Definition: operators.h:1250
HeavyFootprint< ImagePixelT, MaskPixelT, VariancePixelT > makeHeavyFootprint(Footprint const &foot, lsst::afw::image::MaskedImage< ImagePixelT, MaskPixelT, VariancePixelT > const &img, HeavyFootprintCtrl const *ctrl=NULL)
def lsst.meas.deblender.deblend.SourceDeblendTask.deblend (   self,
  exposure,
  srcs,
  psf 
)

Deblend.

Parameters
[in]exposureExposure to process
[in,out]srcsSourceCatalog containing sources detected on this exposure.
[in]psfPSF
Returns
None

Definition at line 240 of file deblend.py.

241  def deblend(self, exposure, srcs, psf):
242  """!
243  Deblend.
244 
245  @param[in] exposure Exposure to process
246  @param[in,out] srcs SourceCatalog containing sources detected on this exposure.
247  @param[in] psf PSF
248 
249  @return None
250  """
251  self.log.info("Deblending %d sources" % len(srcs))
252 
253  from lsst.meas.deblender.baseline import deblend
254  import lsst.meas.algorithms as measAlg
255 
256  # find the median stdev in the image...
257  mi = exposure.getMaskedImage()
258  statsCtrl = afwMath.StatisticsControl()
259  statsCtrl.setAndMask(mi.getMask().getPlaneBitMask(self.config.maskPlanes))
260  stats = afwMath.makeStatistics(mi.getVariance(), mi.getMask(), afwMath.MEDIAN, statsCtrl)
261  sigma1 = math.sqrt(stats.getValue(afwMath.MEDIAN))
262  self.log.logdebug('sigma1: %g' % sigma1)
263 
264  schema = srcs.getSchema()
265 
266  n0 = len(srcs)
267  nparents = 0
268  for i,src in enumerate(srcs):
269  #t0 = time.clock()
270 
271  fp = src.getFootprint()
272  pks = fp.getPeaks()
273  if len(pks) < 2:
274  continue
275 
276  if self.isLargeFootprint(fp):
277  src.set(self.tooBigKey, True)
278  self.skipParent(src, mi.getMask())
279  self.log.logdebug('Parent %i: skipping large footprint' % (int(src.getId()),))
280  continue
281  if self.isMasked(fp, exposure.getMaskedImage().getMask()):
282  src.set(self.maskedKey, True)
283  self.skipParent(src, mi.getMask())
284  self.log.logdebug('Parent %i: skipping masked footprint' % (int(src.getId()),))
285  continue
286 
287  nparents += 1
288  bb = fp.getBBox()
289  psf_fwhm = self._getPsfFwhm(psf, bb)
290 
291  self.log.logdebug('Parent %i: deblending %i peaks' % (int(src.getId()), len(pks)))
292 
293  self.preSingleDeblendHook(exposure, srcs, i, fp, psf, psf_fwhm, sigma1)
294  npre = len(srcs)
295 
296  # This should really be set in deblend, but deblend doesn't have access to the src
297  src.set(self.tooManyPeaksKey, len(fp.getPeaks()) > self.config.maxNumberOfPeaks)
298 
299  try:
300  res = deblend(
301  fp, mi, psf, psf_fwhm, sigma1=sigma1,
302  psfChisqCut1 = self.config.psfChisq1,
303  psfChisqCut2 = self.config.psfChisq2,
304  psfChisqCut2b= self.config.psfChisq2b,
305  maxNumberOfPeaks=self.config.maxNumberOfPeaks,
306  strayFluxToPointSources=self.config.strayFluxToPointSources,
307  assignStrayFlux=self.config.assignStrayFlux,
308  findStrayFlux=(self.config.assignStrayFlux or self.config.findStrayFlux),
309  strayFluxAssignment=self.config.strayFluxRule,
310  rampFluxAtEdge=(self.config.edgeHandling == 'ramp'),
311  patchEdges=(self.config.edgeHandling == 'noclip'),
312  tinyFootprintSize=self.config.tinyFootprintSize,
313  clipStrayFluxFraction=self.config.clipStrayFluxFraction,
314  )
315  if self.config.catchFailures:
316  src.set(self.deblendFailedKey, False)
317  except Exception as e:
318  if self.config.catchFailures:
319  self.log.warn("Unable to deblend source %d: %s" % (src.getId(), e))
320  src.set(self.deblendFailedKey, True)
321  import traceback
322  traceback.print_exc()
323  continue
324  else:
325  raise
326 
327  kids = []
328  nchild = 0
329  for j, peak in enumerate(res.peaks):
330 
331  failed = False
332  if peak.skip:
333  # skip this source?
334  msg = 'Skipping out-of-bounds peak at (%i,%i)' % (pks[j].getIx(), pks[j].getIy())
335  self.log.warn(msg)
336  src.set(self.deblendSkippedKey, True)
337  failed = True
338 
339  heavy = peak.getFluxPortion()
340  if heavy is None:
341  # This can happen for children >= maxNumberOfPeaks
342  msg = 'Skipping peak at (%i,%i), child %i of %i: no flux portion' \
343  % (pks[j].getIx(), pks[j].getIy(), j+1, len(res.peaks))
344  self.log.warn(msg)
345  src.set(self.deblendSkippedKey, True)
346  failed = True
347 
348  if failed:
349  if self.config.propagateAllPeaks:
350  # make sure we have enough info to create a minimal child src
351  if heavy is None:
352  # copy the full footprint and strip out extra peaks
353  foot = afwDet.Footprint(src.getFootprint())
354  peakList = foot.getPeaks()
355  peakList.clear()
356  peakList.append(peak.peak)
357  zeroMimg = afwImage.MaskedImageF(foot.getBBox())
358  heavy = afwDet.makeHeavyFootprint(foot, zeroMimg)
359  if peak.deblendedAsPsf:
360  if peak.psfFitFlux is None:
361  peak.psfFitFlux = 0.0
362  if peak.psfFitCenter is None:
363  peak.psfFitCenter = (peak.peak.getIx(), peak.peak.getIy())
364  self.log.warn("Peak failed. Using minimal default info for child.")
365  else:
366  continue
367 
368  assert(len(heavy.getPeaks()) == 1)
369 
370  src.set(self.deblendSkippedKey, False)
371  child = srcs.addNew(); nchild += 1
372  child.assign(heavy.getPeaks()[0], self.peakSchemaMapper)
373  child.setParent(src.getId())
374  child.setFootprint(heavy)
375  child.set(self.psfKey, peak.deblendedAsPsf)
376  child.set(self.hasStrayFluxKey, peak.strayFlux is not None)
377  if peak.deblendedAsPsf:
378  (cx,cy) = peak.psfFitCenter
379  child.set(self.psfCenterKey, afwGeom.Point2D(cx, cy))
380  child.set(self.psfFluxKey, peak.psfFitFlux)
381  child.set(self.deblendRampedTemplateKey, peak.hasRampedTemplate)
382  child.set(self.deblendPatchedTemplateKey, peak.patched)
383  kids.append(child)
384 
385  # Child footprints may extend beyond the full extent of their parent's which
386  # results in a failure of the replace-by-noise code to reinstate these pixels
387  # to their original values. The following updates the parent footprint
388  # in-place to ensure it contains the full union of itself and all of its
389  # children's footprints.
390  src.getFootprint().include([child.getFootprint() for child in kids])
391 
392  src.set(self.nChildKey, nchild)
393  self.calculateBlendedness(exposure.getMaskedImage(), src, kids)
394 
395  self.postSingleDeblendHook(exposure, srcs, i, npre, kids, fp, psf, psf_fwhm, sigma1, res)
396  #print 'Deblending parent id', src.getId(), 'took', time.clock() - t0
397 
398 
399  n1 = len(srcs)
400  self.log.info('Deblended: of %i sources, %i were deblended, creating %i children, total %i sources'
401  % (n0, nparents, n1-n0, n1))
Pass parameters to a Statistics objectA class to pass parameters which control how the stats are calc...
Definition: Statistics.h:92
A set of pixels in an Image.
Definition: Footprint.h:62
Statistics makeStatistics(afwImage::Mask< afwImage::MaskPixel > const &msk, int const flags, StatisticsControl const &sctrl)
Specialization to handle Masks.
Definition: Statistics.cc:1082
HeavyFootprint< ImagePixelT, MaskPixelT, VariancePixelT > makeHeavyFootprint(Footprint const &foot, lsst::afw::image::MaskedImage< ImagePixelT, MaskPixelT, VariancePixelT > const &img, HeavyFootprintCtrl const *ctrl=NULL)
def lsst.meas.deblender.deblend.SourceDeblendTask.isLargeFootprint (   self,
  footprint 
)
Returns whether a Footprint is large

'Large' is defined by thresholds on the area, size and axis ratio.
These may be disabled independently by configuring them to be non-positive.

This is principally intended to get rid of satellite streaks, which the
deblender or other downstream processing can have trouble dealing with
(e.g., multiple large HeavyFootprints can chew up memory).

Definition at line 408 of file deblend.py.

409  def isLargeFootprint(self, footprint):
410  """Returns whether a Footprint is large
411 
412  'Large' is defined by thresholds on the area, size and axis ratio.
413  These may be disabled independently by configuring them to be non-positive.
414 
415  This is principally intended to get rid of satellite streaks, which the
416  deblender or other downstream processing can have trouble dealing with
417  (e.g., multiple large HeavyFootprints can chew up memory).
418  """
419  if self.config.maxFootprintArea > 0 and footprint.getArea() > self.config.maxFootprintArea:
420  return True
421  if self.config.maxFootprintSize > 0:
422  bbox = footprint.getBBox()
423  if max(bbox.getWidth(), bbox.getHeight()) > self.config.maxFootprintSize:
424  return True
425  if self.config.minFootprintAxisRatio > 0:
426  axes = afwEll.Axes(footprint.getShape())
427  if axes.getB() < self.config.minFootprintAxisRatio*axes.getA():
428  return True
429  return False
def lsst.meas.deblender.deblend.SourceDeblendTask.isMasked (   self,
  footprint,
  mask 
)
Returns whether the footprint violates the mask limits

Definition at line 430 of file deblend.py.

431  def isMasked(self, footprint, mask):
432  """Returns whether the footprint violates the mask limits"""
433  size = float(footprint.getArea())
434  for maskName, limit in self.config.maskLimits.iteritems():
435  maskVal = mask.getPlaneBitMask(maskName)
436  unmasked = afwDet.Footprint(footprint)
437  unmasked.intersectMask(mask, maskVal) # footprint of unmasked pixels
438  if (size - unmasked.getArea())/size > limit:
439  return True
440  return False
A set of pixels in an Image.
Definition: Footprint.h:62
def lsst.meas.deblender.deblend.SourceDeblendTask.postSingleDeblendHook (   self,
  exposure,
  srcs,
  i,
  npre,
  kids,
  fp,
  psf,
  psf_fwhm,
  sigma1,
  res 
)

Definition at line 405 of file deblend.py.

406  def postSingleDeblendHook(self, exposure, srcs, i, npre, kids, fp, psf, psf_fwhm, sigma1, res):
407  pass
def lsst.meas.deblender.deblend.SourceDeblendTask.preSingleDeblendHook (   self,
  exposure,
  srcs,
  i,
  fp,
  psf,
  psf_fwhm,
  sigma1 
)

Definition at line 402 of file deblend.py.

403  def preSingleDeblendHook(self, exposure, srcs, i, fp, psf, psf_fwhm, sigma1):
404  pass
def lsst.meas.deblender.deblend.SourceDeblendTask.run (   self,
  exposure,
  sources,
  psf 
)

Run deblend().

Parameters
[in]exposureExposure to process
[in,out]sourcesSourceCatalog containing sources detected on this exposure.
[in]psfPSF
Returns
None

Definition at line 222 of file deblend.py.

223  def run(self, exposure, sources, psf):
224  """!
225  Run deblend().
226 
227  @param[in] exposure Exposure to process
228  @param[in,out] sources SourceCatalog containing sources detected on this exposure.
229  @param[in] psf PSF
230 
231  @return None
232  """
233  self.deblend(exposure, sources, psf)
def lsst.meas.deblender.deblend.SourceDeblendTask.skipParent (   self,
  source,
  mask 
)
Indicate that the parent source is not being deblended

We set the appropriate flags and mask.

@param source  The source to flag as skipped
@param mask  The mask to update

Definition at line 441 of file deblend.py.

442  def skipParent(self, source, mask):
443  """Indicate that the parent source is not being deblended
444 
445  We set the appropriate flags and mask.
446 
447  @param source The source to flag as skipped
448  @param mask The mask to update
449  """
450  fp = source.getFootprint()
451  source.set(self.deblendSkippedKey, True)
452  source.set(self.nChildKey, len(fp.getPeaks())) # It would have this many if we deblended them all
453  if self.config.notDeblendedMask:
454  mask.addMaskPlane(self.config.notDeblendedMask)
455  afwDet.setMaskFromFootprint(mask, fp, mask.getPlaneBitMask(self.config.notDeblendedMask))
MaskT setMaskFromFootprint(lsst::afw::image::Mask< MaskT > *mask, Footprint const &footprint, MaskT const bitmask)
OR bitmask into all the Mask&#39;s pixels that are in the Footprint.

Member Data Documentation

string lsst.meas.deblender.deblend.SourceDeblendTask._DefaultName = "sourceDeblend"
staticprivate

Definition at line 141 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.blendednessKey

Definition at line 211 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.ConfigClass = SourceDeblendConfig
static

Definition at line 140 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.deblendFailedKey

Definition at line 191 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.deblendPatchedTemplateKey

Definition at line 202 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.deblendRampedTemplateKey

Definition at line 197 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.deblendSkippedKey

Definition at line 194 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.hasStrayFluxKey

Definition at line 207 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.maskedKey

Definition at line 187 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.nChildKey

Definition at line 174 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.peakSchemaMapper

Definition at line 159 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.psfCenterKey

Definition at line 178 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.psfFluxKey

Definition at line 180 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.psfKey

Definition at line 176 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.tooBigKey

Definition at line 185 of file deblend.py.

lsst.meas.deblender.deblend.SourceDeblendTask.tooManyPeaksKey

Definition at line 182 of file deblend.py.


The documentation for this class was generated from the following file: