LSSTApplications  10.0+286,10.0+36,10.0+46,10.0-2-g4f67435,10.1+152,10.1+37,11.0,11.0+1,11.0-1-g47edd16,11.0-1-g60db491,11.0-1-g7418c06,11.0-2-g04d2804,11.0-2-g68503cd,11.0-2-g818369d,11.0-2-gb8b8ce7
LSSTDataManagementBasePackage
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.meas.base.noiseReplacer.NoiseReplacer Class Reference

Class that handles replacing sources with noise during measurement. More...

Inheritance diagram for lsst.meas.base.noiseReplacer.NoiseReplacer:

Public Member Functions

def __init__
 Initialize the NoiseReplacer. More...
 
def insertSource
 Insert the heavy footprint of a given source into the exposure. More...
 
def removeSource
 Remove the heavy footprint of a given source and replace with previous noise. More...
 
def end
 End the NoiseReplacer. More...
 
def getNoiseGenerator
 Generate noise image using parameters given. More...
 

Public Attributes

 noiseSource
 
 noiseOffset
 
 noiseSeedMultiplier
 
 noiseGenMean
 
 noiseGenStd
 
 log
 
 exposure
 
 footprints
 
 removeplanes
 
 otherbitmask
 
 heavies
 
 heavyNoise
 FIXME: the heavy footprint includes the mask and variance planes, which we shouldn't need (I don't think we ever want to modify them in the input image). More...
 

Static Public Attributes

 ConfigClass = NoiseReplacerConfig
 

Detailed Description

Class that handles replacing sources with noise during measurement.

When measuring a source (or the children associated with a parent source), this class is used to replace its neighbors with noise, using the deblender's definition of the sources as stored in HeavyFootprints attached to the SourceRecords. The algorithm works as follows:

This is a functional copy of the code in the older ReplaceWithNoiseTask, but with a slightly different API needed for the new measurement framework; note that it is not a Task, as the lifetime of a NoiseReplacer now corresponds to a single exposure, not an entire processing run.

Definition at line 52 of file noiseReplacer.py.

Constructor & Destructor Documentation

def lsst.meas.base.noiseReplacer.NoiseReplacer.__init__ (   self,
  config,
  exposure,
  footprints,
  noiseImage = None,
  exposureId = None,
  log = None 
)

Initialize the NoiseReplacer.

Parameters
[in]configinstance of NoiseReplacerConfig
[in,out]exposureExposure to be noise replaced. (All sources replaced on return)
[in]footprintsdict of {id: (parent, footprint)};
[in]noiseImagean afw.image.ImageF used as a predictable noise replacement source (for tests only)
[in]logpex.logging.Log object to use for status messages; no status messages will be printed if None

'footprints' is a dict of {id: (parent, footprint)}; when used in SFM, the ID will be the source ID, but in forced photometry, this will be the reference ID, as that's what we used to determine the deblend families. This routine should create HeavyFootprints for any non-Heavy Footprints, and replace them in the dict. It should then create a dict of HeavyFootprints containing noise, but only for parent objects, then replace all sources with noise. This should ignore any footprints that lay outside the bounding box of the exposure, and clip those that lie on the border.

NOTE: as the code currently stands, the heavy footprint for a deblended object must be available from the input catalog. If it is not, it cannot be reproduced here. In that case, the topmost parent in the objects parent chain must be used. The heavy footprint for that source is created in this class from the masked image.

Definition at line 74 of file noiseReplacer.py.

74 
75  def __init__(self, config, exposure, footprints, noiseImage=None, exposureId=None, log=None):
76  """!
77  Initialize the NoiseReplacer.
78 
79  @param[in] config instance of NoiseReplacerConfig
80  @param[in,out] exposure Exposure to be noise replaced. (All sources replaced on return)
81  @param[in] footprints dict of {id: (parent, footprint)};
82  @param[in] noiseImage an afw.image.ImageF used as a predictable noise replacement source
83  (for tests only)
84  @param[in] log pex.logging.Log object to use for status messages; no status messages
85  will be printed if None
86 
87  'footprints' is a dict of {id: (parent, footprint)}; when used in SFM, the ID will be the
88  source ID, but in forced photometry, this will be the reference ID, as that's what we used to
89  determine the deblend families. This routine should create HeavyFootprints for any non-Heavy
90  Footprints, and replace them in the dict. It should then create a dict of HeavyFootprints
91  containing noise, but only for parent objects, then replace all sources with noise.
92  This should ignore any footprints that lay outside the bounding box of the exposure,
93  and clip those that lie on the border.
94 
95  NOTE: as the code currently stands, the heavy footprint for a deblended object must be available
96  from the input catalog. If it is not, it cannot be reproduced here. In that case, the
97  topmost parent in the objects parent chain must be used. The heavy footprint for that source
98  is created in this class from the masked image.
99  """
100  noiseMeanVar=None
101  self.noiseSource = config.noiseSource
102  self.noiseOffset = config.noiseOffset
103  self.noiseSeedMultiplier = config.noiseSeedMultiplier
104  self.noiseGenMean = None
105  self.noiseGenStd = None
106  self.log = log
107 
108  # creates heavies, replaces all footprints with noise
109  # We need the source table to be sorted by ID to do the parent lookups
110  self.exposure = exposure
111  self.footprints = footprints
112  mi = exposure.getMaskedImage()
113  im = mi.getImage()
114  mask = mi.getMask()
115  # Add temporary Mask planes for THISDET and OTHERDET
116  self.removeplanes = []
117  bitmasks = []
118  for maskname in ['THISDET', 'OTHERDET']:
119  try:
120  # does it already exist?
121  plane = mask.getMaskPlane(maskname)
122  if self.log: self.log.logdebug('Mask plane "%s" already existed' % maskname)
123  except:
124  # if not, add it; we should delete it when done.
125  plane = mask.addMaskPlane(maskname)
126  self.removeplanes.append(maskname)
127  mask.clearMaskPlane(plane)
128  bitmask = mask.getPlaneBitMask(maskname)
129  bitmasks.append(bitmask)
130  if self.log: self.log.logdebug('Mask plane "%s": plane %i, bitmask %i = 0x%x'
131  % (maskname, plane, bitmask, bitmask))
132  self.thisbitmask,self.otherbitmask = bitmasks
133  del bitmasks
134  self.heavies = {}
135  # Start by creating HeavyFootprints for each source which has no parent
136  # and just use them for children which do not already have heavy footprints.
137  # If a heavy footprint is available for a child, we will use it. Otherwise,
138  # we use the first parent in the parent chain which has a heavy footprint,
139  # which with the one level deblender will alway be the topmost parent
140  # NOTE: heavy footprints get destroyed by the transform process in forcedPhotImage.py,
141  # so they are never available for forced measurements.
142 
143  # Create in the dict heavies = {id:heavyfootprint}
144  for id in footprints.keys():
145  fp = footprints[id]
146  if fp[1].isHeavy():
147  self.heavies[id] = afwDet.cast_HeavyFootprintF(fp[1])
148  elif fp[0] == 0:
149  self.heavies[id] = afwDet.makeHeavyFootprint(fp[1], mi)
150 
151  ### FIXME: the heavy footprint includes the mask
152  ### and variance planes, which we shouldn't need
153  ### (I don't think we ever want to modify them in
154  ### the input image). Copying them around is
155  ### wasteful.
156 
157  # We now create a noise HeavyFootprint for each source with has a heavy footprint.
158  # We'll put the noise footprints in a dict heavyNoise = {id:heavyNoiseFootprint}
159  self.heavyNoise = {}
160  noisegen = self.getNoiseGenerator(exposure, noiseImage, noiseMeanVar, exposureId=exposureId)
161  # The noiseGenMean and Std are used by the unit tests
162  self.noiseGenMean = noisegen.mean
163  self.noiseGenStd = noisegen.std
164  if self.log: self.log.logdebug('Using noise generator: %s' % (str(noisegen)))
165  for id in self.heavies.keys():
166  fp = footprints[id][1]
167  noiseFp = noisegen.getHeavyFootprint(fp)
168  self.heavyNoise[id] = noiseFp
169  # Also insert the noisy footprint into the image now.
170  # Notice that we're just inserting it into "im", ie,
171  # the Image, not the MaskedImage.
172  noiseFp.insert(im)
173  # Also set the OTHERDET bit
def __init__
Initialize the NoiseReplacer.
MaskT setMaskFromFootprint(lsst::afw::image::Mask< MaskT > *mask, Footprint const &footprint, MaskT const bitmask)
OR bitmask into all the Mask&#39;s pixels that are in the Footprint.
def getNoiseGenerator
Generate noise image using parameters given.
heavyNoise
FIXME: the heavy footprint includes the mask and variance planes, which we shouldn&#39;t need (I don&#39;t th...
HeavyFootprint< ImagePixelT, MaskPixelT, VariancePixelT > makeHeavyFootprint(Footprint const &foot, lsst::afw::image::MaskedImage< ImagePixelT, MaskPixelT, VariancePixelT > const &img, HeavyFootprintCtrl const *ctrl=NULL)

Member Function Documentation

def lsst.meas.base.noiseReplacer.NoiseReplacer.end (   self)

End the NoiseReplacer.

Restore original data to the exposure from the heavies dictionary Restore the mask planes to their original state

Definition at line 225 of file noiseReplacer.py.

226  def end(self):
227  """!
228  End the NoiseReplacer.
229 
230  Restore original data to the exposure from the heavies dictionary
231  Restore the mask planes to their original state
232  """
233  # restores original image, cleans up temporaries
234  # (ie, replace all the top-level pixels)
235  mi = self.exposure.getMaskedImage()
236  im = mi.getImage()
237  mask = mi.getMask()
238  for id in self.footprints.keys():
239  if self.footprints[id][0] != 0:
240  continue
241  self.heavies[id].insert(im)
242  for maskname in self.removeplanes:
243  mask.removeAndClearMaskPlane(maskname, True)
244 
245  del self.removeplanes
246  del self.thisbitmask
247  del self.otherbitmask
248  del self.heavies
249  del self.heavyNoise
heavyNoise
FIXME: the heavy footprint includes the mask and variance planes, which we shouldn&#39;t need (I don&#39;t th...
def lsst.meas.base.noiseReplacer.NoiseReplacer.getNoiseGenerator (   self,
  exposure,
  noiseImage,
  noiseMeanVar,
  exposureId = None 
)

Generate noise image using parameters given.

Definition at line 250 of file noiseReplacer.py.

251  def getNoiseGenerator(self, exposure, noiseImage, noiseMeanVar, exposureId=None):
252  """!
253  Generate noise image using parameters given
254  """
255  if noiseImage is not None:
256  return ImageNoiseGenerator(noiseImage)
257  rand = None
258  if self.noiseSeedMultiplier:
259  # default plugin, our seed
260  if not exposureId is None and not exposureId == 0:
261  seed = exposureId * self.noiseSeedMultiplier
262  else:
263  seed = self.noiseSeedMultiplier
264  rand = afwMath.Random(afwMath.Random.MT19937, seed)
265  if noiseMeanVar is not None:
266  try:
267  # Assume noiseMeanVar is an iterable of floats
268  noiseMean,noiseVar = noiseMeanVar
269  noiseMean = float(noiseMean)
270  noiseVar = float(noiseVar)
271  noiseStd = math.sqrt(noiseVar)
272  if self.log: self.log.logdebug('Using passed-in noise mean = %g, variance = %g -> stdev %g'
273  % (noiseMean, noiseVar, noiseStd))
274  return FixedGaussianNoiseGenerator(noiseMean, noiseStd, rand=rand)
275  except:
276  if self.log: self.log.logdebug('Failed to cast passed-in noiseMeanVar to floats: %s'
277  % (str(noiseMeanVar)))
278  offset = self.noiseOffset
279  noiseSource = self.noiseSource
280 
281  if noiseSource == 'meta':
282  # check the exposure metadata
283  meta = exposure.getMetadata()
284  # this key name correspond to estimateBackground() in detection.py
285  try:
286  bgMean = meta.getAsDouble('BGMEAN')
287  # We would have to adjust for GAIN if ip_isr didn't make it 1.0
288  noiseStd = math.sqrt(bgMean)
289  if self.log: self.log.logdebug('Using noise variance = (BGMEAN = %g) from exposure metadata'
290  % (bgMean))
291  return FixedGaussianNoiseGenerator(offset, noiseStd, rand=rand)
292  except:
293  if self.log: self.log.logdebug('Failed to get BGMEAN from exposure metadata')
294 
295  if noiseSource == 'variance':
296  if self.log: self.log.logdebug('Will draw noise according to the variance plane.')
297  var = exposure.getMaskedImage().getVariance()
298  return VariancePlaneNoiseGenerator(var, mean=offset, rand=rand)
299 
300  # Compute an image-wide clipped variance.
301  im = exposure.getMaskedImage().getImage()
302  s = afwMath.makeStatistics(im, afwMath.MEANCLIP | afwMath.STDEVCLIP)
303  noiseMean = s.getValue(afwMath.MEANCLIP)
304  noiseStd = s.getValue(afwMath.STDEVCLIP)
305  if self.log: self.log.logdebug("Measured from image: clipped mean = %g, stdev = %g"
306  % (noiseMean,noiseStd))
307  return FixedGaussianNoiseGenerator(noiseMean + offset, noiseStd, rand=rand)
Generates Gaussian noise whose variance matches that of the variance plane of the image...
Generates Gaussian noise with a fixed mean and standard deviation.
Statistics makeStatistics(afwImage::Mask< afwImage::MaskPixel > const &msk, int const flags, StatisticsControl const &sctrl)
Specialization to handle Masks.
Definition: Statistics.cc:1082
def getNoiseGenerator
Generate noise image using parameters given.
def lsst.meas.base.noiseReplacer.NoiseReplacer.insertSource (   self,
  id 
)

Insert the heavy footprint of a given source into the exposure.

Parameters
[in]idid for current source to insert from original footprint dict

Also adjusts the mask plane to show the source of this footprint.

Definition at line 175 of file noiseReplacer.py.

176  def insertSource(self, id):
177  """!
178  Insert the heavy footprint of a given source into the exposure
179 
180  @param[in] id id for current source to insert from original footprint dict
181 
182  Also adjusts the mask plane to show the source of this footprint.
183  """
184  # Copy this source's pixels into the image
185  mi = self.exposure.getMaskedImage()
186  im = mi.getImage()
187  mask = mi.getMask()
188  # usedid can point either to this source, or to the first parent in the
189  # parent chain which has a heavy footprint (or to the topmost parent,
190  # which always has one)
191  usedid = id
192  while self.footprints[usedid][0] != 0 and not usedid in self.heavies.keys():
193  usedid = self.footprints[usedid][0]
194  fp = self.heavies[usedid]
195  fp.insert(im)
196  afwDet.setMaskFromFootprint(mask, fp, self.thisbitmask)
MaskT clearMaskFromFootprint(lsst::afw::image::Mask< MaskT > *mask, Footprint const &footprint, MaskT const bitmask)
(AND ~bitmask) all the Mask&#39;s pixels that are in the Footprint; that is, set to zero in the Mask-inte...
MaskT setMaskFromFootprint(lsst::afw::image::Mask< MaskT > *mask, Footprint const &footprint, MaskT const bitmask)
OR bitmask into all the Mask&#39;s pixels that are in the Footprint.
def insertSource
Insert the heavy footprint of a given source into the exposure.
def lsst.meas.base.noiseReplacer.NoiseReplacer.removeSource (   self,
  id 
)

Remove the heavy footprint of a given source and replace with previous noise.

Parameters
[in]idid for current source to insert from original footprint dict

Also restore the mask plane.

Definition at line 198 of file noiseReplacer.py.

199  def removeSource(self, id):
200  """!
201  Remove the heavy footprint of a given source and replace with previous noise
202 
203  @param[in] id id for current source to insert from original footprint dict
204 
205  Also restore the mask plane.
206  """
207  # remove a single source
208  # (Replace this source's pixels by noise again.)
209  # Do this by finding the source's top-level ancestor
210  mi = self.exposure.getMaskedImage()
211  im = mi.getImage()
212  mask = mi.getMask()
213 
214  # use the same algorithm as in remove Source to find the heavy noise footprint
215  # which will undo what insertSource(id) does
216  usedid = id
217  while self.footprints[usedid][0] != 0 and not usedid in self.heavies.keys():
218  usedid = self.footprints[usedid][0]
219  # Re-insert the noise pixels
220  fp = self.heavyNoise[usedid]
221  fp.insert(im)
222  # Clear the THISDET mask plane.
223  afwDet.clearMaskFromFootprint(mask, fp, self.thisbitmask)
MaskT clearMaskFromFootprint(lsst::afw::image::Mask< MaskT > *mask, Footprint const &footprint, MaskT const bitmask)
(AND ~bitmask) all the Mask&#39;s pixels that are in the Footprint; that is, set to zero in the Mask-inte...
MaskT setMaskFromFootprint(lsst::afw::image::Mask< MaskT > *mask, Footprint const &footprint, MaskT const bitmask)
OR bitmask into all the Mask&#39;s pixels that are in the Footprint.
heavyNoise
FIXME: the heavy footprint includes the mask and variance planes, which we shouldn&#39;t need (I don&#39;t th...
def removeSource
Remove the heavy footprint of a given source and replace with previous noise.

Member Data Documentation

lsst.meas.base.noiseReplacer.NoiseReplacer.ConfigClass = NoiseReplacerConfig
static

Definition at line 72 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.exposure

Definition at line 109 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.footprints

Definition at line 110 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.heavies

Definition at line 133 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.heavyNoise

FIXME: the heavy footprint includes the mask and variance planes, which we shouldn't need (I don't think we ever want to modify them in the input image).

Copying them around is wasteful.

Definition at line 158 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.log

Definition at line 105 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.noiseGenMean

Definition at line 103 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.noiseGenStd

Definition at line 104 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.noiseOffset

Definition at line 101 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.noiseSeedMultiplier

Definition at line 102 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.noiseSource

Definition at line 100 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.otherbitmask

Definition at line 131 of file noiseReplacer.py.

lsst.meas.base.noiseReplacer.NoiseReplacer.removeplanes

Definition at line 115 of file noiseReplacer.py.


The documentation for this class was generated from the following file: