lsst.meas.base.noiseReplacer.NoiseReplacer Class Reference
Public Member Functions | |
| __init__ (self, config, exposure, footprints, noiseImage=None, exposureId=None, log=None) | |
| insertSource (self, id) | |
| removeSource (self, id) | |
| end (self) | |
| getNoiseGenerator (self, exposure, noiseImage, noiseMeanVar, exposureId=None) | |
Public Attributes | |
| noiseSource = config.noiseSource | |
| noiseOffset = config.noiseOffset | |
| noiseSeedMultiplier = config.noiseSeedMultiplier | |
| noiseGenMean = None | |
| noiseGenStd = None | |
| list | removeplanes = [] |
| thisbitmask = mask.getPlaneBitMask(maskname) | |
| otherbitmask = bitmasks | |
| dict | heavies = {} |
| dict | heavyNoise = {} |
Static Public Attributes | |
| ConfigClass = NoiseReplacerConfig | |
| exposure = None | |
| footprints = None | |
| log = None | |
Detailed Description
Replace sources with noise during measurement.
Parameters
----------
config : `NoiseReplacerConfig`
Configuration.
exposure : `lsst.afw.image.Exposure`
Image in which sources will be replaced by noise. During operation,
the image will be modified in-place to replace all sources. At the end
of the measurment procedure, the original sources will be replaced.
footprints : `dict`
Mapping of ``id`` to a tuple of ``(parent, Footprint)``. When used in
single-frame measurement, ``id`` is the source ID, but in forced
photometry this is the reference ID (as that is used to determine
deblend families).
noiseImage : `lsst.afw.image.ImageF`
An image used as a predictable noise replacement source. Used during
testing only.
log : `lsst.log.log.log.Log` or `logging.Logger`, optional
Logger to use for status messages; no status messages will be recorded
if `None`.
Notes
-----
When measuring a source (or the children associated with a parent source),
this class is used to replace its neighbors with noise, using the
deblender's definition of the sources as stored in
`~lsst.afw.detection.heavyFootprint.HeavyFootprint`\ s attached to the
`~lsst.afw.table.SourceRecord`\ s. The algorithm works as follows:
#. All pixels in the source `~lsst.afw.detection.Footprint`\ s are replaced
with artificially generated noise (in `NoiseReplacer.__init__`).
#. Before each source is measured, we restore the original pixel data by
inserting that source's
`~lsst.afw.detection.heavyFootprint.HeavyFootprint` (from the deblender)
into the image.
#. After measurement, we again replace the source pixels with (the same)
artificial noise.
#. After measuring all sources, the image is returned to its original
state.
This is a functional copy of the code in the older
``ReplaceWithNoiseTask``, but with a slightly different API needed for the
new measurement framework; note that it is not an `~lsst.pipe.base.Task`,
as the lifetime of a ``NoiseReplacer`` now corresponds to a single
exposure, not an entire processing run.
When processing the ``footprints`` parameter, this routine should create
`~lsst.afw.detection.heavyFootprint.HeavyFootprint`\ s for any non-Heavy
`~lsst.afw.detection.Footprint`\ s, and replace them in the dictionary. It
should then create a dict of
`~lsst.afw.detection.heavyFootprint.HeavyFootprint`\ s containing noise,
but only for parent objects, then replace all sources with noise. This
should ignore any footprints that lay outside the bounding box of the
exposure, and clip those that lie on the border.
As the code currently stands, the heavy footprint for a deblended object
must be available from the input catalog. If it is not, it cannot be
reproduced here. In that case, the topmost parent in the objects parent
chain must be used. The heavy footprint for that source is created in
this class from the masked image.
Definition at line 58 of file noiseReplacer.py.
Constructor & Destructor Documentation
◆ __init__()
| lsst.meas.base.noiseReplacer.NoiseReplacer.__init__ | ( | self, | |
| config, | |||
| exposure, | |||
| footprints, | |||
| noiseImage = None, | |||
| exposureId = None, | |||
| log = None ) |
Definition at line 136 of file noiseReplacer.py.
136 def __init__(self, config, exposure, footprints, noiseImage=None, exposureId=None, log=None):
137 noiseMeanVar = None
138 self.noiseSource = config.noiseSource
139 self.noiseOffset = config.noiseOffset
140 self.noiseSeedMultiplier = config.noiseSeedMultiplier
141 self.noiseGenMean = None
142 self.noiseGenStd = None
143 self.log = log
144
145 # creates heavies, replaces all footprints with noise
146 # We need the source table to be sorted by ID to do the parent lookups
147 self.exposure = exposure
148 self.footprints = footprints
149 mi = exposure.getMaskedImage()
150 im = mi.getImage()
151 mask = mi.getMask()
152 # Add temporary Mask planes for THISDET and OTHERDET
153 self.removeplanes = []
154 bitmasks = []
155 for maskname in ['THISDET', 'OTHERDET']:
156 if maskname in mask.getMaskPlaneDict():
157 # does it already exist?
158 plane = mask.getMaskPlane(maskname)
159 if self.log:
160 self.log.debug('Mask plane "%s" already existed', maskname)
161 else:
162 # if not, add it; we should delete it when done.
163 plane = mask.addMaskPlane(maskname)
164 self.removeplanes.append(maskname)
165 mask.clearMaskPlane(plane)
166 bitmask = mask.getPlaneBitMask(maskname)
167 bitmasks.append(bitmask)
168 if self.log:
169 self.log.debug('Mask plane "%s": plane %i, bitmask %i = 0x%x',
170 maskname, plane, bitmask, bitmask)
171 self.thisbitmask, self.otherbitmask = bitmasks
172 del bitmasks
173 self.heavies = {}
174 # Start by creating HeavyFootprints for each source which has no parent
175 # and just use them for children which do not already have heavy footprints.
176 # If a heavy footprint is available for a child, we will use it. Otherwise,
177 # we use the first parent in the parent chain which has a heavy footprint,
178 # which with the one level deblender will alway be the topmost parent
179 # NOTE: heavy footprints get destroyed by the transform process in forcedPhotCcd.py
180 # or forcedPhotCoadd.py so they are never available for forced measurements.
181
182 # Create in the dict heavies = {id:heavyfootprint}
183 for id, fp in footprints.items():
184 if fp[1].isHeavy():
185 self.heavies[id] = fp[1]
186 elif fp[0] == 0:
187 self.heavies[id] = afwDet.makeHeavyFootprint(fp[1], mi)
188
189 # ## FIXME: the heavy footprint includes the mask
190 # ## and variance planes, which we shouldn't need
191 # ## (I don't think we ever want to modify them in
192 # ## the input image). Copying them around is
193 # ## wasteful.
194
195 # We now create a noise HeavyFootprint for each source with has a heavy footprint.
196 # We'll put the noise footprints in a dict heavyNoise = {id:heavyNoiseFootprint}
197 self.heavyNoise = {}
198 noisegen = self.getNoiseGenerator(exposure, noiseImage, noiseMeanVar, exposureId=exposureId)
199 if self.log:
200 self.log.debug('Using noise generator: %s', str(noisegen))
201 for id in self.heavies:
202 fp = footprints[id][1]
203 noiseFp = noisegen.getHeavyFootprint(fp)
204 self.heavyNoise[id] = noiseFp
205 # Also insert the noisy footprint into the image now.
206 # Notice that we're just inserting it into "im", ie,
207 # the Image, not the MaskedImage.
208 noiseFp.insert(im)
209 # Also set the OTHERDET bit
210 fp.spans.setMask(mask, self.otherbitmask)
211
HeavyFootprint< ImagePixelT, MaskPixelT, VariancePixelT > makeHeavyFootprint(Footprint const &foot, lsst::afw::image::MaskedImage< ImagePixelT, MaskPixelT, VariancePixelT > const &img, HeavyFootprintCtrl const *ctrl=nullptr)
Create a HeavyFootprint with footprint defined by the given Footprint and pixel values from the given...
Definition HeavyFootprint.h:148
Member Function Documentation
◆ end()
| lsst.meas.base.noiseReplacer.NoiseReplacer.end | ( | self | ) |
End the NoiseReplacer. Restores original data to the exposure from the heavies dictionary and the mask planes to their original state.
Definition at line 272 of file noiseReplacer.py.
272 def end(self):
273 """End the NoiseReplacer.
274
275 Restores original data to the exposure from the heavies dictionary and
276 the mask planes to their original state.
277 """
278 # restores original image, cleans up temporaries
279 # (ie, replace all the top-level pixels)
280 mi = self.exposure.getMaskedImage()
281 im = mi.getImage()
282 mask = mi.getMask()
283 for id in self.footprints.keys():
284 if self.footprints[id][0] != 0:
285 continue
286 self.heavies[id].insert(im)
287 for maskname in self.removeplanes:
288 mask.removeAndClearMaskPlane(maskname, True)
289
290 del self.removeplanes
291 del self.thisbitmask
292 del self.otherbitmask
293 del self.heavies
294 del self.heavyNoise
295
◆ getNoiseGenerator()
| lsst.meas.base.noiseReplacer.NoiseReplacer.getNoiseGenerator | ( | self, | |
| exposure, | |||
| noiseImage, | |||
| noiseMeanVar, | |||
| exposureId = None ) |
Return a generator of artificial noise. Returns ------- noiseGenerator : `lsst.afw.image.noiseReplacer.NoiseGenerator`
Definition at line 296 of file noiseReplacer.py.
296 def getNoiseGenerator(self, exposure, noiseImage, noiseMeanVar, exposureId=None):
297 """Return a generator of artificial noise.
298
299 Returns
300 -------
301 noiseGenerator : `lsst.afw.image.noiseReplacer.NoiseGenerator`
302 """
303 if noiseImage is not None:
304 return ImageNoiseGenerator(noiseImage)
305 rand = None
306 if self.noiseSeedMultiplier:
307 # default plugin, our seed
308 if exposureId is not None and exposureId != 0:
309 seed = exposureId*self.noiseSeedMultiplier
310 else:
311 seed = self.noiseSeedMultiplier
312 rand = afwMath.Random(afwMath.Random.MT19937, seed)
313 if noiseMeanVar is not None:
314 try:
315 # Assume noiseMeanVar is an iterable of floats
316 noiseMean, noiseVar = noiseMeanVar
317 noiseMean = float(noiseMean)
318 noiseVar = float(noiseVar)
319 noiseStd = math.sqrt(noiseVar)
320 if self.log:
321 self.log.debug('Using passed-in noise mean = %g, variance = %g -> stdev %g',
322 noiseMean, noiseVar, noiseStd)
323 return FixedGaussianNoiseGenerator(noiseMean, noiseStd, rand=rand)
324 except Exception:
325 if self.log:
326 self.log.debug('Failed to cast passed-in noiseMeanVar to floats: %s',
327 str(noiseMeanVar))
328 offset = self.noiseOffset
329 noiseSource = self.noiseSource
330
331 if noiseSource == 'meta':
332 # check the exposure metadata
333 meta = exposure.getMetadata()
334 # this key name correspond to SubtractBackgroundTask() in meas_algorithms
335 try:
336 bgMean = meta.getAsDouble('BGMEAN')
337 # We would have to adjust for GAIN if ip_isr didn't make it 1.0
338 noiseStd = math.sqrt(bgMean)
339 if self.log:
340 self.log.debug('Using noise variance = (BGMEAN = %g) from exposure metadata',
341 bgMean)
342 return FixedGaussianNoiseGenerator(offset, noiseStd, rand=rand)
343 except Exception:
344 if self.log:
345 self.log.debug('Failed to get BGMEAN from exposure metadata')
346
347 if noiseSource == 'variance':
348 if self.log:
349 self.log.debug('Will draw noise according to the variance plane.')
350 var = exposure.getMaskedImage().getVariance()
351 return VariancePlaneNoiseGenerator(var, mean=offset, rand=rand)
352
353 if noiseSource == 'variance_median':
354 if self.log:
355 self.log.debug('Will draw noise using the median of the variance plane.')
356 var = exposure.getMaskedImage().getVariance()
357 s = afwMath.makeStatistics(var, afwMath.MEDIAN)
358 varMedian = s.getValue(afwMath.MEDIAN)
359 if self.log:
360 self.log.debug("Measured from variance: median variance = %g",
361 varMedian)
362 return FixedGaussianNoiseGenerator(offset, math.sqrt(varMedian), rand=rand)
363
364 # Compute an image-wide clipped variance.
365 im = exposure.getMaskedImage().getImage()
366 s = afwMath.makeStatistics(im, afwMath.MEANCLIP | afwMath.STDEVCLIP)
367 noiseMean = s.getValue(afwMath.MEANCLIP)
368 noiseStd = s.getValue(afwMath.STDEVCLIP)
369 if self.log:
370 self.log.debug("Measured from image: clipped mean = %g, stdev = %g",
371 noiseMean, noiseStd)
372 return FixedGaussianNoiseGenerator(noiseMean + offset, noiseStd, rand=rand)
373
374
A class that can be used to generate sequences of random numbers according to a number of different a...
Definition Random.h:57
Statistics makeStatistics(lsst::afw::image::Image< Pixel > const &img, lsst::afw::image::Mask< image::MaskPixel > const &msk, int const flags, StatisticsControl const &sctrl=StatisticsControl())
Handle a watered-down front-end to the constructor (no variance)
Definition Statistics.h:361
◆ insertSource()
| lsst.meas.base.noiseReplacer.NoiseReplacer.insertSource | ( | self, | |
| id ) |
Insert the heavy footprint of a given source into the exposure.
Parameters
----------
id : `int`
ID of the source to insert from original dictionary of footprints.
Notes
-----
Also adjusts the mask plane to show the source of this footprint.
Definition at line 212 of file noiseReplacer.py.
212 def insertSource(self, id):
213 """Insert the heavy footprint of a given source into the exposure.
214
215 Parameters
216 ----------
217 id : `int`
218 ID of the source to insert from original dictionary of footprints.
219
220 Notes
221 -----
222 Also adjusts the mask plane to show the source of this footprint.
223 """
224 # Copy this source's pixels into the image
225 mi = self.exposure.getMaskedImage()
226 im = mi.getImage()
227 mask = mi.getMask()
228 # usedid can point either to this source, or to the first parent in the
229 # parent chain which has a heavy footprint (or to the topmost parent,
230 # which always has one)
231 usedid = id
232 while self.footprints[usedid][0] != 0 and usedid not in self.heavies:
233 usedid = self.footprints[usedid][0]
234 fp = self.heavies[usedid]
235 fp.insert(im)
236 fp.spans.setMask(mask, self.thisbitmask)
237 fp.spans.clearMask(mask, self.otherbitmask)
238
◆ removeSource()
| lsst.meas.base.noiseReplacer.NoiseReplacer.removeSource | ( | self, | |
| id ) |
Replace the heavy footprint of a given source with noise.
The same artificial noise is used as in the original replacement.
Parameters
----------
id : `int`
ID of the source to replace from original dictionary of footprints.
Notes
-----
Also restores the mask plane.
Definition at line 239 of file noiseReplacer.py.
239 def removeSource(self, id):
240 """Replace the heavy footprint of a given source with noise.
241
242 The same artificial noise is used as in the original replacement.
243
244 Parameters
245 ----------
246 id : `int`
247 ID of the source to replace from original dictionary of footprints.
248
249 Notes
250 -----
251 Also restores the mask plane.
252 """
253 # remove a single source
254 # (Replace this source's pixels by noise again.)
255 # Do this by finding the source's top-level ancestor
256 mi = self.exposure.getMaskedImage()
257 im = mi.getImage()
258 mask = mi.getMask()
259
260 # use the same algorithm as in remove Source to find the heavy noise footprint
261 # which will undo what insertSource(id) does
262 usedid = id
263 while self.footprints[usedid][0] != 0 and usedid not in self.heavies:
264 usedid = self.footprints[usedid][0]
265 # Re-insert the noise pixels
266 fp = self.heavyNoise[usedid]
267 fp.insert(im)
268 # Clear the THISDET mask plane.
269 fp.spans.clearMask(mask, self.thisbitmask)
270 fp.spans.setMask(mask, self.otherbitmask)
271
Member Data Documentation
◆ ConfigClass
|
static |
Definition at line 122 of file noiseReplacer.py.
◆ exposure
|
static |
Definition at line 124 of file noiseReplacer.py.
◆ footprints
|
static |
Definition at line 128 of file noiseReplacer.py.
◆ heavies
| dict lsst.meas.base.noiseReplacer.NoiseReplacer.heavies = {} |
Definition at line 173 of file noiseReplacer.py.
◆ heavyNoise
| dict lsst.meas.base.noiseReplacer.NoiseReplacer.heavyNoise = {} |
Definition at line 197 of file noiseReplacer.py.
◆ log
|
static |
Definition at line 132 of file noiseReplacer.py.
◆ noiseGenMean
| lsst.meas.base.noiseReplacer.NoiseReplacer.noiseGenMean = None |
Definition at line 141 of file noiseReplacer.py.
◆ noiseGenStd
| lsst.meas.base.noiseReplacer.NoiseReplacer.noiseGenStd = None |
Definition at line 142 of file noiseReplacer.py.
◆ noiseOffset
| lsst.meas.base.noiseReplacer.NoiseReplacer.noiseOffset = config.noiseOffset |
Definition at line 139 of file noiseReplacer.py.
◆ noiseSeedMultiplier
| lsst.meas.base.noiseReplacer.NoiseReplacer.noiseSeedMultiplier = config.noiseSeedMultiplier |
Definition at line 140 of file noiseReplacer.py.
◆ noiseSource
| lsst.meas.base.noiseReplacer.NoiseReplacer.noiseSource = config.noiseSource |
Definition at line 138 of file noiseReplacer.py.
◆ otherbitmask
| lsst.meas.base.noiseReplacer.NoiseReplacer.otherbitmask = bitmasks |
Definition at line 171 of file noiseReplacer.py.
◆ removeplanes
| list lsst.meas.base.noiseReplacer.NoiseReplacer.removeplanes = [] |
Definition at line 153 of file noiseReplacer.py.
◆ thisbitmask
| lsst.meas.base.noiseReplacer.NoiseReplacer.thisbitmask = mask.getPlaneBitMask(maskname) |
Definition at line 171 of file noiseReplacer.py.
The documentation for this class was generated from the following file:
- /j/snowflake/release/lsstsw/stack/lsst-scipipe-10.0.0/Linux64/meas_base/g30358e5240+f0e04ebe90/python/lsst/meas/base/noiseReplacer.py
Generated on Thu Mar 27 2025 08:48:30 for LSST Applications by
1.13.2