lsst.ip.diffim.imageMapReduce.ImageReducer Class Reference

Inheritance diagram for lsst.ip.diffim.imageMapReduce.ImageReducer:

Public Member Functions
def	run (self, mapperResults, exposure, **kwargs)

Static Public Attributes
	ConfigClass = ImageReducerConfig

Detailed Description

Base class for any 'reduce' task that is to be
used as `ImageMapReduceConfig.reducer`.

Basic reduce operations are provided by the `run` method
of this class, to be selected by its config.

Definition at line 180 of file imageMapReduce.py.

Member Function Documentation

run()

def lsst.ip.diffim.imageMapReduce.ImageReducer.run	(		self,
			mapperResults,
			exposure,
		**	kwargs
	)

Reduce a list of items produced by `ImageMapper`.

Either stitch the passed `mapperResults` list
together into a new Exposure (default) or pass it through
(if `self.config.reduceOperation` is 'none').

If `self.config.reduceOperation` is not 'none', then expect
that the `pipeBase.Struct`s in the `mapperResults` list
contain sub-exposures named 'subExposure', to be stitched back
into a single Exposure with the same dimensions, PSF, and mask
as the input `exposure`. Otherwise, the `mapperResults` list
is simply returned directly.

Parameters
----------
mapperResults : `list`
    list of `lsst.pipe.base.Struct` returned by `ImageMapper.run`.
exposure : `lsst.afw.image.Exposure`
    the original exposure which is cloned to use as the
    basis for the resulting exposure (if
    ``self.config.mapper.reduceOperation`` is not 'None')
kwargs :
    additional keyword arguments propagated from
    `ImageMapReduceTask.run`.

Returns
-------
A `lsst.pipe.base.Struct` containing either an `lsst.afw.image.Exposure`
(named 'exposure') or a list (named 'result'),
depending on `config.reduceOperation`.

Notes
-----
1. This currently correctly handles overlapping sub-exposures.
   For overlapping sub-exposures, use `config.reduceOperation='average'`.
2. This correctly handles varying PSFs, constructing the resulting
   exposure's PSF via CoaddPsf (DM-9629).

Known issues

1. To be done: correct handling of masks (nearly there)
2. This logic currently makes *two* copies of the original exposure
   (one here and one in `mapper.run()`). Possibly of concern
   for large images on memory-constrained systems.

Definition at line 190 of file imageMapReduce.py.

    def run(self, mapperResults, exposure, **kwargs):
        """Reduce a list of items produced by `ImageMapper`.
 
        Either stitch the passed `mapperResults` list
        together into a new Exposure (default) or pass it through
        (if `self.config.reduceOperation` is 'none').
 
        If `self.config.reduceOperation` is not 'none', then expect
        that the `pipeBase.Struct`s in the `mapperResults` list
        contain sub-exposures named 'subExposure', to be stitched back
        into a single Exposure with the same dimensions, PSF, and mask
        as the input `exposure`. Otherwise, the `mapperResults` list
        is simply returned directly.
 
        Parameters
        ----------
        mapperResults : `list`
            list of `lsst.pipe.base.Struct` returned by `ImageMapper.run`.
        exposure : `lsst.afw.image.Exposure`
            the original exposure which is cloned to use as the
            basis for the resulting exposure (if
            ``self.config.mapper.reduceOperation`` is not 'None')
        kwargs :
            additional keyword arguments propagated from
            `ImageMapReduceTask.run`.
 
        Returns
        -------
        A `lsst.pipe.base.Struct` containing either an `lsst.afw.image.Exposure`
        (named 'exposure') or a list (named 'result'),
        depending on `config.reduceOperation`.
 
        Notes
        -----
        1. This currently correctly handles overlapping sub-exposures.
           For overlapping sub-exposures, use `config.reduceOperation='average'`.
        2. This correctly handles varying PSFs, constructing the resulting
           exposure's PSF via CoaddPsf (DM-9629).
 
        Known issues
 
        1. To be done: correct handling of masks (nearly there)
        2. This logic currently makes *two* copies of the original exposure
           (one here and one in `mapper.run()`). Possibly of concern
           for large images on memory-constrained systems.
        """
        # No-op; simply pass mapperResults directly to ImageMapReduceTask.run
        if self.config.reduceOperation == 'none':
            return pipeBase.Struct(result=mapperResults)
 
        if self.config.reduceOperation == 'coaddPsf':
            # Each element of `mapperResults` should contain 'psf' and 'bbox'
            coaddPsf = self._constructPsf(mapperResults, exposure)
            return pipeBase.Struct(result=coaddPsf)
 
        newExp = exposure.clone()
        newMI = newExp.getMaskedImage()
 
        reduceOp = self.config.reduceOperation
        if reduceOp == 'copy':
            weights = None
            newMI.getImage()[:, :] = np.nan
            newMI.getVariance()[:, :] = np.nan
        else:
            newMI.getImage()[:, :] = 0.
            newMI.getVariance()[:, :] = 0.
            if reduceOp == 'average':  # make an array to keep track of weights
                weights = afwImage.ImageI(newMI.getBBox())
 
        for item in mapperResults:
            item = item.subExposure  # Expected named value in the pipeBase.Struct
            if not (isinstance(item, afwImage.ExposureF) or isinstance(item, afwImage.ExposureI)
                    or isinstance(item, afwImage.ExposureU) or isinstance(item, afwImage.ExposureD)):
                raise TypeError("""Expecting an Exposure type, got %s.
                                   Consider using `reduceOperation="none".""" % str(type(item)))
            subExp = newExp.Factory(newExp, item.getBBox())
            subMI = subExp.getMaskedImage()
            patchMI = item.getMaskedImage()
            isValid = ~np.isnan(patchMI.getImage().getArray() * patchMI.getVariance().getArray())
 
            if reduceOp == 'copy':
                subMI.getImage().getArray()[isValid] = patchMI.getImage().getArray()[isValid]
                subMI.getVariance().getArray()[isValid] = patchMI.getVariance().getArray()[isValid]
                subMI.getMask().getArray()[:, :] |= patchMI.getMask().getArray()
 
            if reduceOp == 'sum' or reduceOp == 'average':  # much of these two options is the same
                subMI.getImage().getArray()[isValid] += patchMI.getImage().getArray()[isValid]
                subMI.getVariance().getArray()[isValid] += patchMI.getVariance().getArray()[isValid]
                subMI.getMask().getArray()[:, :] |= patchMI.getMask().getArray()
                if reduceOp == 'average':
                    # wtsView is a view into the `weights` Image
                    wtsView = afwImage.ImageI(weights, item.getBBox())
                    wtsView.getArray()[isValid] += 1
 
        # New mask plane - for debugging map-reduced images
        mask = newMI.getMask()
        for m in self.config.badMaskPlanes:
            mask.addMaskPlane(m)
        bad = mask.getPlaneBitMask(self.config.badMaskPlanes)
 
        isNan = np.where(np.isnan(newMI.getImage().getArray() * newMI.getVariance().getArray()))
        if len(isNan[0]) > 0:
            # set mask to INVALID for pixels where produced exposure is NaN
            mask.getArray()[isNan[0], isNan[1]] |= bad
 
        if reduceOp == 'average':
            wts = weights.getArray().astype(np.float)
            self.log.info('AVERAGE: Maximum overlap: %f', np.nanmax(wts))
            self.log.info('AVERAGE: Average overlap: %f', np.nanmean(wts))
            self.log.info('AVERAGE: Minimum overlap: %f', np.nanmin(wts))
            wtsZero = np.equal(wts, 0.)
            wtsZeroInds = np.where(wtsZero)
            wtsZeroSum = len(wtsZeroInds[0])
            self.log.info('AVERAGE: Number of zero pixels: %f (%f%%)', wtsZeroSum,
                          wtsZeroSum * 100. / wtsZero.size)
            notWtsZero = ~wtsZero
            tmp = newMI.getImage().getArray()
            np.divide(tmp, wts, out=tmp, where=notWtsZero)
            tmp = newMI.getVariance().getArray()
            np.divide(tmp, wts, out=tmp, where=notWtsZero)
            if len(wtsZeroInds[0]) > 0:
                newMI.getImage().getArray()[wtsZeroInds] = np.nan
                newMI.getVariance().getArray()[wtsZeroInds] = np.nan
                # set mask to something for pixels where wts == 0.
                # happens sometimes if operation failed on a certain subexposure
                mask.getArray()[wtsZeroInds] |= bad
 
        # Not sure how to construct a PSF when reduceOp=='copy'...
        if reduceOp == 'sum' or reduceOp == 'average':
            psf = self._constructPsf(mapperResults, exposure)
            newExp.setPsf(psf)
 
        return pipeBase.Struct(exposure=newExp)
 

Member Data Documentation

ConfigClass

lsst.ip.diffim.imageMapReduce.ImageReducer.ConfigClass = ImageReducerConfig

static

Definition at line 187 of file imageMapReduce.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/0.4.3/Linux64/ip_diffim/21.0.0-11-ga42c5b2+86977b0b17/python/lsst/ip/diffim/imageMapReduce.py