lsst.ip.diffim.imageMapReduce.ImageReducer Class Reference

Inheritance diagram for lsst.ip.diffim.imageMapReduce.ImageReducer:

Public Member Functions
	run (self, mapperResults, exposure, **kwargs)

Static Public Attributes
	ConfigClass = ImageReducerConfig

Protected Member Functions
	_constructPsf (self, mapperResults, exposure)

Static Protected Attributes
str	_DefaultName = "ip_diffim_ImageReducer"

Detailed Description

Base class for any 'reduce' task that is to be
used as `ImageMapReduceConfig.reducer`.

Basic reduce operations are provided by the `run` method
of this class, to be selected by its config.

Definition at line 181 of file imageMapReduce.py.

Member Function Documentation

_constructPsf()

lsst.ip.diffim.imageMapReduce.ImageReducer._constructPsf ( self, mapperResults, exposure )

protected

Construct a CoaddPsf based on PSFs from individual subExposures

Currently uses (and returns) a CoaddPsf. TBD if we want to
create a custom subclass of CoaddPsf to differentiate it.

Parameters
----------
mapperResults : `list`
    list of `pipeBase.Struct` returned by `ImageMapper.run`.
    For this to work, each element of `mapperResults` must contain
    a `subExposure` element, from which the component Psfs are
    extracted (thus the reducerTask cannot have
    `reduceOperation = 'none'`.
exposure : `lsst.afw.image.Exposure`
    the original exposure which is used here solely for its
    bounding-box and WCS.

Returns
-------
psf : `lsst.meas.algorithms.CoaddPsf`
    A psf constructed from the PSFs of the individual subExposures.

Definition at line 325 of file imageMapReduce.py.

    def _constructPsf(self, mapperResults, exposure):
        """Construct a CoaddPsf based on PSFs from individual subExposures
 
        Currently uses (and returns) a CoaddPsf. TBD if we want to
        create a custom subclass of CoaddPsf to differentiate it.
 
        Parameters
        ----------
        mapperResults : `list`
            list of `pipeBase.Struct` returned by `ImageMapper.run`.
            For this to work, each element of `mapperResults` must contain
            a `subExposure` element, from which the component Psfs are
            extracted (thus the reducerTask cannot have
            `reduceOperation = 'none'`.
        exposure : `lsst.afw.image.Exposure`
            the original exposure which is used here solely for its
            bounding-box and WCS.
 
        Returns
        -------
        psf : `lsst.meas.algorithms.CoaddPsf`
            A psf constructed from the PSFs of the individual subExposures.
        """
        schema = afwTable.ExposureTable.makeMinimalSchema()
        schema.addField("weight", type="D", doc="Coadd weight")
        mycatalog = afwTable.ExposureCatalog(schema)
 
        # We're just using the exposure's WCS (assuming that the subExposures'
        # WCSs are the same, which they better be!).
        wcsref = exposure.getWcs()
        for i, res in enumerate(mapperResults):
            record = mycatalog.getTable().makeRecord()
            if 'subExposure' in res.getDict():
                subExp = res.subExposure
                if subExp.getWcs() != wcsref:
                    raise ValueError('Wcs of subExposure is different from exposure')
                record.setPsf(subExp.getPsf())
                record.setWcs(subExp.getWcs())
                record.setBBox(subExp.getBBox())
            elif 'psf' in res.getDict():
                record.setPsf(res.psf)
                record.setWcs(wcsref)
                record.setBBox(res.bbox)
            record['weight'] = 1.0
            record['id'] = i
            mycatalog.append(record)
 
        # create the coaddpsf
        psf = measAlg.CoaddPsf(mycatalog, wcsref, 'weight')
        return psf
 
 

run()

lsst.ip.diffim.imageMapReduce.ImageReducer.run	(		self,
			mapperResults,
			exposure,
		**	kwargs )

Reduce a list of items produced by `ImageMapper`.

Either stitch the passed `mapperResults` list
together into a new Exposure (default) or pass it through
(if `self.config.reduceOperation` is 'none').

If `self.config.reduceOperation` is not 'none', then expect
that the `pipeBase.Struct`s in the `mapperResults` list
contain sub-exposures named 'subExposure', to be stitched back
into a single Exposure with the same dimensions, PSF, and mask
as the input `exposure`. Otherwise, the `mapperResults` list
is simply returned directly.

Parameters
----------
mapperResults : `list`
    list of `lsst.pipe.base.Struct` returned by `ImageMapper.run`.
exposure : `lsst.afw.image.Exposure`
    the original exposure which is cloned to use as the
    basis for the resulting exposure (if
    ``self.config.mapper.reduceOperation`` is not 'None')
kwargs :
    additional keyword arguments propagated from
    `ImageMapReduceTask.run`.

Returns
-------
A `lsst.pipe.base.Struct` containing either an `lsst.afw.image.Exposure`
(named 'exposure') or a list (named 'result'),
depending on `config.reduceOperation`.

Notes
-----
1. This currently correctly handles overlapping sub-exposures.
   For overlapping sub-exposures, use `config.reduceOperation='average'`.
2. This correctly handles varying PSFs, constructing the resulting
   exposure's PSF via CoaddPsf (DM-9629).

Known issues

1. To be done: correct handling of masks (nearly there)
2. This logic currently makes *two* copies of the original exposure
   (one here and one in `mapper.run()`). Possibly of concern
   for large images on memory-constrained systems.

Definition at line 191 of file imageMapReduce.py.

    def run(self, mapperResults, exposure, **kwargs):
        """Reduce a list of items produced by `ImageMapper`.
 
        Either stitch the passed `mapperResults` list
        together into a new Exposure (default) or pass it through
        (if `self.config.reduceOperation` is 'none').
 
        If `self.config.reduceOperation` is not 'none', then expect
        that the `pipeBase.Struct`s in the `mapperResults` list
        contain sub-exposures named 'subExposure', to be stitched back
        into a single Exposure with the same dimensions, PSF, and mask
        as the input `exposure`. Otherwise, the `mapperResults` list
        is simply returned directly.
 
        Parameters
        ----------
        mapperResults : `list`
            list of `lsst.pipe.base.Struct` returned by `ImageMapper.run`.
        exposure : `lsst.afw.image.Exposure`
            the original exposure which is cloned to use as the
            basis for the resulting exposure (if
            ``self.config.mapper.reduceOperation`` is not 'None')
        kwargs :
            additional keyword arguments propagated from
            `ImageMapReduceTask.run`.
 
        Returns
        -------
        A `lsst.pipe.base.Struct` containing either an `lsst.afw.image.Exposure`
        (named 'exposure') or a list (named 'result'),
        depending on `config.reduceOperation`.
 
        Notes
        -----
        1. This currently correctly handles overlapping sub-exposures.
           For overlapping sub-exposures, use `config.reduceOperation='average'`.
        2. This correctly handles varying PSFs, constructing the resulting
           exposure's PSF via CoaddPsf (DM-9629).
 
        Known issues
 
        1. To be done: correct handling of masks (nearly there)
        2. This logic currently makes *two* copies of the original exposure
           (one here and one in `mapper.run()`). Possibly of concern
           for large images on memory-constrained systems.
        """
        # No-op; simply pass mapperResults directly to ImageMapReduceTask.run
        if self.config.reduceOperation == 'none':
            return pipeBase.Struct(result=mapperResults)
 
        if self.config.reduceOperation == 'coaddPsf':
            # Each element of `mapperResults` should contain 'psf' and 'bbox'
            coaddPsf = self._constructPsf(mapperResults, exposure)
            return pipeBase.Struct(result=coaddPsf)
 
        newExp = exposure.clone()
        newMI = newExp.getMaskedImage()
 
        reduceOp = self.config.reduceOperation
        if reduceOp == 'copy':
            weights = None
            newMI.image.array[:, :] = np.nan
            newMI.variance.array[:, :] = np.nan
        else:
            newMI.image.array[:, :] = 0.
            newMI.variance.array[:, :] = 0.
            if reduceOp == 'average':  # make an array to keep track of weights
                weights = afwImage.ImageI(newMI.getBBox())
 
        for item in mapperResults:
            item = item.subExposure  # Expected named value in the pipeBase.Struct
            if not (isinstance(item, afwImage.ExposureF) or isinstance(item, afwImage.ExposureI)
                    or isinstance(item, afwImage.ExposureU) or isinstance(item, afwImage.ExposureD)):
                raise TypeError("""Expecting an Exposure type, got %s.
                                   Consider using `reduceOperation="none".""" % str(type(item)))
            subExp = newExp.Factory(newExp, item.getBBox())
            subMI = subExp.getMaskedImage()
            patchMI = item.getMaskedImage()
            isValid = ~np.isnan(patchMI.image.array * patchMI.variance.array)
 
            if reduceOp == 'copy':
                subMI.image.array[isValid] = patchMI.image.array[isValid]
                subMI.variance.array[isValid] = patchMI.variance.array[isValid]
                subMI.mask.array[:, :] |= patchMI.mask.array
 
            if reduceOp == 'sum' or reduceOp == 'average':  # much of these two options is the same
                subMI.image.array[isValid] += patchMI.image.array[isValid]
                subMI.variance.array[isValid] += patchMI.variance.array[isValid]
                subMI.mask.array[:, :] |= patchMI.mask.array
                if reduceOp == 'average':
                    # wtsView is a view into the `weights` Image
                    wtsView = afwImage.ImageI(weights, item.getBBox())
                    wtsView.array[isValid] += 1
 
        # New mask plane - for debugging map-reduced images
        mask = newMI.mask
        for m in self.config.badMaskPlanes:
            mask.addMaskPlane(m)
        bad = mask.getPlaneBitMask(self.config.badMaskPlanes)
 
        isNan = np.where(np.isnan(newMI.image.array * newMI.variance.array))
        if len(isNan[0]) > 0:
            # set mask to INVALID for pixels where produced exposure is NaN
            mask.array[isNan[0], isNan[1]] |= bad
 
        if reduceOp == 'average':
            wts = weights.array.astype(float)
            self.log.info('AVERAGE: Maximum overlap: %f', np.nanmax(wts))
            self.log.info('AVERAGE: Average overlap: %f', np.nanmean(wts))
            self.log.info('AVERAGE: Minimum overlap: %f', np.nanmin(wts))
            wtsZero = np.equal(wts, 0.)
            wtsZeroInds = np.where(wtsZero)
            wtsZeroSum = len(wtsZeroInds[0])
            self.log.info('AVERAGE: Number of zero pixels: %f (%f%%)', wtsZeroSum,
                          wtsZeroSum * 100. / wtsZero.size)
            notWtsZero = ~wtsZero
            tmp = newMI.image.array
            np.divide(tmp, wts, out=tmp, where=notWtsZero)
            tmp = newMI.variance.array
            np.divide(tmp, wts, out=tmp, where=notWtsZero)
            if len(wtsZeroInds[0]) > 0:
                newMI.image.array[wtsZeroInds] = np.nan
                newMI.variance.array[wtsZeroInds] = np.nan
                # set mask to something for pixels where wts == 0.
                # happens sometimes if operation failed on a certain subexposure
                mask.array[wtsZeroInds] |= bad
 
        # Not sure how to construct a PSF when reduceOp=='copy'...
        if reduceOp == 'sum' or reduceOp == 'average':
            psf = self._constructPsf(mapperResults, exposure)
            newExp.setPsf(psf)
 
        return pipeBase.Struct(exposure=newExp)
 

Member Data Documentation

_DefaultName

str lsst.ip.diffim.imageMapReduce.ImageReducer._DefaultName = "ip_diffim_ImageReducer"

staticprotected

Definition at line 189 of file imageMapReduce.py.

ConfigClass

lsst.ip.diffim.imageMapReduce.ImageReducer.ConfigClass = ImageReducerConfig

static

Definition at line 188 of file imageMapReduce.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-8.0.0/Linux64/ip_diffim/g1470d8bcf6+3ea6592b6f/python/lsst/ip/diffim/imageMapReduce.py