lsst.cp.pipe.defects.MeasureDefectsTask Class Reference

Inheritance diagram for lsst.cp.pipe.defects.MeasureDefectsTask:

Public Member Functions
def	run (self, inputExp, camera)
def	findHotAndColdPixels (self, exp, nSigma)
def	maskBlocksIfIntermitentBadPixelsInColumn (self, defects)
def	debugView (self, stepname, ampImage, defects, detector)
def	debugHistogram (self, stepname, ampImage, nSigmaUsed, exp)

Static Public Attributes
	ConfigClass = MeasureDefectsTaskConfig

Detailed Description

Measure the defects from one exposure.

Definition at line 126 of file defects.py.

Member Function Documentation

debugHistogram()

def lsst.cp.pipe.defects.MeasureDefectsTask.debugHistogram	(		self,
			stepname,
			ampImage,
			nSigmaUsed,
			exp
	)

Make a histogram of the distribution of pixel values for each amp.

The main image data histogram is plotted in blue. Edge pixels,
if masked, are in red. Note that masked edge pixels do not contribute
to the underflow and overflow numbers.

Note that this currently only supports the 16-amp LSST detectors.

Parameters
----------
dataRef : `lsst.daf.persistence.ButlerDataRef`
    dataRef for the detector.
exp : `lsst.afw.image.exposure.Exposure`
    The exposure in which the defects were found.
visit : `int`
    The visit number.
nSigmaUsed : `float`
    The number of sigma used for detection

Definition at line 439 of file defects.py.

    def debugHistogram(self, stepname, ampImage, nSigmaUsed, exp):
        """
        Make a histogram of the distribution of pixel values for each amp.
 
        The main image data histogram is plotted in blue. Edge pixels,
        if masked, are in red. Note that masked edge pixels do not contribute
        to the underflow and overflow numbers.
 
        Note that this currently only supports the 16-amp LSST detectors.
 
        Parameters
        ----------
        dataRef : `lsst.daf.persistence.ButlerDataRef`
            dataRef for the detector.
        exp : `lsst.afw.image.exposure.Exposure`
            The exposure in which the defects were found.
        visit : `int`
            The visit number.
        nSigmaUsed : `float`
            The number of sigma used for detection
        """
        frame = getDebugFrame(self._display, stepname)
        if frame:
            import matplotlib.pyplot as plt
 
            detector = exp.getDetector()
            nX = np.floor(np.sqrt(len(detector)))
            nY = len(detector) // nX
            fig, ax = plt.subplots(nrows=nY, ncols=nX, sharex='col', sharey='row', figsize=(13, 10))
 
            expTime = exp.getInfo().getVisitInfo().getExposureTime()
 
            for (amp, a) in zip(reversed(detector), ax.flatten()):
                mi = exp.maskedImage[amp.getBBox()]
 
                # normalize by expTime as we plot in ADU/s and don't always work with master calibs
                mi.image.array /= expTime
                stats = afwMath.makeStatistics(mi, afwMath.MEANCLIP | afwMath.STDEVCLIP)
                mean, sigma = stats.getValue(afwMath.MEANCLIP), stats.getValue(afwMath.STDEVCLIP)
                # Get array of pixels
                EDGEBIT = exp.maskedImage.mask.getPlaneBitMask("EDGE")
                imgData = mi.image.array[(mi.mask.array & EDGEBIT) == 0].flatten()
                edgeData = mi.image.array[(mi.mask.array & EDGEBIT) != 0].flatten()
 
                thrUpper = mean + nSigmaUsed*sigma
                thrLower = mean - nSigmaUsed*sigma
 
                nRight = len(imgData[imgData > thrUpper])
                nLeft = len(imgData[imgData < thrLower])
 
                nsig = nSigmaUsed + 1.2  # add something small so the edge of the plot is out from level used
                leftEdge = mean - nsig * nSigmaUsed*sigma
                rightEdge = mean + nsig * nSigmaUsed*sigma
                nbins = np.linspace(leftEdge, rightEdge, 1000)
                ey, bin_borders, patches = a.hist(edgeData, histtype='step', bins=nbins,
                                                  lw=1, edgecolor='red')
                y, bin_borders, patches = a.hist(imgData, histtype='step', bins=nbins,
                                                 lw=3, edgecolor='blue')
 
                # Report number of entries in over-and -underflow bins, i.e. off the edges of the histogram
                nOverflow = len(imgData[imgData > rightEdge])
                nUnderflow = len(imgData[imgData < leftEdge])
 
                # Put v-lines and textboxes in
                a.axvline(thrUpper, c='k')
                a.axvline(thrLower, c='k')
                msg = f"{amp.getName()}\nmean:{mean: .2f}\n$\\sigma$:{sigma: .2f}"
                a.text(0.65, 0.6, msg, transform=a.transAxes, fontsize=11)
                msg = f"nLeft:{nLeft}\nnRight:{nRight}\nnOverflow:{nOverflow}\nnUnderflow:{nUnderflow}"
                a.text(0.03, 0.6, msg, transform=a.transAxes, fontsize=11.5)
 
                # set axis limits and scales
                a.set_ylim([1., 1.7*np.max(y)])
                lPlot, rPlot = a.get_xlim()
                a.set_xlim(np.array([lPlot, rPlot]))
                a.set_yscale('log')
                a.set_xlabel("ADU/s")
        return
 
 

debugView()

def lsst.cp.pipe.defects.MeasureDefectsTask.debugView	(		self,
			stepname,
			ampImage,
			defects,
			detector
	)

Plot the defects found by the task.

Parameters
----------
exp : `lsst.afw.image.exposure.Exposure`
    The exposure in which the defects were found.
visit : `int`
    The visit number.
defects : `lsst.ip.isr.Defect`
    The defects to plot.
imageType : `str`
    The type of image, either 'dark' or 'flat'.

Definition at line 399 of file defects.py.

    def debugView(self, stepname, ampImage, defects, detector):
        # def _plotDefects(self, exp, visit, defects, imageType):  # pragma: no cover
        """Plot the defects found by the task.
 
        Parameters
        ----------
        exp : `lsst.afw.image.exposure.Exposure`
            The exposure in which the defects were found.
        visit : `int`
            The visit number.
        defects : `lsst.ip.isr.Defect`
            The defects to plot.
        imageType : `str`
            The type of image, either 'dark' or 'flat'.
        """
        frame = getDebugFrame(self._display, stepname)
        if frame:
            disp = afwDisplay.Display(frame=frame)
            disp.scale('asinh', 'zscale')
            disp.setMaskTransparency(80)
            disp.setMaskPlaneColor("BAD", afwDisplay.RED)
 
            maskedIm = ampImage.clone()
            defects.maskPixels(maskedIm, "BAD")
 
            mpDict = maskedIm.mask.getMaskPlaneDict()
            for plane in mpDict.keys():
                if plane in ['BAD']:
                    continue
                disp.setMaskPlaneColor(plane, afwDisplay.IGNORE)
 
            disp.setImageColormap('gray')
            disp.mtv(maskedIm)
            cameraGeom.utils.overlayCcdBoxes(detector, isTrimmed=True, display=disp)
            prompt = "Press Enter to continue [c]... "
            while True:
                ans = input(prompt).lower()
                if ans in ('', 'c', ):
                    break
 

findHotAndColdPixels()

def lsst.cp.pipe.defects.MeasureDefectsTask.findHotAndColdPixels	(		self,
			exp,
			nSigma
	)

Find hot and cold pixels in an image.

Using config-defined thresholds on a per-amp basis, mask
pixels that are nSigma above threshold in dark frames (hot
pixels), or nSigma away from the clipped mean in flats (hot &
cold pixels).

Parameters
----------
exp : `lsst.afw.image.exposure.Exposure`
    The exposure in which to find defects.
nSigma : `list [ `float` ]
    Detection threshold to use.  Positive for DETECTED pixels,
    negative for DETECTED_NEGATIVE pixels.

Returns
-------
defects : `lsst.ip.isr.Defect`
    The defects found in the image.

Definition at line 190 of file defects.py.

    def findHotAndColdPixels(self, exp, nSigma):
        """Find hot and cold pixels in an image.
 
        Using config-defined thresholds on a per-amp basis, mask
        pixels that are nSigma above threshold in dark frames (hot
        pixels), or nSigma away from the clipped mean in flats (hot &
        cold pixels).
 
        Parameters
        ----------
        exp : `lsst.afw.image.exposure.Exposure`
            The exposure in which to find defects.
        nSigma : `list [ `float` ]
            Detection threshold to use.  Positive for DETECTED pixels,
            negative for DETECTED_NEGATIVE pixels.
 
        Returns
        -------
        defects : `lsst.ip.isr.Defect`
            The defects found in the image.
 
        """
 
        self._setEdgeBits(exp)
        maskedIm = exp.maskedImage
 
        # the detection polarity for afwDetection, True for positive,
        # False for negative, and therefore True for darks as they only have
        # bright pixels, and both for flats, as they have bright and dark pix
        footprintList = []
 
        for amp in exp.getDetector():
            ampImg = maskedIm[amp.getBBox()].clone()
 
            # crop ampImage depending on where the amp lies in the image
            if self.config.nPixBorderLeftRight:
                if ampImg.getX0() == 0:
                    ampImg = ampImg[self.config.nPixBorderLeftRight:, :, afwImage.LOCAL]
                else:
                    ampImg = ampImg[:-self.config.nPixBorderLeftRight, :, afwImage.LOCAL]
            if self.config.nPixBorderUpDown:
                if ampImg.getY0() == 0:
                    ampImg = ampImg[:, self.config.nPixBorderUpDown:, afwImage.LOCAL]
                else:
                    ampImg = ampImg[:, :-self.config.nPixBorderUpDown, afwImage.LOCAL]
 
            if self._getNumGoodPixels(ampImg) == 0:  # amp contains no usable pixels
                continue
 
            # Remove a background estimate
            ampImg -= afwMath.makeStatistics(ampImg, afwMath.MEANCLIP, ).getValue()
 
            mergedSet = None
            for sigma in nSigma:
                nSig = np.abs(sigma)
                self.debugHistogram('ampFlux', ampImg, nSig, exp)
                polarity = {-1: False, 1: True}[np.sign(sigma)]
 
                threshold = afwDetection.createThreshold(nSig, 'stdev', polarity=polarity)
 
                footprintSet = afwDetection.FootprintSet(ampImg, threshold)
                footprintSet.setMask(maskedIm.mask, ("DETECTED" if polarity else "DETECTED_NEGATIVE"))
 
                if mergedSet is None:
                    mergedSet = footprintSet
                else:
                    mergedSet.merge(footprintSet)
 
            footprintList += mergedSet.getFootprints()
 
            self.debugView('defectMap', ampImg,
                           Defects.fromFootprintList(mergedSet.getFootprints()), exp.getDetector())
 
        defects = Defects.fromFootprintList(footprintList)
        defects = self.maskBlocksIfIntermitentBadPixelsInColumn(defects)
 
        return defects
 

maskBlocksIfIntermitentBadPixelsInColumn()

def lsst.cp.pipe.defects.MeasureDefectsTask.maskBlocksIfIntermitentBadPixelsInColumn	(		self,
			defects
	)

Mask blocks in a column if there are on-and-off bad pixels

If there's a column with on and off bad pixels, mask all the
pixels in between, except if there is a large enough gap of
consecutive good pixels between two bad pixels in the column.

Parameters
---------
defects: `lsst.ip.isr.Defect`
    The defects found in the image so far

Returns
------
defects: `lsst.ip.isr.Defect`
    If the number of bad pixels in a column is not larger or
    equal than self.config.badPixelColumnThreshold, the iput
    list is returned. Otherwise, the defects list returned
    will include boxes that mask blocks of on-and-of pixels.

Definition at line 298 of file defects.py.

    def maskBlocksIfIntermitentBadPixelsInColumn(self, defects):
        """Mask blocks in a column if there are on-and-off bad pixels
 
        If there's a column with on and off bad pixels, mask all the
        pixels in between, except if there is a large enough gap of
        consecutive good pixels between two bad pixels in the column.
 
        Parameters
        ---------
        defects: `lsst.ip.isr.Defect`
            The defects found in the image so far
 
        Returns
        ------
        defects: `lsst.ip.isr.Defect`
            If the number of bad pixels in a column is not larger or
            equal than self.config.badPixelColumnThreshold, the iput
            list is returned. Otherwise, the defects list returned
            will include boxes that mask blocks of on-and-of pixels.
 
        """
        # Get the (x, y) values of each bad pixel in amp.
        coordinates = []
        for defect in defects:
            bbox = defect.getBBox()
            x0, y0 = bbox.getMinX(), bbox.getMinY()
            deltaX0, deltaY0 = bbox.getDimensions()
            for j in np.arange(y0, y0+deltaY0):
                for i in np.arange(x0, x0 + deltaX0):
                    coordinates.append((i, j))
 
        x, y = [], []
        for coordinatePair in coordinates:
            x.append(coordinatePair[0])
            y.append(coordinatePair[1])
 
        x = np.array(x)
        y = np.array(y)
        # Find the defects with same "x" (vertical) coordinate (column).
        unique, counts = np.unique(x, return_counts=True)
        multipleX = []
        for (a, b) in zip(unique, counts):
            if b >= self.config.badOnAndOffPixelColumnThreshold:
                multipleX.append(a)
        if len(multipleX) != 0:
            defects = self._markBlocksInBadColumn(x, y, multipleX, defects)
 
        return defects
 

run()

def lsst.cp.pipe.defects.MeasureDefectsTask.run	(		self,
			inputExp,
			camera
	)

Measure one exposure for defects.

Parameters
----------
inputExp : `lsst.afw.image.Exposure`
     Exposure to examine.
camera : `lsst.afw.cameraGeom.Camera`
     Camera to use for metadata.

Returns
-------
results : `lsst.pipe.base.Struct`
     Results struct containing:
     - ``outputDefects` : `lsst.ip.isr.Defects`
         The defects measured from this exposure.

Definition at line 132 of file defects.py.

    def run(self, inputExp, camera):
        """Measure one exposure for defects.
 
        Parameters
        ----------
        inputExp : `lsst.afw.image.Exposure`
             Exposure to examine.
        camera : `lsst.afw.cameraGeom.Camera`
             Camera to use for metadata.
 
        Returns
        -------
        results : `lsst.pipe.base.Struct`
             Results struct containing:
             - ``outputDefects` : `lsst.ip.isr.Defects`
                 The defects measured from this exposure.
        """
        detector = inputExp.getDetector()
 
        filterName = inputExp.getFilterLabel().physicalLabel
        datasetType = inputExp.getMetadata().get('IMGTYPE', 'UNKNOWN')
 
        if datasetType.lower() == 'dark':
            nSigmaList = [self.config.nSigmaBright]
        else:
            nSigmaList = [self.config.nSigmaBright, self.config.nSigmaDark]
        defects = self.findHotAndColdPixels(inputExp, nSigmaList)
 
        msg = "Found %s defects containing %s pixels in %s"
        self.log.info(msg, len(defects), self._nPixFromDefects(defects), datasetType)
 
        defects.updateMetadata(camera=camera, detector=detector, filterName=filterName,
                               setCalibId=True, setDate=True,
                               cpDefectGenImageType=datasetType)
 
        return pipeBase.Struct(
            outputDefects=defects,
        )
 

Member Data Documentation

ConfigClass

lsst.cp.pipe.defects.MeasureDefectsTask.ConfigClass = MeasureDefectsTaskConfig

static

Definition at line 129 of file defects.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-0.6.0/Linux64/cp_pipe/22.0.1-18-gb17765a+2264247a6b/python/lsst/cp/pipe/defects.py