Inheritance diagram for lsst.cp.pipe.cpFlatNormTask.CpFlatNormalizationTask:

Public Member Functions
def	runQuantum (self, butlerQC, inputRefs, outputRefs)

def	run (self, inputMDs, inputDims, camera)

def	measureScales (self, bgMatrix, bgCounts=None, iterations=10)

Static Public Attributes
	ConfigClass = CpFlatNormalizationTaskConfig

Detailed Description

Rescale merged flat frames to remove unequal screen illumination.

Definition at line 182 of file cpFlatNormTask.py.

Member Function Documentation

◆ measureScales()

def lsst.cp.pipe.cpFlatNormTask.CpFlatNormalizationTask.measureScales	(	self,
		bgMatrix,
		bgCounts = `None`,
		iterations = `10`
	)

Convert backgrounds to exposure and detector components.

Parameters
----------
bgMatrix : `np.ndarray`, (nDetectors, nExposures)
    Input backgrounds indexed by exposure (axis=0) and
    detector (axis=1).
bgCounts : `np.ndarray`, (nDetectors, nExposures), optional
    Input pixel counts used to in measuring bgMatrix, indexed
    identically.
iterations : `int`, optional
    Number of iterations to use in decomposition.

Returns
-------
scaleResult : `lsst.pipe.base.Struct`
    Result struct containing fields:

    ``vectorE``
        Output E vector of exposure level scalings
        (`np.array`, (nExposures)).
    ``vectorG``
        Output G vector of detector level scalings
        (`np.array`, (nExposures)).
    ``bgModel``
        Expected model bgMatrix values, calculated from E and G
        (`np.ndarray`, (nDetectors, nExposures)).

Notes
-----

The set of background measurements B[exposure, detector] of
flat frame data should be defined by a "Cartesian" product of
two vectors, E[exposure] and G[detector].  The E vector
represents the total flux incident on the focal plane.  In a
perfect camera, this is simply the sum along the columns of B
(np.sum(B, axis=0)).

However, this simple model ignores differences in detector
gains, the vignetting of the detectors, and the illumination
pattern of the source lamp.  The G vector describes these
detector dependent differences, which should be identical over
different exposures.  For a perfect lamp of unit total
intensity, this is simply the sum along the rows of B
(np.sum(B, axis=1)).  This algorithm divides G by the total
flux level, to provide the relative (not absolute) scales
between detectors.

The algorithm here, from pipe_drivers/constructCalibs.py and
from there from Eugene Magnier/PanSTARRS [1]_, attempts to
iteratively solve this decomposition from initial "perfect" E
and G vectors.  The operation is performed in log space to
reduce the multiply and divides to linear additions and
subtractions.

References
----------
.. [1] https://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/browser/trunk/psModules/src/detrend/pmFlatNormalize.c  # noqa: E501

Definition at line 316 of file cpFlatNormTask.py.

     def measureScales(self, bgMatrix, bgCounts=None, iterations=10):
         """Convert backgrounds to exposure and detector components.
  
         Parameters
         ----------
         bgMatrix : `np.ndarray`, (nDetectors, nExposures)
             Input backgrounds indexed by exposure (axis=0) and
             detector (axis=1).
         bgCounts : `np.ndarray`, (nDetectors, nExposures), optional
             Input pixel counts used to in measuring bgMatrix, indexed
             identically.
         iterations : `int`, optional
             Number of iterations to use in decomposition.
  
         Returns
         -------
         scaleResult : `lsst.pipe.base.Struct`
             Result struct containing fields:
  
             ``vectorE``
                 Output E vector of exposure level scalings
                 (`np.array`, (nExposures)).
             ``vectorG``
                 Output G vector of detector level scalings
                 (`np.array`, (nExposures)).
             ``bgModel``
                 Expected model bgMatrix values, calculated from E and G
                 (`np.ndarray`, (nDetectors, nExposures)).
  
         Notes
         -----
  
         The set of background measurements B[exposure, detector] of
         flat frame data should be defined by a "Cartesian" product of
         two vectors, E[exposure] and G[detector].  The E vector
         represents the total flux incident on the focal plane.  In a
         perfect camera, this is simply the sum along the columns of B
         (np.sum(B, axis=0)).
  
         However, this simple model ignores differences in detector
         gains, the vignetting of the detectors, and the illumination
         pattern of the source lamp.  The G vector describes these
         detector dependent differences, which should be identical over
         different exposures.  For a perfect lamp of unit total
         intensity, this is simply the sum along the rows of B
         (np.sum(B, axis=1)).  This algorithm divides G by the total
         flux level, to provide the relative (not absolute) scales
         between detectors.
  
         The algorithm here, from pipe_drivers/constructCalibs.py and
         from there from Eugene Magnier/PanSTARRS [1]_, attempts to
         iteratively solve this decomposition from initial "perfect" E
         and G vectors.  The operation is performed in log space to
         reduce the multiply and divides to linear additions and
         subtractions.
  
         References
         ----------
         .. [1] https://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/browser/trunk/psModules/src/detrend/pmFlatNormalize.c  # noqa: E501
  
         """
         numExps = bgMatrix.shape[1]
         numChips = bgMatrix.shape[0]
         if bgCounts is None:
             bgCounts = np.ones_like(bgMatrix)
  
         logMeas = np.log(bgMatrix)
         logMeas = np.ma.masked_array(logMeas, ~np.isfinite(logMeas))
         logG = np.zeros(numChips)
         logE = np.array([np.average(logMeas[:, iexp] - logG,
                                     weights=bgCounts[:, iexp]) for iexp in range(numExps)])
  
         for iter in range(iterations):
             logG = np.array([np.average(logMeas[ichip, :] - logE,
                                         weights=bgCounts[ichip, :]) for ichip in range(numChips)])
  
             bad = np.isnan(logG)
             if np.any(bad):
                 logG[bad] = logG[~bad].mean()
  
             logE = np.array([np.average(logMeas[:, iexp] - logG,
                                         weights=bgCounts[:, iexp]) for iexp in range(numExps)])
             fluxLevel = np.average(np.exp(logG), weights=np.sum(bgCounts, axis=1))
  
             logG -= np.log(fluxLevel)
             self.log.debug(f"ITER {iter}: Flux: {fluxLevel}")
             self.log.debug(f"Exps: {np.exp(logE)}")
             self.log.debug(f"{np.mean(logG)}")
  
         logE = np.array([np.average(logMeas[:, iexp] - logG,
                                     weights=bgCounts[:, iexp]) for iexp in range(numExps)])
  
         bgModel = np.exp(logE[np.newaxis, :] - logG[:, np.newaxis])
         return pipeBase.Struct(
             expScales=np.exp(logE),
             detScales=np.exp(logG),
             bgModel=bgModel,
         )

◆ run()

def lsst.cp.pipe.cpFlatNormTask.CpFlatNormalizationTask.run	(	self,
		inputMDs,
		inputDims,
		camera
	)

Normalize FLAT exposures to a consistent level.

Parameters
----------
inputMDs : `list` [`lsst.daf.base.PropertyList`]
    Amplifier-level metadata used to construct scales.
inputDims : `list` [`dict`]
    List of dictionaries of input data dimensions/values.
    Each list entry should contain:

    ``"exposure"``
        exposure id value (`int`)
    ``"detector"``
        detector id value (`int`)

Returns
-------
outputScales : `dict` [`dict` [`dict` [`float`]]]
    Dictionary of scales, indexed by detector (`int`),
    amplifier (`int`), and exposure (`int`).

Raises
------
KeyError
    Raised if the input dimensions do not contain detector and
    exposure, or if the metadata does not contain the expected
    statistic entry.

Definition at line 200 of file cpFlatNormTask.py.

     def run(self, inputMDs, inputDims, camera):
         """Normalize FLAT exposures to a consistent level.
  
         Parameters
         ----------
         inputMDs : `list` [`lsst.daf.base.PropertyList`]
             Amplifier-level metadata used to construct scales.
         inputDims : `list` [`dict`]
             List of dictionaries of input data dimensions/values.
             Each list entry should contain:
  
             ``"exposure"``
                 exposure id value (`int`)
             ``"detector"``
                 detector id value (`int`)
  
         Returns
         -------
         outputScales : `dict` [`dict` [`dict` [`float`]]]
             Dictionary of scales, indexed by detector (`int`),
             amplifier (`int`), and exposure (`int`).
  
         Raises
         ------
         KeyError
             Raised if the input dimensions do not contain detector and
             exposure, or if the metadata does not contain the expected
             statistic entry.
         """
         expSet = sorted(set([d['exposure'] for d in inputDims]))
         detSet = sorted(set([d['detector'] for d in inputDims]))
  
         expMap = {exposureId: idx for idx, exposureId in enumerate(expSet)}
         detMap = {detectorId: idx for idx, detectorId in enumerate(detSet)}
  
         nExp = len(expSet)
         nDet = len(detSet)
         if self.config.level == 'DETECTOR':
             bgMatrix = np.zeros((nDet, nExp))
             bgCounts = np.ones((nDet, nExp))
         elif self.config.level == 'AMP':
             nAmp = len(camera[detSet[0]])
             bgMatrix = np.zeros((nDet * nAmp, nExp))
             bgCounts = np.ones((nDet * nAmp, nExp))
  
         for inMetadata, inDimensions in zip(inputMDs, inputDims):
             try:
                 exposureId = inDimensions['exposure']
                 detectorId = inDimensions['detector']
             except Exception as e:
                 raise KeyError("Cannot find expected dimensions in %s" % (inDimensions, )) from e
  
             if self.config.level == 'DETECTOR':
                 detIdx = detMap[detectorId]
                 expIdx = expMap[exposureId]
                 try:
                     value = inMetadata.get('DETECTOR_MEDIAN')
                     count = inMetadata.get('DETECTOR_N')
                 except Exception as e:
                     raise KeyError("Cannot read expected metadata string.") from e
  
                 if np.isfinite(value):
                     bgMatrix[detIdx][expIdx] = value
                     bgCounts[detIdx][expIdx] = count
                 else:
                     bgMatrix[detIdx][expIdx] = np.nan
                     bgCounts[detIdx][expIdx] = 1
  
             elif self.config.level == 'AMP':
                 detector = camera[detectorId]
                 nAmp = len(detector)
  
                 detIdx = detMap[detectorId] * nAmp
                 expIdx = expMap[exposureId]
  
                 for ampIdx, amp in enumerate(detector):
                     try:
                         value = inMetadata.get(f'AMP_MEDIAN_{ampIdx}')
                         count = inMetadata.get(f'AMP_N_{ampIdx}')
                     except Exception as e:
                         raise KeyError("cannot read expected metadata string.") from e
  
                     detAmpIdx = detIdx + ampIdx
                     if np.isfinite(value):
                         bgMatrix[detAmpIdx][expIdx] = value
                         bgCounts[detAmpIdx][expIdx] = count
                     else:
                         bgMatrix[detAmpIdx][expIdx] = np.nan
                         bgMatrix[detAmpIdx][expIdx] = 1
  
         scaleResult = self.measureScales(bgMatrix, bgCounts, iterations=self.config.scaleMaxIter)
         expScales = scaleResult.expScales
         detScales = scaleResult.detScales
  
         outputScales = defaultdict(lambda: defaultdict(lambda: defaultdict(lambda: defaultdict(float))))
  
         # Note that the enumerated "detId"/"expId" here index the
         # "detScales" and "expScales" arrays.
         if self.config.level == 'DETECTOR':
             for detIdx, det in enumerate(detSet):
                 for amp in camera[det]:
                     for expIdx, exp in enumerate(expSet):
                         outputScales['expScale'][det][amp.getName()][exp] = expScales[expIdx].tolist()
                 outputScales['detScale'][det] = detScales[detIdx].tolist()
         elif self.config.level == 'AMP':
             for detIdx, det in enumerate(detSet):
                 for ampIdx, amp in enumerate(camera[det]):
                     for expIdx, exp in enumerate(expSet):
                         outputScales['expScale'][det][amp.getName()][exp] = expScales[expIdx].tolist()
                     detAmpIdx = detIdx + ampIdx
                     outputScales['detScale'][det][amp.getName()] = detScales[detAmpIdx].tolist()
  
         return pipeBase.Struct(
             outputScales=ddict2dict(outputScales),
         )
  

◆ runQuantum()

def lsst.cp.pipe.cpFlatNormTask.CpFlatNormalizationTask.runQuantum	(	self,
		butlerQC,
		inputRefs,
		outputRefs
	)

Definition at line 189 of file cpFlatNormTask.py.

     def runQuantum(self, butlerQC, inputRefs, outputRefs):
         inputs = butlerQC.get(inputRefs)
  
         # Use the dimensions of the inputs for generating
         # output scales.
         dimensions = [exp.dataId.byName() for exp in inputRefs.inputMDs]
         inputs['inputDims'] = dimensions
  
         outputs = self.run(**inputs)
         butlerQC.put(outputs, outputRefs)
  

Member Data Documentation

◆ ConfigClass

lsst.cp.pipe.cpFlatNormTask.CpFlatNormalizationTask.ConfigClass = CpFlatNormalizationTaskConfig

static

Definition at line 186 of file cpFlatNormTask.py.

The documentation for this class was generated from the following file:

/j/snowflake/release/lsstsw/stack/lsst-scipipe-0.6.0/Linux64/cp_pipe/22.0.1-18-gb17765a+2264247a6b/python/lsst/cp/pipe/cpFlatNormTask.py

Public Member Functions