lsst.ip.isr.linearize.Linearizer Class Reference

Inheritance diagram for lsst.ip.isr.linearize.Linearizer:

Public Member Functions
def	__init__ (self, table=None, **kwargs)
def	updateMetadata (self, setDate=False, **kwargs)
def	fromDetector (self, detector)
def	fromDict (cls, dictionary)
def	toDict (self)
def	fromTable (cls, tableList)
def	toTable (self)
def	getLinearityTypeByName (self, linearityTypeName)
def	validate (self, detector=None, amplifier=None)
def	applyLinearity (self, image, detector=None, log=None)
def	requiredAttributes (self)
def	requiredAttributes (self, value)
def	__str__ (self)
def	__eq__ (self, other)
def	getMetadata (self)
def	setMetadata (self, metadata)
def	updateMetadata (self, camera=None, detector=None, filterName=None, setCalibId=False, setCalibInfo=False, setDate=False, **kwargs)
def	calibInfoFromDict (self, dictionary)
def	determineCalibClass (cls, metadata, message)
def	readText (cls, filename, **kwargs)
def	writeText (self, filename, format="auto")
def	readFits (cls, filename, **kwargs)
def	writeFits (self, filename)
def	fromDict (cls, dictionary, **kwargs)
def	fromTable (cls, tableList, **kwargs)
def	validate (self, other=None)
def	apply (self, target)

Public Attributes
	hasLinearity
	override
	ampNames
	linearityCoeffs
	linearityType
	linearityBBox
	fitParams
	fitParamsErr
	fitChiSq
	tableData
	requiredAttributes
	log

Detailed Description

Parameter set for linearization.

These parameters are included in cameraGeom.Amplifier, but
should be accessible externally to allow for testing.

Parameters
----------
table : `numpy.array`, optional
    Lookup table; a 2-dimensional array of floats:
        - one row for each row index (value of coef[0] in the amplifier)
        - one column for each image value
    To avoid copying the table the last index should vary fastest
    (numpy default "C" order)
detector : `lsst.afw.cameraGeom.Detector`, optional
    Detector object.  Passed to self.fromDetector() on init.
log : `logging.Logger`, optional
    Logger to handle messages.
kwargs : `dict`, optional
    Other keyword arguments to pass to the parent init.

Raises
------
RuntimeError :
    Raised if the supplied table is not 2D, or if the table has fewer
    columns than rows (indicating that the indices are swapped).

Notes
-----
The linearizer attributes stored are:

hasLinearity : `bool`
    Whether a linearity correction is defined for this detector.
override : `bool`
    Whether the detector parameters should be overridden.
ampNames : `list` [`str`]
    List of amplifier names to correct.
linearityCoeffs : `dict` [`str`, `numpy.array`]
    Coefficients to use in correction.  Indexed by amplifier
    names.  The format of the array depends on the type of
    correction to apply.
linearityType : `dict` [`str`, `str`]
    Type of correction to use, indexed by amplifier names.
linearityBBox : `dict` [`str`, `lsst.geom.Box2I`]
    Bounding box the correction is valid over, indexed by
    amplifier names.
fitParams : `dict` [`str`, `numpy.array`], optional
    Linearity fit parameters used to construct the correction
    coefficients, indexed as above.
fitParamsErr : `dict` [`str`, `numpy.array`], optional
    Uncertainty values of the linearity fit parameters used to
    construct the correction coefficients, indexed as above.
fitChiSq : `dict` [`str`, `float`], optional
    Chi-squared value of the linearity fit, indexed as above.
tableData : `numpy.array`, optional
    Lookup table data for the linearity correction.

Definition at line 38 of file linearize.py.

Constructor & Destructor Documentation

__init__()

def lsst.ip.isr.linearize.Linearizer.__init__	(		self,
			table = None,
		**	kwargs
	)

Definition at line 99 of file linearize.py.

    def __init__(self, table=None, **kwargs):
        self.hasLinearity = False
        self.override = False
 
        self.ampNames = list()
        self.linearityCoeffs = dict()
        self.linearityType = dict()
        self.linearityBBox = dict()
 
        self.fitParams = dict()
        self.fitParamsErr = dict()
        self.fitChiSq = dict()
 
        self.tableData = None
        if table is not None:
            if len(table.shape) != 2:
                raise RuntimeError("table shape = %s; must have two dimensions" % (table.shape,))
            if table.shape[1] < table.shape[0]:
                raise RuntimeError("table shape = %s; indices are switched" % (table.shape,))
            self.tableData = np.array(table, order="C")
 
        super().__init__(**kwargs)
        self.requiredAttributes.update(['hasLinearity', 'override',
                                        'ampNames',
                                        'linearityCoeffs', 'linearityType', 'linearityBBox',
                                        'fitParams', 'fitParamsErr', 'fitChiSq',
                                        'tableData'])
 

Member Function Documentation

__eq__()

def lsst.ip.isr.calibType.IsrCalib.__eq__ (   self,   other  )

inherited

Calibration equivalence.

Running ``calib.log.setLevel(0)`` enables debug statements to
identify problematic fields.

Reimplemented in lsst.ip.isr.defects.Defects, and lsst.ip.isr.calibType.IsrProvenance.

Definition at line 95 of file calibType.py.

    def __eq__(self, other):
        """Calibration equivalence.
 
        Running ``calib.log.setLevel(0)`` enables debug statements to
        identify problematic fields.
        """
        if not isinstance(other, self.__class__):
            self.log.debug("Incorrect class type: %s %s", self.__class__, other.__class__)
            return False
 
        for attr in self._requiredAttributes:
            attrSelf = getattr(self, attr)
            attrOther = getattr(other, attr)
 
            if isinstance(attrSelf, dict):
                # Dictionary of arrays.
                if attrSelf.keys() != attrOther.keys():
                    self.log.debug("Dict Key Failure: %s %s %s", attr, attrSelf.keys(), attrOther.keys())
                    return False
                for key in attrSelf:
                    if not np.allclose(attrSelf[key], attrOther[key], equal_nan=True):
                        self.log.debug("Array Failure: %s %s %s", key, attrSelf[key], attrOther[key])
                        return False
            elif isinstance(attrSelf, np.ndarray):
                # Bare array.
                if not np.allclose(attrSelf, attrOther, equal_nan=True):
                    self.log.debug("Array Failure: %s %s %s", attr, attrSelf, attrOther)
                    return False
            elif type(attrSelf) != type(attrOther):
                if set([attrSelf, attrOther]) == set([None, ""]):
                    # Fits converts None to "", but None is not "".
                    continue
                self.log.debug("Type Failure: %s %s %s %s %s", attr, type(attrSelf), type(attrOther),
                               attrSelf, attrOther)
                return False
            else:
                if attrSelf != attrOther:
                    self.log.debug("Value Failure: %s %s %s", attr, attrSelf, attrOther)
                    return False
 
        return True
 

__str__()

def lsst.ip.isr.calibType.IsrCalib.__str__ (   self )

inherited

Reimplemented in lsst.ip.isr.defects.Defects, and lsst.ip.isr.calibType.IsrProvenance.

Definition at line 92 of file calibType.py.

    def __str__(self):
        return f"{self.__class__.__name__}(obstype={self._OBSTYPE}, detector={self._detectorName}, )"
 

apply()

def lsst.ip.isr.calibType.IsrCalib.apply (   self,   target  )

inherited

Method to apply the calibration to the target object.

Parameters
----------
target : `object`
    Thing to validate against.

Returns
-------
valid : `bool`
    Returns true if the calibration was applied correctly.

Raises
------
NotImplementedError :
    Raised if not implemented.

Definition at line 612 of file calibType.py.

    def apply(self, target):
        """Method to apply the calibration to the target object.
 
        Parameters
        ----------
        target : `object`
            Thing to validate against.
 
        Returns
        -------
        valid : `bool`
            Returns true if the calibration was applied correctly.
 
        Raises
        ------
        NotImplementedError :
            Raised if not implemented.
        """
        raise NotImplementedError("Must be implemented by subclass.")
 
 

applyLinearity()

def lsst.ip.isr.linearize.Linearizer.applyLinearity	(		self,
			image,
			detector = None,
			log = None
	)

Apply the linearity to an image.

If the linearity parameters are populated, use those,
otherwise use the values from the detector.

Parameters
----------
image : `~lsst.afw.image.image`
    Image to correct.
detector : `~lsst.afw.cameraGeom.detector`
    Detector to use for linearity parameters if not already
    populated.
log : `~logging.Logger`, optional
    Log object to use for logging.

Definition at line 428 of file linearize.py.

    def applyLinearity(self, image, detector=None, log=None):
        """Apply the linearity to an image.
 
        If the linearity parameters are populated, use those,
        otherwise use the values from the detector.
 
        Parameters
        ----------
        image : `~lsst.afw.image.image`
            Image to correct.
        detector : `~lsst.afw.cameraGeom.detector`
            Detector to use for linearity parameters if not already
            populated.
        log : `~logging.Logger`, optional
            Log object to use for logging.
        """
        if log is None:
            log = self.log
        if detector and not self.hasLinearity:
            self.fromDetector(detector)
 
        self.validate(detector)
 
        numAmps = 0
        numLinearized = 0
        numOutOfRange = 0
        for ampName in self.linearityType.keys():
            linearizer = self.getLinearityTypeByName(self.linearityType[ampName])
            numAmps += 1
            if linearizer is not None:
                ampView = image.Factory(image, self.linearityBBox[ampName])
                success, outOfRange = linearizer()(ampView, **{'coeffs': self.linearityCoeffs[ampName],
                                                               'table': self.tableData,
                                                               'log': self.log})
                numOutOfRange += outOfRange
                if success:
                    numLinearized += 1
                elif log is not None:
                    log.warning("Amplifier %s did not linearize.",
                                ampName)
        return Struct(
            numAmps=numAmps,
            numLinearized=numLinearized,
            numOutOfRange=numOutOfRange
        )
 
 

calibInfoFromDict()

def lsst.ip.isr.calibType.IsrCalib.calibInfoFromDict (   self,   dictionary  )

inherited

Handle common keywords.

This isn't an ideal solution, but until all calibrations
expect to find everything in the metadata, they still need to
search through dictionaries.

Parameters
----------
dictionary : `dict` or `lsst.daf.base.PropertyList`
    Source for the common keywords.

Raises
------
RuntimeError :
    Raised if the dictionary does not match the expected OBSTYPE.

Definition at line 261 of file calibType.py.

    def calibInfoFromDict(self, dictionary):
        """Handle common keywords.
 
        This isn't an ideal solution, but until all calibrations
        expect to find everything in the metadata, they still need to
        search through dictionaries.
 
        Parameters
        ----------
        dictionary : `dict` or `lsst.daf.base.PropertyList`
            Source for the common keywords.
 
        Raises
        ------
        RuntimeError :
            Raised if the dictionary does not match the expected OBSTYPE.
 
        """
 
        def search(haystack, needles):
            """Search dictionary 'haystack' for an entry in 'needles'
            """
            test = [haystack.get(x) for x in needles]
            test = set([x for x in test if x is not None])
            if len(test) == 0:
                if "metadata" in haystack:
                    return search(haystack["metadata"], needles)
                else:
                    return None
            elif len(test) == 1:
                value = list(test)[0]
                if value == "":
                    return None
                else:
                    return value
            else:
                raise ValueError(f"Too many values found: {len(test)} {test} {needles}")
 
        if "metadata" in dictionary:
            metadata = dictionary["metadata"]
 
            if self._OBSTYPE != metadata["OBSTYPE"]:
                raise RuntimeError(f"Incorrect calibration supplied.  Expected {self._OBSTYPE}, "
                                   f"found {metadata['OBSTYPE']}")
 
        self._instrument = search(dictionary, ["INSTRUME", "instrument"])
        self._raftName = search(dictionary, ["RAFTNAME"])
        self._slotName = search(dictionary, ["SLOTNAME"])
        self._detectorId = search(dictionary, ["DETECTOR", "detectorId"])
        self._detectorName = search(dictionary, ["DET_NAME", "DETECTOR_NAME", "detectorName"])
        self._detectorSerial = search(dictionary, ["DET_SER", "DETECTOR_SERIAL", "detectorSerial"])
        self._filter = search(dictionary, ["FILTER", "filterName"])
        self._calibId = search(dictionary, ["CALIB_ID"])
 

determineCalibClass()

def lsst.ip.isr.calibType.IsrCalib.determineCalibClass (   cls,   metadata,   message  )

inherited

Attempt to find calibration class in metadata.

Parameters
----------
metadata : `dict` or `lsst.daf.base.PropertyList`
    Metadata possibly containing a calibration class entry.
message : `str`
    Message to include in any errors.

Returns
-------
calibClass : `object`
    The class to use to read the file contents.  Should be an
    `lsst.ip.isr.IsrCalib` subclass.

Raises
------
ValueError :
    Raised if the resulting calibClass is the base
    `lsst.ip.isr.IsrClass` (which does not implement the
    content methods).

Definition at line 316 of file calibType.py.

    def determineCalibClass(cls, metadata, message):
        """Attempt to find calibration class in metadata.
 
        Parameters
        ----------
        metadata : `dict` or `lsst.daf.base.PropertyList`
            Metadata possibly containing a calibration class entry.
        message : `str`
            Message to include in any errors.
 
        Returns
        -------
        calibClass : `object`
            The class to use to read the file contents.  Should be an
            `lsst.ip.isr.IsrCalib` subclass.
 
        Raises
        ------
        ValueError :
            Raised if the resulting calibClass is the base
            `lsst.ip.isr.IsrClass` (which does not implement the
            content methods).
        """
        calibClassName = metadata.get("CALIBCLS")
        calibClass = doImport(calibClassName) if calibClassName is not None else cls
        if calibClass is IsrCalib:
            raise ValueError(f"Cannot use base class to read calibration data: {msg}")
        return calibClass
 

fromDetector()

def lsst.ip.isr.linearize.Linearizer.fromDetector	(		self,
			detector
	)

Read linearity parameters from a detector.

Parameters
----------
detector : `lsst.afw.cameraGeom.detector`
    Input detector with parameters to use.

Returns
-------
calib : `lsst.ip.isr.Linearizer`
    The calibration constructed from the detector.

Reimplemented from lsst.ip.isr.calibType.IsrCalib.

Definition at line 147 of file linearize.py.

    def fromDetector(self, detector):
        """Read linearity parameters from a detector.
 
        Parameters
        ----------
        detector : `lsst.afw.cameraGeom.detector`
            Input detector with parameters to use.
 
        Returns
        -------
        calib : `lsst.ip.isr.Linearizer`
            The calibration constructed from the detector.
        """
        self._detectorName = detector.getName()
        self._detectorSerial = detector.getSerial()
        self._detectorId = detector.getId()
        self.hasLinearity = True
 
        # Do not translate Threshold, Maximum, Units.
        for amp in detector.getAmplifiers():
            ampName = amp.getName()
            self.ampNames.append(ampName)
            self.linearityType[ampName] = amp.getLinearityType()
            self.linearityCoeffs[ampName] = amp.getLinearityCoeffs()
            self.linearityBBox[ampName] = amp.getBBox()
 
        return self
 

fromDict() [1/2]

def lsst.ip.isr.linearize.Linearizer.fromDict	(		cls,
			dictionary
	)

Construct a calibration from a dictionary of properties

Parameters
----------
dictionary : `dict`
    Dictionary of properties

Returns
-------
calib : `lsst.ip.isr.Linearity`
    Constructed calibration.

Raises
------
RuntimeError
    Raised if the supplied dictionary is for a different
    calibration.

Definition at line 176 of file linearize.py.

    def fromDict(cls, dictionary):
        """Construct a calibration from a dictionary of properties
 
        Parameters
        ----------
        dictionary : `dict`
            Dictionary of properties
 
        Returns
        -------
        calib : `lsst.ip.isr.Linearity`
            Constructed calibration.
 
        Raises
        ------
        RuntimeError
            Raised if the supplied dictionary is for a different
            calibration.
        """
 
        calib = cls()
 
        if calib._OBSTYPE != dictionary['metadata']['OBSTYPE']:
            raise RuntimeError(f"Incorrect linearity supplied.  Expected {calib._OBSTYPE}, "
                               f"found {dictionary['metadata']['OBSTYPE']}")
 
        calib.setMetadata(dictionary['metadata'])
 
        calib.hasLinearity = dictionary.get('hasLinearity',
                                            dictionary['metadata'].get('HAS_LINEARITY', False))
        calib.override = dictionary.get('override', True)
 
        if calib.hasLinearity:
            for ampName in dictionary['amplifiers']:
                amp = dictionary['amplifiers'][ampName]
                calib.ampNames.append(ampName)
                calib.linearityCoeffs[ampName] = np.array(amp.get('linearityCoeffs', [0.0]))
                calib.linearityType[ampName] = amp.get('linearityType', 'None')
                calib.linearityBBox[ampName] = amp.get('linearityBBox', None)
 
                calib.fitParams[ampName] = np.array(amp.get('fitParams', [0.0]))
                calib.fitParamsErr[ampName] = np.array(amp.get('fitParamsErr', [0.0]))
                calib.fitChiSq[ampName] = amp.get('fitChiSq', np.nan)
 
            calib.tableData = dictionary.get('tableData', None)
            if calib.tableData:
                calib.tableData = np.array(calib.tableData)
 
        return calib
 

fromDict() [2/2]

def lsst.ip.isr.calibType.IsrCalib.fromDict (   cls,   dictionary, **  kwargs  )

inherited

Construct a calibration from a dictionary of properties.

Must be implemented by the specific calibration subclasses.

Parameters
----------
dictionary : `dict`
    Dictionary of properties.
kwargs : `dict` or collections.abc.Mapping`, optional
    Set of key=value options.

Returns
------
calib : `lsst.ip.isr.CalibType`
    Constructed calibration.

Raises
------
NotImplementedError :
    Raised if not implemented.

Definition at line 512 of file calibType.py.

    def fromDict(cls, dictionary, **kwargs):
        """Construct a calibration from a dictionary of properties.
 
        Must be implemented by the specific calibration subclasses.
 
        Parameters
        ----------
        dictionary : `dict`
            Dictionary of properties.
        kwargs : `dict` or collections.abc.Mapping`, optional
            Set of key=value options.
 
        Returns
        ------
        calib : `lsst.ip.isr.CalibType`
            Constructed calibration.
 
        Raises
        ------
        NotImplementedError :
            Raised if not implemented.
        """
        raise NotImplementedError("Must be implemented by subclass.")
 

fromTable() [1/2]

def lsst.ip.isr.linearize.Linearizer.fromTable	(		cls,
			tableList
	)

Read linearity from a FITS file.

This method uses the `fromDict` method to create the
calibration, after constructing an appropriate dictionary from
the input tables.

Parameters
----------
tableList : `list` [`astropy.table.Table`]
    afwTable read from input file name.

Returns
-------
linearity : `~lsst.ip.isr.linearize.Linearizer``
    Linearity parameters.

Notes
-----
The method reads a FITS file with 1 or 2 extensions. The metadata is read from the header of
extension 1, which must exist.  Then the table is loaded, and  the ['AMPLIFIER_NAME', 'TYPE',
'COEFFS', 'BBOX_X0', 'BBOX_Y0', 'BBOX_DX', 'BBOX_DY'] columns are read and used to
set each dictionary by looping over rows.
Eextension 2 is then attempted to read in the try block (which only exists for lookup tables).
It has a column named 'LOOKUP_VALUES' that contains a vector of the lookup entries in each row.

Definition at line 255 of file linearize.py.

    def fromTable(cls, tableList):
        """Read linearity from a FITS file.
 
        This method uses the `fromDict` method to create the
        calibration, after constructing an appropriate dictionary from
        the input tables.
 
        Parameters
        ----------
        tableList : `list` [`astropy.table.Table`]
            afwTable read from input file name.
 
        Returns
        -------
        linearity : `~lsst.ip.isr.linearize.Linearizer``
            Linearity parameters.
 
        Notes
        -----
        The method reads a FITS file with 1 or 2 extensions. The metadata is read from the header of
        extension 1, which must exist.  Then the table is loaded, and  the ['AMPLIFIER_NAME', 'TYPE',
        'COEFFS', 'BBOX_X0', 'BBOX_Y0', 'BBOX_DX', 'BBOX_DY'] columns are read and used to
        set each dictionary by looping over rows.
        Eextension 2 is then attempted to read in the try block (which only exists for lookup tables).
        It has a column named 'LOOKUP_VALUES' that contains a vector of the lookup entries in each row.
 
        """
        coeffTable = tableList[0]
 
        metadata = coeffTable.meta
        inDict = dict()
        inDict['metadata'] = metadata
        inDict['hasLinearity'] = metadata.get('HAS_LINEARITY', False)
        inDict['amplifiers'] = dict()
 
        for record in coeffTable:
            ampName = record['AMPLIFIER_NAME']
 
            fitParams = record['FIT_PARAMS'] if 'FIT_PARAMS' in record.columns else np.array([0.0])
            fitParamsErr = record['FIT_PARAMS_ERR'] if 'FIT_PARAMS_ERR' in record.columns else np.array([0.0])
            fitChiSq = record['RED_CHI_SQ'] if 'RED_CHI_SQ' in record.columns else np.nan
 
            inDict['amplifiers'][ampName] = {
                'linearityType': record['TYPE'],
                'linearityCoeffs': record['COEFFS'],
                'linearityBBox': Box2I(Point2I(record['BBOX_X0'], record['BBOX_Y0']),
                                       Extent2I(record['BBOX_DX'], record['BBOX_DY'])),
                'fitParams': fitParams,
                'fitParamsErr': fitParamsErr,
                'fitChiSq': fitChiSq,
            }
 
        if len(tableList) > 1:
            tableData = tableList[1]
            inDict['tableData'] = [record['LOOKUP_VALUES'] for record in tableData]
 
        return cls().fromDict(inDict)
 

fromTable() [2/2]

def lsst.ip.isr.calibType.IsrCalib.fromTable (   cls,   tableList, **  kwargs  )

inherited

Construct a calibration from a dictionary of properties.

Must be implemented by the specific calibration subclasses.

Parameters
----------
tableList : `list` [`lsst.afw.table.Table`]
    List of tables of properties.
kwargs : `dict` or collections.abc.Mapping`, optional
    Set of key=value options.

Returns
------
calib : `lsst.ip.isr.CalibType`
    Constructed calibration.

Raises
------
NotImplementedError :
    Raised if not implemented.

Definition at line 555 of file calibType.py.

    def fromTable(cls, tableList, **kwargs):
        """Construct a calibration from a dictionary of properties.
 
        Must be implemented by the specific calibration subclasses.
 
        Parameters
        ----------
        tableList : `list` [`lsst.afw.table.Table`]
            List of tables of properties.
        kwargs : `dict` or collections.abc.Mapping`, optional
            Set of key=value options.
 
        Returns
        ------
        calib : `lsst.ip.isr.CalibType`
            Constructed calibration.
 
        Raises
        ------
        NotImplementedError :
            Raised if not implemented.
        """
        raise NotImplementedError("Must be implemented by subclass.")
 

getLinearityTypeByName()

def lsst.ip.isr.linearize.Linearizer.getLinearityTypeByName	(		self,
			linearityTypeName
	)

Determine the linearity class to use from the type name.

Parameters
----------
linearityTypeName : str
    String name of the linearity type that is needed.

Returns
-------
linearityType : `~lsst.ip.isr.linearize.LinearizeBase`
    The appropriate linearity class to use.  If no matching class
    is found, `None` is returned.

Definition at line 347 of file linearize.py.

    def getLinearityTypeByName(self, linearityTypeName):
        """Determine the linearity class to use from the type name.
 
        Parameters
        ----------
        linearityTypeName : str
            String name of the linearity type that is needed.
 
        Returns
        -------
        linearityType : `~lsst.ip.isr.linearize.LinearizeBase`
            The appropriate linearity class to use.  If no matching class
            is found, `None` is returned.
        """
        for t in [LinearizeLookupTable,
                  LinearizeSquared,
                  LinearizePolynomial,
                  LinearizeProportional,
                  LinearizeSpline,
                  LinearizeNone]:
            if t.LinearityType == linearityTypeName:
                return t
        return None
 

getMetadata()

def lsst.ip.isr.calibType.IsrCalib.getMetadata (   self )

inherited

Retrieve metadata associated with this calibration.

Returns
-------
meta : `lsst.daf.base.PropertyList`
    Metadata. The returned `~lsst.daf.base.PropertyList` can be
    modified by the caller and the changes will be written to
    external files.

Definition at line 145 of file calibType.py.

    def getMetadata(self):
        """Retrieve metadata associated with this calibration.
 
        Returns
        -------
        meta : `lsst.daf.base.PropertyList`
            Metadata. The returned `~lsst.daf.base.PropertyList` can be
            modified by the caller and the changes will be written to
            external files.
        """
        return self._metadata
 

readFits()

def lsst.ip.isr.calibType.IsrCalib.readFits (   cls,   filename, **  kwargs  )

inherited

Read calibration data from a FITS file.

Parameters
----------
filename : `str`
    Filename to read data from.
kwargs : `dict` or collections.abc.Mapping`, optional
    Set of key=value pairs to pass to the ``fromTable``
    method.

Returns
-------
calib : `lsst.ip.isr.IsrCalib`
    Calibration contained within the file.

Definition at line 432 of file calibType.py.

    def readFits(cls, filename, **kwargs):
        """Read calibration data from a FITS file.
 
        Parameters
        ----------
        filename : `str`
            Filename to read data from.
        kwargs : `dict` or collections.abc.Mapping`, optional
            Set of key=value pairs to pass to the ``fromTable``
            method.
 
        Returns
        -------
        calib : `lsst.ip.isr.IsrCalib`
            Calibration contained within the file.
        """
        tableList = []
        tableList.append(Table.read(filename, hdu=1))
        extNum = 2  # Fits indices start at 1, we've read one already.
        keepTrying = True
 
        while keepTrying:
            with warnings.catch_warnings():
                warnings.simplefilter("error")
                try:
                    newTable = Table.read(filename, hdu=extNum)
                    tableList.append(newTable)
                    extNum += 1
                except Exception:
                    keepTrying = False
 
        for table in tableList:
            for k, v in table.meta.items():
                if isinstance(v, fits.card.Undefined):
                    table.meta[k] = None
 
        calibClass = cls.determineCalibClass(tableList[0].meta, "readFits")
        return calibClass.fromTable(tableList, **kwargs)
 

readText()

def lsst.ip.isr.calibType.IsrCalib.readText (   cls,   filename, **  kwargs  )

inherited

Read calibration representation from a yaml/ecsv file.

Parameters
----------
filename : `str`
    Name of the file containing the calibration definition.
kwargs : `dict` or collections.abc.Mapping`, optional
    Set of key=value pairs to pass to the ``fromDict`` or
    ``fromTable`` methods.

Returns
-------
calib : `~lsst.ip.isr.IsrCalibType`
    Calibration class.

Raises
------
RuntimeError :
    Raised if the filename does not end in ".ecsv" or ".yaml".

Definition at line 346 of file calibType.py.

    def readText(cls, filename, **kwargs):
        """Read calibration representation from a yaml/ecsv file.
 
        Parameters
        ----------
        filename : `str`
            Name of the file containing the calibration definition.
        kwargs : `dict` or collections.abc.Mapping`, optional
            Set of key=value pairs to pass to the ``fromDict`` or
            ``fromTable`` methods.
 
        Returns
        -------
        calib : `~lsst.ip.isr.IsrCalibType`
            Calibration class.
 
        Raises
        ------
        RuntimeError :
            Raised if the filename does not end in ".ecsv" or ".yaml".
        """
        if filename.endswith((".ecsv", ".ECSV")):
            data = Table.read(filename, format="ascii.ecsv")
            calibClass = cls.determineCalibClass(data.meta, "readText/ECSV")
            return calibClass.fromTable([data], **kwargs)
        elif filename.endswith((".yaml", ".YAML")):
            with open(filename, "r") as f:
                data = yaml.load(f, Loader=yaml.CLoader)
            calibClass = cls.determineCalibClass(data["metadata"], "readText/YAML")
            return calibClass.fromDict(data, **kwargs)
        else:
            raise RuntimeError(f"Unknown filename extension: {filename}")
 

requiredAttributes() [1/2]

def lsst.ip.isr.calibType.IsrCalib.requiredAttributes (   self )

inherited

Definition at line 138 of file calibType.py.

    def requiredAttributes(self):
        return self._requiredAttributes
 

requiredAttributes() [2/2]

def lsst.ip.isr.calibType.IsrCalib.requiredAttributes (   self,   value  )

inherited

Definition at line 142 of file calibType.py.

    def requiredAttributes(self, value):
        self._requiredAttributes = value
 

setMetadata()

def lsst.ip.isr.calibType.IsrCalib.setMetadata (   self,   metadata  )

inherited

Store a copy of the supplied metadata with this calibration.

Parameters
----------
metadata : `lsst.daf.base.PropertyList`
    Metadata to associate with the calibration.  Will be copied and
    overwrite existing metadata.

Definition at line 157 of file calibType.py.

    def setMetadata(self, metadata):
        """Store a copy of the supplied metadata with this calibration.
 
        Parameters
        ----------
        metadata : `lsst.daf.base.PropertyList`
            Metadata to associate with the calibration.  Will be copied and
            overwrite existing metadata.
        """
        if metadata is not None:
            self._metadata.update(metadata)
 
        # Ensure that we have the obs type required by calibration ingest
        self._metadata["OBSTYPE"] = self._OBSTYPE
        self._metadata[self._OBSTYPE + "_SCHEMA"] = self._SCHEMA
        self._metadata[self._OBSTYPE + "_VERSION"] = self._VERSION
 
        if isinstance(metadata, dict):
            self.calibInfoFromDict(metadata)
        elif isinstance(metadata, PropertyList):
            self.calibInfoFromDict(metadata.toDict())
 

toDict()

def lsst.ip.isr.linearize.Linearizer.toDict

(

self

)

Return linearity parameters as a dict.

Returns
-------
outDict : `dict`:

Reimplemented from lsst.ip.isr.calibType.IsrCalib.

Definition at line 226 of file linearize.py.

    def toDict(self):
        """Return linearity parameters as a dict.
 
        Returns
        -------
        outDict : `dict`:
        """
        self.updateMetadata()
 
        outDict = {'metadata': self.getMetadata(),
                   'detectorName': self._detectorName,
                   'detectorSerial': self._detectorSerial,
                   'detectorId': self._detectorId,
                   'hasTable': self.tableData is not None,
                   'amplifiers': dict(),
                   }
        for ampName in self.linearityType:
            outDict['amplifiers'][ampName] = {'linearityType': self.linearityType[ampName],
                                              'linearityCoeffs': self.linearityCoeffs[ampName].tolist(),
                                              'linearityBBox': self.linearityBBox[ampName],
                                              'fitParams': self.fitParams[ampName].tolist(),
                                              'fitParamsErr': self.fitParamsErr[ampName].tolist(),
                                              'fitChiSq': self.fitChiSq[ampName]}
        if self.tableData is not None:
            outDict['tableData'] = self.tableData.tolist()
 
        return outDict
 

toTable()

def lsst.ip.isr.linearize.Linearizer.toTable

(

self

)

Construct a list of tables containing the information in this calibration

The list of tables should create an identical calibration
after being passed to this class's fromTable method.

Returns
-------
tableList : `list` [`astropy.table.Table`]
    List of tables containing the linearity calibration
    information.

Reimplemented from lsst.ip.isr.calibType.IsrCalib.

Definition at line 313 of file linearize.py.

    def toTable(self):
        """Construct a list of tables containing the information in this calibration
 
        The list of tables should create an identical calibration
        after being passed to this class's fromTable method.
 
        Returns
        -------
        tableList : `list` [`astropy.table.Table`]
            List of tables containing the linearity calibration
            information.
        """
 
        tableList = []
        self.updateMetadata()
        catalog = Table([{'AMPLIFIER_NAME': ampName,
                          'TYPE': self.linearityType[ampName],
                          'COEFFS': self.linearityCoeffs[ampName],
                          'BBOX_X0': self.linearityBBox[ampName].getMinX(),
                          'BBOX_Y0': self.linearityBBox[ampName].getMinY(),
                          'BBOX_DX': self.linearityBBox[ampName].getWidth(),
                          'BBOX_DY': self.linearityBBox[ampName].getHeight(),
                          'FIT_PARAMS': self.fitParams[ampName],
                          'FIT_PARAMS_ERR': self.fitParamsErr[ampName],
                          'RED_CHI_SQ': self.fitChiSq[ampName],
                          } for ampName in self.ampNames])
        catalog.meta = self.getMetadata().toDict()
        tableList.append(catalog)
 
        if self.tableData is not None:
            catalog = Table([{'LOOKUP_VALUES': value} for value in self.tableData])
            tableList.append(catalog)
        return(tableList)
 

updateMetadata() [1/2]

def lsst.ip.isr.calibType.IsrCalib.updateMetadata (   self,   camera = None,   detector = None,   filterName = None,   setCalibId = False,   setCalibInfo = False,   setDate = False, **  kwargs  )

inherited

Update metadata keywords with new values.

Parameters
----------
camera : `lsst.afw.cameraGeom.Camera`, optional
    Reference camera to use to set _instrument field.
detector : `lsst.afw.cameraGeom.Detector`, optional
    Reference detector to use to set _detector* fields.
filterName : `str`, optional
    Filter name to assign to this calibration.
setCalibId : `bool`, optional
    Construct the _calibId field from other fields.
setCalibInfo : `bool`, optional
    Set calibration parameters from metadata.
setDate : `bool`, optional
    Ensure the metadata CALIBDATE fields are set to the current datetime.
kwargs : `dict` or `collections.abc.Mapping`, optional
    Set of key=value pairs to assign to the metadata.

Definition at line 179 of file calibType.py.

                       **kwargs):
        """Update metadata keywords with new values.
 
        Parameters
        ----------
        camera : `lsst.afw.cameraGeom.Camera`, optional
            Reference camera to use to set _instrument field.
        detector : `lsst.afw.cameraGeom.Detector`, optional
            Reference detector to use to set _detector* fields.
        filterName : `str`, optional
            Filter name to assign to this calibration.
        setCalibId : `bool`, optional
            Construct the _calibId field from other fields.
        setCalibInfo : `bool`, optional
            Set calibration parameters from metadata.
        setDate : `bool`, optional
            Ensure the metadata CALIBDATE fields are set to the current datetime.
        kwargs : `dict` or `collections.abc.Mapping`, optional
            Set of key=value pairs to assign to the metadata.
        """
        mdOriginal = self.getMetadata()
        mdSupplemental = dict()
 
        for k, v in kwargs.items():
            if isinstance(v, fits.card.Undefined):
                kwargs[k] = None
 
        if setCalibInfo:
            self.calibInfoFromDict(kwargs)
 
        if camera:
            self._instrument = camera.getName()
 
        if detector:
            self._detectorName = detector.getName()
            self._detectorSerial = detector.getSerial()
            self._detectorId = detector.getId()
            if "_" in self._detectorName:
                (self._raftName, self._slotName) = self._detectorName.split("_")
 
        if filterName:
            # TOD0 DM-28093: I think this whole comment can go away, if we
            # always use physicalLabel everywhere in ip_isr.
            # If set via:
            # exposure.getInfo().getFilter().getName()
            # then this will hold the abstract filter.
            self._filter = filterName
 
        if setDate:
            date = datetime.datetime.now()
            mdSupplemental["CALIBDATE"] = date.isoformat()
            mdSupplemental["CALIB_CREATION_DATE"] = date.date().isoformat()
            mdSupplemental["CALIB_CREATION_TIME"] = date.time().isoformat()
 
        if setCalibId:
            values = []
            values.append(f"instrument={self._instrument}") if self._instrument else None
            values.append(f"raftName={self._raftName}") if self._raftName else None
            values.append(f"detectorName={self._detectorName}") if self._detectorName else None
            values.append(f"detector={self._detectorId}") if self._detectorId else None
            values.append(f"filter={self._filter}") if self._filter else None
 
            calibDate = mdOriginal.get("CALIBDATE", mdSupplemental.get("CALIBDATE", None))
            values.append(f"calibDate={calibDate}") if calibDate else None
 
            self._calibId = " ".join(values)
 
        self._metadata["INSTRUME"] = self._instrument if self._instrument else None
        self._metadata["RAFTNAME"] = self._raftName if self._raftName else None
        self._metadata["SLOTNAME"] = self._slotName if self._slotName else None
        self._metadata["DETECTOR"] = self._detectorId
        self._metadata["DET_NAME"] = self._detectorName if self._detectorName else None
        self._metadata["DET_SER"] = self._detectorSerial if self._detectorSerial else None
        self._metadata["FILTER"] = self._filter if self._filter else None
        self._metadata["CALIB_ID"] = self._calibId if self._calibId else None
        self._metadata["CALIBCLS"] = get_full_type_name(self)
 
        mdSupplemental.update(kwargs)
        mdOriginal.update(mdSupplemental)
 

updateMetadata() [2/2]

def lsst.ip.isr.linearize.Linearizer.updateMetadata	(		self,
			setDate = False,
		**	kwargs
	)

Update metadata keywords with new values.

This calls the base class's method after ensuring the required
calibration keywords will be saved.

Parameters
----------
setDate : `bool`, optional
    Update the CALIBDATE fields in the metadata to the current
    time. Defaults to False.
kwargs :
    Other keyword parameters to set in the metadata.

Definition at line 127 of file linearize.py.

    def updateMetadata(self, setDate=False, **kwargs):
        """Update metadata keywords with new values.
 
        This calls the base class's method after ensuring the required
        calibration keywords will be saved.
 
        Parameters
        ----------
        setDate : `bool`, optional
            Update the CALIBDATE fields in the metadata to the current
            time. Defaults to False.
        kwargs :
            Other keyword parameters to set in the metadata.
        """
        kwargs['HAS_LINEARITY'] = self.hasLinearity
        kwargs['OVERRIDE'] = self.override
        kwargs['HAS_TABLE'] = self.tableData is not None
 
        super().updateMetadata(setDate=setDate, **kwargs)
 

validate() [1/2]

def lsst.ip.isr.linearize.Linearizer.validate	(		self,
			detector = None,
			amplifier = None
	)

Validate linearity for a detector/amplifier.

Parameters
----------
detector : `lsst.afw.cameraGeom.Detector`, optional
    Detector to validate, along with its amplifiers.
amplifier : `lsst.afw.cameraGeom.Amplifier`, optional
    Single amplifier to validate.

Raises
------
RuntimeError :
    Raised if there is a mismatch in linearity parameters, and
    the cameraGeom parameters are not being overridden.

Definition at line 371 of file linearize.py.

    def validate(self, detector=None, amplifier=None):
        """Validate linearity for a detector/amplifier.
 
        Parameters
        ----------
        detector : `lsst.afw.cameraGeom.Detector`, optional
            Detector to validate, along with its amplifiers.
        amplifier : `lsst.afw.cameraGeom.Amplifier`, optional
            Single amplifier to validate.
 
        Raises
        ------
        RuntimeError :
            Raised if there is a mismatch in linearity parameters, and
            the cameraGeom parameters are not being overridden.
        """
        amplifiersToCheck = []
        if detector:
            if self._detectorName != detector.getName():
                raise RuntimeError("Detector names don't match: %s != %s" %
                                   (self._detectorName, detector.getName()))
            if int(self._detectorId) != int(detector.getId()):
                raise RuntimeError("Detector IDs don't match: %s != %s" %
                                   (int(self._detectorId), int(detector.getId())))
            if self._detectorSerial != detector.getSerial():
                raise RuntimeError("Detector serial numbers don't match: %s != %s" %
                                   (self._detectorSerial, detector.getSerial()))
            if len(detector.getAmplifiers()) != len(self.linearityCoeffs.keys()):
                raise RuntimeError("Detector number of amps = %s does not match saved value %s" %
                                   (len(detector.getAmplifiers()),
                                    len(self.linearityCoeffs.keys())))
            amplifiersToCheck.extend(detector.getAmplifiers())
 
        if amplifier:
            amplifiersToCheck.extend(amplifier)
 
        for amp in amplifiersToCheck:
            ampName = amp.getName()
            if ampName not in self.linearityCoeffs.keys():
                raise RuntimeError("Amplifier %s is not in linearity data" %
                                   (ampName, ))
            if amp.getLinearityType() != self.linearityType[ampName]:
                if self.override:
                    self.log.warning("Overriding amplifier defined linearityType (%s) for %s",
                                     self.linearityType[ampName], ampName)
                else:
                    raise RuntimeError("Amplifier %s type %s does not match saved value %s" %
                                       (ampName, amp.getLinearityType(), self.linearityType[ampName]))
            if (amp.getLinearityCoeffs().shape != self.linearityCoeffs[ampName].shape or not
                    np.allclose(amp.getLinearityCoeffs(), self.linearityCoeffs[ampName], equal_nan=True)):
                if self.override:
                    self.log.warning("Overriding amplifier defined linearityCoeffs (%s) for %s",
                                     self.linearityCoeffs[ampName], ampName)
                else:
                    raise RuntimeError("Amplifier %s coeffs %s does not match saved value %s" %
                                       (ampName, amp.getLinearityCoeffs(), self.linearityCoeffs[ampName]))
 

validate() [2/2]

def lsst.ip.isr.calibType.IsrCalib.validate (   self,   other = None  )

inherited

Validate that this calibration is defined and can be used.

Parameters
----------
other : `object`, optional
    Thing to validate against.

Returns
-------
valid : `bool`
    Returns true if the calibration is valid and appropriate.

Definition at line 597 of file calibType.py.

    def validate(self, other=None):
        """Validate that this calibration is defined and can be used.
 
        Parameters
        ----------
        other : `object`, optional
            Thing to validate against.
 
        Returns
        -------
        valid : `bool`
            Returns true if the calibration is valid and appropriate.
        """
        return False
 

writeFits()

def lsst.ip.isr.calibType.IsrCalib.writeFits (   self,   filename  )

inherited

Write calibration data to a FITS file.

Parameters
----------
filename : `str`
    Filename to write data to.

Returns
-------
used : `str`
    The name of the file used to write the data.

Definition at line 471 of file calibType.py.

    def writeFits(self, filename):
        """Write calibration data to a FITS file.
 
        Parameters
        ----------
        filename : `str`
            Filename to write data to.
 
        Returns
        -------
        used : `str`
            The name of the file used to write the data.
 
        """
        tableList = self.toTable()
        with warnings.catch_warnings():
            warnings.filterwarnings("ignore", category=Warning, module="astropy.io")
            astropyList = [fits.table_to_hdu(table) for table in tableList]
            astropyList.insert(0, fits.PrimaryHDU())
 
            writer = fits.HDUList(astropyList)
            writer.writeto(filename, overwrite=True)
        return filename
 

writeText()

def lsst.ip.isr.calibType.IsrCalib.writeText (   self,   filename,   format = "auto"  )

inherited

Write the calibration data to a text file.

Parameters
----------
filename : `str`
    Name of the file to write.
format : `str`
    Format to write the file as.  Supported values are:
        ``"auto"`` : Determine filetype from filename.
        ``"yaml"`` : Write as yaml.
        ``"ecsv"`` : Write as ecsv.
Returns
-------
used : `str`
    The name of the file used to write the data.  This may
    differ from the input if the format is explicitly chosen.

Raises
------
RuntimeError :
    Raised if filename does not end in a known extension, or
    if all information cannot be written.

Notes
-----
The file is written to YAML/ECSV format and will include any
associated metadata.

Definition at line 379 of file calibType.py.

    def writeText(self, filename, format="auto"):
        """Write the calibration data to a text file.
 
        Parameters
        ----------
        filename : `str`
            Name of the file to write.
        format : `str`
            Format to write the file as.  Supported values are:
                ``"auto"`` : Determine filetype from filename.
                ``"yaml"`` : Write as yaml.
                ``"ecsv"`` : Write as ecsv.
        Returns
        -------
        used : `str`
            The name of the file used to write the data.  This may
            differ from the input if the format is explicitly chosen.
 
        Raises
        ------
        RuntimeError :
            Raised if filename does not end in a known extension, or
            if all information cannot be written.
 
        Notes
        -----
        The file is written to YAML/ECSV format and will include any
        associated metadata.
        """
        if format == "yaml" or (format == "auto" and filename.lower().endswith((".yaml", ".YAML"))):
            outDict = self.toDict()
            path, ext = os.path.splitext(filename)
            filename = path + ".yaml"
            with open(filename, "w") as f:
                yaml.dump(outDict, f)
        elif format == "ecsv" or (format == "auto" and filename.lower().endswith((".ecsv", ".ECSV"))):
            tableList = self.toTable()
            if len(tableList) > 1:
                # ECSV doesn't support multiple tables per file, so we
                # can only write the first table.
                raise RuntimeError(f"Unable to persist {len(tableList)}tables in ECSV format.")
 
            table = tableList[0]
            path, ext = os.path.splitext(filename)
            filename = path + ".ecsv"
            table.write(filename, format="ascii.ecsv")
        else:
            raise RuntimeError(f"Attempt to write to a file {filename} "
                               "that does not end in '.yaml' or '.ecsv'")
 
        return filename
 

Member Data Documentation

ampNames

lsst.ip.isr.linearize.Linearizer.ampNames

Definition at line 103 of file linearize.py.

fitChiSq

lsst.ip.isr.linearize.Linearizer.fitChiSq

Definition at line 110 of file linearize.py.

fitParams

lsst.ip.isr.linearize.Linearizer.fitParams

Definition at line 108 of file linearize.py.

fitParamsErr

lsst.ip.isr.linearize.Linearizer.fitParamsErr

Definition at line 109 of file linearize.py.

hasLinearity

lsst.ip.isr.linearize.Linearizer.hasLinearity

Definition at line 100 of file linearize.py.

linearityBBox

lsst.ip.isr.linearize.Linearizer.linearityBBox

Definition at line 106 of file linearize.py.

linearityCoeffs

lsst.ip.isr.linearize.Linearizer.linearityCoeffs

Definition at line 104 of file linearize.py.

linearityType

lsst.ip.isr.linearize.Linearizer.linearityType

Definition at line 105 of file linearize.py.

log

lsst.ip.isr.calibType.IsrCalib.log

inherited

Definition at line 86 of file calibType.py.

override

lsst.ip.isr.linearize.Linearizer.override

Definition at line 101 of file linearize.py.

requiredAttributes

lsst.ip.isr.calibType.IsrCalib.requiredAttributes

inherited

Definition at line 81 of file calibType.py.

tableData

lsst.ip.isr.linearize.Linearizer.tableData

Definition at line 112 of file linearize.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-0.7.0/Linux64/ip_isr/22.0.1-27-g97737f7+2a20fdde0d/python/lsst/ip/isr/linearize.py