LSST Applications  21.0.0-172-gfb10e10a+18fedfabac,22.0.0+297cba6710,22.0.0+80564b0ff1,22.0.0+8d77f4f51a,22.0.0+a28f4c53b1,22.0.0+dcf3732eb2,22.0.1-1-g7d6de66+2a20fdde0d,22.0.1-1-g8e32f31+297cba6710,22.0.1-1-geca5380+7fa3b7d9b6,22.0.1-12-g44dc1dc+2a20fdde0d,22.0.1-15-g6a90155+515f58c32b,22.0.1-16-g9282f48+790f5f2caa,22.0.1-2-g92698f7+dcf3732eb2,22.0.1-2-ga9b0f51+7fa3b7d9b6,22.0.1-2-gd1925c9+bf4f0e694f,22.0.1-24-g1ad7a390+a9625a72a8,22.0.1-25-g5bf6245+3ad8ecd50b,22.0.1-25-gb120d7b+8b5510f75f,22.0.1-27-g97737f7+2a20fdde0d,22.0.1-32-gf62ce7b1+aa4237961e,22.0.1-4-g0b3f228+2a20fdde0d,22.0.1-4-g243d05b+871c1b8305,22.0.1-4-g3a563be+32dcf1063f,22.0.1-4-g44f2e3d+9e4ab0f4fa,22.0.1-42-gca6935d93+ba5e5ca3eb,22.0.1-5-g15c806e+85460ae5f3,22.0.1-5-g58711c4+611d128589,22.0.1-5-g75bb458+99c117b92f,22.0.1-6-g1c63a23+7fa3b7d9b6,22.0.1-6-g50866e6+84ff5a128b,22.0.1-6-g8d3140d+720564cf76,22.0.1-6-gd805d02+cc5644f571,22.0.1-8-ge5750ce+85460ae5f3,master-g6e05de7fdc+babf819c66,master-g99da0e417a+8d77f4f51a,w.2021.48
LSST Data Management Base Package
Public Member Functions | Public Attributes | List of all members
lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset Class Reference
Inheritance diagram for lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset:
lsst.ip.isr.calibType.IsrCalib

Public Member Functions

def __init__ (self, ampNames=[], ptcFitType=None, covMatrixSide=1, **kwargs)
 
def setAmpValues (self, ampName, inputExpIdPair=[(np.nan, np.nan)], expIdMask=[np.nan], rawExpTime=[np.nan], rawMean=[np.nan], rawVar=[np.nan], photoCharge=[np.nan], gain=np.nan, gainErr=np.nan, noise=np.nan, noiseErr=np.nan, ptcFitPars=[np.nan], ptcFitParsError=[np.nan], ptcFitChiSq=np.nan, covArray=[], covArrayModel=[], covSqrtWeights=[], aMatrix=[], bMatrix=[], covArrayModelNoB=[], aMatrixNoB=[], finalVar=[np.nan], finalModelVar=[np.nan], finalMean=[np.nan])
 
def updateMetadata (self, setDate=False, **kwargs)
 
def fromDict (cls, dictionary)
 
def toDict (self)
 
def fromTable (cls, tableList)
 
def toTable (self)
 
def getExpIdsUsed (self, ampName)
 
def getGoodAmps (self)
 
def requiredAttributes (self)
 
def requiredAttributes (self, value)
 
def __str__ (self)
 
def __eq__ (self, other)
 
def getMetadata (self)
 
def setMetadata (self, metadata)
 
def updateMetadata (self, camera=None, detector=None, filterName=None, setCalibId=False, setCalibInfo=False, setDate=False, **kwargs)
 
def calibInfoFromDict (self, dictionary)
 
def determineCalibClass (cls, metadata, message)
 
def readText (cls, filename, **kwargs)
 
def writeText (self, filename, format="auto")
 
def readFits (cls, filename, **kwargs)
 
def writeFits (self, filename)
 
def fromDetector (self, detector)
 
def fromDict (cls, dictionary, **kwargs)
 
def fromTable (cls, tableList, **kwargs)
 
def validate (self, other=None)
 
def apply (self, target)
 

Public Attributes

 ptcFitType
 
 ampNames
 
 covMatrixSide
 
 badAmps
 
 inputExpIdPairs
 
 expIdMask
 
 rawExpTimes
 
 rawMeans
 
 rawVars
 
 photoCharge
 
 gain
 
 gainErr
 
 noise
 
 noiseErr
 
 ptcFitPars
 
 ptcFitParsError
 
 ptcFitChiSq
 
 covariances
 
 covariancesModel
 
 covariancesSqrtWeights
 
 aMatrix
 
 bMatrix
 
 covariancesModelNoB
 
 aMatrixNoB
 
 finalVars
 
 finalModelVars
 
 finalMeans
 
 requiredAttributes
 
 log
 

Detailed Description

A simple class to hold the output data from the PTC task.
The dataset is made up of a dictionary for each item, keyed by the
amplifiers' names, which much be supplied at construction time.
New items cannot be added to the class to save accidentally saving to the
wrong property, and the class can be frozen if desired.
inputExpIdPairs records the exposures used to produce the data.
When fitPtc() or fitCovariancesAstier() is run, a mask is built up, which
is by definition always the same length as inputExpIdPairs, rawExpTimes,
rawMeans and rawVars, and is a list of bools, which are incrementally set
to False as points are discarded from the fits.
PTC fit parameters for polynomials are stored in a list in ascending order
of polynomial term, i.e. par[0]*x^0 + par[1]*x + par[2]*x^2 etc
with the length of the list corresponding to the order of the polynomial
plus one.

Parameters
----------
ampNames : `list`
    List with the names of the amplifiers of the detector at hand.

ptcFitType : `str`
    Type of model fitted to the PTC: "POLYNOMIAL", "EXPAPPROXIMATION",
    or "FULLCOVARIANCE".

covMatrixSide : `int`
    Maximum lag of covariances (size of square covariance matrices).

kwargs : `dict`, optional
    Other keyword arguments to pass to the parent init.

Notes
-----
The stored attributes are:
badAmps : `list`
    List with bad amplifiers names.
inputExpIdPairs : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the input exposures IDs.
expIdMask : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the mask produced after
    outlier rejection. The mask produced by the "FULLCOVARIANCE"
    option may differ from the one produced in the other two PTC
    fit types.
rawExpTimes : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the unmasked exposure times.
rawMeans : `dict`, [`str`, `list`]
    Dictionary keyed by amp namescontaining the unmasked average of the
    means of the exposures in each flat pair.
rawVars : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the variance of the
    difference image of the exposures in each flat pair.
gain : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the fitted gains.
gainErr : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the errors on the
    fitted gains.
noise : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the fitted noise.
noiseErr : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the errors on the fitted noise.
ptcFitPars : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the fitted parameters of the
    PTC model for ptcFitTye in ["POLYNOMIAL", "EXPAPPROXIMATION"].
ptcFitParsError : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the errors on the fitted
    parameters of the PTC model for ptcFitTye in
    ["POLYNOMIAL", "EXPAPPROXIMATION"].
ptcFitChiSq : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the reduced chi squared
    of the fit for ptcFitTye in ["POLYNOMIAL", "EXPAPPROXIMATION"].
covariances : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing a list of measured
    covariances per mean flux.
covariancesModel : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containinging covariances model
    (Eq. 20 of Astier+19) per mean flux.
covariancesSqrtWeights : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containinging sqrt. of covariances
    weights.
aMatrix : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the "a" parameters from
    the model in Eq. 20 of Astier+19.
bMatrix : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the "b" parameters from
    the model in Eq. 20 of Astier+19.
covariancesModelNoB : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing covariances model
    (with 'b'=0 in Eq. 20 of Astier+19)
    per mean flux.
aMatrixNoB : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the "a" parameters from the
    model in Eq. 20 of Astier+19
    (and 'b' = 0).
finalVars : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the masked variance of the
    difference image of each flat
    pair. If needed, each array will be right-padded with
    np.nan to match the length of rawExpTimes.
finalModelVars : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the masked modeled
    variance of the difference image of each flat pair. If needed, each
    array will be right-padded with np.nan to match the length of
    rawExpTimes.
finalMeans : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the masked average of the
    means of the exposures in each flat pair. If needed, each array
    will be right-padded with np.nan to match the length of
    rawExpTimes.
photoCharge : `dict`, [`str`, `list`]
     Dictionary keyed by amp names containing the integrated photocharge
     for linearity calibration.

Returns
-------
`lsst.cp.pipe.ptc.PhotonTransferCurveDataset`
    Output dataset from MeasurePhotonTransferCurveTask.

Definition at line 33 of file ptcDataset.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.__init__ (   self,
  ampNames = [],
  ptcFitType = None,
  covMatrixSide = 1,
**  kwargs 
)

Reimplemented from lsst.ip.isr.calibType.IsrCalib.

Definition at line 155 of file ptcDataset.py.

155  def __init__(self, ampNames=[], ptcFitType=None, covMatrixSide=1, **kwargs):
156 
157  self.ptcFitType = ptcFitType
158  self.ampNames = ampNames
159  self.covMatrixSide = covMatrixSide
160 
161  self.badAmps = [np.nan]
162 
163  self.inputExpIdPairs = {ampName: [] for ampName in ampNames}
164  self.expIdMask = {ampName: [] for ampName in ampNames}
165  self.rawExpTimes = {ampName: [] for ampName in ampNames}
166  self.rawMeans = {ampName: [] for ampName in ampNames}
167  self.rawVars = {ampName: [] for ampName in ampNames}
168  self.photoCharge = {ampName: [] for ampName in ampNames}
169 
170  self.gain = {ampName: np.nan for ampName in ampNames}
171  self.gainErr = {ampName: np.nan for ampName in ampNames}
172  self.noise = {ampName: np.nan for ampName in ampNames}
173  self.noiseErr = {ampName: np.nan for ampName in ampNames}
174 
175  self.ptcFitPars = {ampName: [] for ampName in ampNames}
176  self.ptcFitParsError = {ampName: [] for ampName in ampNames}
177  self.ptcFitChiSq = {ampName: np.nan for ampName in ampNames}
178 
179  self.covariances = {ampName: [] for ampName in ampNames}
180  self.covariancesModel = {ampName: [] for ampName in ampNames}
181  self.covariancesSqrtWeights = {ampName: [] for ampName in ampNames}
182  self.aMatrix = {ampName: np.nan for ampName in ampNames}
183  self.bMatrix = {ampName: np.nan for ampName in ampNames}
184  self.covariancesModelNoB = {ampName: [] for ampName in ampNames}
185  self.aMatrixNoB = {ampName: np.nan for ampName in ampNames}
186 
187  self.finalVars = {ampName: [] for ampName in ampNames}
188  self.finalModelVars = {ampName: [] for ampName in ampNames}
189  self.finalMeans = {ampName: [] for ampName in ampNames}
190 
191  super().__init__(**kwargs)
192  self.requiredAttributes.update(['badAmps', 'inputExpIdPairs', 'expIdMask', 'rawExpTimes',
193  'rawMeans', 'rawVars', 'gain', 'gainErr', 'noise', 'noiseErr',
194  'ptcFitPars', 'ptcFitParsError', 'ptcFitChiSq', 'aMatrixNoB',
195  'covariances', 'covariancesModel', 'covariancesSqrtWeights',
196  'covariancesModelNoB',
197  'aMatrix', 'bMatrix', 'finalVars', 'finalModelVars', 'finalMeans',
198  'photoCharge'])
199 

Member Function Documentation

◆ __eq__()

def lsst.ip.isr.calibType.IsrCalib.__eq__ (   self,
  other 
)
inherited
Calibration equivalence.

Running ``calib.log.setLevel(0)`` enables debug statements to
identify problematic fields.

Reimplemented in lsst.ip.isr.defects.Defects, and lsst.ip.isr.calibType.IsrProvenance.

Definition at line 95 of file calibType.py.

95  def __eq__(self, other):
96  """Calibration equivalence.
97 
98  Running ``calib.log.setLevel(0)`` enables debug statements to
99  identify problematic fields.
100  """
101  if not isinstance(other, self.__class__):
102  self.log.debug("Incorrect class type: %s %s", self.__class__, other.__class__)
103  return False
104 
105  for attr in self._requiredAttributes:
106  attrSelf = getattr(self, attr)
107  attrOther = getattr(other, attr)
108 
109  if isinstance(attrSelf, dict):
110  # Dictionary of arrays.
111  if attrSelf.keys() != attrOther.keys():
112  self.log.debug("Dict Key Failure: %s %s %s", attr, attrSelf.keys(), attrOther.keys())
113  return False
114  for key in attrSelf:
115  if not np.allclose(attrSelf[key], attrOther[key], equal_nan=True):
116  self.log.debug("Array Failure: %s %s %s", key, attrSelf[key], attrOther[key])
117  return False
118  elif isinstance(attrSelf, np.ndarray):
119  # Bare array.
120  if not np.allclose(attrSelf, attrOther, equal_nan=True):
121  self.log.debug("Array Failure: %s %s %s", attr, attrSelf, attrOther)
122  return False
123  elif type(attrSelf) != type(attrOther):
124  if set([attrSelf, attrOther]) == set([None, ""]):
125  # Fits converts None to "", but None is not "".
126  continue
127  self.log.debug("Type Failure: %s %s %s %s %s", attr, type(attrSelf), type(attrOther),
128  attrSelf, attrOther)
129  return False
130  else:
131  if attrSelf != attrOther:
132  self.log.debug("Value Failure: %s %s %s", attr, attrSelf, attrOther)
133  return False
134 
135  return True
136 
table::Key< int > type
Definition: Detector.cc:163
daf::base::PropertySet * set
Definition: fits.cc:912

◆ __str__()

def lsst.ip.isr.calibType.IsrCalib.__str__ (   self)
inherited

Reimplemented in lsst.ip.isr.defects.Defects, and lsst.ip.isr.calibType.IsrProvenance.

Definition at line 92 of file calibType.py.

92  def __str__(self):
93  return f"{self.__class__.__name__}(obstype={self._OBSTYPE}, detector={self._detectorName}, )"
94 

◆ apply()

def lsst.ip.isr.calibType.IsrCalib.apply (   self,
  target 
)
inherited
Method to apply the calibration to the target object.

Parameters
----------
target : `object`
    Thing to validate against.

Returns
-------
valid : `bool`
    Returns true if the calibration was applied correctly.

Raises
------
NotImplementedError :
    Raised if not implemented.

Definition at line 612 of file calibType.py.

612  def apply(self, target):
613  """Method to apply the calibration to the target object.
614 
615  Parameters
616  ----------
617  target : `object`
618  Thing to validate against.
619 
620  Returns
621  -------
622  valid : `bool`
623  Returns true if the calibration was applied correctly.
624 
625  Raises
626  ------
627  NotImplementedError :
628  Raised if not implemented.
629  """
630  raise NotImplementedError("Must be implemented by subclass.")
631 
632 

◆ calibInfoFromDict()

def lsst.ip.isr.calibType.IsrCalib.calibInfoFromDict (   self,
  dictionary 
)
inherited
Handle common keywords.

This isn't an ideal solution, but until all calibrations
expect to find everything in the metadata, they still need to
search through dictionaries.

Parameters
----------
dictionary : `dict` or `lsst.daf.base.PropertyList`
    Source for the common keywords.

Raises
------
RuntimeError :
    Raised if the dictionary does not match the expected OBSTYPE.

Definition at line 261 of file calibType.py.

261  def calibInfoFromDict(self, dictionary):
262  """Handle common keywords.
263 
264  This isn't an ideal solution, but until all calibrations
265  expect to find everything in the metadata, they still need to
266  search through dictionaries.
267 
268  Parameters
269  ----------
270  dictionary : `dict` or `lsst.daf.base.PropertyList`
271  Source for the common keywords.
272 
273  Raises
274  ------
275  RuntimeError :
276  Raised if the dictionary does not match the expected OBSTYPE.
277 
278  """
279 
280  def search(haystack, needles):
281  """Search dictionary 'haystack' for an entry in 'needles'
282  """
283  test = [haystack.get(x) for x in needles]
284  test = set([x for x in test if x is not None])
285  if len(test) == 0:
286  if "metadata" in haystack:
287  return search(haystack["metadata"], needles)
288  else:
289  return None
290  elif len(test) == 1:
291  value = list(test)[0]
292  if value == "":
293  return None
294  else:
295  return value
296  else:
297  raise ValueError(f"Too many values found: {len(test)} {test} {needles}")
298 
299  if "metadata" in dictionary:
300  metadata = dictionary["metadata"]
301 
302  if self._OBSTYPE != metadata["OBSTYPE"]:
303  raise RuntimeError(f"Incorrect calibration supplied. Expected {self._OBSTYPE}, "
304  f"found {metadata['OBSTYPE']}")
305 
306  self._instrument = search(dictionary, ["INSTRUME", "instrument"])
307  self._raftName = search(dictionary, ["RAFTNAME"])
308  self._slotName = search(dictionary, ["SLOTNAME"])
309  self._detectorId = search(dictionary, ["DETECTOR", "detectorId"])
310  self._detectorName = search(dictionary, ["DET_NAME", "DETECTOR_NAME", "detectorName"])
311  self._detectorSerial = search(dictionary, ["DET_SER", "DETECTOR_SERIAL", "detectorSerial"])
312  self._filter = search(dictionary, ["FILTER", "filterName"])
313  self._calibId = search(dictionary, ["CALIB_ID"])
314 
daf::base::PropertyList * list
Definition: fits.cc:913

◆ determineCalibClass()

def lsst.ip.isr.calibType.IsrCalib.determineCalibClass (   cls,
  metadata,
  message 
)
inherited
Attempt to find calibration class in metadata.

Parameters
----------
metadata : `dict` or `lsst.daf.base.PropertyList`
    Metadata possibly containing a calibration class entry.
message : `str`
    Message to include in any errors.

Returns
-------
calibClass : `object`
    The class to use to read the file contents.  Should be an
    `lsst.ip.isr.IsrCalib` subclass.

Raises
------
ValueError :
    Raised if the resulting calibClass is the base
    `lsst.ip.isr.IsrClass` (which does not implement the
    content methods).

Definition at line 316 of file calibType.py.

316  def determineCalibClass(cls, metadata, message):
317  """Attempt to find calibration class in metadata.
318 
319  Parameters
320  ----------
321  metadata : `dict` or `lsst.daf.base.PropertyList`
322  Metadata possibly containing a calibration class entry.
323  message : `str`
324  Message to include in any errors.
325 
326  Returns
327  -------
328  calibClass : `object`
329  The class to use to read the file contents. Should be an
330  `lsst.ip.isr.IsrCalib` subclass.
331 
332  Raises
333  ------
334  ValueError :
335  Raised if the resulting calibClass is the base
336  `lsst.ip.isr.IsrClass` (which does not implement the
337  content methods).
338  """
339  calibClassName = metadata.get("CALIBCLS")
340  calibClass = doImport(calibClassName) if calibClassName is not None else cls
341  if calibClass is IsrCalib:
342  raise ValueError(f"Cannot use base class to read calibration data: {msg}")
343  return calibClass
344 

◆ fromDetector()

def lsst.ip.isr.calibType.IsrCalib.fromDetector (   self,
  detector 
)
inherited
Modify the calibration parameters to match the supplied detector.

Parameters
----------
detector : `lsst.afw.cameraGeom.Detector`
    Detector to use to set parameters from.

Raises
------
NotImplementedError
    This needs to be implemented by subclasses for each
    calibration type.

Reimplemented in lsst.ip.isr.linearize.Linearizer.

Definition at line 495 of file calibType.py.

495  def fromDetector(self, detector):
496  """Modify the calibration parameters to match the supplied detector.
497 
498  Parameters
499  ----------
500  detector : `lsst.afw.cameraGeom.Detector`
501  Detector to use to set parameters from.
502 
503  Raises
504  ------
505  NotImplementedError
506  This needs to be implemented by subclasses for each
507  calibration type.
508  """
509  raise NotImplementedError("Must be implemented by subclass.")
510 

◆ fromDict() [1/2]

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.fromDict (   cls,
  dictionary 
)
Construct a calibration from a dictionary of properties.
Must be implemented by the specific calibration subclasses.
Parameters
----------
dictionary : `dict`
    Dictionary of properties.
Returns
-------
calib : `lsst.ip.isr.CalibType`
    Constructed calibration.
Raises
------
RuntimeError :
    Raised if the supplied dictionary is for a different
    calibration.

Definition at line 272 of file ptcDataset.py.

272  def fromDict(cls, dictionary):
273  """Construct a calibration from a dictionary of properties.
274  Must be implemented by the specific calibration subclasses.
275  Parameters
276  ----------
277  dictionary : `dict`
278  Dictionary of properties.
279  Returns
280  -------
281  calib : `lsst.ip.isr.CalibType`
282  Constructed calibration.
283  Raises
284  ------
285  RuntimeError :
286  Raised if the supplied dictionary is for a different
287  calibration.
288  """
289  calib = cls()
290  if calib._OBSTYPE != dictionary['metadata']['OBSTYPE']:
291  raise RuntimeError(f"Incorrect Photon Transfer Curve dataset supplied. "
292  f"Expected {calib._OBSTYPE}, found {dictionary['metadata']['OBSTYPE']}")
293  calib.setMetadata(dictionary['metadata'])
294  calib.ptcFitType = dictionary['ptcFitType']
295  calib.covMatrixSide = dictionary['covMatrixSide']
296  calib.badAmps = np.array(dictionary['badAmps'], 'str').tolist()
297  # The cov matrices are square
298  covMatrixSide = calib.covMatrixSide
299  # Number of final signal levels
300  covDimensionsProduct = len(np.array(list(dictionary['covariances'].values())[0]).ravel())
301  nSignalPoints = int(covDimensionsProduct/(covMatrixSide*covMatrixSide))
302 
303  for ampName in dictionary['ampNames']:
304  calib.ampNames.append(ampName)
305  calib.inputExpIdPairs[ampName] = np.array(dictionary['inputExpIdPairs'][ampName]).tolist()
306  calib.expIdMask[ampName] = np.array(dictionary['expIdMask'][ampName]).tolist()
307  calib.rawExpTimes[ampName] = np.array(dictionary['rawExpTimes'][ampName]).tolist()
308  calib.rawMeans[ampName] = np.array(dictionary['rawMeans'][ampName]).tolist()
309  calib.rawVars[ampName] = np.array(dictionary['rawVars'][ampName]).tolist()
310  calib.gain[ampName] = np.array(dictionary['gain'][ampName]).tolist()
311  calib.gainErr[ampName] = np.array(dictionary['gainErr'][ampName]).tolist()
312  calib.noise[ampName] = np.array(dictionary['noise'][ampName]).tolist()
313  calib.noiseErr[ampName] = np.array(dictionary['noiseErr'][ampName]).tolist()
314  calib.ptcFitPars[ampName] = np.array(dictionary['ptcFitPars'][ampName]).tolist()
315  calib.ptcFitParsError[ampName] = np.array(dictionary['ptcFitParsError'][ampName]).tolist()
316  calib.ptcFitChiSq[ampName] = np.array(dictionary['ptcFitChiSq'][ampName]).tolist()
317  calib.covariances[ampName] = np.array(dictionary['covariances'][ampName]).reshape(
318  (nSignalPoints, covMatrixSide, covMatrixSide)).tolist()
319  calib.covariancesModel[ampName] = np.array(
320  dictionary['covariancesModel'][ampName]).reshape(
321  (nSignalPoints, covMatrixSide, covMatrixSide)).tolist()
322  calib.covariancesSqrtWeights[ampName] = np.array(
323  dictionary['covariancesSqrtWeights'][ampName]).reshape(
324  (nSignalPoints, covMatrixSide, covMatrixSide)).tolist()
325  calib.aMatrix[ampName] = np.array(dictionary['aMatrix'][ampName]).reshape(
326  (covMatrixSide, covMatrixSide)).tolist()
327  calib.bMatrix[ampName] = np.array(dictionary['bMatrix'][ampName]).reshape(
328  (covMatrixSide, covMatrixSide)).tolist()
329  calib.covariancesModelNoB[ampName] = np.array(
330  dictionary['covariancesModelNoB'][ampName]).reshape(
331  (nSignalPoints, covMatrixSide, covMatrixSide)).tolist()
332  calib.aMatrixNoB[ampName] = np.array(
333  dictionary['aMatrixNoB'][ampName]).reshape((covMatrixSide, covMatrixSide)).tolist()
334  calib.finalVars[ampName] = np.array(dictionary['finalVars'][ampName]).tolist()
335  calib.finalModelVars[ampName] = np.array(dictionary['finalModelVars'][ampName]).tolist()
336  calib.finalMeans[ampName] = np.array(dictionary['finalMeans'][ampName]).tolist()
337  calib.photoCharge[ampName] = np.array(dictionary['photoCharge'][ampName]).tolist()
338  calib.updateMetadata()
339  return calib
340 

◆ fromDict() [2/2]

def lsst.ip.isr.calibType.IsrCalib.fromDict (   cls,
  dictionary,
**  kwargs 
)
inherited
Construct a calibration from a dictionary of properties.

Must be implemented by the specific calibration subclasses.

Parameters
----------
dictionary : `dict`
    Dictionary of properties.
kwargs : `dict` or collections.abc.Mapping`, optional
    Set of key=value options.

Returns
------
calib : `lsst.ip.isr.CalibType`
    Constructed calibration.

Raises
------
NotImplementedError :
    Raised if not implemented.

Definition at line 512 of file calibType.py.

512  def fromDict(cls, dictionary, **kwargs):
513  """Construct a calibration from a dictionary of properties.
514 
515  Must be implemented by the specific calibration subclasses.
516 
517  Parameters
518  ----------
519  dictionary : `dict`
520  Dictionary of properties.
521  kwargs : `dict` or collections.abc.Mapping`, optional
522  Set of key=value options.
523 
524  Returns
525  ------
526  calib : `lsst.ip.isr.CalibType`
527  Constructed calibration.
528 
529  Raises
530  ------
531  NotImplementedError :
532  Raised if not implemented.
533  """
534  raise NotImplementedError("Must be implemented by subclass.")
535 

◆ fromTable() [1/2]

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.fromTable (   cls,
  tableList 
)
Construct calibration from a list of tables.
This method uses the `fromDict` method to create the
calibration, after constructing an appropriate dictionary from
the input tables.
Parameters
----------
tableList : `list` [`lsst.afw.table.Table`]
    List of tables to use to construct the datasetPtc.
Returns
-------
calib : `lsst.cp.pipe.`
    The calibration defined in the tables.

Definition at line 387 of file ptcDataset.py.

387  def fromTable(cls, tableList):
388  """Construct calibration from a list of tables.
389  This method uses the `fromDict` method to create the
390  calibration, after constructing an appropriate dictionary from
391  the input tables.
392  Parameters
393  ----------
394  tableList : `list` [`lsst.afw.table.Table`]
395  List of tables to use to construct the datasetPtc.
396  Returns
397  -------
398  calib : `lsst.cp.pipe.`
399  The calibration defined in the tables.
400  """
401  ptcTable = tableList[0]
402 
403  metadata = ptcTable.meta
404  inDict = dict()
405  inDict['metadata'] = metadata
406  inDict['ampNames'] = []
407  inDict['ptcFitType'] = []
408  inDict['covMatrixSide'] = []
409  inDict['inputExpIdPairs'] = dict()
410  inDict['expIdMask'] = dict()
411  inDict['rawExpTimes'] = dict()
412  inDict['rawMeans'] = dict()
413  inDict['rawVars'] = dict()
414  inDict['gain'] = dict()
415  inDict['gainErr'] = dict()
416  inDict['noise'] = dict()
417  inDict['noiseErr'] = dict()
418  inDict['ptcFitPars'] = dict()
419  inDict['ptcFitParsError'] = dict()
420  inDict['ptcFitChiSq'] = dict()
421  inDict['covariances'] = dict()
422  inDict['covariancesModel'] = dict()
423  inDict['covariancesSqrtWeights'] = dict()
424  inDict['aMatrix'] = dict()
425  inDict['bMatrix'] = dict()
426  inDict['covariancesModelNoB'] = dict()
427  inDict['aMatrixNoB'] = dict()
428  inDict['finalVars'] = dict()
429  inDict['finalModelVars'] = dict()
430  inDict['finalMeans'] = dict()
431  inDict['badAmps'] = []
432  inDict['photoCharge'] = dict()
433 
434  for record in ptcTable:
435  ampName = record['AMPLIFIER_NAME']
436 
437  inDict['ptcFitType'] = record['PTC_FIT_TYPE']
438  inDict['covMatrixSide'] = record['COV_MATRIX_SIDE']
439  inDict['ampNames'].append(ampName)
440  inDict['inputExpIdPairs'][ampName] = record['INPUT_EXP_ID_PAIRS']
441  inDict['expIdMask'][ampName] = record['EXP_ID_MASK']
442  inDict['rawExpTimes'][ampName] = record['RAW_EXP_TIMES']
443  inDict['rawMeans'][ampName] = record['RAW_MEANS']
444  inDict['rawVars'][ampName] = record['RAW_VARS']
445  inDict['gain'][ampName] = record['GAIN']
446  inDict['gainErr'][ampName] = record['GAIN_ERR']
447  inDict['noise'][ampName] = record['NOISE']
448  inDict['noiseErr'][ampName] = record['NOISE_ERR']
449  inDict['ptcFitPars'][ampName] = record['PTC_FIT_PARS']
450  inDict['ptcFitParsError'][ampName] = record['PTC_FIT_PARS_ERROR']
451  inDict['ptcFitChiSq'][ampName] = record['PTC_FIT_CHI_SQ']
452  inDict['covariances'][ampName] = record['COVARIANCES']
453  inDict['covariancesModel'][ampName] = record['COVARIANCES_MODEL']
454  inDict['covariancesSqrtWeights'][ampName] = record['COVARIANCES_SQRT_WEIGHTS']
455  inDict['aMatrix'][ampName] = record['A_MATRIX']
456  inDict['bMatrix'][ampName] = record['B_MATRIX']
457  inDict['covariancesModelNoB'][ampName] = record['COVARIANCES_MODEL_NO_B']
458  inDict['aMatrixNoB'][ampName] = record['A_MATRIX_NO_B']
459  inDict['finalVars'][ampName] = record['FINAL_VARS']
460  inDict['finalModelVars'][ampName] = record['FINAL_MODEL_VARS']
461  inDict['finalMeans'][ampName] = record['FINAL_MEANS']
462  inDict['badAmps'] = record['BAD_AMPS']
463  inDict['photoCharge'][ampName] = record['PHOTO_CHARGE']
464  return cls().fromDict(inDict)
465 
std::shared_ptr< FrameSet > append(FrameSet const &first, FrameSet const &second)
Construct a FrameSet that performs two transformations in series.
Definition: functional.cc:33

◆ fromTable() [2/2]

def lsst.ip.isr.calibType.IsrCalib.fromTable (   cls,
  tableList,
**  kwargs 
)
inherited
Construct a calibration from a dictionary of properties.

Must be implemented by the specific calibration subclasses.

Parameters
----------
tableList : `list` [`lsst.afw.table.Table`]
    List of tables of properties.
kwargs : `dict` or collections.abc.Mapping`, optional
    Set of key=value options.

Returns
------
calib : `lsst.ip.isr.CalibType`
    Constructed calibration.

Raises
------
NotImplementedError :
    Raised if not implemented.

Definition at line 555 of file calibType.py.

555  def fromTable(cls, tableList, **kwargs):
556  """Construct a calibration from a dictionary of properties.
557 
558  Must be implemented by the specific calibration subclasses.
559 
560  Parameters
561  ----------
562  tableList : `list` [`lsst.afw.table.Table`]
563  List of tables of properties.
564  kwargs : `dict` or collections.abc.Mapping`, optional
565  Set of key=value options.
566 
567  Returns
568  ------
569  calib : `lsst.ip.isr.CalibType`
570  Constructed calibration.
571 
572  Raises
573  ------
574  NotImplementedError :
575  Raised if not implemented.
576  """
577  raise NotImplementedError("Must be implemented by subclass.")
578 

◆ getExpIdsUsed()

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.getExpIdsUsed (   self,
  ampName 
)
Get the exposures used, i.e. not discarded, for a given amp.
If no mask has been created yet, all exposures are returned.

Definition at line 549 of file ptcDataset.py.

549  def getExpIdsUsed(self, ampName):
550  """Get the exposures used, i.e. not discarded, for a given amp.
551  If no mask has been created yet, all exposures are returned.
552  """
553  if len(self.expIdMask[ampName]) == 0:
554  return self.inputExpIdPairs[ampName]
555 
556  # if the mask exists it had better be the same length as the expIdPairs
557  assert len(self.expIdMask[ampName]) == len(self.inputExpIdPairs[ampName])
558 
559  pairs = self.inputExpIdPairs[ampName]
560  mask = self.expIdMask[ampName]
561  # cast to bool required because numpy
562  return [(exp1, exp2) for ((exp1, exp2), m) in zip(pairs, mask) if bool(m) is True]
563 

◆ getGoodAmps()

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.getGoodAmps (   self)

Definition at line 564 of file ptcDataset.py.

564  def getGoodAmps(self):
565  return [amp for amp in self.ampNames if amp not in self.badAmps]

◆ getMetadata()

def lsst.ip.isr.calibType.IsrCalib.getMetadata (   self)
inherited
Retrieve metadata associated with this calibration.

Returns
-------
meta : `lsst.daf.base.PropertyList`
    Metadata. The returned `~lsst.daf.base.PropertyList` can be
    modified by the caller and the changes will be written to
    external files.

Definition at line 145 of file calibType.py.

145  def getMetadata(self):
146  """Retrieve metadata associated with this calibration.
147 
148  Returns
149  -------
150  meta : `lsst.daf.base.PropertyList`
151  Metadata. The returned `~lsst.daf.base.PropertyList` can be
152  modified by the caller and the changes will be written to
153  external files.
154  """
155  return self._metadata
156 

◆ readFits()

def lsst.ip.isr.calibType.IsrCalib.readFits (   cls,
  filename,
**  kwargs 
)
inherited
Read calibration data from a FITS file.

Parameters
----------
filename : `str`
    Filename to read data from.
kwargs : `dict` or collections.abc.Mapping`, optional
    Set of key=value pairs to pass to the ``fromTable``
    method.

Returns
-------
calib : `lsst.ip.isr.IsrCalib`
    Calibration contained within the file.

Definition at line 432 of file calibType.py.

432  def readFits(cls, filename, **kwargs):
433  """Read calibration data from a FITS file.
434 
435  Parameters
436  ----------
437  filename : `str`
438  Filename to read data from.
439  kwargs : `dict` or collections.abc.Mapping`, optional
440  Set of key=value pairs to pass to the ``fromTable``
441  method.
442 
443  Returns
444  -------
445  calib : `lsst.ip.isr.IsrCalib`
446  Calibration contained within the file.
447  """
448  tableList = []
449  tableList.append(Table.read(filename, hdu=1))
450  extNum = 2 # Fits indices start at 1, we've read one already.
451  keepTrying = True
452 
453  while keepTrying:
454  with warnings.catch_warnings():
455  warnings.simplefilter("error")
456  try:
457  newTable = Table.read(filename, hdu=extNum)
458  tableList.append(newTable)
459  extNum += 1
460  except Exception:
461  keepTrying = False
462 
463  for table in tableList:
464  for k, v in table.meta.items():
465  if isinstance(v, fits.card.Undefined):
466  table.meta[k] = None
467 
468  calibClass = cls.determineCalibClass(tableList[0].meta, "readFits")
469  return calibClass.fromTable(tableList, **kwargs)
470 

◆ readText()

def lsst.ip.isr.calibType.IsrCalib.readText (   cls,
  filename,
**  kwargs 
)
inherited
Read calibration representation from a yaml/ecsv file.

Parameters
----------
filename : `str`
    Name of the file containing the calibration definition.
kwargs : `dict` or collections.abc.Mapping`, optional
    Set of key=value pairs to pass to the ``fromDict`` or
    ``fromTable`` methods.

Returns
-------
calib : `~lsst.ip.isr.IsrCalibType`
    Calibration class.

Raises
------
RuntimeError :
    Raised if the filename does not end in ".ecsv" or ".yaml".

Definition at line 346 of file calibType.py.

346  def readText(cls, filename, **kwargs):
347  """Read calibration representation from a yaml/ecsv file.
348 
349  Parameters
350  ----------
351  filename : `str`
352  Name of the file containing the calibration definition.
353  kwargs : `dict` or collections.abc.Mapping`, optional
354  Set of key=value pairs to pass to the ``fromDict`` or
355  ``fromTable`` methods.
356 
357  Returns
358  -------
359  calib : `~lsst.ip.isr.IsrCalibType`
360  Calibration class.
361 
362  Raises
363  ------
364  RuntimeError :
365  Raised if the filename does not end in ".ecsv" or ".yaml".
366  """
367  if filename.endswith((".ecsv", ".ECSV")):
368  data = Table.read(filename, format="ascii.ecsv")
369  calibClass = cls.determineCalibClass(data.meta, "readText/ECSV")
370  return calibClass.fromTable([data], **kwargs)
371  elif filename.endswith((".yaml", ".YAML")):
372  with open(filename, "r") as f:
373  data = yaml.load(f, Loader=yaml.CLoader)
374  calibClass = cls.determineCalibClass(data["metadata"], "readText/YAML")
375  return calibClass.fromDict(data, **kwargs)
376  else:
377  raise RuntimeError(f"Unknown filename extension: {filename}")
378 

◆ requiredAttributes() [1/2]

def lsst.ip.isr.calibType.IsrCalib.requiredAttributes (   self)
inherited

Definition at line 138 of file calibType.py.

138  def requiredAttributes(self):
139  return self._requiredAttributes
140 

◆ requiredAttributes() [2/2]

def lsst.ip.isr.calibType.IsrCalib.requiredAttributes (   self,
  value 
)
inherited

Definition at line 142 of file calibType.py.

142  def requiredAttributes(self, value):
143  self._requiredAttributes = value
144 

◆ setAmpValues()

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.setAmpValues (   self,
  ampName,
  inputExpIdPair = [(np.nan, np.nan)],
  expIdMask = [np.nan],
  rawExpTime = [np.nan],
  rawMean = [np.nan],
  rawVar = [np.nan],
  photoCharge = [np.nan],
  gain = np.nan,
  gainErr = np.nan,
  noise = np.nan,
  noiseErr = np.nan,
  ptcFitPars = [np.nan],
  ptcFitParsError = [np.nan],
  ptcFitChiSq = np.nan,
  covArray = [],
  covArrayModel = [],
  covSqrtWeights = [],
  aMatrix = [],
  bMatrix = [],
  covArrayModelNoB = [],
  aMatrixNoB = [],
  finalVar = [np.nan],
  finalModelVar = [np.nan],
  finalMean = [np.nan] 
)
Function to initialize an amp of a PhotonTransferCurveDataset.

Notes
-----
The parameters are all documented in `init`.

Definition at line 200 of file ptcDataset.py.

205  finalVar=[np.nan], finalModelVar=[np.nan], finalMean=[np.nan]):
206  """Function to initialize an amp of a PhotonTransferCurveDataset.
207 
208  Notes
209  -----
210  The parameters are all documented in `init`.
211  """
212  nanMatrix = np.full((self.covMatrixSide, self.covMatrixSide), np.nan)
213  if len(covArray) == 0:
214  covArray = [nanMatrix]
215  if len(covArrayModel) == 0:
216  covArrayModel = [nanMatrix]
217  if len(covSqrtWeights) == 0:
218  covSqrtWeights = [nanMatrix]
219  if len(covArrayModelNoB) == 0:
220  covArrayModelNoB = [nanMatrix]
221  if len(aMatrix) == 0:
222  aMatrix = nanMatrix
223  if len(bMatrix) == 0:
224  bMatrix = nanMatrix
225  if len(aMatrixNoB) == 0:
226  aMatrixNoB = nanMatrix
227 
228  self.inputExpIdPairs[ampName] = inputExpIdPair
229  self.expIdMask[ampName] = expIdMask
230  self.rawExpTimes[ampName] = rawExpTime
231  self.rawMeans[ampName] = rawMean
232  self.rawVars[ampName] = rawVar
233  self.photoCharge[ampName] = photoCharge
234  self.gain[ampName] = gain
235  self.gainErr[ampName] = gainErr
236  self.noise[ampName] = noise
237  self.noiseErr[ampName] = noiseErr
238  self.ptcFitPars[ampName] = ptcFitPars
239  self.ptcFitParsError[ampName] = ptcFitParsError
240  self.ptcFitChiSq[ampName]
241  self.covariances[ampName] = covArray
242  self.covariancesSqrtWeights[ampName] = covSqrtWeights
243  self.covariancesModel[ampName] = covArrayModel
244  self.covariancesModelNoB[ampName] = covArrayModelNoB
245  self.aMatrix[ampName] = aMatrix
246  self.bMatrix[ampName] = bMatrix
247  self.aMatrixNoB[ampName] = aMatrixNoB
248  self.ptcFitPars[ampName] = ptcFitPars
249  self.ptcFitParsError[ampName] = ptcFitParsError
250  self.ptcFitChiSq[ampName] = ptcFitChiSq
251  self.finalVars[ampName] = finalVar
252  self.finalModelVars[ampName] = finalModelVar
253  self.finalMeans[ampName] = finalMean
254 

◆ setMetadata()

def lsst.ip.isr.calibType.IsrCalib.setMetadata (   self,
  metadata 
)
inherited
Store a copy of the supplied metadata with this calibration.

Parameters
----------
metadata : `lsst.daf.base.PropertyList`
    Metadata to associate with the calibration.  Will be copied and
    overwrite existing metadata.

Definition at line 157 of file calibType.py.

157  def setMetadata(self, metadata):
158  """Store a copy of the supplied metadata with this calibration.
159 
160  Parameters
161  ----------
162  metadata : `lsst.daf.base.PropertyList`
163  Metadata to associate with the calibration. Will be copied and
164  overwrite existing metadata.
165  """
166  if metadata is not None:
167  self._metadata.update(metadata)
168 
169  # Ensure that we have the obs type required by calibration ingest
170  self._metadata["OBSTYPE"] = self._OBSTYPE
171  self._metadata[self._OBSTYPE + "_SCHEMA"] = self._SCHEMA
172  self._metadata[self._OBSTYPE + "_VERSION"] = self._VERSION
173 
174  if isinstance(metadata, dict):
175  self.calibInfoFromDict(metadata)
176  elif isinstance(metadata, PropertyList):
177  self.calibInfoFromDict(metadata.toDict())
178 

◆ toDict()

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.toDict (   self)
Return a dictionary containing the calibration properties.
The dictionary should be able to be round-tripped through
`fromDict`.
Returns
-------
dictionary : `dict`
    Dictionary of properties.

Reimplemented from lsst.ip.isr.calibType.IsrCalib.

Definition at line 341 of file ptcDataset.py.

341  def toDict(self):
342  """Return a dictionary containing the calibration properties.
343  The dictionary should be able to be round-tripped through
344  `fromDict`.
345  Returns
346  -------
347  dictionary : `dict`
348  Dictionary of properties.
349  """
350  self.updateMetadata()
351 
352  outDict = dict()
353  metadata = self.getMetadata()
354  outDict['metadata'] = metadata
355 
356  outDict['ptcFitType'] = self.ptcFitType
357  outDict['covMatrixSide'] = self.covMatrixSide
358  outDict['ampNames'] = self.ampNames
359  outDict['badAmps'] = self.badAmps
360  outDict['inputExpIdPairs'] = self.inputExpIdPairs
361  outDict['expIdMask'] = self.expIdMask
362  outDict['rawExpTimes'] = self.rawExpTimes
363  outDict['rawMeans'] = self.rawMeans
364  outDict['rawVars'] = self.rawVars
365  outDict['gain'] = self.gain
366  outDict['gainErr'] = self.gainErr
367  outDict['noise'] = self.noise
368  outDict['noiseErr'] = self.noiseErr
369  outDict['ptcFitPars'] = self.ptcFitPars
370  outDict['ptcFitParsError'] = self.ptcFitParsError
371  outDict['ptcFitChiSq'] = self.ptcFitChiSq
372  outDict['covariances'] = self.covariances
373  outDict['covariancesModel'] = self.covariancesModel
374  outDict['covariancesSqrtWeights'] = self.covariancesSqrtWeights
375  outDict['aMatrix'] = self.aMatrix
376  outDict['bMatrix'] = self.bMatrix
377  outDict['covariancesModelNoB'] = self.covariancesModelNoB
378  outDict['aMatrixNoB'] = self.aMatrixNoB
379  outDict['finalVars'] = self.finalVars
380  outDict['finalModelVars'] = self.finalModelVars
381  outDict['finalMeans'] = self.finalMeans
382  outDict['photoCharge'] = self.photoCharge
383 
384  return outDict
385 

◆ toTable()

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.toTable (   self)
Construct a list of tables containing the information in this
calibration.

The list of tables should create an identical calibration
after being passed to this class's fromTable method.
Returns
-------
tableList : `list` [`astropy.table.Table`]
    List of tables containing the linearity calibration
    information.

Reimplemented from lsst.ip.isr.calibType.IsrCalib.

Definition at line 466 of file ptcDataset.py.

466  def toTable(self):
467  """Construct a list of tables containing the information in this
468  calibration.
469 
470  The list of tables should create an identical calibration
471  after being passed to this class's fromTable method.
472  Returns
473  -------
474  tableList : `list` [`astropy.table.Table`]
475  List of tables containing the linearity calibration
476  information.
477  """
478  tableList = []
479  self.updateMetadata()
480  nPoints = []
481  for i, ampName in enumerate(self.ampNames):
482  nPoints.append(len(list(self.covariances.values())[i]))
483  nSignalPoints = max(nPoints)
484  nPadPoints = {}
485  for i, ampName in enumerate(self.ampNames):
486  nPadPoints[ampName] = nSignalPoints - len(list(self.covariances.values())[i])
487  covMatrixSide = self.covMatrixSide
488 
489  catalog = Table([{'AMPLIFIER_NAME': ampName,
490  'PTC_FIT_TYPE': self.ptcFitType,
491  'COV_MATRIX_SIDE': self.covMatrixSide,
492  'INPUT_EXP_ID_PAIRS': self.inputExpIdPairs[ampName]
493  if len(self.expIdMask[ampName]) else np.nan,
494  'EXP_ID_MASK': self.expIdMask[ampName]
495  if len(self.expIdMask[ampName]) else np.nan,
496  'RAW_EXP_TIMES': np.array(self.rawExpTimes[ampName]).tolist()
497  if len(self.rawExpTimes[ampName]) else np.nan,
498  'RAW_MEANS': np.array(self.rawMeans[ampName]).tolist()
499  if len(self.rawMeans[ampName]) else np.nan,
500  'RAW_VARS': np.array(self.rawVars[ampName]).tolist()
501  if len(self.rawVars[ampName]) else np.nan,
502  'GAIN': self.gain[ampName],
503  'GAIN_ERR': self.gainErr[ampName],
504  'NOISE': self.noise[ampName],
505  'NOISE_ERR': self.noiseErr[ampName],
506  'PTC_FIT_PARS': np.array(self.ptcFitPars[ampName]).tolist(),
507  'PTC_FIT_PARS_ERROR': np.array(self.ptcFitParsError[ampName]).tolist(),
508  'PTC_FIT_CHI_SQ': self.ptcFitChiSq[ampName],
509  'COVARIANCES': np.pad(np.array(self.covariances[ampName]),
510  ((0, nPadPoints[ampName]), (0, 0), (0, 0)),
511  'constant', constant_values=np.nan).reshape(
512  nSignalPoints*covMatrixSide**2).tolist(),
513  'COVARIANCES_MODEL': np.pad(np.array(self.covariancesModel[ampName]),
514  ((0, nPadPoints[ampName]), (0, 0), (0, 0)),
515  'constant', constant_values=np.nan).reshape(
516  nSignalPoints*covMatrixSide**2).tolist(),
517  'COVARIANCES_SQRT_WEIGHTS': np.pad(np.array(self.covariancesSqrtWeights[ampName]),
518  ((0, nPadPoints[ampName]), (0, 0), (0, 0)),
519  'constant', constant_values=0.0).reshape(
520  nSignalPoints*covMatrixSide**2).tolist(),
521  'A_MATRIX': np.array(self.aMatrix[ampName]).reshape(covMatrixSide**2).tolist(),
522  'B_MATRIX': np.array(self.bMatrix[ampName]).reshape(covMatrixSide**2).tolist(),
523  'COVARIANCES_MODEL_NO_B':
524  np.pad(np.array(self.covariancesModelNoB[ampName]),
525  ((0, nPadPoints[ampName]), (0, 0), (0, 0)),
526  'constant', constant_values=np.nan).reshape(
527  nSignalPoints*covMatrixSide**2).tolist(),
528  'A_MATRIX_NO_B': np.array(self.aMatrixNoB[ampName]).reshape(
529  covMatrixSide**2).tolist(),
530  'FINAL_VARS': np.pad(np.array(self.finalVars[ampName]), (0, nPadPoints[ampName]),
531  'constant', constant_values=np.nan).tolist(),
532  'FINAL_MODEL_VARS': np.pad(np.array(self.finalModelVars[ampName]),
533  (0, nPadPoints[ampName]),
534  'constant', constant_values=np.nan).tolist(),
535  'FINAL_MEANS': np.pad(np.array(self.finalMeans[ampName]),
536  (0, nPadPoints[ampName]),
537  'constant', constant_values=np.nan).tolist(),
538  'BAD_AMPS': np.array(self.badAmps).tolist() if len(self.badAmps) else np.nan,
539  'PHOTO_CHARGE': np.array(self.photoCharge[ampName]).tolist(),
540  } for ampName in self.ampNames])
541  inMeta = self.getMetadata().toDict()
542  outMeta = {k: v for k, v in inMeta.items() if v is not None}
543  outMeta.update({k: "" for k, v in inMeta.items() if v is None})
544  catalog.meta = outMeta
545  tableList.append(catalog)
546 
547  return(tableList)
548 
int max

◆ updateMetadata() [1/2]

def lsst.ip.isr.calibType.IsrCalib.updateMetadata (   self,
  camera = None,
  detector = None,
  filterName = None,
  setCalibId = False,
  setCalibInfo = False,
  setDate = False,
**  kwargs 
)
inherited
Update metadata keywords with new values.

Parameters
----------
camera : `lsst.afw.cameraGeom.Camera`, optional
    Reference camera to use to set _instrument field.
detector : `lsst.afw.cameraGeom.Detector`, optional
    Reference detector to use to set _detector* fields.
filterName : `str`, optional
    Filter name to assign to this calibration.
setCalibId : `bool`, optional
    Construct the _calibId field from other fields.
setCalibInfo : `bool`, optional
    Set calibration parameters from metadata.
setDate : `bool`, optional
    Ensure the metadata CALIBDATE fields are set to the current datetime.
kwargs : `dict` or `collections.abc.Mapping`, optional
    Set of key=value pairs to assign to the metadata.

Definition at line 179 of file calibType.py.

181  **kwargs):
182  """Update metadata keywords with new values.
183 
184  Parameters
185  ----------
186  camera : `lsst.afw.cameraGeom.Camera`, optional
187  Reference camera to use to set _instrument field.
188  detector : `lsst.afw.cameraGeom.Detector`, optional
189  Reference detector to use to set _detector* fields.
190  filterName : `str`, optional
191  Filter name to assign to this calibration.
192  setCalibId : `bool`, optional
193  Construct the _calibId field from other fields.
194  setCalibInfo : `bool`, optional
195  Set calibration parameters from metadata.
196  setDate : `bool`, optional
197  Ensure the metadata CALIBDATE fields are set to the current datetime.
198  kwargs : `dict` or `collections.abc.Mapping`, optional
199  Set of key=value pairs to assign to the metadata.
200  """
201  mdOriginal = self.getMetadata()
202  mdSupplemental = dict()
203 
204  for k, v in kwargs.items():
205  if isinstance(v, fits.card.Undefined):
206  kwargs[k] = None
207 
208  if setCalibInfo:
209  self.calibInfoFromDict(kwargs)
210 
211  if camera:
212  self._instrument = camera.getName()
213 
214  if detector:
215  self._detectorName = detector.getName()
216  self._detectorSerial = detector.getSerial()
217  self._detectorId = detector.getId()
218  if "_" in self._detectorName:
219  (self._raftName, self._slotName) = self._detectorName.split("_")
220 
221  if filterName:
222  # TOD0 DM-28093: I think this whole comment can go away, if we
223  # always use physicalLabel everywhere in ip_isr.
224  # If set via:
225  # exposure.getInfo().getFilter().getName()
226  # then this will hold the abstract filter.
227  self._filter = filterName
228 
229  if setDate:
230  date = datetime.datetime.now()
231  mdSupplemental["CALIBDATE"] = date.isoformat()
232  mdSupplemental["CALIB_CREATION_DATE"] = date.date().isoformat()
233  mdSupplemental["CALIB_CREATION_TIME"] = date.time().isoformat()
234 
235  if setCalibId:
236  values = []
237  values.append(f"instrument={self._instrument}") if self._instrument else None
238  values.append(f"raftName={self._raftName}") if self._raftName else None
239  values.append(f"detectorName={self._detectorName}") if self._detectorName else None
240  values.append(f"detector={self._detectorId}") if self._detectorId else None
241  values.append(f"filter={self._filter}") if self._filter else None
242 
243  calibDate = mdOriginal.get("CALIBDATE", mdSupplemental.get("CALIBDATE", None))
244  values.append(f"calibDate={calibDate}") if calibDate else None
245 
246  self._calibId = " ".join(values)
247 
248  self._metadata["INSTRUME"] = self._instrument if self._instrument else None
249  self._metadata["RAFTNAME"] = self._raftName if self._raftName else None
250  self._metadata["SLOTNAME"] = self._slotName if self._slotName else None
251  self._metadata["DETECTOR"] = self._detectorId
252  self._metadata["DET_NAME"] = self._detectorName if self._detectorName else None
253  self._metadata["DET_SER"] = self._detectorSerial if self._detectorSerial else None
254  self._metadata["FILTER"] = self._filter if self._filter else None
255  self._metadata["CALIB_ID"] = self._calibId if self._calibId else None
256  self._metadata["CALIBCLS"] = get_full_type_name(self)
257 
258  mdSupplemental.update(kwargs)
259  mdOriginal.update(mdSupplemental)
260 

◆ updateMetadata() [2/2]

def lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.updateMetadata (   self,
  setDate = False,
**  kwargs 
)
Update calibration metadata.
This calls the base class's method after ensuring the required
calibration keywords will be saved.
Parameters
----------
setDate : `bool`, optional
    Update the CALIBDATE fields in the metadata to the current
    time. Defaults to False.
kwargs :
    Other keyword parameters to set in the metadata.

Definition at line 255 of file ptcDataset.py.

255  def updateMetadata(self, setDate=False, **kwargs):
256  """Update calibration metadata.
257  This calls the base class's method after ensuring the required
258  calibration keywords will be saved.
259  Parameters
260  ----------
261  setDate : `bool`, optional
262  Update the CALIBDATE fields in the metadata to the current
263  time. Defaults to False.
264  kwargs :
265  Other keyword parameters to set in the metadata.
266  """
267  kwargs['PTC_FIT_TYPE'] = self.ptcFitType
268 
269  super().updateMetadata(setDate=setDate, **kwargs)
270 

◆ validate()

def lsst.ip.isr.calibType.IsrCalib.validate (   self,
  other = None 
)
inherited
Validate that this calibration is defined and can be used.

Parameters
----------
other : `object`, optional
    Thing to validate against.

Returns
-------
valid : `bool`
    Returns true if the calibration is valid and appropriate.

Definition at line 597 of file calibType.py.

597  def validate(self, other=None):
598  """Validate that this calibration is defined and can be used.
599 
600  Parameters
601  ----------
602  other : `object`, optional
603  Thing to validate against.
604 
605  Returns
606  -------
607  valid : `bool`
608  Returns true if the calibration is valid and appropriate.
609  """
610  return False
611 

◆ writeFits()

def lsst.ip.isr.calibType.IsrCalib.writeFits (   self,
  filename 
)
inherited
Write calibration data to a FITS file.

Parameters
----------
filename : `str`
    Filename to write data to.

Returns
-------
used : `str`
    The name of the file used to write the data.

Definition at line 471 of file calibType.py.

471  def writeFits(self, filename):
472  """Write calibration data to a FITS file.
473 
474  Parameters
475  ----------
476  filename : `str`
477  Filename to write data to.
478 
479  Returns
480  -------
481  used : `str`
482  The name of the file used to write the data.
483 
484  """
485  tableList = self.toTable()
486  with warnings.catch_warnings():
487  warnings.filterwarnings("ignore", category=Warning, module="astropy.io")
488  astropyList = [fits.table_to_hdu(table) for table in tableList]
489  astropyList.insert(0, fits.PrimaryHDU())
490 
491  writer = fits.HDUList(astropyList)
492  writer.writeto(filename, overwrite=True)
493  return filename
494 
def writeFits(filename, stamps, metadata, type_name, write_mask, write_variance, write_archive=False)
Definition: stamps.py:42

◆ writeText()

def lsst.ip.isr.calibType.IsrCalib.writeText (   self,
  filename,
  format = "auto" 
)
inherited
Write the calibration data to a text file.

Parameters
----------
filename : `str`
    Name of the file to write.
format : `str`
    Format to write the file as.  Supported values are:
        ``"auto"`` : Determine filetype from filename.
        ``"yaml"`` : Write as yaml.
        ``"ecsv"`` : Write as ecsv.
Returns
-------
used : `str`
    The name of the file used to write the data.  This may
    differ from the input if the format is explicitly chosen.

Raises
------
RuntimeError :
    Raised if filename does not end in a known extension, or
    if all information cannot be written.

Notes
-----
The file is written to YAML/ECSV format and will include any
associated metadata.

Definition at line 379 of file calibType.py.

379  def writeText(self, filename, format="auto"):
380  """Write the calibration data to a text file.
381 
382  Parameters
383  ----------
384  filename : `str`
385  Name of the file to write.
386  format : `str`
387  Format to write the file as. Supported values are:
388  ``"auto"`` : Determine filetype from filename.
389  ``"yaml"`` : Write as yaml.
390  ``"ecsv"`` : Write as ecsv.
391  Returns
392  -------
393  used : `str`
394  The name of the file used to write the data. This may
395  differ from the input if the format is explicitly chosen.
396 
397  Raises
398  ------
399  RuntimeError :
400  Raised if filename does not end in a known extension, or
401  if all information cannot be written.
402 
403  Notes
404  -----
405  The file is written to YAML/ECSV format and will include any
406  associated metadata.
407  """
408  if format == "yaml" or (format == "auto" and filename.lower().endswith((".yaml", ".YAML"))):
409  outDict = self.toDict()
410  path, ext = os.path.splitext(filename)
411  filename = path + ".yaml"
412  with open(filename, "w") as f:
413  yaml.dump(outDict, f)
414  elif format == "ecsv" or (format == "auto" and filename.lower().endswith((".ecsv", ".ECSV"))):
415  tableList = self.toTable()
416  if len(tableList) > 1:
417  # ECSV doesn't support multiple tables per file, so we
418  # can only write the first table.
419  raise RuntimeError(f"Unable to persist {len(tableList)}tables in ECSV format.")
420 
421  table = tableList[0]
422  path, ext = os.path.splitext(filename)
423  filename = path + ".ecsv"
424  table.write(filename, format="ascii.ecsv")
425  else:
426  raise RuntimeError(f"Attempt to write to a file {filename} "
427  "that does not end in '.yaml' or '.ecsv'")
428 
429  return filename
430 

Member Data Documentation

◆ aMatrix

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.aMatrix

Definition at line 182 of file ptcDataset.py.

◆ aMatrixNoB

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.aMatrixNoB

Definition at line 185 of file ptcDataset.py.

◆ ampNames

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.ampNames

Definition at line 158 of file ptcDataset.py.

◆ badAmps

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.badAmps

Definition at line 161 of file ptcDataset.py.

◆ bMatrix

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.bMatrix

Definition at line 183 of file ptcDataset.py.

◆ covariances

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.covariances

Definition at line 179 of file ptcDataset.py.

◆ covariancesModel

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.covariancesModel

Definition at line 180 of file ptcDataset.py.

◆ covariancesModelNoB

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.covariancesModelNoB

Definition at line 184 of file ptcDataset.py.

◆ covariancesSqrtWeights

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.covariancesSqrtWeights

Definition at line 181 of file ptcDataset.py.

◆ covMatrixSide

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.covMatrixSide

Definition at line 159 of file ptcDataset.py.

◆ expIdMask

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.expIdMask

Definition at line 164 of file ptcDataset.py.

◆ finalMeans

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.finalMeans

Definition at line 189 of file ptcDataset.py.

◆ finalModelVars

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.finalModelVars

Definition at line 188 of file ptcDataset.py.

◆ finalVars

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.finalVars

Definition at line 187 of file ptcDataset.py.

◆ gain

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.gain

Definition at line 170 of file ptcDataset.py.

◆ gainErr

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.gainErr

Definition at line 171 of file ptcDataset.py.

◆ inputExpIdPairs

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.inputExpIdPairs

Definition at line 163 of file ptcDataset.py.

◆ log

lsst.ip.isr.calibType.IsrCalib.log
inherited

Definition at line 86 of file calibType.py.

◆ noise

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.noise

Definition at line 172 of file ptcDataset.py.

◆ noiseErr

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.noiseErr

Definition at line 173 of file ptcDataset.py.

◆ photoCharge

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.photoCharge

Definition at line 168 of file ptcDataset.py.

◆ ptcFitChiSq

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.ptcFitChiSq

Definition at line 177 of file ptcDataset.py.

◆ ptcFitPars

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.ptcFitPars

Definition at line 175 of file ptcDataset.py.

◆ ptcFitParsError

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.ptcFitParsError

Definition at line 176 of file ptcDataset.py.

◆ ptcFitType

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.ptcFitType

Definition at line 157 of file ptcDataset.py.

◆ rawExpTimes

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.rawExpTimes

Definition at line 165 of file ptcDataset.py.

◆ rawMeans

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.rawMeans

Definition at line 166 of file ptcDataset.py.

◆ rawVars

lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset.rawVars

Definition at line 167 of file ptcDataset.py.

◆ requiredAttributes

lsst.ip.isr.calibType.IsrCalib.requiredAttributes
inherited

Definition at line 81 of file calibType.py.


The documentation for this class was generated from the following file: