Public Member Functions |
Static Public Member Functions |
Public Attributes |
Static Protected Member Functions |
Protected Attributes |
Static Protected Attributes |
List of all members
lsst.ip.isr.crosstalk.CrosstalkCalib Class Reference
Inheritance diagram for lsst.ip.isr.crosstalk.CrosstalkCalib:
Public Member Functions | |
| __init__ (self, detector=None, nAmp=0, **kwargs) | |
| updateMetadata (self, setDate=False, **kwargs) | |
| fromDetector (self, detector, coeffVector=None, coeffSqrVector=None) | |
| fromDict (cls, dictionary) | |
| toDict (self) | |
| fromTable (cls, tableList) | |
| toTable (self) | |
| subtractCrosstalk (self, thisExposure, sourceExposure=None, crosstalkCoeffs=None, crosstalkCoeffsSqr=None, crosstalkCoeffsValid=None, badPixels=["BAD"], minPixelToMask=45000, doSubtrahendMasking=False, crosstalkStr="CROSSTALK", isTrimmed=None, backgroundMethod="None", doSqrCrosstalk=False, fullAmplifier=False, parallelOverscan=None, detectorConfig=None, badAmpDict=None, ignoreVariance=False) | |
| requiredAttributes (self) | |
| requiredAttributes (self, value) | |
| __str__ (self) | |
| __eq__ (self, other) | |
| metadata (self) | |
| getMetadata (self) | |
| setMetadata (self, metadata) | |
| updateMetadataFromExposures (self, exposures) | |
| calibInfoFromDict (self, dictionary) | |
| determineCalibClass (cls, metadata, message) | |
| readText (cls, filename, **kwargs) | |
| writeText (self, filename, format="auto") | |
| readFits (cls, filename, **kwargs) | |
| writeFits (self, filename) | |
| validate (self, other=None) | |
| apply (self, target) | |
Static Public Member Functions | |
| extractAmp (image, ampToFlip, ampTarget, isTrimmed=False, fullAmplifier=False, parallelOverscan=None) | |
| calculateBackground (mi, badPixels=["BAD"]) | |
Public Attributes | |
| bool | hasCrosstalk = False |
| int | nAmp = nAmp if nAmp else 0 |
| tuple | crosstalkShape = (self.nAmp, self.nAmp) |
| int | coeffs = np.zeros(self.crosstalkShape) if self.nAmp else None |
| int | coeffErr = np.zeros(self.crosstalkShape) if self.nAmp else None |
| int | coeffNum |
| int | coeffValid |
| int | coeffsSqr = np.zeros(self.crosstalkShape) if self.nAmp else None |
| int | coeffErrSqr = np.zeros(self.crosstalkShape) if self.nAmp else None |
| int | ampGainRatios = np.zeros(self.crosstalkShape) if self.nAmp else None |
| int | fitGains = np.zeros(self.nAmp) if self.nAmp else None |
| str | crosstalkRatiosUnits = 'adu' if self.nAmp else None |
| dict | interChip = {} |
| requiredAttributes = set(["_OBSTYPE", "_SCHEMA", "_VERSION"]) | |
| log = log if log else logging.getLogger(__name__) | |
Static Protected Member Functions | |
| _getAppropriateBBox (amp, isTrimmed, fullAmplifier) | |
Protected Attributes | |
| _instrument = None | |
| _raftName = None | |
| _slotName = None | |
| _detectorName = None | |
| _detectorSerial = None | |
| _detectorId = None | |
| _filter = None | |
| str | _calibId = None |
| _seqfile = None | |
| _seqname = None | |
| _seqcksum = None | |
| _metadata = PropertyList() | |
| _requiredAttributes | |
Static Protected Attributes | |
| str | _OBSTYPE = "generic" |
| str | _SCHEMA = "NO SCHEMA" |
| int | _VERSION = 0 |
Detailed Description
Calibration of amp-to-amp crosstalk coefficients.
Parameters
----------
detector : `lsst.afw.cameraGeom.Detector`, optional
Detector to use to pull coefficients from.
nAmp : `int`, optional
Number of amplifiers to initialize.
log : `logging.Logger`, optional
Log to write messages to.
**kwargs :
Parameters to pass to parent constructor.
Notes
-----
The crosstalk attributes stored are:
hasCrosstalk : `bool`
Whether there is crosstalk defined for this detector.
nAmp : `int`
Number of amplifiers in this detector.
crosstalkShape : `tuple` [`int`, `int`]
A tuple containing the shape of the ``coeffs`` matrix. This
should be equivalent to (``nAmp``, ``nAmp``).
coeffs : `numpy.ndarray`
A matrix containing the crosstalk coefficients. coeff[i][j]
contains the coefficients to calculate the contribution
amplifier_j has on amplifier_i (each row[i] contains the
corrections for detector_i).
coeffErr : `numpy.ndarray`, optional
A matrix (as defined by ``coeffs``) containing the standard
distribution of the crosstalk measurements.
coeffNum : `numpy.ndarray`, optional
A matrix containing the number of pixel pairs used to measure
the ``coeffs`` and ``coeffErr``.
coeffValid : `numpy.ndarray`, optional
A matrix of Boolean values indicating if the coefficient is
valid, defined as abs(coeff) > coeffErr / sqrt(coeffNum).
coeffsSqr : `numpy.ndarray`, optional
A matrix containing potential quadratic crosstalk coefficients
(see e.g., Snyder+21, 2001.03223). coeffsSqr[i][j]
contains the coefficients to calculate the contribution
amplifier_j has on amplifier_i (each row[i] contains the
corrections for detector_i).
coeffErrSqr : `numpy.ndarray`, optional
A matrix (as defined by ``coeffsSqr``) containing the standard
distribution of the quadratic term of the crosstalk measurements.
interChip : `dict` [`numpy.ndarray`]
A dictionary keyed by detectorName containing ``coeffs``
matrices used to correct for inter-chip crosstalk with a
source on the detector indicated.
Version 1.1 adds quadratic coefficients, a matrix with the ratios
of amplifiers gains per detector, and a field to indicate the units
of the numerator and denominator of the source and target signals, with
"adu" meaning "ADU / ADU" and "electron" meaning "e- / e-".
Version 1.2 adds the original gains used in the crosstalk fit.
Definition at line 43 of file crosstalk.py.
Constructor & Destructor Documentation
◆ __init__()
| lsst.ip.isr.crosstalk.CrosstalkCalib.__init__ | ( | self, | |
| detector = None, | |||
| nAmp = 0, | |||
| ** | kwargs ) |
Definition at line 107 of file crosstalk.py.
107 def __init__(self, detector=None, nAmp=0, **kwargs):
108 self.hasCrosstalk = False
109 self.nAmp = nAmp if nAmp else 0
110 self.crosstalkShape = (self.nAmp, self.nAmp)
111
112 self.coeffs = np.zeros(self.crosstalkShape) if self.nAmp else None
113 self.coeffErr = np.zeros(self.crosstalkShape) if self.nAmp else None
114 self.coeffNum = np.zeros(self.crosstalkShape,
115 dtype=int) if self.nAmp else None
116 self.coeffValid = np.ones(self.crosstalkShape,
117 dtype=bool) if self.nAmp else None
118 # Quadratic terms, if any.
119 self.coeffsSqr = np.zeros(self.crosstalkShape) if self.nAmp else None
120 self.coeffErrSqr = np.zeros(self.crosstalkShape) if self.nAmp else None
121
122 # Gain ratios
123 self.ampGainRatios = np.zeros(self.crosstalkShape) if self.nAmp else None
124
125 # Gains used for fit.
126 self.fitGains = np.zeros(self.nAmp) if self.nAmp else None
127
128 # Units
129 self.crosstalkRatiosUnits = 'adu' if self.nAmp else None
130
131 self.interChip = {}
132
133 super().__init__(**kwargs)
134 self.requiredAttributes.update(['hasCrosstalk', 'nAmp', 'coeffs',
135 'coeffErr', 'coeffNum', 'coeffValid',
136 'coeffsSqr', 'coeffErrSqr',
137 'ampGainRatios', 'crosstalkRatiosUnits',
138 'fitGains', 'interChip'])
139 if detector:
140 self.fromDetector(detector)
141
Member Function Documentation
◆ __eq__()
|
inherited |
Calibration equivalence. Running ``calib.log.setLevel(0)`` enables debug statements to identify problematic fields.
Definition at line 103 of file calibType.py.
103 def __eq__(self, other):
104 """Calibration equivalence.
105
106 Running ``calib.log.setLevel(0)`` enables debug statements to
107 identify problematic fields.
108 """
109 if not isinstance(other, self.__class__):
110 self.log.debug("Incorrect class type: %s %s", self.__class__, other.__class__)
111 return False
112
113 for attr in self._requiredAttributes:
114 attrSelf = getattr(self, attr)
115 attrOther = getattr(other, attr)
116
117 if isinstance(attrSelf, dict):
118 # Dictionary of arrays.
119 if attrSelf.keys() != attrOther.keys():
120 self.log.debug("Dict Key Failure: %s %s %s", attr, attrSelf.keys(), attrOther.keys())
121 return False
122 for key in attrSelf:
123 try:
124 if not np.allclose(attrSelf[key], attrOther[key], equal_nan=True):
125 self.log.debug("Array Failure: %s %s %s", key, attrSelf[key], attrOther[key])
126 return False
127 except TypeError:
128 # If it is not something numpy can handle
129 # (it's not a number or array of numbers),
130 # then it needs to have its own equivalence
131 # operator.
132 if np.all(attrSelf[key] != attrOther[key]):
133 return False
134 elif isinstance(attrSelf, np.ndarray):
135 # Bare array.
136 if isinstance(attrSelf[0], (str, np.str_, np.bytes_)):
137 if not np.all(attrSelf == attrOther):
138 self.log.debug("Array Failure: %s %s %s", attr, attrSelf, attrOther)
139 return False
140 else:
141 if not np.allclose(attrSelf, attrOther, equal_nan=True):
142 self.log.debug("Array Failure: %s %s %s", attr, attrSelf, attrOther)
143 return False
144 elif type(attrSelf) is not type(attrOther):
145 if set([attrSelf, attrOther]) == set([None, ""]):
146 # Fits converts None to "", but None is not "".
147 continue
148 self.log.debug("Type Failure: %s %s %s %s %s", attr, type(attrSelf), type(attrOther),
149 attrSelf, attrOther)
150 return False
151 else:
152 if attrSelf != attrOther:
153 self.log.debug("Value Failure: %s %s %s", attr, attrSelf, attrOther)
154 return False
155
156 return True
157
◆ __str__()
|
inherited |
Definition at line 100 of file calibType.py.
100 def __str__(self):
101 return f"{self.__class__.__name__}(obstype={self._OBSTYPE}, detector={self._detectorName}, )"
102
◆ _getAppropriateBBox()
|
staticprotected |
Get the appropriate bounding box from an amplifier.
Parameters
----------
amp : `lsst.afw.cameraGeom.Amplifier`
Amplifier to get bounding box.
isTrimmed : `bool`
Is this a trimmed image?
fullAmplifier : `bool`
Extract full amplifier for an untrimmed image?
Definition at line 544 of file crosstalk.py.
544 def _getAppropriateBBox(amp, isTrimmed, fullAmplifier):
545 """Get the appropriate bounding box from an amplifier.
546
547 Parameters
548 ----------
549 amp : `lsst.afw.cameraGeom.Amplifier`
550 Amplifier to get bounding box.
551 isTrimmed : `bool`
552 Is this a trimmed image?
553 fullAmplifier : `bool`
554 Extract full amplifier for an untrimmed image?
555 """
556 if isTrimmed:
557 return amp.getBBox()
558 elif fullAmplifier and not isTrimmed:
559 return amp.getRawBBox()
560 else:
561 return amp.getRawDataBBox()
562
◆ apply()
|
inherited |
Method to apply the calibration to the target object.
Parameters
----------
target : `object`
Thing to validate against.
Returns
-------
valid : `bool`
Returns true if the calibration was applied correctly.
Raises
------
NotImplementedError
Raised if not implemented.
Definition at line 685 of file calibType.py.
685 def apply(self, target):
686 """Method to apply the calibration to the target object.
687
688 Parameters
689 ----------
690 target : `object`
691 Thing to validate against.
692
693 Returns
694 -------
695 valid : `bool`
696 Returns true if the calibration was applied correctly.
697
698 Raises
699 ------
700 NotImplementedError
701 Raised if not implemented.
702 """
703 raise NotImplementedError("Must be implemented by subclass.")
704
705
◆ calculateBackground()
|
static |
Estimate median background in image.
Getting a great background model isn't important for crosstalk
correction, since the crosstalk is at a low level. The median should
be sufficient.
Parameters
----------
mi : `lsst.afw.image.MaskedImage`
MaskedImage for which to measure background.
badPixels : `list` of `str`
Mask planes to ignore.
Returns
-------
bg : `float`
Median background level.
Definition at line 564 of file crosstalk.py.
564 def calculateBackground(mi, badPixels=["BAD"]):
565 """Estimate median background in image.
566
567 Getting a great background model isn't important for crosstalk
568 correction, since the crosstalk is at a low level. The median should
569 be sufficient.
570
571 Parameters
572 ----------
573 mi : `lsst.afw.image.MaskedImage`
574 MaskedImage for which to measure background.
575 badPixels : `list` of `str`
576 Mask planes to ignore.
577 Returns
578 -------
579 bg : `float`
580 Median background level.
581 """
582 mask = mi.getMask()
583 stats = lsst.afw.math.StatisticsControl()
584 stats.setAndMask(mask.getPlaneBitMask(badPixels))
586
Statistics makeStatistics(lsst::afw::image::Image< Pixel > const &img, lsst::afw::image::Mask< image::MaskPixel > const &msk, int const flags, StatisticsControl const &sctrl=StatisticsControl())
Handle a watered-down front-end to the constructor (no variance)
Definition Statistics.h:361
◆ calibInfoFromDict()
|
inherited |
Handle common keywords.
This isn't an ideal solution, but until all calibrations
expect to find everything in the metadata, they still need to
search through dictionaries.
Parameters
----------
dictionary : `dict` or `lsst.daf.base.PropertyList`
Source for the common keywords.
Raises
------
RuntimeError
Raised if the dictionary does not match the expected OBSTYPE.
Definition at line 322 of file calibType.py.
322 def calibInfoFromDict(self, dictionary):
323 """Handle common keywords.
324
325 This isn't an ideal solution, but until all calibrations
326 expect to find everything in the metadata, they still need to
327 search through dictionaries.
328
329 Parameters
330 ----------
331 dictionary : `dict` or `lsst.daf.base.PropertyList`
332 Source for the common keywords.
333
334 Raises
335 ------
336 RuntimeError
337 Raised if the dictionary does not match the expected OBSTYPE.
338 """
339
340 def search(haystack, needles):
341 """Search dictionary 'haystack' for an entry in 'needles'
342 """
343 test = [haystack.get(x) for x in needles]
344 test = set([x for x in test if x is not None])
345 if len(test) == 0:
346 if "metadata" in haystack:
347 return search(haystack["metadata"], needles)
348 else:
349 return None
350 elif len(test) == 1:
351 value = list(test)[0]
352 if value == "":
353 return None
354 else:
355 return value
356 else:
357 raise ValueError(f"Too many values found: {len(test)} {test} {needles}")
358
359 if "metadata" in dictionary:
360 metadata = dictionary["metadata"]
361
362 if self._OBSTYPE != metadata["OBSTYPE"]:
363 raise RuntimeError(f"Incorrect calibration supplied. Expected {self._OBSTYPE}, "
364 f"found {metadata['OBSTYPE']}")
365
366 if (value := search(dictionary, ["INSTRUME", "instrument"])) is not None:
367 self._instrument = value
368 if (value := search(dictionary, ["RAFTNAME"])) is not None:
369 self._slotName = value
370 if (value := search(dictionary, ["DETECTOR", "detectorId"])) is not None:
371 self._detectorId = value
372 if (value := search(dictionary, ["DET_NAME", "DETECTOR_NAME", "detectorName"])) is not None:
373 self._detectorName = value
374 if (value := search(dictionary, ["DET_SER", "DETECTOR_SERIAL", "detectorSerial"])) is not None:
375 self._detectorSerial = value
376 if (value := search(dictionary, ["FILTER", "filterName"])) is not None:
377 self._filter = value
378 if (value := search(dictionary, ["CALIB_ID"])) is not None:
379 self._calibId = value
380 if (value := search(dictionary, ["SEQFILE"])) is not None:
381 self._seqfile = value
382 if (value := search(dictionary, ["SEQNAME"])) is not None:
383 self._seqname = value
384 if (value := search(dictionary, ["SEQCKSUM"])) is not None:
385 self._seqcksum = value
386
◆ determineCalibClass()
|
inherited |
Attempt to find calibration class in metadata.
Parameters
----------
metadata : `dict` or `lsst.daf.base.PropertyList`
Metadata possibly containing a calibration class entry.
message : `str`
Message to include in any errors.
Returns
-------
calibClass : `object`
The class to use to read the file contents. Should be an
`lsst.ip.isr.IsrCalib` subclass.
Raises
------
ValueError
Raised if the resulting calibClass is the base
`lsst.ip.isr.IsrClass` (which does not implement the
content methods).
Definition at line 388 of file calibType.py.
388 def determineCalibClass(cls, metadata, message):
389 """Attempt to find calibration class in metadata.
390
391 Parameters
392 ----------
393 metadata : `dict` or `lsst.daf.base.PropertyList`
394 Metadata possibly containing a calibration class entry.
395 message : `str`
396 Message to include in any errors.
397
398 Returns
399 -------
400 calibClass : `object`
401 The class to use to read the file contents. Should be an
402 `lsst.ip.isr.IsrCalib` subclass.
403
404 Raises
405 ------
406 ValueError
407 Raised if the resulting calibClass is the base
408 `lsst.ip.isr.IsrClass` (which does not implement the
409 content methods).
410 """
411 calibClassName = metadata.get("CALIBCLS")
412 calibClass = doImport(calibClassName) if calibClassName is not None else cls
413 if calibClass is IsrCalib:
414 raise ValueError(f"Cannot use base class to read calibration data: {message}")
415 return calibClass
416
◆ extractAmp()
|
static |
Extract the image data from an amp, flipped to match ampTarget.
Parameters
----------
image : `lsst.afw.image.Image` or `lsst.afw.image.MaskedImage`
Image containing the amplifier of interest.
amp : `lsst.afw.cameraGeom.Amplifier`
Amplifier on image to extract.
ampTarget : `lsst.afw.cameraGeom.Amplifier`
Target amplifier that the extracted image will be flipped
to match.
isTrimmed : `bool`, optional
The image is already trimmed.
fullAmplifier : `bool`, optional
Use full amplifier and not just imaging region.
parallelOverscan : `bool`, optional
This has been deprecated and is unused, and will be removed
after v29.
Returns
-------
output : `lsst.afw.image.Image`
Amplifier from image, flipped to desired configuration.
This will always return a copy of the original data.
Definition at line 489 of file crosstalk.py.
489 def extractAmp(image, ampToFlip, ampTarget, isTrimmed=False, fullAmplifier=False, parallelOverscan=None):
490 """Extract the image data from an amp, flipped to match ampTarget.
491
492 Parameters
493 ----------
494 image : `lsst.afw.image.Image` or `lsst.afw.image.MaskedImage`
495 Image containing the amplifier of interest.
496 amp : `lsst.afw.cameraGeom.Amplifier`
497 Amplifier on image to extract.
498 ampTarget : `lsst.afw.cameraGeom.Amplifier`
499 Target amplifier that the extracted image will be flipped
500 to match.
501 isTrimmed : `bool`, optional
502 The image is already trimmed.
503 fullAmplifier : `bool`, optional
504 Use full amplifier and not just imaging region.
505 parallelOverscan : `bool`, optional
506 This has been deprecated and is unused, and will be removed
507 after v29.
508
509 Returns
510 -------
511 output : `lsst.afw.image.Image`
512 Amplifier from image, flipped to desired configuration.
513 This will always return a copy of the original data.
514 """
515 if parallelOverscan is not None:
516 warnings.warn(
517 "The parallelOverscan option has been deprecated and will be removed after v29.",
518 FutureWarning,
519 )
520
521 X_FLIP = {lsst.afw.cameraGeom.ReadoutCorner.LL: False,
522 lsst.afw.cameraGeom.ReadoutCorner.LR: True,
523 lsst.afw.cameraGeom.ReadoutCorner.UL: False,
524 lsst.afw.cameraGeom.ReadoutCorner.UR: True}
525 Y_FLIP = {lsst.afw.cameraGeom.ReadoutCorner.LL: False,
526 lsst.afw.cameraGeom.ReadoutCorner.LR: False,
527 lsst.afw.cameraGeom.ReadoutCorner.UL: True,
528 lsst.afw.cameraGeom.ReadoutCorner.UR: True}
529
530 bbox = CrosstalkCalib._getAppropriateBBox(ampToFlip, isTrimmed, fullAmplifier)
531 output = image[bbox]
532
533 sourceAmpCorner = ampToFlip.getReadoutCorner()
534 targetAmpCorner = ampTarget.getReadoutCorner()
535
536 # Flipping is necessary only if the desired configuration doesn't match
537 # what we currently have.
538 xFlip = X_FLIP[targetAmpCorner] ^ X_FLIP[sourceAmpCorner]
539 yFlip = Y_FLIP[targetAmpCorner] ^ Y_FLIP[sourceAmpCorner]
540 # This always makes a copy of the image.
542
std::shared_ptr< ImageT > flipImage(ImageT const &inImage, bool flipLR, bool flipTB)
Flip an image left–right and/or top–bottom.
Definition rotateImage.cc:92
◆ fromDetector()
| lsst.ip.isr.crosstalk.CrosstalkCalib.fromDetector | ( | self, | |
| detector, | |||
| coeffVector = None, | |||
| coeffSqrVector = None ) |
Set calibration parameters from the detector.
Parameters
----------
detector : `lsst.afw.cameraGeom.Detector`
Detector to use to set parameters from.
coeffVector : `numpy.array`, optional
Use the detector geometry (bounding boxes and flip
information), but use ``coeffVector`` instead of the
output of ``detector.getCrosstalk()``.
coeffSqrVector : `numpy.array`, optional
Quadratic crosstalk coefficients.
Returns
-------
calib : `lsst.ip.isr.CrosstalkCalib`
The calibration constructed from the detector.
Reimplemented from lsst.ip.isr.calibType.IsrCalib.
Definition at line 167 of file crosstalk.py.
167 def fromDetector(self, detector, coeffVector=None, coeffSqrVector=None):
168 """Set calibration parameters from the detector.
169
170 Parameters
171 ----------
172 detector : `lsst.afw.cameraGeom.Detector`
173 Detector to use to set parameters from.
174 coeffVector : `numpy.array`, optional
175 Use the detector geometry (bounding boxes and flip
176 information), but use ``coeffVector`` instead of the
177 output of ``detector.getCrosstalk()``.
178 coeffSqrVector : `numpy.array`, optional
179 Quadratic crosstalk coefficients.
180
181 Returns
182 -------
183 calib : `lsst.ip.isr.CrosstalkCalib`
184 The calibration constructed from the detector.
185
186 """
187 self._detectorId = detector.getId()
188 self._detectorName = detector.getName()
189 self._detectorSerial = detector.getSerial()
190
191 self.nAmp = len(detector)
192 self.crosstalkShape = (self.nAmp, self.nAmp)
193
194 if coeffVector is not None:
195 crosstalkCoeffs = coeffVector
196 else:
197 crosstalkCoeffs = detector.getCrosstalk()
198 if coeffSqrVector is not None:
199 self.coeffsSqr = coeffSqrVector
200 else:
201 self.coeffsSqr = np.zeros(self.crosstalkShape)
202 if len(crosstalkCoeffs) == 1 and crosstalkCoeffs[0] == 0.0:
203 return self
204 self.coeffs = np.array(crosstalkCoeffs).reshape(self.crosstalkShape)
205
206 if self.coeffs.shape != self.crosstalkShape:
207 raise RuntimeError("Crosstalk coefficients do not match detector shape. "
208 f"{self.crosstalkShape} {self.nAmp}")
209 # Set default as in __init__
210 self.coeffErr = np.zeros(self.crosstalkShape)
211 self.coeffNum = np.zeros(self.crosstalkShape, dtype=int)
212 self.coeffValid = np.ones(self.crosstalkShape, dtype=bool)
213 self.coeffErrSqr = np.zeros(self.crosstalkShape)
214 self.ampGainRatios = np.zeros(self.crosstalkShape)
215 self.fitGains = np.zeros(self.nAmp)
216 self.crosstalkRatiosUnits = 'adu'
217
218 self.interChip = {}
219
220 self.hasCrosstalk = True
221 self.updateMetadata()
222
223 return self
224
◆ fromDict()
| lsst.ip.isr.crosstalk.CrosstalkCalib.fromDict | ( | cls, | |
| dictionary ) |
Construct a calibration from a dictionary of properties.
Must be implemented by the specific calibration subclasses.
Parameters
----------
dictionary : `dict`
Dictionary of properties.
Returns
-------
calib : `lsst.ip.isr.CalibType`
Constructed calibration.
Raises
------
RuntimeError
Raised if the supplied dictionary is for a different
calibration.
Reimplemented from lsst.ip.isr.calibType.IsrCalib.
Definition at line 226 of file crosstalk.py.
226 def fromDict(cls, dictionary):
227 """Construct a calibration from a dictionary of properties.
228
229 Must be implemented by the specific calibration subclasses.
230
231 Parameters
232 ----------
233 dictionary : `dict`
234 Dictionary of properties.
235
236 Returns
237 -------
238 calib : `lsst.ip.isr.CalibType`
239 Constructed calibration.
240
241 Raises
242 ------
243 RuntimeError
244 Raised if the supplied dictionary is for a different
245 calibration.
246 """
247 calib = cls()
248
249 if calib._OBSTYPE != dictionary['metadata']['OBSTYPE']:
250 raise RuntimeError(f"Incorrect crosstalk supplied. Expected {calib._OBSTYPE}, "
251 f"found {dictionary['metadata']['OBSTYPE']}")
252
253 calib.setMetadata(dictionary['metadata'])
254
255 if 'detectorName' in dictionary:
256 calib._detectorName = dictionary.get('detectorName')
257 elif 'DETECTOR_NAME' in dictionary:
258 calib._detectorName = dictionary.get('DETECTOR_NAME')
259 elif 'DET_NAME' in dictionary['metadata']:
260 calib._detectorName = dictionary['metadata']['DET_NAME']
261 else:
262 calib._detectorName = None
263
264 if 'detectorSerial' in dictionary:
265 calib._detectorSerial = dictionary.get('detectorSerial')
266 elif 'DETECTOR_SERIAL' in dictionary:
267 calib._detectorSerial = dictionary.get('DETECTOR_SERIAL')
268 elif 'DET_SER' in dictionary['metadata']:
269 calib._detectorSerial = dictionary['metadata']['DET_SER']
270 else:
271 calib._detectorSerial = None
272
273 if 'detectorId' in dictionary:
274 calib._detectorId = dictionary.get('detectorId')
275 elif 'DETECTOR' in dictionary:
276 calib._detectorId = dictionary.get('DETECTOR')
277 elif 'DETECTOR' in dictionary['metadata']:
278 calib._detectorId = dictionary['metadata']['DETECTOR']
279 elif calib._detectorSerial:
280 calib._detectorId = calib._detectorSerial
281 else:
282 calib._detectorId = None
283
284 if 'instrument' in dictionary:
285 calib._instrument = dictionary.get('instrument')
286 elif 'INSTRUME' in dictionary['metadata']:
287 calib._instrument = dictionary['metadata']['INSTRUME']
288 else:
289 calib._instrument = None
290
291 calib.hasCrosstalk = dictionary.get('hasCrosstalk',
292 dictionary['metadata'].get('HAS_CROSSTALK', False))
293 if calib.hasCrosstalk:
294 calib.nAmp = dictionary.get('nAmp', dictionary['metadata'].get('NAMP', 0))
295 calib.crosstalkShape = (calib.nAmp, calib.nAmp)
296 calib.coeffs = np.array(dictionary['coeffs']).reshape(calib.crosstalkShape)
297 calib.crosstalkRatiosUnits = dictionary.get(
298 'crosstalkRatiosUnits',
299 dictionary['metadata'].get('CROSSTALK_RATIOS_UNITS', 'adu'))
300 if 'coeffErr' in dictionary:
301 calib.coeffErr = np.array(dictionary['coeffErr']).reshape(calib.crosstalkShape)
302 else:
303 calib.coeffErr = np.zeros_like(calib.coeffs)
304 if 'coeffNum' in dictionary:
305 calib.coeffNum = np.array(dictionary['coeffNum']).reshape(calib.crosstalkShape)
306 else:
307 calib.coeffNum = np.zeros_like(calib.coeffs, dtype=int)
308 if 'coeffValid' in dictionary:
309 calib.coeffValid = np.array(dictionary['coeffValid']).reshape(calib.crosstalkShape)
310 else:
311 calib.coeffValid = np.ones_like(calib.coeffs, dtype=bool)
312 if 'coeffsSqr' in dictionary:
313 calib.coeffsSqr = np.array(dictionary['coeffsSqr']).reshape(calib.crosstalkShape)
314 else:
315 calib.coeffsSqr = np.zeros_like(calib.coeffs)
316 if 'coeffErrSqr' in dictionary:
317 calib.coeffErrSqr = np.array(dictionary['coeffErrSqr']).reshape(calib.crosstalkShape)
318 else:
319 calib.coeffErrSqr = np.zeros_like(calib.coeffs)
320 if 'ampGainRatios' in dictionary:
321 calib.ampGainRatios = np.array(dictionary['ampGainRatios']).reshape(calib.crosstalkShape)
322 else:
323 calib.ampGainRatios = np.zeros_like(calib.coeffs)
324 if 'fitGains' in dictionary:
325 # Compatibility for matrices that were stored with fitGains
326 # of length 1.
327 fitGains = np.array(dictionary['fitGains'])
328 if len(fitGains) == 1:
329 # Expand to the correct number of amps, all zero (unknown).
330 calib.fitGains = np.zeros(calib.nAmp)
331 else:
332 calib.fitGains = np.array(dictionary['fitGains']).reshape(calib.nAmp)
333 else:
334 calib.fitGains = np.zeros(calib.nAmp)
335
336 calib.interChip = dictionary.get('interChip', None)
337 if calib.interChip:
338 for detector in calib.interChip:
339 coeffVector = calib.interChip[detector]
340 calib.interChip[detector] = np.array(coeffVector).reshape(calib.crosstalkShape)
341
342 calib.updateMetadata()
343 return calib
344
◆ fromTable()
| lsst.ip.isr.crosstalk.CrosstalkCalib.fromTable | ( | cls, | |
| tableList ) |
Construct calibration from a list of tables.
This method uses the `fromDict` method to create the
calibration, after constructing an appropriate dictionary from
the input tables.
Parameters
----------
tableList : `list` [`lsst.afw.table.Table`]
List of tables to use to construct the crosstalk
calibration.
Returns
-------
calib : `lsst.ip.isr.CrosstalkCalib`
The calibration defined in the tables.
Reimplemented from lsst.ip.isr.calibType.IsrCalib.
Definition at line 393 of file crosstalk.py.
393 def fromTable(cls, tableList):
394 """Construct calibration from a list of tables.
395
396 This method uses the `fromDict` method to create the
397 calibration, after constructing an appropriate dictionary from
398 the input tables.
399
400 Parameters
401 ----------
402 tableList : `list` [`lsst.afw.table.Table`]
403 List of tables to use to construct the crosstalk
404 calibration.
405
406 Returns
407 -------
408 calib : `lsst.ip.isr.CrosstalkCalib`
409 The calibration defined in the tables.
410
411 """
412 coeffTable = tableList[0]
413
414 metadata = coeffTable.meta
415 inDict = dict()
416 inDict['metadata'] = metadata
417 inDict['hasCrosstalk'] = metadata['HAS_CROSSTALK']
418 inDict['nAmp'] = metadata['NAMP']
419 calibVersion = metadata['CROSSTALK_VERSION']
420 if calibVersion < 1.1:
421 inDict['crosstalkRatiosUnits'] = 'adu'
422 else:
423 inDict['crosstalkRatiosUnits'] = metadata['CROSSTALK_RATIOS_UNITS']
424 inDict['coeffs'] = coeffTable['CT_COEFFS']
425 if 'CT_ERRORS' in coeffTable.columns:
426 inDict['coeffErr'] = coeffTable['CT_ERRORS']
427 if 'CT_COUNTS' in coeffTable.columns:
428 inDict['coeffNum'] = coeffTable['CT_COUNTS']
429 if 'CT_VALID' in coeffTable.columns:
430 inDict['coeffValid'] = coeffTable['CT_VALID']
431 if 'CT_COEFFS_SQR' in coeffTable.columns:
432 inDict['coeffsSqr'] = coeffTable['CT_COEFFS_SQR']
433 if 'CT_ERRORS_SQR' in coeffTable.columns:
434 inDict['coeffErrSqr'] = coeffTable['CT_ERRORS_SQR']
435 if 'CT_AMP_GAIN_RATIOS' in coeffTable.columns:
436 inDict['ampGainRatios'] = coeffTable['CT_AMP_GAIN_RATIOS']
437 if 'CT_FIT_GAINS' in coeffTable.columns:
438 inDict['fitGains'] = coeffTable['CT_FIT_GAINS']
439
440 if len(tableList) > 1:
441 inDict['interChip'] = dict()
442 interChipTable = tableList[1]
443 for record in interChipTable:
444 inDict['interChip'][record['IC_SOURCE_DET']] = record['IC_COEFFS']
445
446 return cls().fromDict(inDict)
447
◆ getMetadata()
|
inherited |
Retrieve metadata associated with this calibration.
Returns
-------
meta : `lsst.daf.base.PropertyList`
Metadata. The returned `~lsst.daf.base.PropertyList` can be
modified by the caller and the changes will be written to
external files.
Definition at line 171 of file calibType.py.
171 def getMetadata(self):
172 """Retrieve metadata associated with this calibration.
173
174 Returns
175 -------
176 meta : `lsst.daf.base.PropertyList`
177 Metadata. The returned `~lsst.daf.base.PropertyList` can be
178 modified by the caller and the changes will be written to
179 external files.
180 """
181 return self._metadata
182
◆ metadata()
|
inherited |
Definition at line 168 of file calibType.py.
168 def metadata(self):
169 return self._metadata
170
◆ readFits()
|
inherited |
Read calibration data from a FITS file.
Parameters
----------
filename : `str`
Filename to read data from.
kwargs : `dict` or collections.abc.Mapping`, optional
Set of key=value pairs to pass to the ``fromTable``
method.
Returns
-------
calib : `lsst.ip.isr.IsrCalib`
Calibration contained within the file.
Definition at line 505 of file calibType.py.
505 def readFits(cls, filename, **kwargs):
506 """Read calibration data from a FITS file.
507
508 Parameters
509 ----------
510 filename : `str`
511 Filename to read data from.
512 kwargs : `dict` or collections.abc.Mapping`, optional
513 Set of key=value pairs to pass to the ``fromTable``
514 method.
515
516 Returns
517 -------
518 calib : `lsst.ip.isr.IsrCalib`
519 Calibration contained within the file.
520 """
521 tableList = []
522 tableList.append(Table.read(filename, hdu=1, mask_invalid=False))
523 extNum = 2 # Fits indices start at 1, we've read one already.
524 keepTrying = True
525
526 while keepTrying:
527 with warnings.catch_warnings():
528 warnings.simplefilter("error")
529 try:
530 newTable = Table.read(filename, hdu=extNum, mask_invalid=False)
531 tableList.append(newTable)
532 extNum += 1
533 except Exception:
534 keepTrying = False
535
536 for table in tableList:
537 for k, v in table.meta.items():
538 if isinstance(v, fits.card.Undefined):
539 table.meta[k] = None
540
541 calibClass = cls.determineCalibClass(tableList[0].meta, "readFits")
542 return calibClass.fromTable(tableList, **kwargs)
543
◆ readText()
|
inherited |
Read calibration representation from a yaml/ecsv file.
Parameters
----------
filename : `str`
Name of the file containing the calibration definition.
kwargs : `dict` or collections.abc.Mapping`, optional
Set of key=value pairs to pass to the ``fromDict`` or
``fromTable`` methods.
Returns
-------
calib : `~lsst.ip.isr.IsrCalibType`
Calibration class.
Raises
------
RuntimeError
Raised if the filename does not end in ".ecsv" or ".yaml".
Definition at line 418 of file calibType.py.
418 def readText(cls, filename, **kwargs):
419 """Read calibration representation from a yaml/ecsv file.
420
421 Parameters
422 ----------
423 filename : `str`
424 Name of the file containing the calibration definition.
425 kwargs : `dict` or collections.abc.Mapping`, optional
426 Set of key=value pairs to pass to the ``fromDict`` or
427 ``fromTable`` methods.
428
429 Returns
430 -------
431 calib : `~lsst.ip.isr.IsrCalibType`
432 Calibration class.
433
434 Raises
435 ------
436 RuntimeError
437 Raised if the filename does not end in ".ecsv" or ".yaml".
438 """
439 if filename.endswith((".ecsv", ".ECSV")):
440 data = Table.read(filename, format="ascii.ecsv")
441 calibClass = cls.determineCalibClass(data.meta, "readText/ECSV")
442 return calibClass.fromTable([data], **kwargs)
443 elif filename.endswith((".yaml", ".YAML")):
444 with open(filename, "r") as f:
445 data = yaml.load(f, Loader=yaml.CLoader)
446 calibClass = cls.determineCalibClass(data["metadata"], "readText/YAML")
447 return calibClass.fromDict(data, **kwargs)
448 else:
449 raise RuntimeError(f"Unknown filename extension: {filename}")
450
◆ requiredAttributes() [1/2]
|
inherited |
Definition at line 159 of file calibType.py.
159 def requiredAttributes(self):
160 return self._requiredAttributes
161
◆ requiredAttributes() [2/2]
|
inherited |
Definition at line 163 of file calibType.py.
163 def requiredAttributes(self, value):
164 self._requiredAttributes = value
165
◆ setMetadata()
|
inherited |
Store a copy of the supplied metadata with this calibration.
Parameters
----------
metadata : `lsst.daf.base.PropertyList`
Metadata to associate with the calibration. Will be copied and
overwrite existing metadata.
Reimplemented in lsst.ip.isr.transmissionCurve.IntermediateTransmissionCurve.
Definition at line 183 of file calibType.py.
183 def setMetadata(self, metadata):
184 """Store a copy of the supplied metadata with this calibration.
185
186 Parameters
187 ----------
188 metadata : `lsst.daf.base.PropertyList`
189 Metadata to associate with the calibration. Will be copied and
190 overwrite existing metadata.
191 """
192 if metadata is not None:
193 self._metadata.update(metadata)
194
195 # Ensure that we have the obs type required by calibration ingest
196 self._metadata["OBSTYPE"] = self._OBSTYPE
197 self._metadata[self._OBSTYPE + "_SCHEMA"] = self._SCHEMA
198 self._metadata[self._OBSTYPE + "_VERSION"] = self._VERSION
199
200 if isinstance(metadata, dict):
201 self.calibInfoFromDict(metadata)
202 elif isinstance(metadata, PropertyList):
203 self.calibInfoFromDict(metadata.toDict())
204
◆ subtractCrosstalk()
| lsst.ip.isr.crosstalk.CrosstalkCalib.subtractCrosstalk | ( | self, | |
| thisExposure, | |||
| sourceExposure = None, | |||
| crosstalkCoeffs = None, | |||
| crosstalkCoeffsSqr = None, | |||
| crosstalkCoeffsValid = None, | |||
| badPixels = ["BAD"], | |||
| minPixelToMask = 45000, | |||
| doSubtrahendMasking = False, | |||
| crosstalkStr = "CROSSTALK", | |||
| isTrimmed = None, | |||
| backgroundMethod = "None", | |||
| doSqrCrosstalk = False, | |||
| fullAmplifier = False, | |||
| parallelOverscan = None, | |||
| detectorConfig = None, | |||
| badAmpDict = None, | |||
| ignoreVariance = False ) |
Subtract the crosstalk from thisExposure, optionally using a
different source.
We set the mask plane indicated by ``crosstalkStr`` in a
target amplifier for pixels in a source amplifier that exceed
``minPixelToMask``, if ``doSubtrahendMasking`` is False. With
that enabled, the mask is only set if the absolute value of
the correction applied exceeds ``minPixelToMask``. Note that
the correction is applied to all pixels in the amplifier, but
only those that have a substantial crosstalk are masked with
``crosstalkStr``.
The uncorrected image is used as a template for correction. This is
good enough if the crosstalk is small (e.g., coefficients < ~ 1e-3),
but if it's larger you may want to iterate.
Parameters
----------
thisExposure : `lsst.afw.image.Exposure`
Exposure for which to subtract crosstalk.
sourceExposure : `lsst.afw.image.Exposure`, optional
Exposure to use as the source of the crosstalk. If not set,
thisExposure is used as the source (intra-detector crosstalk).
crosstalkCoeffs : `numpy.ndarray`, optional.
Coefficients to use to correct crosstalk.
crosstalkCoeffsSqr : `numpy.ndarray`, optional.
Quadratic coefficients to use to correct crosstalk.
crosstalkCoeffsValid : `numpy.ndarray`, optional
Boolean array that is True where coefficients are valid.
badPixels : `list` of `str`, optional
Mask planes to ignore.
minPixelToMask : `float`, optional
Minimum pixel value to set the ``crosstalkStr`` mask
plane. If doSubtrahendMasking is True, this is calculated
from the absolute magnitude of the subtrahend image.
Otherwise, this sets the minimum source value to use to
set that mask.
doSubtrahendMasking : `bool`, optional
If true, the mask is calculated from the properties of the
subtrahend image, not from the brightness of the source
pixel.
crosstalkStr : `str`, optional
Mask plane name for pixels greatly modified by crosstalk
(above minPixelToMask).
isTrimmed : `bool`, optional
This option has been deprecated and does not do anything.
It will be removed after v29.
backgroundMethod : `str`, optional
Method used to subtract the background. "AMP" uses
amplifier-by-amplifier background levels, "DETECTOR" uses full
exposure/maskedImage levels. Any other value results in no
background subtraction.
doSqrCrosstalk: `bool`, optional
Should the quadratic crosstalk coefficients be used for the
crosstalk correction?
fullAmplifier : `bool`, optional
Use full amplifier and not just imaging region.
parallelOverscan : `bool`, optional
This option is deprecated and will be removed after v29.
detectorConfig : `lsst.ip.isr.overscanDetectorConfig`, optional
Per-amplifier configs to use if parallelOverscan is True.
This option is deprecated and will be removed after v29.
badAmpDict : `dict` [`str`, `bool`], optional
Dictionary to identify bad amplifiers that should not be
source or target for crosstalk correction.
ignoreVariance : `bool`, optional
Ignore the variance plane when doing crosstalk correction?
Notes
-----
For a given image I, we want to find the crosstalk subtrahend
image CT, such that
I_corrected = I - CT
The subtrahend image is the sum of all crosstalk contributions
that appear in I, so we can build it up by amplifier. Each
amplifier A in image I sees the contributions from all other
amplifiers B_v != A. For the current linear model, we set `sImage`
equal to the segment of the subtrahend image CT corresponding to
amplifier A, and then build it up as:
simage_linear = sum_v coeffsA_v * (B_v - bkg_v) where coeffsA_v
is the vector of crosstalk coefficients for sources that cause
images in amplifier A. The bkg_v term in this equation is
identically 0.0 for all cameras except obs_subaru (and is only
non-zero there for historical reasons).
To include the non-linear term, we can again add to the subtrahend
image using the same loop, as:
simage_nonlinear = sum_v (coeffsA_v * B_v) + (NLcoeffsA_v * B_v * B_v)
= sum_v linear_term_v + nonlinear_term_v
where coeffsA_v is the linear term, and NLcoeffsA_v are the quadratic
component. For LSSTCam, it has been observed that the linear_term_v >>
nonlinear_term_v.
Definition at line 587 of file crosstalk.py.
593 ignoreVariance=False):
594 """Subtract the crosstalk from thisExposure, optionally using a
595 different source.
596
597 We set the mask plane indicated by ``crosstalkStr`` in a
598 target amplifier for pixels in a source amplifier that exceed
599 ``minPixelToMask``, if ``doSubtrahendMasking`` is False. With
600 that enabled, the mask is only set if the absolute value of
601 the correction applied exceeds ``minPixelToMask``. Note that
602 the correction is applied to all pixels in the amplifier, but
603 only those that have a substantial crosstalk are masked with
604 ``crosstalkStr``.
605
606 The uncorrected image is used as a template for correction. This is
607 good enough if the crosstalk is small (e.g., coefficients < ~ 1e-3),
608 but if it's larger you may want to iterate.
609
610 Parameters
611 ----------
612 thisExposure : `lsst.afw.image.Exposure`
613 Exposure for which to subtract crosstalk.
614 sourceExposure : `lsst.afw.image.Exposure`, optional
615 Exposure to use as the source of the crosstalk. If not set,
616 thisExposure is used as the source (intra-detector crosstalk).
617 crosstalkCoeffs : `numpy.ndarray`, optional.
618 Coefficients to use to correct crosstalk.
619 crosstalkCoeffsSqr : `numpy.ndarray`, optional.
620 Quadratic coefficients to use to correct crosstalk.
621 crosstalkCoeffsValid : `numpy.ndarray`, optional
622 Boolean array that is True where coefficients are valid.
623 badPixels : `list` of `str`, optional
624 Mask planes to ignore.
625 minPixelToMask : `float`, optional
626 Minimum pixel value to set the ``crosstalkStr`` mask
627 plane. If doSubtrahendMasking is True, this is calculated
628 from the absolute magnitude of the subtrahend image.
629 Otherwise, this sets the minimum source value to use to
630 set that mask.
631 doSubtrahendMasking : `bool`, optional
632 If true, the mask is calculated from the properties of the
633 subtrahend image, not from the brightness of the source
634 pixel.
635 crosstalkStr : `str`, optional
636 Mask plane name for pixels greatly modified by crosstalk
637 (above minPixelToMask).
638 isTrimmed : `bool`, optional
639 This option has been deprecated and does not do anything.
640 It will be removed after v29.
641 backgroundMethod : `str`, optional
642 Method used to subtract the background. "AMP" uses
643 amplifier-by-amplifier background levels, "DETECTOR" uses full
644 exposure/maskedImage levels. Any other value results in no
645 background subtraction.
646 doSqrCrosstalk: `bool`, optional
647 Should the quadratic crosstalk coefficients be used for the
648 crosstalk correction?
649 fullAmplifier : `bool`, optional
650 Use full amplifier and not just imaging region.
651 parallelOverscan : `bool`, optional
652 This option is deprecated and will be removed after v29.
653 detectorConfig : `lsst.ip.isr.overscanDetectorConfig`, optional
654 Per-amplifier configs to use if parallelOverscan is True.
655 This option is deprecated and will be removed after v29.
656 badAmpDict : `dict` [`str`, `bool`], optional
657 Dictionary to identify bad amplifiers that should not be
658 source or target for crosstalk correction.
659 ignoreVariance : `bool`, optional
660 Ignore the variance plane when doing crosstalk correction?
661
662 Notes
663 -----
664
665 For a given image I, we want to find the crosstalk subtrahend
666 image CT, such that
667 I_corrected = I - CT
668 The subtrahend image is the sum of all crosstalk contributions
669 that appear in I, so we can build it up by amplifier. Each
670 amplifier A in image I sees the contributions from all other
671 amplifiers B_v != A. For the current linear model, we set `sImage`
672 equal to the segment of the subtrahend image CT corresponding to
673 amplifier A, and then build it up as:
674 simage_linear = sum_v coeffsA_v * (B_v - bkg_v) where coeffsA_v
675 is the vector of crosstalk coefficients for sources that cause
676 images in amplifier A. The bkg_v term in this equation is
677 identically 0.0 for all cameras except obs_subaru (and is only
678 non-zero there for historical reasons).
679 To include the non-linear term, we can again add to the subtrahend
680 image using the same loop, as:
681
682 simage_nonlinear = sum_v (coeffsA_v * B_v) + (NLcoeffsA_v * B_v * B_v)
683 = sum_v linear_term_v + nonlinear_term_v
684
685 where coeffsA_v is the linear term, and NLcoeffsA_v are the quadratic
686 component. For LSSTCam, it has been observed that the linear_term_v >>
687 nonlinear_term_v.
688 """
689 targetMaskedImage = thisExposure.maskedImage
690 targetDetector = thisExposure.getDetector()
691 if self.hasCrosstalk is False:
692 self.fromDetector(targetDetector, coeffVector=crosstalkCoeffs)
693
694 # TODO: Remove on DM-48394
695 if isTrimmed is not None:
696 warnings.warn(
697 "The isTrimmed option has been deprecated and will be removed after v29.",
698 FutureWarning,
699 )
700 isTrimmed = isTrimmedImage(targetMaskedImage, targetDetector)
701
702 # TODO: Remove on DM-48394
703 if parallelOverscan is not None:
704 warnings.warn(
705 "The parallelOverscan option has been deprecated and will be removed after v29.",
706 FutureWarning,
707 )
708 if detectorConfig is not None:
709 warnings.warn(
710 "The detectorConfig option has been deprecated and will be removed after v29.",
711 FutureWarning,
712 )
713
714 numAmps = len(targetDetector)
715 if numAmps != self.nAmp:
716 raise RuntimeError(f"Crosstalk built for {self.nAmp} in {self._detectorName}, received "
717 f"{numAmps} in {targetDetector.getName()}")
718
719 if doSqrCrosstalk and crosstalkCoeffsSqr is None:
720 raise RuntimeError("Attempted to perform NL crosstalk correction without NL "
721 "crosstalk coefficients.")
722
723 if fullAmplifier and (backgroundMethod != "None"):
724 raise RuntimeError("Cannot do full amplifier with background subtraction.")
725
726 if sourceExposure:
727 sourceMaskedImage = sourceExposure.maskedImage
728 sourceDetector = sourceExposure.getDetector()
729 else:
730 sourceMaskedImage = targetMaskedImage
731 sourceDetector = targetDetector
732
733 if crosstalkCoeffs is not None:
734 coeffs = np.asarray(crosstalkCoeffs).copy()
735 else:
736 coeffs = np.asarray(self.coeffs).copy()
737 self.log.debug("CT COEFF: %s", coeffs)
738
739 if doSqrCrosstalk:
740 coeffsSqr = np.asarray(crosstalkCoeffsSqr).copy()
741 self.log.debug("CT COEFF SQR: %s", coeffsSqr)
742 else:
743 coeffsSqr = np.zeros_like(coeffs)
744
745 # Check for valid values; set to 0 otherwise.
746 badCoeffs = ~np.isfinite(coeffs) | (coeffs == 0.0)
747 coeffs[badCoeffs] = 0.0
748 coeffsSqr[badCoeffs] = 0.0
749 if crosstalkCoeffsValid is not None:
750 coeffs[~crosstalkCoeffsValid] = 0.0
751 coeffsSqr[~crosstalkCoeffsValid] = 0.0
752
753 if badAmpDict:
754 for index, amp in enumerate(sourceDetector):
755 if badAmpDict[amp.getName()]:
756 coeffs[index, :] = 0.0
757 coeffs[:, index] = 0.0
758
759 # Compute backgrounds if requested.
760 backgrounds = {amp.getName(): 0.0 for amp in sourceDetector}
761 if backgroundMethod == "AMP":
762 backgrounds = {
763 amp.getName(): self.calculateBackground(
764 sourceMaskedImage[self._getAppropriateBBox(
765 amp,
766 isTrimmed,
767 False,
768 )], badPixels)
769 for amp in sourceDetector
770 }
771 elif backgroundMethod == "DETECTOR":
772 background = self.calculateBackground(sourceMaskedImage, badPixels)
773 backgrounds = {amp.getName(): background for amp in sourceDetector}
774
775 # Add the crosstalk mask plane to the target mask.
776 sourceCrosstalkPlane = sourceMaskedImage.mask.addMaskPlane(crosstalkStr)
777 if sourceExposure is not None:
778 targetCrosstalkPlane = targetMaskedImage.mask.addMaskPlane(crosstalkStr)
779 else:
780 targetCrosstalkPlane = sourceCrosstalkPlane
781
782 # If we are not doing subtrahend masking, the CROSSTALK mask bit
783 # is set according to the counts in the source pixel (above
784 # background), regardless of the crosstalk coefficient value.
785 if not doSubtrahendMasking:
786 # When we are not using subtrahend masking, we set the mask.
787 thresholdBackground = self.calculateBackground(sourceMaskedImage, badPixels)
788 toMask = (sourceMaskedImage.image.array >= (minPixelToMask + thresholdBackground))
789 sourceMaskedImage.mask.array[toMask] |= sourceMaskedImage.mask.getPlaneBitMask(crosstalkStr)
790
791 crosstalk = sourceMaskedImage.mask.getPlaneBitMask(crosstalkStr)
792
793 # Define a subtrahend image to contain all the scaled crosstalk
794 # signals. These will be applied to the target image.
795 subtrahend = targetMaskedImage.Factory(targetMaskedImage.getBBox())
796 subtrahend.set((0, 0, 0))
797
798 # If we are ignoring variance and doing subtrahend masking, then
799 # we can work on only the image plane (3x speed increase).
800 imageOnly = ignoreVariance and doSubtrahendMasking
801
802 for sourceIndex, sourceAmp in enumerate(sourceDetector):
803 for targetIndex, targetAmp in enumerate(targetDetector):
804 coeff = coeffs[sourceIndex, targetIndex]
805 coeffSqr = coeffsSqr[sourceIndex, targetIndex]
806
807 if coeff == 0.0:
808 continue
809
810 targetBBox = self._getAppropriateBBox(targetAmp, isTrimmed, fullAmplifier)
811
812 # The extractAmp() method will always make a copy of the source
813 # amplifier data.
814 if imageOnly:
815 sourceImage = self.extractAmp(
816 sourceMaskedImage.image,
817 sourceAmp,
818 targetAmp,
819 isTrimmed=isTrimmed,
820 fullAmplifier=fullAmplifier,
821 )
822 targetImage = subtrahend[targetBBox].image
823 else:
824 sourceImage = self.extractAmp(
825 sourceMaskedImage,
826 sourceAmp,
827 targetAmp,
828 isTrimmed=isTrimmed,
829 fullAmplifier=fullAmplifier,
830 )
831 targetImage = subtrahend[targetBBox]
832
833 # Remove all other masks from copied sourceImage.
834 sourceImage.mask.array[:] &= crosstalk
835
836 if backgrounds[sourceAmp.getName()] != 0.0:
837 sourceImage -= backgrounds[sourceAmp.getName()]
838
839 # This operation will also transfer the CROSSTALK mask bit from
840 # above to the target (subtrahend) image if we are using a
841 # masked image.
842 targetImage.scaledPlus(coeff, sourceImage)
843 if coeffSqr != 0.0:
844 sourceImage.scaledMultiplies(1.0, sourceImage)
845 targetImage.scaledPlus(coeffSqr, sourceImage)
846
847 # Clear the mask in the target image, because the subtrahend image
848 # contains the crosstalk mask.
849 sourceMaskedImage.mask.clearMaskPlane(sourceCrosstalkPlane)
850 if sourceExposure is not None:
851 targetMaskedImage.mask.clearMaskPlane(targetCrosstalkPlane)
852
853 if doSubtrahendMasking:
854 # Set crosstalkStr bit only for those pixels that have
855 # been significantly modified (i.e., those masked as such
856 # in 'subtrahend'), not necessarily those that are bright
857 # originally.
858
859 # The existing mask in the subtrahend comes from the
860 # threshold set above. It should be cleared so we can
861 # recalculate it.
862 subtrahend.mask.clearMaskPlane(targetCrosstalkPlane)
863
864 # For masking purposes, we only really care when the
865 # correction is significantly different than the median
866 # value on that amplifier (which includes the contribution
867 # of every other amplifier background that crosstalks onto
868 # that amplifier). Subtract and save this "background"
869 # level, so we can threshold to set the mask relative to
870 # that background, but still include that contribution in
871 # the correction we're applying.
872 subtrahendBackgrounds = {}
873 for amp in targetDetector:
874 # Note that we never want the full amplifier for background
875 # calculations.
876 bbox = self._getAppropriateBBox(amp, isTrimmed, False)
877 ampData = subtrahend[bbox]
878 background = np.median(ampData.image.array)
879 subtrahendBackgrounds[amp.getName()] = background
880 ampData.image.array[:, :] -= background
881 self.log.debug(f"Subtrahend background level: {amp.getName()} {background}")
882
883 toMask = (subtrahend.image.array >= minPixelToMask) | (subtrahend.image.array <= -minPixelToMask)
884 subtrahend.mask.array[toMask] |= subtrahend.mask.getPlaneBitMask(crosstalkStr)
885
886 # Put the backgrounds back.
887 for amp in targetDetector:
888 bbox = self._getAppropriateBBox(amp, isTrimmed, False)
889 ampData = subtrahend[bbox]
890 background = subtrahendBackgrounds[amp.getName()]
891 ampData.image.array[:, :] += background
892
893 # Subtract subtrahend from input. The mask plane is fully
894 # populated, so this operation also sets the ``crosstalkStr``
895 # bit where applicable.
896 targetMaskedImage -= subtrahend
897
898
◆ toDict()
| lsst.ip.isr.crosstalk.CrosstalkCalib.toDict | ( | self | ) |
Return a dictionary containing the calibration properties.
The dictionary should be able to be round-tripped through
`fromDict`.
Returns
-------
dictionary : `dict`
Dictionary of properties.
Reimplemented from lsst.ip.isr.calibType.IsrCalib.
Definition at line 345 of file crosstalk.py.
345 def toDict(self):
346 """Return a dictionary containing the calibration properties.
347
348 The dictionary should be able to be round-tripped through
349 `fromDict`.
350
351 Returns
352 -------
353 dictionary : `dict`
354 Dictionary of properties.
355 """
356 self.updateMetadata()
357
358 outDict = {}
359 metadata = self.getMetadata()
360 outDict['metadata'] = metadata
361
362 outDict['hasCrosstalk'] = self.hasCrosstalk
363 outDict['nAmp'] = self.nAmp
364 outDict['crosstalkShape'] = self.crosstalkShape
365 outDict['crosstalkRatiosUnits'] = self.crosstalkRatiosUnits
366
367 ctLength = self.nAmp*self.nAmp
368 outDict['coeffs'] = self.coeffs.reshape(ctLength).tolist()
369
370 if self.coeffErr is not None:
371 outDict['coeffErr'] = self.coeffErr.reshape(ctLength).tolist()
372 if self.coeffNum is not None:
373 outDict['coeffNum'] = self.coeffNum.reshape(ctLength).tolist()
374 if self.coeffValid is not None:
375 outDict['coeffValid'] = self.coeffValid.reshape(ctLength).tolist()
376 if self.coeffsSqr is not None:
377 outDict['coeffsSqr'] = self.coeffsSqr.reshape(ctLength).tolist()
378 if self.coeffErrSqr is not None:
379 outDict['coeffErrSqr'] = self.coeffErrSqr.reshape(ctLength).tolist()
380 if self.ampGainRatios is not None:
381 outDict['ampGainRatios'] = self.ampGainRatios.reshape(ctLength).tolist()
382 if self.fitGains is not None:
383 outDict['fitGains'] = self.fitGains.tolist()
384
385 if self.interChip:
386 outDict['interChip'] = dict()
387 for detector in self.interChip:
388 outDict['interChip'][detector] = self.interChip[detector].reshape(ctLength).tolist()
389
390 return outDict
391
◆ toTable()
| lsst.ip.isr.crosstalk.CrosstalkCalib.toTable | ( | self | ) |
Construct a list of tables containing the information in this
calibration.
The list of tables should create an identical calibration
after being passed to this class's fromTable method.
Returns
-------
tableList : `list` [`lsst.afw.table.Table`]
List of tables containing the crosstalk calibration
information.
Reimplemented from lsst.ip.isr.calibType.IsrCalib.
Definition at line 448 of file crosstalk.py.
448 def toTable(self):
449 """Construct a list of tables containing the information in this
450 calibration.
451
452 The list of tables should create an identical calibration
453 after being passed to this class's fromTable method.
454
455 Returns
456 -------
457 tableList : `list` [`lsst.afw.table.Table`]
458 List of tables containing the crosstalk calibration
459 information.
460
461 """
462 tableList = []
463 self.updateMetadata()
464 catalog = Table([{'CT_COEFFS': self.coeffs.reshape(self.nAmp*self.nAmp),
465 'CT_ERRORS': self.coeffErr.reshape(self.nAmp*self.nAmp),
466 'CT_COUNTS': self.coeffNum.reshape(self.nAmp*self.nAmp),
467 'CT_VALID': self.coeffValid.reshape(self.nAmp*self.nAmp),
468 'CT_COEFFS_SQR': self.coeffsSqr.reshape(self.nAmp*self.nAmp),
469 'CT_ERRORS_SQR': self.coeffErrSqr.reshape(self.nAmp*self.nAmp),
470 'CT_AMP_GAIN_RATIOS': self.ampGainRatios.reshape(self.nAmp*self.nAmp),
471 'CT_FIT_GAINS': self.fitGains,
472 }])
473 # filter None, because astropy can't deal.
474 inMeta = self.getMetadata().toDict()
475 outMeta = {k: v for k, v in inMeta.items() if v is not None}
476 outMeta.update({k: "" for k, v in inMeta.items() if v is None})
477 catalog.meta = outMeta
478 tableList.append(catalog)
479
480 if self.interChip:
481 interChipTable = Table([{'IC_SOURCE_DET': sourceDet,
482 'IC_COEFFS': self.interChip[sourceDet].reshape(self.nAmp*self.nAmp)}
483 for sourceDet in self.interChip.keys()])
484 tableList.append(interChipTable)
485 return tableList
486
◆ updateMetadata()
| lsst.ip.isr.crosstalk.CrosstalkCalib.updateMetadata | ( | self, | |
| setDate = False, | |||
| ** | kwargs ) |
Update calibration metadata.
This calls the base class's method after ensuring the required
calibration keywords will be saved.
Parameters
----------
setDate : `bool`, optional
Update the CALIBDATE fields in the metadata to the current
time. Defaults to False.
kwargs :
Other keyword parameters to set in the metadata.
Reimplemented from lsst.ip.isr.calibType.IsrCalib.
Definition at line 142 of file crosstalk.py.
142 def updateMetadata(self, setDate=False, **kwargs):
143 """Update calibration metadata.
144
145 This calls the base class's method after ensuring the required
146 calibration keywords will be saved.
147
148 Parameters
149 ----------
150 setDate : `bool`, optional
151 Update the CALIBDATE fields in the metadata to the current
152 time. Defaults to False.
153 kwargs :
154 Other keyword parameters to set in the metadata.
155 """
156 kwargs['DETECTOR'] = self._detectorId
157 kwargs['DETECTOR_NAME'] = self._detectorName
158 kwargs['DETECTOR_SERIAL'] = self._detectorSerial
159 kwargs['HAS_CROSSTALK'] = self.hasCrosstalk
160 kwargs['NAMP'] = self.nAmp
161 self.crosstalkShape = (self.nAmp, self.nAmp)
162 kwargs['CROSSTALK_SHAPE'] = self.crosstalkShape
163 kwargs['CROSSTALK_RATIOS_UNITS'] = self.crosstalkRatiosUnits
164
165 super().updateMetadata(setDate=setDate, **kwargs)
166
◆ updateMetadataFromExposures()
|
inherited |
Extract and unify metadata information.
Parameters
----------
exposures : `list`
Exposures or other calibrations to scan.
Definition at line 291 of file calibType.py.
291 def updateMetadataFromExposures(self, exposures):
292 """Extract and unify metadata information.
293
294 Parameters
295 ----------
296 exposures : `list`
297 Exposures or other calibrations to scan.
298 """
299 # This list of keywords is the set of header entries that
300 # should be checked and propagated. Not having an entry is
301 # not a failure, as they may not be defined for the exposures
302 # being used.
303 keywords = ["SEQNAME", "SEQFILE", "SEQCKSUM", "ODP", "AP0_RC"]
304 metadata = {}
305
306 for exp in exposures:
307 try:
308 expMeta = exp.getMetadata()
309 except AttributeError:
310 continue
311 for key in keywords:
312 if key in expMeta:
313 if key in metadata:
314 if metadata[key] != expMeta[key]:
315 self.log.warning("Metadata mismatch! Have: %s Found %s",
316 metadata[key], expMeta[key])
317 else:
318 metadata[key] = expMeta[key]
319
320 self.updateMetadata(**metadata, setCalibInfo=True)
321
◆ validate()
|
inherited |
Validate that this calibration is defined and can be used.
Parameters
----------
other : `object`, optional
Thing to validate against.
Returns
-------
valid : `bool`
Returns true if the calibration is valid and appropriate.
Reimplemented in lsst.ip.isr.linearize.Linearizer, and lsst.ip.isr.photodiodeCorrection.PhotodiodeCorrection.
Definition at line 670 of file calibType.py.
670 def validate(self, other=None):
671 """Validate that this calibration is defined and can be used.
672
673 Parameters
674 ----------
675 other : `object`, optional
676 Thing to validate against.
677
678 Returns
679 -------
680 valid : `bool`
681 Returns true if the calibration is valid and appropriate.
682 """
683 return False
684
◆ writeFits()
|
inherited |
Write calibration data to a FITS file.
Parameters
----------
filename : `str`
Filename to write data to.
Returns
-------
used : `str`
The name of the file used to write the data.
Reimplemented in lsst.ip.isr.transmissionCurve.IntermediateTransmissionCurve.
Definition at line 544 of file calibType.py.
544 def writeFits(self, filename):
545 """Write calibration data to a FITS file.
546
547 Parameters
548 ----------
549 filename : `str`
550 Filename to write data to.
551
552 Returns
553 -------
554 used : `str`
555 The name of the file used to write the data.
556 """
557 tableList = self.toTable()
558 with warnings.catch_warnings():
559 warnings.filterwarnings("ignore", category=Warning, module="astropy.io")
560 astropyList = [fits.table_to_hdu(table) for table in tableList]
561 astropyList.insert(0, fits.PrimaryHDU())
562
563 writer = fits.HDUList(astropyList)
564 writer.writeto(filename, overwrite=True)
565 return filename
566
◆ writeText()
|
inherited |
Write the calibration data to a text file.
Parameters
----------
filename : `str`
Name of the file to write.
format : `str`
Format to write the file as. Supported values are:
``"auto"`` : Determine filetype from filename.
``"yaml"`` : Write as yaml.
``"ecsv"`` : Write as ecsv.
Returns
-------
used : `str`
The name of the file used to write the data. This may
differ from the input if the format is explicitly chosen.
Raises
------
RuntimeError
Raised if filename does not end in a known extension, or
if all information cannot be written.
Notes
-----
The file is written to YAML/ECSV format and will include any
associated metadata.
Definition at line 451 of file calibType.py.
451 def writeText(self, filename, format="auto"):
452 """Write the calibration data to a text file.
453
454 Parameters
455 ----------
456 filename : `str`
457 Name of the file to write.
458 format : `str`
459 Format to write the file as. Supported values are:
460 ``"auto"`` : Determine filetype from filename.
461 ``"yaml"`` : Write as yaml.
462 ``"ecsv"`` : Write as ecsv.
463
464 Returns
465 -------
466 used : `str`
467 The name of the file used to write the data. This may
468 differ from the input if the format is explicitly chosen.
469
470 Raises
471 ------
472 RuntimeError
473 Raised if filename does not end in a known extension, or
474 if all information cannot be written.
475
476 Notes
477 -----
478 The file is written to YAML/ECSV format and will include any
479 associated metadata.
480 """
481 if format == "yaml" or (format == "auto" and filename.lower().endswith((".yaml", ".YAML"))):
482 outDict = self.toDict()
483 path, ext = os.path.splitext(filename)
484 filename = path + ".yaml"
485 with open(filename, "w") as f:
486 yaml.dump(outDict, f)
487 elif format == "ecsv" or (format == "auto" and filename.lower().endswith((".ecsv", ".ECSV"))):
488 tableList = self.toTable()
489 if len(tableList) > 1:
490 # ECSV doesn't support multiple tables per file, so we
491 # can only write the first table.
492 raise RuntimeError(f"Unable to persist {len(tableList)}tables in ECSV format.")
493
494 table = tableList[0]
495 path, ext = os.path.splitext(filename)
496 filename = path + ".ecsv"
497 table.write(filename, format="ascii.ecsv")
498 else:
499 raise RuntimeError(f"Attempt to write to a file {filename} "
500 "that does not end in '.yaml' or '.ecsv'")
501
502 return filename
503
Member Data Documentation
◆ _calibId
|
protectedinherited |
Definition at line 74 of file calibType.py.
◆ _detectorId
|
protectedinherited |
Definition at line 72 of file calibType.py.
◆ _detectorName
|
protectedinherited |
Definition at line 70 of file calibType.py.
◆ _detectorSerial
|
protectedinherited |
Definition at line 71 of file calibType.py.
◆ _filter
|
protectedinherited |
Definition at line 73 of file calibType.py.
◆ _instrument
|
protectedinherited |
Definition at line 67 of file calibType.py.
◆ _metadata
|
protectedinherited |
Definition at line 78 of file calibType.py.
◆ _OBSTYPE
|
staticprotectedinherited |
Definition at line 62 of file calibType.py.
◆ _raftName
|
protectedinherited |
Definition at line 68 of file calibType.py.
◆ _requiredAttributes
|
protectedinherited |
Definition at line 113 of file calibType.py.
◆ _SCHEMA
|
staticprotectedinherited |
Definition at line 63 of file calibType.py.
◆ _seqcksum
|
protectedinherited |
Definition at line 77 of file calibType.py.
◆ _seqfile
|
protectedinherited |
Definition at line 75 of file calibType.py.
◆ _seqname
|
protectedinherited |
Definition at line 76 of file calibType.py.
◆ _slotName
|
protectedinherited |
Definition at line 69 of file calibType.py.
◆ _VERSION
|
staticprotectedinherited |
Definition at line 64 of file calibType.py.
◆ ampGainRatios
| int lsst.ip.isr.crosstalk.CrosstalkCalib.ampGainRatios = np.zeros(self.crosstalkShape) if self.nAmp else None |
Definition at line 123 of file crosstalk.py.
◆ coeffErr
| int lsst.ip.isr.crosstalk.CrosstalkCalib.coeffErr = np.zeros(self.crosstalkShape) if self.nAmp else None |
Definition at line 113 of file crosstalk.py.
◆ coeffErrSqr
| int lsst.ip.isr.crosstalk.CrosstalkCalib.coeffErrSqr = np.zeros(self.crosstalkShape) if self.nAmp else None |
Definition at line 120 of file crosstalk.py.
◆ coeffNum
| int lsst.ip.isr.crosstalk.CrosstalkCalib.coeffNum |
Initial value:
= np.zeros(self.crosstalkShape,
dtype=int) if self.nAmp else None
Definition at line 114 of file crosstalk.py.
◆ coeffs
| int lsst.ip.isr.crosstalk.CrosstalkCalib.coeffs = np.zeros(self.crosstalkShape) if self.nAmp else None |
Definition at line 112 of file crosstalk.py.
◆ coeffsSqr
| int lsst.ip.isr.crosstalk.CrosstalkCalib.coeffsSqr = np.zeros(self.crosstalkShape) if self.nAmp else None |
Definition at line 119 of file crosstalk.py.
◆ coeffValid
| int lsst.ip.isr.crosstalk.CrosstalkCalib.coeffValid |
Initial value:
= np.ones(self.crosstalkShape,
dtype=bool) if self.nAmp else None
Definition at line 116 of file crosstalk.py.
◆ crosstalkRatiosUnits
| str lsst.ip.isr.crosstalk.CrosstalkCalib.crosstalkRatiosUnits = 'adu' if self.nAmp else None |
Definition at line 129 of file crosstalk.py.
◆ crosstalkShape
| tuple lsst.ip.isr.crosstalk.CrosstalkCalib.crosstalkShape = (self.nAmp, self.nAmp) |
Definition at line 110 of file crosstalk.py.
◆ fitGains
| int lsst.ip.isr.crosstalk.CrosstalkCalib.fitGains = np.zeros(self.nAmp) if self.nAmp else None |
Definition at line 126 of file crosstalk.py.
◆ hasCrosstalk
| bool lsst.ip.isr.crosstalk.CrosstalkCalib.hasCrosstalk = False |
Definition at line 108 of file crosstalk.py.
◆ interChip
| dict lsst.ip.isr.crosstalk.CrosstalkCalib.interChip = {} |
Definition at line 131 of file crosstalk.py.
◆ log
|
inherited |
Definition at line 94 of file calibType.py.
◆ nAmp
| int lsst.ip.isr.crosstalk.CrosstalkCalib.nAmp = nAmp if nAmp else 0 |
Definition at line 109 of file crosstalk.py.
◆ requiredAttributes
|
inherited |
Definition at line 88 of file calibType.py.
The documentation for this class was generated from the following file:
- /j/snowflake/release/lsstsw/stack/lsst-scipipe-10.0.0/Linux64/ip_isr/ge1ad929117+3bfb9058a5/python/lsst/ip/isr/crosstalk.py
Generated on Fri Apr 25 2025 08:03:50 for LSST Applications by
1.13.2