LSST Applications
21.0.0-172-gfb10e10a+18fedfabac,22.0.0+297cba6710,22.0.0+80564b0ff1,22.0.0+8d77f4f51a,22.0.0+a28f4c53b1,22.0.0+dcf3732eb2,22.0.1-1-g7d6de66+2a20fdde0d,22.0.1-1-g8e32f31+297cba6710,22.0.1-1-geca5380+7fa3b7d9b6,22.0.1-12-g44dc1dc+2a20fdde0d,22.0.1-15-g6a90155+515f58c32b,22.0.1-16-g9282f48+790f5f2caa,22.0.1-2-g92698f7+dcf3732eb2,22.0.1-2-ga9b0f51+7fa3b7d9b6,22.0.1-2-gd1925c9+bf4f0e694f,22.0.1-24-g1ad7a390+a9625a72a8,22.0.1-25-g5bf6245+3ad8ecd50b,22.0.1-25-gb120d7b+8b5510f75f,22.0.1-27-g97737f7+2a20fdde0d,22.0.1-32-gf62ce7b1+aa4237961e,22.0.1-4-g0b3f228+2a20fdde0d,22.0.1-4-g243d05b+871c1b8305,22.0.1-4-g3a563be+32dcf1063f,22.0.1-4-g44f2e3d+9e4ab0f4fa,22.0.1-42-gca6935d93+ba5e5ca3eb,22.0.1-5-g15c806e+85460ae5f3,22.0.1-5-g58711c4+611d128589,22.0.1-5-g75bb458+99c117b92f,22.0.1-6-g1c63a23+7fa3b7d9b6,22.0.1-6-g50866e6+84ff5a128b,22.0.1-6-g8d3140d+720564cf76,22.0.1-6-gd805d02+cc5644f571,22.0.1-8-ge5750ce+85460ae5f3,master-g6e05de7fdc+babf819c66,master-g99da0e417a+8d77f4f51a,w.2021.48
LSST Data Management Base Package
|
Public Member Functions | |
def | applyOverrides (cls, config) |
def | __init__ (self, *args, **kwargs) |
def | processSingle (self, sensorRef) |
def | batchWallTime (cls, time, parsedCmd, numCores) |
Return walltime request for batch job. More... | |
def | runDataRef (self, expRefList, butler, calibId) |
Construct a calib from a list of exposure references. More... | |
def | getOutputId (self, expRefList, calibId) |
Generate the data identifier for the output calib. More... | |
def | getMjd (self, butler, dataId, timescale=dafBase.DateTime.UTC) |
def | getFilter (self, butler, dataId) |
def | addMissingKeys (self, dataId, butler, missingKeys=None, calibName=None) |
def | updateMetadata (self, calibImage, exposureTime, darkTime=None, **kwargs) |
Update the metadata from the VisitInfo. More... | |
def | scatterProcess (self, pool, ccdIdLists) |
Scatter the processing among the nodes. More... | |
def | process (self, cache, ccdId, outputName="postISRCCD", **kwargs) |
Process a CCD, specified by a data identifier. More... | |
def | processWrite (self, dataRef, exposure, outputName="postISRCCD") |
Write the processed CCD. More... | |
def | processResult (self, exposure) |
def | scale (self, ccdIdLists, data) |
Determine scaling across CCDs and exposures. More... | |
def | scatterCombine (self, pool, outputId, ccdIdLists, scales) |
Scatter the combination of exposures across multiple nodes. More... | |
def | getFullyQualifiedOutputId (self, ccdName, butler, outputId) |
def | combine (self, cache, struct, outputId) |
Combine multiple exposures of a particular CCD and write the output. More... | |
def | calculateOutputHeaderFromRaws (self, butler, calib, dataIdList, outputId) |
Calculate the output header from the raw headers. More... | |
def | recordCalibInputs (self, butler, calib, dataIdList, outputId) |
Record metadata including the inputs and creation details. More... | |
def | interpolateNans (self, image) |
def | write (self, butler, exposure, dataId) |
Write the final combined calib. More... | |
def | makeCameraImage (self, camera, dataId, calibs) |
Create and write an image of the entire camera. More... | |
def | checkCcdIdLists (self, ccdIdLists) |
def | parseAndRun (cls, *args, **kwargs) |
def | parseAndSubmit (cls, args=None, **kwargs) |
def | batchCommand (cls, args) |
Return command to run CmdLineTask. More... | |
def | logOperation (self, operation, catch=False, trace=True) |
Provide a context manager for logging an operation. More... | |
Static Public Attributes | |
ConfigClass = FringeConfig | |
string | calibName = "fringe" |
RunnerClass = CalibTaskRunner | |
filterName = None | |
float | exposureTime = 1.0 |
Fringe construction task The principal change from the base class is that the images are background-subtracted and rescaled by the background. XXX This is probably not right for a straight-up combination, as we are currently doing, since the fringe amplitudes need not scale with the continuum. XXX Would like to have this do PCA and generate multiple images, but that will take a bit of work with the persistence code.
Definition at line 1123 of file constructCalibs.py.
def lsst.pipe.drivers.constructCalibs.FringeTask.__init__ | ( | self, | |
* | args, | ||
** | kwargs | ||
) |
Constructor
Reimplemented from lsst.pipe.drivers.constructCalibs.CalibTask.
Definition at line 1145 of file constructCalibs.py.
|
inherited |
Definition at line 549 of file constructCalibs.py.
def lsst.pipe.drivers.constructCalibs.FringeTask.applyOverrides | ( | cls, | |
config | |||
) |
Overrides for fringe construction
Definition at line 1141 of file constructCalibs.py.
|
inherited |
Return command to run CmdLineTask.
@param cls: Class @param args: Parsed batch job arguments (from BatchArgumentParser)
Definition at line 476 of file parallel.py.
|
inherited |
Return walltime request for batch job.
Subclasses should override if the walltime should be calculated differently (e.g., addition of some serial time). @param cls: Class @param time: Requested time per iteration @param parsedCmd: Results of argument parsing @param numCores: Number of cores
Reimplemented from lsst.ctrl.pool.parallel.BatchCmdLineTask.
Definition at line 410 of file constructCalibs.py.
|
inherited |
Calculate the output header from the raw headers.
This metadata will go into the output FITS header. It will include all headers that are identical in all inputs. @param butler Data butler @param calib Combined calib exposure. @param dataIdList List of data identifiers for calibration inputs @param outputId Data identifier for output
Definition at line 766 of file constructCalibs.py.
|
inherited |
Check that the list of CCD dataIds is consistent @param ccdIdLists Dict of data identifier lists for each CCD name @return Number of exposures, number of CCDs
Definition at line 886 of file constructCalibs.py.
|
inherited |
Combine multiple exposures of a particular CCD and write the output.
Only the slave nodes execute this method. @param cache Process pool cache @param struct Parameters for the combination, which has the following components: * ccdName Name tuple for CCD * ccdIdList List of data identifiers for combination * scales Scales to apply (expScales are scalings for each exposure, ccdScale is final scale for combined image) @param outputId Data identifier for combined image (exposure part only) @return binned calib image
Reimplemented in lsst.pipe.drivers.constructCalibs.SkyTask.
Definition at line 726 of file constructCalibs.py.
|
inherited |
|
inherited |
Get fully-qualified output data identifier We may need to look up keys that aren't in the output dataId. @param ccdName Name tuple for CCD @param butler Data butler @param outputId Data identifier for combined image (exposure part only) @return fully-qualified output dataId
Definition at line 710 of file constructCalibs.py.
|
inherited |
Determine the Modified Julian Date (MJD; in TAI) from a data identifier
Definition at line 531 of file constructCalibs.py.
|
inherited |
Generate the data identifier for the output calib.
The mean date and the common filter are included, using keywords from the configuration. The CCD-specific part is not included in the data identifier. @param expRefList List of data references at exposure level @param calibId Data identifier elements for the calib provided by the user @return data identifier
Definition at line 496 of file constructCalibs.py.
|
inherited |
Interpolate over NANs in the combined image NANs can result from masked areas on the CCD. We don't want them getting into our science images, so we replace them with the median of the image.
Definition at line 847 of file constructCalibs.py.
|
inherited |
Provide a context manager for logging an operation.
@param operation: description of operation (string) @param catch: Catch all exceptions? @param trace: Log a traceback of caught exception? Note that if 'catch' is True, all exceptions are swallowed, but there may be other side-effects such as undefined variables.
Definition at line 502 of file parallel.py.
|
inherited |
Create and write an image of the entire camera.
This is useful for judging the quality or getting an overview of the features of the calib. @param camera Camera object @param dataId Data identifier for output @param calibs Dict mapping CCD detector ID to calib image
Definition at line 874 of file constructCalibs.py.
|
inherited |
Run with a MPI process pool
Definition at line 534 of file parallel.py.
|
inherited |
Definition at line 435 of file parallel.py.
|
inherited |
Process a CCD, specified by a data identifier.
After processing, optionally returns a result (produced by the 'processResult' method) calculated from the processed exposure. These results will be gathered by the master node, and is a means for coordinated scaling of all CCDs for flats, etc. Only slave nodes execute this method. @param cache Process pool cache @param ccdId Data identifier for CCD @param outputName Output dataset name for butler @return result from 'processResult'
Definition at line 602 of file constructCalibs.py.
|
inherited |
Extract processing results from a processed exposure This method generates what is gathered by the master node. This can be a background measurement or similar for scaling flat-fields. It must be picklable! Only slave nodes execute this method.
Reimplemented in lsst.pipe.drivers.constructCalibs.FlatTask.
Definition at line 660 of file constructCalibs.py.
def lsst.pipe.drivers.constructCalibs.FringeTask.processSingle | ( | self, | |
sensorRef | |||
) |
Subtract the background and normalise by the background level
Reimplemented from lsst.pipe.drivers.constructCalibs.CalibTask.
Definition at line 1151 of file constructCalibs.py.
|
inherited |
Write the processed CCD.
We need to write these out because we can't hold them all in memory at once. Only slave nodes execute this method. @param dataRef Data reference @param exposure CCD exposure to write @param outputName Output dataset name for butler.
Definition at line 646 of file constructCalibs.py.
|
inherited |
Record metadata including the inputs and creation details.
This metadata will go into the FITS header. @param butler Data butler @param calib Combined calib exposure. @param dataIdList List of data identifiers for calibration inputs @param outputId Data identifier for output
Definition at line 819 of file constructCalibs.py.
|
inherited |
Construct a calib from a list of exposure references.
This is the entry point, called by the TaskRunner.__call__ Only the master node executes this method. @param expRefList List of data references at the exposure level @param butler Data butler @param calibId Identifier dict for calib
Definition at line 422 of file constructCalibs.py.
|
inherited |
Determine scaling across CCDs and exposures.
This is necessary mainly for flats, so as to determine a consistent scaling across the entire focal plane. This implementation is simply a placeholder. Only the master node executes this method. @param ccdIdLists Dict of data identifier lists for each CCD tuple @param data Dict of lists of returned data for each CCD tuple @return dict of Struct(ccdScale: scaling for CCD, expScales: scaling for each exposure ) for each CCD tuple
Reimplemented in lsst.pipe.drivers.constructCalibs.FlatTask.
Definition at line 671 of file constructCalibs.py.
|
inherited |
Scatter the combination of exposures across multiple nodes.
In this case, we can only scatter across as many nodes as there are CCDs. Only the master node executes this method. @param pool Process pool @param outputId Output identifier (exposure part only) @param ccdIdLists Dict of data identifier lists for each CCD name @param scales Dict of structs with scales, for each CCD name @param dict of binned images
Definition at line 690 of file constructCalibs.py.
|
inherited |
Scatter the processing among the nodes.
We scatter each CCD independently (exposures aren't grouped together), to make full use of all available processors. This necessitates piecing everything back together in the same format as ccdIdLists afterwards. Only the master node executes this method. @param pool Process pool @param ccdIdLists Dict of data identifier lists for each CCD name @return Dict of lists of returned data for each CCD name
Reimplemented in lsst.pipe.drivers.constructCalibs.SkyTask.
Definition at line 586 of file constructCalibs.py.
|
inherited |
Update the metadata from the VisitInfo.
@param calibImage The image whose metadata is to be set @param exposureTime The exposure time for the image @param darkTime The time since the last read (default: exposureTime)
Definition at line 570 of file constructCalibs.py.
|
inherited |
Write the final combined calib.
Only the slave nodes execute this method @param butler Data butler @param exposure CCD exposure to write @param dataId Data identifier for output
Definition at line 862 of file constructCalibs.py.
|
static |
Definition at line 1138 of file constructCalibs.py.
|
static |
Definition at line 1136 of file constructCalibs.py.
|
staticinherited |
Definition at line 401 of file constructCalibs.py.
|
staticinherited |
Definition at line 399 of file constructCalibs.py.
|
staticinherited |
Definition at line 398 of file constructCalibs.py.