LSST Applications
21.0.0+75b29a8a7f,21.0.0+e70536a077,21.0.0-1-ga51b5d4+62c747d40b,21.0.0-10-gbfb87ad6+3307648ee3,21.0.0-15-gedb9d5423+47cba9fc36,21.0.0-2-g103fe59+fdf0863a2a,21.0.0-2-g1367e85+d38a93257c,21.0.0-2-g45278ab+e70536a077,21.0.0-2-g5242d73+d38a93257c,21.0.0-2-g7f82c8f+e682ffb718,21.0.0-2-g8dde007+d179fbfa6a,21.0.0-2-g8f08a60+9402881886,21.0.0-2-ga326454+e682ffb718,21.0.0-2-ga63a54e+08647d4b1b,21.0.0-2-gde069b7+26c92b3210,21.0.0-2-gecfae73+0445ed2f95,21.0.0-2-gfc62afb+d38a93257c,21.0.0-27-gbbd0d29+ae871e0f33,21.0.0-28-g5fc5e037+feb0e9397b,21.0.0-3-g21c7a62+f4b9c0ff5c,21.0.0-3-g357aad2+57b0bddf0b,21.0.0-3-g4be5c26+d38a93257c,21.0.0-3-g65f322c+3f454acf5d,21.0.0-3-g7d9da8d+75b29a8a7f,21.0.0-3-gaa929c8+9e4ef6332c,21.0.0-3-ge02ed75+4b120a55c4,21.0.0-4-g3300ddd+e70536a077,21.0.0-4-g591bb35+4b120a55c4,21.0.0-4-gc004bbf+4911b9cd27,21.0.0-4-gccdca77+f94adcd104,21.0.0-4-ge8fba5a+2b3a696ff9,21.0.0-5-gb155db7+2c5429117a,21.0.0-5-gdf36809+637e4641ee,21.0.0-6-g00874e7+c9fd7f7160,21.0.0-6-g4e60332+4b120a55c4,21.0.0-7-gc8ca178+40eb9cf840,21.0.0-8-gfbe0b4b+9e4ef6332c,21.0.0-9-g2fd488a+d83b7cd606,w.2021.05
LSST Data Management Base Package
|
Public Member Functions | |
def | __init__ (self, policy, repositoryDir, root=None, registry=None, calibRoot=None, calibRegistry=None, provided=None, parentRegistry=None, repositoryCfg=None) |
def | backup (self, datasetType, dataId) |
def | keys (self) |
def | getKeys (self, datasetType, level) |
def | getDefaultLevel (self) |
def | getDefaultSubLevel (self, level) |
def | getCameraName (cls) |
def | getPackageName (cls) |
def | getGen3Instrument (cls) |
def | getPackageDir (cls) |
def | map_camera (self, dataId, write=False) |
def | bypass_camera (self, datasetType, pythonType, butlerLocation, dataId) |
def | map_expIdInfo (self, dataId, write=False) |
def | bypass_expIdInfo (self, datasetType, pythonType, location, dataId) |
def | std_bfKernel (self, item, dataId) |
def | std_raw (self, item, dataId) |
def | map_skypolicy (self, dataId) |
def | std_skypolicy (self, item, dataId) |
def | getRegistry (self) |
def | getImageCompressionSettings (self, datasetType, dataId) |
def | __new__ (cls, *args, **kwargs) |
def | __getstate__ (self) |
def | __setstate__ (self, state) |
def | queryMetadata (self, datasetType, format, dataId) |
def | getDatasetTypes (self) |
def | map (self, datasetType, dataId, write=False) |
def | canStandardize (self, datasetType) |
def | standardize (self, datasetType, item, dataId) |
def | validate (self, dataId) |
Static Public Member Functions | |
def | getShortCcdName (ccdName) |
Public Attributes | |
log | |
root | |
levels | |
defaultLevel | |
defaultSubLevels | |
rootStorage | |
registry | |
calibRegistry | |
keyDict | |
cameraDataLocation | |
camera | |
filters | |
makeRawVisitInfo | |
mappings | |
Static Public Attributes | |
packageName = None | |
MakeRawVisitInfoClass = MakeRawVisitInfo | |
PupilFactoryClass = afwCameraGeom.PupilFactory | |
translatorClass = None | |
CameraMapper is a base class for mappers that handle images from a camera and products derived from them. This provides an abstraction layer between the data on disk and the code. Public methods: keys, queryMetadata, getDatasetTypes, map, canStandardize, standardize Mappers for specific data sources (e.g., CFHT Megacam, LSST simulations, etc.) should inherit this class. The CameraMapper manages datasets within a "root" directory. Note that writing to a dataset present in the input root will hide the existing dataset but not overwrite it. See #2160 for design discussion. A camera is assumed to consist of one or more rafts, each composed of multiple CCDs. Each CCD is in turn composed of one or more amplifiers (amps). A camera is also assumed to have a camera geometry description (CameraGeom object) as a policy file, a filter description (Filter class static configuration) as another policy file. Information from the camera geometry and defects are inserted into all Exposure objects returned. The mapper uses one or two registries to retrieve metadata about the images. The first is a registry of all raw exposures. This must contain the time of the observation. One or more tables (or the equivalent) within the registry are used to look up data identifier components that are not specified by the user (e.g. filter) and to return results for metadata queries. The second is an optional registry of all calibration data. This should contain validity start and end entries for each calibration dataset in the same timescale as the observation time. Subclasses will typically set MakeRawVisitInfoClass and optionally the metadata translator class: MakeRawVisitInfoClass: a class variable that points to a subclass of MakeRawVisitInfo, a functor that creates an lsst.afw.image.VisitInfo from the FITS metadata of a raw image. translatorClass: The `~astro_metadata_translator.MetadataTranslator` class to use for fixing metadata values. If it is not set an attempt will be made to infer the class from ``MakeRawVisitInfoClass``, failing that the metadata fixup will try to infer the translator class from the header itself. Subclasses must provide the following methods: _extractDetectorName(self, dataId): returns the detector name for a CCD (e.g., "CFHT 21", "R:1,2 S:3,4") as used in the AFW CameraGeom class given a dataset identifier referring to that CCD or a subcomponent of it. _computeCcdExposureId(self, dataId): see below _computeCoaddExposureId(self, dataId, singleFilter): see below Subclasses may also need to override the following methods: _transformId(self, dataId): transformation of a data identifier from colloquial usage (e.g., "ccdname") to proper/actual usage (e.g., "ccd"), including making suitable for path expansion (e.g. removing commas). The default implementation does nothing. Note that this method should not modify its input parameter. getShortCcdName(self, ccdName): a static method that returns a shortened name suitable for use as a filename. The default version converts spaces to underscores. _mapActualToPath(self, template, actualId): convert a template path to an actual path, using the actual dataset identifier. The mapper's behaviors are largely specified by the policy file. See the MapperDictionary.paf for descriptions of the available items. The 'exposures', 'calibrations', and 'datasets' subpolicies configure mappings (see Mappings class). Common default mappings for all subclasses can be specified in the "policy/{images,exposures,calibrations,datasets}.yaml" files. This provides a simple way to add a product to all camera mappers. Functions to map (provide a path to the data given a dataset identifier dictionary) and standardize (convert data into some standard format or type) may be provided in the subclass as "map_{dataset type}" and "std_{dataset type}", respectively. If non-Exposure datasets cannot be retrieved using standard daf_persistence methods alone, a "bypass_{dataset type}" function may be provided in the subclass to return the dataset instead of using the "datasets" subpolicy. Implementations of map_camera and bypass_camera that should typically be sufficient are provided in this base class. Notes ----- .. todo:: Instead of auto-loading the camera at construction time, load it from the calibration registry Parameters ---------- policy : daf_persistence.Policy, Policy with per-camera defaults already merged. repositoryDir : string Policy repository for the subclassing module (obtained with getRepositoryPath() on the per-camera default dictionary). root : string, optional Path to the root directory for data. registry : string, optional Path to registry with data's metadata. calibRoot : string, optional Root directory for calibrations. calibRegistry : string, optional Path to registry with calibrations' metadata. provided : list of string, optional Keys provided by the mapper. parentRegistry : Registry subclass, optional Registry from a parent repository that may be used to look up data's metadata. repositoryCfg : daf_persistence.RepositoryCfg or None, optional The configuration information for the repository this mapper is being used with.
Definition at line 51 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.__init__ | ( | self, | |
policy, | |||
repositoryDir, | |||
root = None , |
|||
registry = None , |
|||
calibRoot = None , |
|||
calibRegistry = None , |
|||
provided = None , |
|||
parentRegistry = None , |
|||
repositoryCfg = None |
|||
) |
Definition at line 193 of file cameraMapper.py.
|
inherited |
|
inherited |
Create a new Mapper, saving arguments for pickling. This is in __new__ instead of __init__ to save the user from having to save the arguments themselves (either explicitly, or by calling the super's __init__ with all their *args,**kwargs. The resulting pickling system (of __new__, __getstate__ and __setstate__ is similar to how __reduce__ is usually used, except that we save the user from any responsibility (except when overriding __new__, but that is not common).
Definition at line 82 of file mapper.py.
|
inherited |
def lsst.obs.base.cameraMapper.CameraMapper.backup | ( | self, | |
datasetType, | |||
dataId | |||
) |
Rename any existing object with the given type and dataId. The CameraMapper implementation saves objects in a sequence of e.g.: - foo.fits - foo.fits~1 - foo.fits~2 All of the backups will be placed in the output repo, however, and will not be removed if they are found elsewhere in the _parent chain. This means that the same file will be stored twice if the previous version was found in an input repo.
Reimplemented from lsst.daf.persistence.mapper.Mapper.
Definition at line 623 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.bypass_camera | ( | self, | |
datasetType, | |||
pythonType, | |||
butlerLocation, | |||
dataId | |||
) |
Return the (preloaded) camera object.
Definition at line 775 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.bypass_expIdInfo | ( | self, | |
datasetType, | |||
pythonType, | |||
location, | |||
dataId | |||
) |
Hook to retrieve an lsst.obs.base.ExposureIdInfo for an exposure
Definition at line 793 of file cameraMapper.py.
|
inherited |
Return true if this mapper can standardize an object of the given dataset type.
def lsst.obs.base.cameraMapper.CameraMapper.getCameraName | ( | cls | ) |
Return the name of the camera that this CameraMapper is for.
Definition at line 718 of file cameraMapper.py.
|
inherited |
def lsst.obs.base.cameraMapper.CameraMapper.getDefaultLevel | ( | self | ) |
Definition at line 709 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.getDefaultSubLevel | ( | self, | |
level | |||
) |
Definition at line 712 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.getGen3Instrument | ( | cls | ) |
Return the gen3 Instrument class equivalent for this gen2 Mapper. Returns ------- instr : `type` A `~lsst.obs.base.Instrument` class.
Definition at line 736 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.getImageCompressionSettings | ( | self, | |
datasetType, | |||
dataId | |||
) |
Stuff image compression settings into a daf.base.PropertySet This goes into the ButlerLocation's "additionalData", which gets passed into the boost::persistence framework. Parameters ---------- datasetType : `str` Type of dataset for which to get the image compression settings. dataId : `dict` Dataset identifier. Returns ------- additionalData : `lsst.daf.base.PropertySet` Image compression settings.
Definition at line 1327 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.getKeys | ( | self, | |
datasetType, | |||
level | |||
) |
Return a dict of supported keys and their value types for a given dataset type at a given level of the key hierarchy. Parameters ---------- datasetType : `str` Dataset type or None for all dataset types. level : `str` or None Level or None for all levels or '' for the default level for the camera. Returns ------- `dict` Keys are strings usable in a dataset identifier, values are their value types.
Definition at line 674 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.getPackageDir | ( | cls | ) |
Return the base directory of this package
Reimplemented in lsst.obs.base.test.baseMapper.BaseMapper.
Definition at line 756 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.getPackageName | ( | cls | ) |
Return the name of the package containing this CameraMapper.
Definition at line 729 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.getRegistry | ( | self | ) |
Get the registry used by this mapper. Returns ------- Registry or None The registry used by this mapper for this mapper's repository.
Reimplemented from lsst.daf.persistence.mapper.Mapper.
Definition at line 1317 of file cameraMapper.py.
|
static |
Convert a CCD name to a form useful as a filename The default implementation converts spaces to underscores.
Definition at line 987 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.keys | ( | self | ) |
Return supported keys. Returns ------- iterable List of keys usable in a dataset identifier
Reimplemented from lsst.daf.persistence.mapper.Mapper.
Definition at line 664 of file cameraMapper.py.
|
inherited |
Map a data id using the mapping method for its dataset type. Parameters ---------- datasetType : string The datasetType to map dataId : DataId instance The dataId to use when mapping write : bool, optional Indicates if the map is being performed for a read operation (False) or a write operation (True) Returns ------- ButlerLocation or a list of ButlerLocation The location(s) found for the map operation. If write is True, a list is returned. If write is False a single ButlerLocation is returned. Raises ------ NoResults If no locaiton was found for this map operation, the derived mapper class may raise a lsst.daf.persistence.NoResults exception. Butler catches this and will look in the next Repository if there is one.
Definition at line 135 of file mapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.map_camera | ( | self, | |
dataId, | |||
write = False |
|||
) |
def lsst.obs.base.cameraMapper.CameraMapper.map_expIdInfo | ( | self, | |
dataId, | |||
write = False |
|||
) |
Definition at line 782 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.map_skypolicy | ( | self, | |
dataId | |||
) |
|
inherited |
|
inherited |
def lsst.obs.base.cameraMapper.CameraMapper.std_bfKernel | ( | self, | |
item, | |||
dataId | |||
) |
Disable standardization for bfKernel bfKernel is a calibration product that is numpy array, unlike other calibration products that are all images; all calibration images are sent through _standardizeExposure due to CalibrationMapping, but we don't want that to happen to bfKernel
Definition at line 799 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.std_raw | ( | self, | |
item, | |||
dataId | |||
) |
Standardize a raw dataset by converting it to an Exposure instead of an Image
Reimplemented in lsst.obs.decam.decamMapper.DecamMapper.
Definition at line 809 of file cameraMapper.py.
def lsst.obs.base.cameraMapper.CameraMapper.std_skypolicy | ( | self, | |
item, | |||
dataId | |||
) |
Standardize a sky policy by returning the one we use.
Definition at line 821 of file cameraMapper.py.
|
inherited |
Validate a dataId's contents. If the dataId is valid, return it. If an invalid component can be transformed into a valid one, copy the dataId, fix the component, and return the copy. Otherwise, raise an exception.
Reimplemented in lsst.obs.test.testMapper.TestMapper.
Definition at line 180 of file mapper.py.
lsst.obs.base.cameraMapper.CameraMapper.calibRegistry |
Definition at line 259 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.camera |
Definition at line 277 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.cameraDataLocation |
Definition at line 276 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.defaultLevel |
Definition at line 218 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.defaultSubLevels |
Definition at line 219 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.filters |
Definition at line 280 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.keyDict |
Definition at line 270 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.levels |
Definition at line 213 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.log |
Definition at line 199 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.makeRawVisitInfo |
Definition at line 287 of file cameraMapper.py.
|
static |
Definition at line 181 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.mappings |
Definition at line 338 of file cameraMapper.py.
|
static |
Definition at line 177 of file cameraMapper.py.
|
static |
Definition at line 184 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.registry |
Definition at line 251 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.root |
Definition at line 202 of file cameraMapper.py.
lsst.obs.base.cameraMapper.CameraMapper.rootStorage |
Definition at line 228 of file cameraMapper.py.
|
static |
Definition at line 187 of file cameraMapper.py.