lsst.obs.base.mapping.CalibrationMapping Class Reference

Inheritance diagram for lsst.obs.base.mapping.CalibrationMapping:

Public Member Functions
def	__init__ (self, datasetType, policy, registry, calibRegistry, calibRoot, dataRoot=None, **kwargs)
def	map (self, mapper, dataId, write=False)
def	lookup (self, properties, dataId)
def	standardize (self, mapper, item, dataId)
def	template (self)
def	keys (self)
def	have (self, properties, dataId)
def	need (self, properties, dataId)

Public Attributes
	reference
	refCols
	refRegistry
	dataRoot
	range
	columns
	setFilter
	metadataKeys
	datasetType
	registry
	rootStorage
	keyDict
	python
	persistable
	storage
	level
	tables
	obsTimeName
	recipe

Detailed Description

CalibrationMapping is a Mapping subclass for calibration-type products.

The difference is that data properties in the query or template
can be looked up using a reference Mapping in addition to this one.

CalibrationMapping Policies can contain the following:

reference (string, optional)
    a list of tables for finding missing dataset
    identifier components (including the observation time, if a validity
    range is required) in the exposure registry; note that the "tables"
    entry refers to the calibration registry

refCols (string, optional)
    a list of dataset properties required from the
    reference tables for lookups in the calibration registry

validRange (bool)
    true if the calibration dataset has a validity range
    specified by a column in the tables of the reference dataset in the
    exposure registry) and two columns in the tables of this calibration
    dataset in the calibration registry)

obsTimeName (string, optional)
    the name of the column in the reference
    dataset tables containing the observation time (default "taiObs")

validStartName (string, optional)
    the name of the column in the
    calibration dataset tables containing the start of the validity range
    (default "validStart")

validEndName (string, optional)
    the name of the column in the
    calibration dataset tables containing the end of the validity range
    (default "validEnd")

Parameters
----------
datasetType : `str`
    Butler dataset type to be mapped.
policy : `daf_persistence.Policy`
    Mapping Policy.
registry : `lsst.obs.base.Registry`
    Registry for metadata lookups
calibRegistry : `lsst.obs.base.Registry`
    Registry for calibration metadata lookups.
calibRoot : `str`
    Path of calibration root directory.
dataRoot : `str`
    Path of data root directory; used for outputs only.

Definition at line 389 of file mapping.py.

Constructor & Destructor Documentation

__init__()

def lsst.obs.base.mapping.CalibrationMapping.__init__	(		self,
			datasetType,
			policy,
			registry,
			calibRegistry,
			calibRoot,
			dataRoot = None,
		**	kwargs
	)

Definition at line 443 of file mapping.py.

    def __init__(self, datasetType, policy, registry, calibRegistry, calibRoot, dataRoot=None, **kwargs):
        Mapping.__init__(self, datasetType, policy, calibRegistry, calibRoot, **kwargs)
        self.reference = policy.asArray("reference") if "reference" in policy else None
        self.refCols = policy.asArray("refCols") if "refCols" in policy else None
        self.refRegistry = registry
        self.dataRoot = dataRoot
        if "validRange" in policy and policy["validRange"]:
            self.range = ("?", policy["validStartName"], policy["validEndName"])
        if "columns" in policy:
            self.columns = policy.asArray("columns")
        if "filter" in policy:
            self.setFilter = policy["filter"]
        self.metadataKeys = None
        if "metadataKey" in policy:
            self.metadataKeys = policy.asArray("metadataKey")
 

Member Function Documentation

have()

def lsst.obs.base.mapping.Mapping.have (   self,   properties,   dataId  )

inherited

Returns whether the provided data identifier has all
the properties in the provided list.

Parameters
----------
properties : `list of `str`
    Properties required.
dataId : `dict`
    Dataset identifier.

Returns
-------
bool
    True if all properties are present.

Definition at line 275 of file mapping.py.

    def have(self, properties, dataId):
        """Returns whether the provided data identifier has all
        the properties in the provided list.
 
        Parameters
        ----------
        properties : `list of `str`
            Properties required.
        dataId : `dict`
            Dataset identifier.
 
        Returns
        -------
        bool
            True if all properties are present.
        """
        for prop in properties:
            if prop not in dataId:
                return False
        return True
 

keys()

def lsst.obs.base.mapping.Mapping.keys (   self )

inherited

Return the dict of keys and value types required for this mapping.

Definition at line 135 of file mapping.py.

    def keys(self):
        """Return the dict of keys and value types required for this mapping.
        """
        return self.keyDict
 

lookup()

def lsst.obs.base.mapping.CalibrationMapping.lookup	(		self,
			properties,
			dataId
	)

Look up properties for in a metadata registry given a partial
dataset identifier.

Parameters
----------
properties : `list` of `str`
    Properties to look up.
dataId : `dict`
    Dataset identifier.

Returns
-------
`list` of `tuple`
    Values of properties.

Reimplemented from lsst.obs.base.mapping.Mapping.

Definition at line 466 of file mapping.py.

    def lookup(self, properties, dataId):
        """Look up properties for in a metadata registry given a partial
        dataset identifier.
 
        Parameters
        ----------
        properties : `list` of `str`
            Properties to look up.
        dataId : `dict`
            Dataset identifier.
 
        Returns
        -------
        `list` of `tuple`
            Values of properties.
        """
 
# Either look up taiObs in reference and then all in calibRegistry
# Or look up all in registry
 
        newId = dataId.copy()
        if self.reference is not None:
            where = []
            values = []
            for k, v in dataId.items():
                if self.refCols and k not in self.refCols:
                    continue
                where.append(k)
                values.append(v)
 
            # Columns we need from the regular registry
            if self.columns is not None:
                columns = set(self.columns)
                for k in dataId.keys():
                    columns.discard(k)
            else:
                columns = set(properties)
 
            if not columns:
                # Nothing to lookup in reference registry; continue with calib
                # registry
                return Mapping.lookup(self, properties, newId)
 
            lookupDataId = dict(zip(where, values))
            lookups = self.refRegistry.lookup(columns, self.reference, lookupDataId)
            if len(lookups) != 1:
                raise RuntimeError("No unique lookup for %s from %s: %d matches" %
                                   (columns, dataId, len(lookups)))
            if columns == set(properties):
                # Have everything we need
                return lookups
            for i, prop in enumerate(columns):
                newId[prop] = lookups[0][i]
        return Mapping.lookup(self, properties, newId)
 

map()

def lsst.obs.base.mapping.CalibrationMapping.map	(		self,
			mapper,
			dataId,
			write = False
	)

Standard implementation of map function.

Parameters
----------
mapper: `lsst.daf.persistence.Mapper`
    Object to be mapped.
dataId: `dict`
    Dataset identifier.

Returns
-------
lsst.daf.persistence.ButlerLocation
    Location of object that was mapped.

Reimplemented from lsst.obs.base.mapping.Mapping.

Definition at line 459 of file mapping.py.

    def map(self, mapper, dataId, write=False):
        location = Mapping.map(self, mapper, dataId, write=write)
        # Want outputs to be in the output directory
        if write and self.dataRoot:
            location.storage = self.dataRoot
        return location
 

need()

def lsst.obs.base.mapping.Mapping.need (   self,   properties,   dataId  )

inherited

Ensures all properties in the provided list are present in
the data identifier, looking them up as needed.  This is only
possible for the case where the data identifies a single
exposure.

Parameters
----------
properties : `list` of `str`
    Properties required.
dataId : `dict`
    Partial dataset identifier

Returns
-------
`dict`
    Copy of dataset identifier with enhanced values.

Definition at line 296 of file mapping.py.

    def need(self, properties, dataId):
        """Ensures all properties in the provided list are present in
        the data identifier, looking them up as needed.  This is only
        possible for the case where the data identifies a single
        exposure.
 
        Parameters
        ----------
        properties : `list` of `str`
            Properties required.
        dataId : `dict`
            Partial dataset identifier
 
        Returns
        -------
        `dict`
            Copy of dataset identifier with enhanced values.
        """
        newId = dataId.copy()
        newProps = []                    # Properties we don't already have
        for prop in properties:
            if prop not in newId:
                newProps.append(prop)
        if len(newProps) == 0:
            return newId
 
        lookups = self.lookup(newProps, newId)
        if len(lookups) != 1:
            raise NoResults("No unique lookup for %s from %s: %d matches" %
                            (newProps, newId, len(lookups)),
                            self.datasetType, dataId)
        for i, prop in enumerate(newProps):
            newId[prop] = lookups[0][i]
        return newId
 
 

standardize()

def lsst.obs.base.mapping.CalibrationMapping.standardize	(		self,
			mapper,
			item,
			dataId
	)

Default standardization function for calibration datasets.

If the item is of a type that should be standardized, the base class
``standardizeExposure`` method is called, otherwise the item is
returned unmodified.

Parameters
----------
mapping : `lsst.obs.base.Mapping`
    Mapping object to pass through.
item : object
    Will be standardized if of type lsst.afw.image.Exposure,
    lsst.afw.image.DecoratedImage, lsst.afw.image.Image
    or lsst.afw.image.MaskedImage

dataId : `dict`
    Dataset identifier

Returns
-------
`lsst.afw.image.Exposure` or item
    The standardized object.

Definition at line 521 of file mapping.py.

    def standardize(self, mapper, item, dataId):
        """Default standardization function for calibration datasets.
 
        If the item is of a type that should be standardized, the base class
        ``standardizeExposure`` method is called, otherwise the item is
        returned unmodified.
 
        Parameters
        ----------
        mapping : `lsst.obs.base.Mapping`
            Mapping object to pass through.
        item : object
            Will be standardized if of type lsst.afw.image.Exposure,
            lsst.afw.image.DecoratedImage, lsst.afw.image.Image
            or lsst.afw.image.MaskedImage
 
        dataId : `dict`
            Dataset identifier
 
        Returns
        -------
        `lsst.afw.image.Exposure` or item
            The standardized object.
        """
        if issubclass(doImport(self.python), (Exposure, MaskedImage, Image, DecoratedImage)):
            return mapper._standardizeExposure(self, item, dataId, filter=self.setFilter)
        return item
 
 

template()

def lsst.obs.base.mapping.Mapping.template (   self )

inherited

Definition at line 128 of file mapping.py.

    def template(self):
        if self._template:  # template must not be an empty string or None
            return self._template
        else:
            raise RuntimeError(f"Template is not defined for the {self.datasetType} dataset type, "
                               "it must be set before it can be used.")
 

Member Data Documentation

columns

lsst.obs.base.mapping.CalibrationMapping.columns

Definition at line 452 of file mapping.py.

dataRoot

lsst.obs.base.mapping.CalibrationMapping.dataRoot

Definition at line 448 of file mapping.py.

datasetType

lsst.obs.base.mapping.Mapping.datasetType

inherited

Definition at line 89 of file mapping.py.

keyDict

lsst.obs.base.mapping.Mapping.keyDict

inherited

Definition at line 102 of file mapping.py.

level

lsst.obs.base.mapping.Mapping.level

inherited

Definition at line 117 of file mapping.py.

metadataKeys

lsst.obs.base.mapping.CalibrationMapping.metadataKeys

Definition at line 455 of file mapping.py.

obsTimeName

lsst.obs.base.mapping.Mapping.obsTimeName

inherited

Definition at line 124 of file mapping.py.

persistable

lsst.obs.base.mapping.Mapping.persistable

inherited

Definition at line 114 of file mapping.py.

python

lsst.obs.base.mapping.Mapping.python

inherited

Definition at line 113 of file mapping.py.

range

lsst.obs.base.mapping.CalibrationMapping.range

Definition at line 450 of file mapping.py.

recipe

lsst.obs.base.mapping.Mapping.recipe

inherited

Definition at line 125 of file mapping.py.

refCols

lsst.obs.base.mapping.CalibrationMapping.refCols

Definition at line 446 of file mapping.py.

reference

lsst.obs.base.mapping.CalibrationMapping.reference

Definition at line 445 of file mapping.py.

refRegistry

lsst.obs.base.mapping.CalibrationMapping.refRegistry

Definition at line 447 of file mapping.py.

registry

lsst.obs.base.mapping.Mapping.registry

inherited

Definition at line 90 of file mapping.py.

rootStorage

lsst.obs.base.mapping.Mapping.rootStorage

inherited

Definition at line 91 of file mapping.py.

setFilter

lsst.obs.base.mapping.CalibrationMapping.setFilter

Definition at line 454 of file mapping.py.

storage

lsst.obs.base.mapping.Mapping.storage

inherited

Definition at line 115 of file mapping.py.

tables

lsst.obs.base.mapping.Mapping.tables

inherited

Definition at line 119 of file mapping.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/0.2.1/Linux64/obs_base/21.0.0-27-gbbd0d29+ae871e0f33/python/lsst/obs/base/mapping.py