lsst.pipe.base.connectionTypes.PrerequisiteInput Class Reference

Inheritance diagram for lsst.pipe.base.connectionTypes.PrerequisiteInput:

Public Member Functions
def	__post_init__ (self)
def	makeDatasetType (self, DimensionUniverse universe, Optional[StorageClass] parentStorageClass=None)
def	__get__ (self, inst, klass)

Static Public Attributes
	bool
	str

Detailed Description

Class used for declaring PipelineTask prerequisite connections

Parameters
----------
name : `str`
    The default name used to identify the dataset type
storageClass : `str`
    The storage class used when (un)/persisting the dataset type
multiple : `bool`
    Indicates if this connection should expect to contain multiple objects
    of the given dataset type
dimensions : iterable of `str`
    The `lsst.daf.butler.Butler` `lsst.daf.butler.Registry` dimensions used
    to identify the dataset type identified by the specified name
deferLoad : `bool`
    Indicates that this dataset type will be loaded as a
    `lsst.daf.butler.DeferredDatasetHandle`. PipelineTasks can use this
    object to load the object at a later time.
lookupFunction: `typing.Callable`, optional
    An optional callable function that will look up PrerequisiteInputs
    using the DatasetType, registry, quantum dataId, and input collections
    passed to it. If no function is specified, the default temporal spatial
    lookup will be used.

Notes
-----
Prerequisite inputs are used for datasets that must exist in the data
repository before a pipeline including this is run; they cannot be produced
by another task in the same pipeline.

In exchange for this limitation, they have a number of advantages relative
to regular `Input` connections:

- The query used to find them then during `QuantumGraph` generation can be
  fully customized by providing a ``lookupFunction``.
- Failed searches for prerequisites during `QuantumGraph` generation will
  usually generate more helpful diagnostics than those for regular `Input`
  connections.
- The default query for prerequisite inputs relates the quantum dimensions
  directly to the dimensions of its dataset type, without being constrained
  by any of the other dimensions in the pipeline.  This allows them to be
  used for temporal calibration lookups (which regular `Input` connections
  cannot do at present) and to work around `QuantumGraph` generation
  limitations involving cases where naive spatial overlap relationships
  between dimensions are not desired (e.g. a task that wants all detectors
  in each visit for which the visit overlaps a tract, not just those where
  that detector+visit combination overlaps the tract).

Definition at line 204 of file connectionTypes.py.

Member Function Documentation

__get__()

def lsst.pipe.base.connectionTypes.BaseConnection.__get__ (   self,   inst,   klass  )

inherited

Descriptor method

This is a method used to turn a connection into a descriptor.
When a connection is added to a connection class, it is a class level
variable. This method makes accessing this connection, on the
instance of the connection class owning this connection, return a
result specialized for that instance. In the case of connections
this specifically means names specified in a config instance will
be visible instead of the default names for the connection.

Definition at line 63 of file connectionTypes.py.

    def __get__(self, inst, klass):
        """Descriptor method
 
        This is a method used to turn a connection into a descriptor.
        When a connection is added to a connection class, it is a class level
        variable. This method makes accessing this connection, on the
        instance of the connection class owning this connection, return a
        result specialized for that instance. In the case of connections
        this specifically means names specified in a config instance will
        be visible instead of the default names for the connection.
        """
        # If inst is None, this is being accessed by the class and not an
        # instance, return this connection itself
        if inst is None:
            return self
        # If no object cache exists, create one to track the instances this
        # connection has been accessed by
        if not hasattr(inst, '_connectionCache'):
            object.__setattr__(inst, '_connectionCache', {})
        # Look up an existing cached instance
        idSelf = id(self)
        if idSelf in inst._connectionCache:
            return inst._connectionCache[idSelf]
        # Accumulate the parameters that define this connection
        params = {}
        for field in dataclasses.fields(self):
            params[field.name] = getattr(self, field.name)
        # Get the name override defined by the instance of the connection class
        params['name'] = inst._nameOverrides[self.varName]
        # Return a new instance of this connection specialized with the
        # information provided by the connection class instance
        return inst._connectionCache.setdefault(idSelf, self.__class__(**params))
 

__post_init__()

def lsst.pipe.base.connectionTypes.DimensionedConnection.__post_init__ (   self )

inherited

Definition at line 144 of file connectionTypes.py.

    def __post_init__(self):
        if isinstance(self.dimensions, str):
            raise TypeError("Dimensions must be iterable of dimensions, got str,"
                            "possibly omitted trailing comma")
        if not isinstance(self.dimensions, typing.Iterable):
            raise TypeError("Dimensions must be iterable of dimensions")
 

makeDatasetType()

def lsst.pipe.base.connectionTypes.DimensionedConnection.makeDatasetType (   self, DimensionUniverse  universe, Optional[StorageClass]   parentStorageClass = None  )

inherited

Construct a true `DatasetType` instance with normalized dimensions.

Parameters
----------
universe : `lsst.daf.butler.DimensionUniverse`
    Set of all known dimensions to be used to normalize the dimension
    names specified in config.
parentStorageClass : `lsst.daf.butler.StorageClass`, optional
    Parent storage class for component datasets; `None` otherwise.

Returns
-------
datasetType : `DatasetType`
    The `DatasetType` defined by this connection.

Reimplemented from lsst.pipe.base.connectionTypes.BaseConnection.

Definition at line 151 of file connectionTypes.py.

                        parentStorageClass: Optional[StorageClass] = None):
        """Construct a true `DatasetType` instance with normalized dimensions.
 
        Parameters
        ----------
        universe : `lsst.daf.butler.DimensionUniverse`
            Set of all known dimensions to be used to normalize the dimension
            names specified in config.
        parentStorageClass : `lsst.daf.butler.StorageClass`, optional
            Parent storage class for component datasets; `None` otherwise.
 
        Returns
        -------
        datasetType : `DatasetType`
            The `DatasetType` defined by this connection.
        """
        return DatasetType(self.name,
                           universe.extract(self.dimensions),
                           self.storageClass, isCalibration=self.isCalibration,
                           parentStorageClass=parentStorageClass)
 
 
@dataclasses.dataclass(frozen=True)

Member Data Documentation

bool

lsst.pipe.base.connectionTypes.BaseInput.bool

staticinherited

Definition at line 195 of file connectionTypes.py.

str

lsst.pipe.base.connectionTypes.BaseConnection.str

staticinherited

Definition at line 60 of file connectionTypes.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-0.4.3/Linux64/pipe_base/22.0.1+94e66cc9ed/python/lsst/pipe/base/connectionTypes.py