lsst.daf.persistence.butlerSubset.ButlerDataRef Class Reference

Public Member Functions
def	__init__ (self, butlerSubset, dataId)
def	__repr__ (self)
def	get (self, datasetType=None, **rest)
def	put (self, obj, datasetType=None, doBackup=False, **rest)
def	getUri (self, datasetType=None, write=False, **rest)
def	subLevels (self)
def	subItems (self, level=None)
def	datasetExists (self, datasetType=None, write=False, **rest)
def	getButler (self)

Public Attributes
	butlerSubset
	dataId

Static Public Attributes
int	GENERATION = 2

Detailed Description

A ButlerDataRef is a reference to a potential dataset or group of datasets
that is portable between compatible dataset types.  As such, it can be
used to create or retrieve datasets.

ButlerDataRefs are (conceptually) created as elements of a ButlerSubset by
Butler.subset().  They are initially specific to the dataset type passed
to that call, but they may be used with any other compatible dataset type.
Dataset type compatibility must be determined externally (or by trial and
error).

ButlerDataRefs may be created at any level of a data identifier hierarchy.
If the level is not one at which datasets exist, a ButlerSubset
with lower-level ButlerDataRefs can be created using
ButlerDataRef.subItems().

Public methods:

get(self, datasetType=None, **rest)

put(self, obj, datasetType=None, **rest)

subItems(self, level=None)

datasetExists(self, datasetType=None, **rest)

getButler(self)

Definition at line 146 of file butlerSubset.py.

Constructor & Destructor Documentation

__init__()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.__init__	(		self,
			butlerSubset,
			dataId
	)

For internal use only.  ButlerDataRefs should only be created by
ButlerSubset and ButlerSubsetIterator.

Definition at line 180 of file butlerSubset.py.

    def __init__(self, butlerSubset, dataId):
        """
        For internal use only.  ButlerDataRefs should only be created by
        ButlerSubset and ButlerSubsetIterator.
        """
 
        self.butlerSubset = butlerSubset
        self.dataId = dataId
 

Member Function Documentation

__repr__()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.__repr__

(

self

)

Definition at line 189 of file butlerSubset.py.

    def __repr__(self):
        return 'ButlerDataRef(butlerSubset=%s, dataId=%s)' % (self.butlerSubset, self.dataId)
 

datasetExists()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.datasetExists	(		self,
			datasetType = None,
			write = False,
		**	rest
	)

Determine if a dataset exists of the given type (or the type used when
creating the ButlerSubset, if None) as specified by the ButlerDataRef.

@param datasetType (str) dataset type to check.
@param write (bool) if True, search only in output repositories
@param **rest            keywords arguments with data identifiers
@returns bool

Definition at line 301 of file butlerSubset.py.

    def datasetExists(self, datasetType=None, write=False, **rest):
        """
        Determine if a dataset exists of the given type (or the type used when
        creating the ButlerSubset, if None) as specified by the ButlerDataRef.
 
        @param datasetType (str) dataset type to check.
        @param write (bool) if True, search only in output repositories
        @param **rest            keywords arguments with data identifiers
        @returns bool
        """
        if datasetType is None:
            datasetType = self.butlerSubset.datasetType
        return self.butlerSubset.butler.datasetExists(
            datasetType, self.dataId, write=write, **rest)
 

get()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.get	(		self,
			datasetType = None,
		**	rest
	)

Retrieve a dataset of the given type (or the type used when creating
the ButlerSubset, if None) as specified by the ButlerDataRef.

@param datasetType (str)  dataset type to retrieve.
@param **rest             keyword arguments with data identifiers
@returns object corresponding to the given dataset type.

Definition at line 192 of file butlerSubset.py.

    def get(self, datasetType=None, **rest):
        """
        Retrieve a dataset of the given type (or the type used when creating
        the ButlerSubset, if None) as specified by the ButlerDataRef.
 
        @param datasetType (str)  dataset type to retrieve.
        @param **rest             keyword arguments with data identifiers
        @returns object corresponding to the given dataset type.
        """
        if datasetType is None:
            datasetType = self.butlerSubset.datasetType
        return self.butlerSubset.butler.get(datasetType, self.dataId, **rest)
 

getButler()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.getButler

(

self

)

Return the butler associated with this data reference.

Definition at line 316 of file butlerSubset.py.

    def getButler(self):
        """
        Return the butler associated with this data reference.
        """
        return self.butlerSubset.butler

getUri()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.getUri	(		self,
			datasetType = None,
			write = False,
		**	rest
	)

Return the URL for a dataset

.. warning:: This is intended only for debugging. The URI should
never be used for anything other than printing.

.. note:: In the event there are multiple URIs, we return only
the first.

.. note:: getUri() does not currently support composite datasets.

Parameters
----------
datasetType : `str`, optional
   The dataset type of interest.
write : `bool`, optional
   Return the URI for writing?
rest : `dict`, optional
   Keyword arguments for the data id.

Returns
-------
uri : `str`
   URI for dataset

Definition at line 223 of file butlerSubset.py.

    def getUri(self, datasetType=None, write=False, **rest):
        """Return the URL for a dataset
 
        .. warning:: This is intended only for debugging. The URI should
        never be used for anything other than printing.
 
        .. note:: In the event there are multiple URIs, we return only
        the first.
 
        .. note:: getUri() does not currently support composite datasets.
 
        Parameters
        ----------
        datasetType : `str`, optional
           The dataset type of interest.
        write : `bool`, optional
           Return the URI for writing?
        rest : `dict`, optional
           Keyword arguments for the data id.
 
        Returns
        -------
        uri : `str`
           URI for dataset
        """
 
        if datasetType is None:
            datasetType = self.butlerSubset.datasetType
        return self.butlerSubset.butler.getUri(datasetType, self.dataId, write=write, **rest)
 

put()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.put	(		self,
			obj,
			datasetType = None,
			doBackup = False,
		**	rest
	)

Persist a dataset of the given type (or the type used when creating
the ButlerSubset, if None) as specified by the ButlerDataRef.

@param obj                object to persist.
@param datasetType (str)  dataset type to persist.
@param doBackup           if True, rename existing instead of overwriting
@param **rest             keyword arguments with data identifiers

WARNING: Setting doBackup=True is not safe for parallel processing, as it
may be subject to race conditions.

Definition at line 205 of file butlerSubset.py.

    def put(self, obj, datasetType=None, doBackup=False, **rest):
        """
        Persist a dataset of the given type (or the type used when creating
        the ButlerSubset, if None) as specified by the ButlerDataRef.
 
        @param obj                object to persist.
        @param datasetType (str)  dataset type to persist.
        @param doBackup           if True, rename existing instead of overwriting
        @param **rest             keyword arguments with data identifiers
 
        WARNING: Setting doBackup=True is not safe for parallel processing, as it
        may be subject to race conditions.
        """
 
        if datasetType is None:
            datasetType = self.butlerSubset.datasetType
        self.butlerSubset.butler.put(obj, datasetType, self.dataId, doBackup=doBackup, **rest)
 

subItems()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.subItems	(		self,
			level = None
	)

Generate a ButlerSubset at a lower level of the hierarchy than this
ButlerDataRef, using it as a partial data id.  If level is None, a
default lower level for the original ButlerSubset level and dataset
type is used.

As currently implemented, the default sublevels for all the
repositories used by this Butler instance must match for the Butler to
be able to select a default sublevel to get the subset.

@param level (str)   the hierarchy level to descend to.
@returns (ButlerSubset) resulting from the lower-level query or () if
                        there is no lower level.

Definition at line 271 of file butlerSubset.py.

    def subItems(self, level=None):
        """
        Generate a ButlerSubset at a lower level of the hierarchy than this
        ButlerDataRef, using it as a partial data id.  If level is None, a
        default lower level for the original ButlerSubset level and dataset
        type is used.
 
        As currently implemented, the default sublevels for all the
        repositories used by this Butler instance must match for the Butler to
        be able to select a default sublevel to get the subset.
 
        @param level (str)   the hierarchy level to descend to.
        @returns (ButlerSubset) resulting from the lower-level query or () if
                                there is no lower level.
        """
 
        if level is None:
            levelSet = set()
            for repoData in self.butlerSubset.butler._repos.all():
                levelSet.add(repoData.repo._mapper.getDefaultSubLevel(
                             self.butlerSubset.level))
            if len(levelSet) > 1:
                raise RuntimeError(
                    "Support for multiple levels not implemented.")
            level = levelSet.pop()
            if level is None:
                return ()
        return self.butlerSubset.butler.subset(self.butlerSubset.datasetType,
                                               level, self.dataId)
 

subLevels()

def lsst.daf.persistence.butlerSubset.ButlerDataRef.subLevels

(

self

)

Return a list of the lower levels of the hierarchy than this
ButlerDataRef.

@returns (iterable)  list of strings with level keys.

Definition at line 253 of file butlerSubset.py.

    def subLevels(self):
        """
        Return a list of the lower levels of the hierarchy than this
        ButlerDataRef.
 
        @returns (iterable)  list of strings with level keys."""
 
        return set(
            self.butlerSubset.butler.getKeys(
                self.butlerSubset.datasetType,
                tag=self.butlerSubset.dataId.tag).keys()
        ) - set(
            self.butlerSubset.butler.getKeys(
                self.butlerSubset.datasetType,
                self.butlerSubset.level,
                tag=self.butlerSubset.dataId.tag).keys()
        )
 

Member Data Documentation

butlerSubset

lsst.daf.persistence.butlerSubset.ButlerDataRef.butlerSubset

Definition at line 186 of file butlerSubset.py.

dataId

lsst.daf.persistence.butlerSubset.ButlerDataRef.dataId

Definition at line 187 of file butlerSubset.py.

GENERATION

int lsst.daf.persistence.butlerSubset.ButlerDataRef.GENERATION = 2

static

Definition at line 176 of file butlerSubset.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-0.7.0/Linux64/daf_persistence/22.0.1-4-g243d05b+871c1b8305/python/lsst/daf/persistence/butlerSubset.py