lsst.obs.base.gen2to3.repoWalker.handlers.SubdirectoryHandler Class Reference

Inheritance diagram for lsst.obs.base.gen2to3.repoWalker.handlers.SubdirectoryHandler:

Public Member Functions
def	__init__ (self, PathElementParser parser)
bool	isForFiles (self)
def	handle (self, str path, nextDataId2, Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]] datasets, *Callable[[DataCoordinate], bool] predicate)
Tuple[Optional[DataCoordinate], Optional[str]]	translate (self, dict dataId2, *bool partial=False)
def	__str__ (self)
bool	__call__ (self, str path, str name, Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]] datasets, *Callable[[DataCoordinate], bool] predicate)
int	rank (self)
def	handle (self, str path, dict nextDataId2, Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]] datasets, *Callable[[DataCoordinate], bool] predicate)
def	__lt__ (self, PathElementHandler other)

Public Attributes
	scanner
	lastDataId2

Detailed Description

A `PathElementHandler` that uses a `DirectoryScanner` to recurse.

Parameters
----------
parser : `PathElementParser`
    An object that matches the path element this handler is responsible for
    and extracts a (partial) Gen2 data ID from it.

Notes
-----
The nested `DirectoryScanner` is default-constructed and should be
populated with child handlers after the `SubdirectoryHandler` is created.

Definition at line 201 of file handlers.py.

Constructor & Destructor Documentation

__init__()

def lsst.obs.base.gen2to3.repoWalker.handlers.SubdirectoryHandler.__init__	(		self,
		PathElementParser	parser
	)

Reimplemented from lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.

Definition at line 216 of file handlers.py.

    def __init__(self, parser: PathElementParser):
        super().__init__(parser=parser)
        self.scanner = DirectoryScanner()
 

Member Function Documentation

__call__()

bool lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.__call__ (   self, str  path, str  name, Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]]  datasets, *Callable[[DataCoordinate], bool]  predicate  )

inherited

Apply the handler to a file path.

Parameters
----------
path : `str`
    Full path of the file or directory.
name : `str`
    Local name of the file or directory within its parent directory.
datasets : `dict` [`DatasetType`, `dict` ]
    Dictionary that found datasets should be added to.  Nested dicts
    are keyed by either `None` (for most datasets) or a `str`
    "CALIBDATE" for calibration datasets.
predicate : `~collections.abc.Callable`
    A callable taking a single `DataCoordinate` argument and returning
    `bool`, indicating whether that (Gen3) data ID represents one
    that should be included in the scan.'

Returns
-------
matched : `bool`
    `True` if this handler was a match for the given path and no other
    handlers need to be tried on it, `False` otherwise.

Reimplemented from lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.

Definition at line 120 of file handlers.py.

                 predicate: Callable[[DataCoordinate], bool]) -> bool:
        # Docstring inherited from PathElementParser.
        nextDataId2 = self._parser.parse(name, self.lastDataId2)
        if nextDataId2 is None:
            return False
        self.handle(path, nextDataId2, datasets, predicate=predicate)
        return True
 

__lt__()

def lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.__lt__ (   self, PathElementHandler  other  )

inherited

Handlers are sorted by rank to reduce the possibility that more
flexible handlers will have a chance to match something they shouldn't.

Definition at line 140 of file scanner.py.

    def __lt__(self, other: PathElementHandler):
        """Handlers are sorted by rank to reduce the possibility that more
        flexible handlers will have a chance to match something they shouldn't.
        """
        return self.rank < other.rank
 

__str__()

def lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.__str__ (   self )

inherited

Reimplemented in lsst.obs.base.gen2to3.repoWalker.handlers.TargetFileHandler.

Definition at line 117 of file handlers.py.

    def __str__(self):
        return f"{type(self).__name__}(parser={self._parser})"
 

handle() [1/2]

def lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.handle (   self, str  path, dict  nextDataId2, Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]]  datasets, *Callable[[DataCoordinate], bool]  predicate  )

inherited

Customization hook for ``__call__``.

Subclasses must override this method, while external callers (i.e.
`DirectoryScanner` should instead invoke `__call__`.

Parameters
----------
path : `str`
    Full path of the file or directory.
nextDataId2 : `dict`
    Gen2 data ID (usually partial) extracted from the path so far.
datasets : `dict` [`DatasetType`, `list` [`FileDataset`] ]
    Dictionary that found datasets should be added to.
predicate : `~collections.abc.Callable`
    A callable taking a single `DataCoordinate` argument and returning
    `bool`, indicating whether that (Gen3) data ID represents one
    that should be included in the scan.
formatterMap : `dict`, optional
    Map dataset type to specialist formatter.

Reimplemented in lsst.obs.base.gen2to3.repoWalker.handlers.SkipHandler.

Definition at line 136 of file handlers.py.

               predicate: Callable[[DataCoordinate], bool]):
        """Customization hook for ``__call__``.
 
        Subclasses must override this method, while external callers (i.e.
        `DirectoryScanner` should instead invoke `__call__`.
 
        Parameters
        ----------
        path : `str`
            Full path of the file or directory.
        nextDataId2 : `dict`
            Gen2 data ID (usually partial) extracted from the path so far.
        datasets : `dict` [`DatasetType`, `list` [`FileDataset`] ]
            Dictionary that found datasets should be added to.
        predicate : `~collections.abc.Callable`
            A callable taking a single `DataCoordinate` argument and returning
            `bool`, indicating whether that (Gen3) data ID represents one
            that should be included in the scan.
        formatterMap : `dict`, optional
            Map dataset type to specialist formatter.
        """
        raise NotImplementedError()
 
 

handle() [2/2]

def lsst.obs.base.gen2to3.repoWalker.handlers.SubdirectoryHandler.handle	(		self,
		str	path,
			nextDataId2,
		Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]]	datasets,
		*Callable[[DataCoordinate], bool]	predicate
	)

Definition at line 226 of file handlers.py.

               predicate: Callable[[DataCoordinate], bool]):
        # Docstring inherited from ParsedPathElementHandler.
        if not nextDataId2:
            # We matched, and there's no data ID at all yet.  That means the
            # full path so far is just a fixed string so we should descend
            # and the match is exclusive.
            scan = True
        else:
            dataId3, _ = self.translate(nextDataId2, partial=True)
            if dataId3 is not None:
                scan = predicate(dataId3)
            else:
                scan = True
        if scan:
            for handler in self.scanner:
                handler.lastDataId2 = nextDataId2
            self.scanner.scan(path, datasets, predicate=predicate)
 

isForFiles()

bool lsst.obs.base.gen2to3.repoWalker.handlers.SubdirectoryHandler.isForFiles

(

self

)

Report what kind of path element this object handlers.

Returns
-------
Return `True` if this handler is for file entries, or `False` if it
is for directories.

Reimplemented from lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.

Definition at line 222 of file handlers.py.

    def isForFiles(self) -> bool:
        # Docstring inherited from PathElementHandler.
        return False
 

rank()

int lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.rank (   self )

inherited

Return a rough indication of how flexible this handler is in terms
of the path element names it can match.

Handlers that match a constant path element should always return zero.

Reimplemented from lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.

Definition at line 131 of file handlers.py.

    def rank(self) -> int:
        # Docstring inherited from PathElementParser.
        return len(self._parser.keys)
 

translate()

Tuple[Optional[DataCoordinate], Optional[str]] lsst.obs.base.gen2to3.repoWalker.handlers.SubdirectoryHandler.translate	(		self,
		dict	dataId2,
		*bool	partial = False
	)

Translate the given data ID from Gen2 to Gen3.

The default implementation returns `None`.  Subclasses that are able
to translate data IDs should override this method.

Parameters
----------
dataId2 : `dict`
    Gen2 data ID.
partial : `bool`, optional
    If `True` (`False` is default) this is a partial data ID for some
    dataset, and missing keys are expected.

Returns
-------
dataId3 : `lsst.daf.butler.DataCoordinate` or `None`
    A Gen3 data ID, or `None` if this handler cannot translate data
    IDs.
calibDate : `str` or `None`
    A Gen2 calibration "CALIBDATE" value, or `None` if there was no
    such value in the template.

Reimplemented from lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.

Definition at line 246 of file handlers.py.

                  ) -> Tuple[Optional[DataCoordinate], Optional[str]]:
        # Docstring inherited from PathElementHandler.
        for handler in self.scanner:
            # Since we're recursing, we're always asking for a partial match,
            # because the data ID we have corresponds to different level than
            # the one child handlers operate at.
            result, calibDate = handler.translate(dataId2, partial=True)
            if result is not None:
                return result, calibDate
        return None, None
 

Member Data Documentation

lastDataId2

lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.lastDataId2

inherited

Definition at line 60 of file scanner.py.

scanner

lsst.obs.base.gen2to3.repoWalker.handlers.SubdirectoryHandler.scanner

Definition at line 218 of file handlers.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/0.2.1/Linux64/obs_base/21.0.0-27-gbbd0d29+ae871e0f33/python/lsst/obs/base/gen2to3/repoWalker/handlers.py