LSST Applications  21.0.0+75b29a8a7f,21.0.0+e70536a077,21.0.0-1-ga51b5d4+62c747d40b,21.0.0-10-gbfb87ad6+3307648ee3,21.0.0-15-gedb9d5423+47cba9fc36,21.0.0-2-g103fe59+fdf0863a2a,21.0.0-2-g1367e85+d38a93257c,21.0.0-2-g45278ab+e70536a077,21.0.0-2-g5242d73+d38a93257c,21.0.0-2-g7f82c8f+e682ffb718,21.0.0-2-g8dde007+d179fbfa6a,21.0.0-2-g8f08a60+9402881886,21.0.0-2-ga326454+e682ffb718,21.0.0-2-ga63a54e+08647d4b1b,21.0.0-2-gde069b7+26c92b3210,21.0.0-2-gecfae73+0445ed2f95,21.0.0-2-gfc62afb+d38a93257c,21.0.0-27-gbbd0d29+ae871e0f33,21.0.0-28-g5fc5e037+feb0e9397b,21.0.0-3-g21c7a62+f4b9c0ff5c,21.0.0-3-g357aad2+57b0bddf0b,21.0.0-3-g4be5c26+d38a93257c,21.0.0-3-g65f322c+3f454acf5d,21.0.0-3-g7d9da8d+75b29a8a7f,21.0.0-3-gaa929c8+9e4ef6332c,21.0.0-3-ge02ed75+4b120a55c4,21.0.0-4-g3300ddd+e70536a077,21.0.0-4-g591bb35+4b120a55c4,21.0.0-4-gc004bbf+4911b9cd27,21.0.0-4-gccdca77+f94adcd104,21.0.0-4-ge8fba5a+2b3a696ff9,21.0.0-5-gb155db7+2c5429117a,21.0.0-5-gdf36809+637e4641ee,21.0.0-6-g00874e7+c9fd7f7160,21.0.0-6-g4e60332+4b120a55c4,21.0.0-7-gc8ca178+40eb9cf840,21.0.0-8-gfbe0b4b+9e4ef6332c,21.0.0-9-g2fd488a+d83b7cd606,w.2021.05
LSST Data Management Base Package
Public Member Functions | Public Attributes | List of all members
lsst.obs.base.gen2to3.repoWalker.handlers.TargetFileHandler Class Reference
Inheritance diagram for lsst.obs.base.gen2to3.repoWalker.handlers.TargetFileHandler:
lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler lsst.obs.base.gen2to3.repoWalker.handlers.MultiExtensionFileHandler

Public Member Functions

def __init__ (self, PathElementParser parser, Translator translator, DatasetType datasetType, FormatterParameter formatter=None)
 
def __str__ (self)
 
bool isForFiles (self)
 
def handle (self, str path, nextDataId2, Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]] datasets, *Callable[[DataCoordinate], bool] predicate)
 
Tuple[Optional[DataCoordinate], Optional[str]] translate (self, dict dataId2, *bool partial=False)
 
bool __call__ (self, str path, str name, Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]] datasets, *Callable[[DataCoordinate], bool] predicate)
 
int rank (self)
 
def handle (self, str path, dict nextDataId2, Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]] datasets, *Callable[[DataCoordinate], bool] predicate)
 
def __lt__ (self, PathElementHandler other)
 

Public Attributes

 lastDataId2
 

Detailed Description

A `PathElementHandler` that matches files that correspond to target
datasets and outputs `FileDataset` instances for them.

Parameters
----------
parser : `PathElementParser`
    An object that matches the path element this handler is responsible for
    and extracts a (partial) Gen2 data ID from it.
translator : `Translator`
    Object that translates data IDs from Gen2 to Gen3.
datasetType : `lsst.daf.butler.DatasetType`
    Gen3 dataset type for the datasets this handler matches.
formatter : `lsst.daf.butler.Formatter` or `str`, optional
    A Gen 3 formatter class or fully-qualified name.

Definition at line 264 of file handlers.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.obs.base.gen2to3.repoWalker.handlers.TargetFileHandler.__init__ (   self,
PathElementParser  parser,
Translator  translator,
DatasetType  datasetType,
FormatterParameter   formatter = None 
)

Definition at line 280 of file handlers.py.

281  formatter: FormatterParameter = None):
282  super().__init__(parser=parser)
283  self._translator = translator
284  self._datasetType = datasetType
285  self._formatter = formatter
286 

Member Function Documentation

◆ __call__()

bool lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.__call__ (   self,
str  path,
str  name,
Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]]  datasets,
*Callable[[DataCoordinate], bool]  predicate 
)
inherited
Apply the handler to a file path.

Parameters
----------
path : `str`
    Full path of the file or directory.
name : `str`
    Local name of the file or directory within its parent directory.
datasets : `dict` [`DatasetType`, `dict` ]
    Dictionary that found datasets should be added to.  Nested dicts
    are keyed by either `None` (for most datasets) or a `str`
    "CALIBDATE" for calibration datasets.
predicate : `~collections.abc.Callable`
    A callable taking a single `DataCoordinate` argument and returning
    `bool`, indicating whether that (Gen3) data ID represents one
    that should be included in the scan.'

Returns
-------
matched : `bool`
    `True` if this handler was a match for the given path and no other
    handlers need to be tried on it, `False` otherwise.

Reimplemented from lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.

Definition at line 120 of file handlers.py.

122  predicate: Callable[[DataCoordinate], bool]) -> bool:
123  # Docstring inherited from PathElementParser.
124  nextDataId2 = self._parser.parse(name, self.lastDataId2)
125  if nextDataId2 is None:
126  return False
127  self.handle(path, nextDataId2, datasets, predicate=predicate)
128  return True
129 

◆ __lt__()

def lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.__lt__ (   self,
PathElementHandler  other 
)
inherited
Handlers are sorted by rank to reduce the possibility that more
flexible handlers will have a chance to match something they shouldn't.

Definition at line 140 of file scanner.py.

140  def __lt__(self, other: PathElementHandler):
141  """Handlers are sorted by rank to reduce the possibility that more
142  flexible handlers will have a chance to match something they shouldn't.
143  """
144  return self.rank < other.rank
145 

◆ __str__()

def lsst.obs.base.gen2to3.repoWalker.handlers.TargetFileHandler.__str__ (   self)

Reimplemented from lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.

Definition at line 289 of file handlers.py.

289  def __str__(self):
290  return f"{type(self).__name__}({self._translator}, {self._datasetType})"
291 

◆ handle() [1/2]

def lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.handle (   self,
str  path,
dict  nextDataId2,
Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]]  datasets,
*Callable[[DataCoordinate], bool]  predicate 
)
inherited
Customization hook for ``__call__``.

Subclasses must override this method, while external callers (i.e.
`DirectoryScanner` should instead invoke `__call__`.

Parameters
----------
path : `str`
    Full path of the file or directory.
nextDataId2 : `dict`
    Gen2 data ID (usually partial) extracted from the path so far.
datasets : `dict` [`DatasetType`, `list` [`FileDataset`] ]
    Dictionary that found datasets should be added to.
predicate : `~collections.abc.Callable`
    A callable taking a single `DataCoordinate` argument and returning
    `bool`, indicating whether that (Gen3) data ID represents one
    that should be included in the scan.
formatterMap : `dict`, optional
    Map dataset type to specialist formatter.

Reimplemented in lsst.obs.base.gen2to3.repoWalker.handlers.SkipHandler.

Definition at line 136 of file handlers.py.

138  predicate: Callable[[DataCoordinate], bool]):
139  """Customization hook for ``__call__``.
140 
141  Subclasses must override this method, while external callers (i.e.
142  `DirectoryScanner` should instead invoke `__call__`.
143 
144  Parameters
145  ----------
146  path : `str`
147  Full path of the file or directory.
148  nextDataId2 : `dict`
149  Gen2 data ID (usually partial) extracted from the path so far.
150  datasets : `dict` [`DatasetType`, `list` [`FileDataset`] ]
151  Dictionary that found datasets should be added to.
152  predicate : `~collections.abc.Callable`
153  A callable taking a single `DataCoordinate` argument and returning
154  `bool`, indicating whether that (Gen3) data ID represents one
155  that should be included in the scan.
156  formatterMap : `dict`, optional
157  Map dataset type to specialist formatter.
158  """
159  raise NotImplementedError()
160 
161 

◆ handle() [2/2]

def lsst.obs.base.gen2to3.repoWalker.handlers.TargetFileHandler.handle (   self,
str  path,
  nextDataId2,
Mapping[DatasetType, Mapping[Optional[str], List[FileDataset]]]  datasets,
*Callable[[DataCoordinate], bool]  predicate 
)

Reimplemented in lsst.obs.base.gen2to3.repoWalker.handlers.MultiExtensionFileHandler.

Definition at line 296 of file handlers.py.

298  predicate: Callable[[DataCoordinate], bool]):
299  # Docstring inherited from ParsedPathElementHandler.
300  dataId3, calibDate = self.translate(nextDataId2, partial=False)
301  if predicate(dataId3):
302  datasets[self._datasetType][calibDate].append(
303  FileDataset(
304  refs=[DatasetRef(self._datasetType, dataId3)],
305  path=path, formatter=self._formatter
306  )
307  )
308 
std::shared_ptr< FrameSet > append(FrameSet const &first, FrameSet const &second)
Construct a FrameSet that performs two transformations in series.
Definition: functional.cc:33

◆ isForFiles()

bool lsst.obs.base.gen2to3.repoWalker.handlers.TargetFileHandler.isForFiles (   self)
Report what kind of path element this object handlers.

Returns
-------
Return `True` if this handler is for file entries, or `False` if it
is for directories.

Reimplemented from lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.

Definition at line 292 of file handlers.py.

292  def isForFiles(self) -> bool:
293  # Docstring inherited from PathElementHandler.
294  return True
295 

◆ rank()

int lsst.obs.base.gen2to3.repoWalker.handlers.ParsedPathElementHandler.rank (   self)
inherited
Return a rough indication of how flexible this handler is in terms
of the path element names it can match.

Handlers that match a constant path element should always return zero.

Reimplemented from lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.

Definition at line 131 of file handlers.py.

131  def rank(self) -> int:
132  # Docstring inherited from PathElementParser.
133  return len(self._parser.keys)
134 

◆ translate()

Tuple[Optional[DataCoordinate], Optional[str]] lsst.obs.base.gen2to3.repoWalker.handlers.TargetFileHandler.translate (   self,
dict  dataId2,
*bool   partial = False 
)
Translate the given data ID from Gen2 to Gen3.

The default implementation returns `None`.  Subclasses that are able
to translate data IDs should override this method.

Parameters
----------
dataId2 : `dict`
    Gen2 data ID.
partial : `bool`, optional
    If `True` (`False` is default) this is a partial data ID for some
    dataset, and missing keys are expected.

Returns
-------
dataId3 : `lsst.daf.butler.DataCoordinate` or `None`
    A Gen3 data ID, or `None` if this handler cannot translate data
    IDs.
calibDate : `str` or `None`
    A Gen2 calibration "CALIBDATE" value, or `None` if there was no
    such value in the template.

Reimplemented from lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.

Reimplemented in lsst.obs.base.gen2to3.repoWalker.handlers.MultiExtensionFileHandler.

Definition at line 309 of file handlers.py.

310  ) -> Tuple[Optional[DataCoordinate], Optional[str]]:
311  # Docstring inherited from PathElementHandler.
312  rawDataId3, calibDate = self._translator(dataId2, partial=partial)
313  if partial:
314  return (
315  DataCoordinate.standardize(rawDataId3, universe=self._datasetType.dimensions.universe),
316  calibDate,
317  )
318  else:
319  return (
320  DataCoordinate.standardize(rawDataId3, graph=self._datasetType.dimensions),
321  calibDate
322  )
323 
324 

Member Data Documentation

◆ lastDataId2

lsst.obs.base.gen2to3.repoWalker.scanner.PathElementHandler.lastDataId2
inherited

Definition at line 60 of file scanner.py.


The documentation for this class was generated from the following file: