lsst.obs.base.gen2to3.repoWalker.parser.PathElementParser Class Reference

Public Member Functions
def	__init__ (self, str template, Dict[str, type] allKeys, *Optional[Dict[str, type]] previousKeys=None)
def	__str__ (self)
Optional[dict]	parse (self, str name, dict lastDataId, *Optional[Log] log=None)

Public Attributes
	template
	keys
	regex

Detailed Description

An object that matches Gen2 file names and extracts Gen2 data IDs.

Parameters
----------
target : `str`
    Either a full Gen2 path template or the part of one the corresponds to
    a single path element (a subdirectory or file name).
allKeys : `dict` [`str`, `type`]
    A dictionary that provides types for all Gen2 data ID keys that are
    substituted into the given template.  Additional key-value pairs may
    be present and will be ignored.
previousKeys : `dict` [`str`, `type`], optional
    A dictionary containing key strings and types for Gen2 data ID keys
    that have been extracted from previous path elements of the same
    template.  Values for these keys must be provided via the
    ``lastDataId`` argument when calling `parse`.

Definition at line 116 of file parser.py.

Constructor & Destructor Documentation

__init__()

def lsst.obs.base.gen2to3.repoWalker.parser.PathElementParser.__init__	(		self,
		str	template,
		Dict[str, type]	allKeys,
		*Optional[Dict[str, type]]	previousKeys = None
	)

Definition at line 134 of file parser.py.

                 previousKeys: Optional[Dict[str, type]] = None):
        self.template = template
        self.keys = {}
        # For each template path element, we iterate over each %-tagged
        # substitution string.
        last = 0
        self.regex = SubstitutableRegEx()
        for match in self.TEMPLATE_RE.finditer(self.template):
            # Copy the (escaped) regular string between the last substitution
            # and this one, escaping it appropriately.
            self.regex.addRegexTerm(re.escape(self.template[last:match.start()]))
            # Pull out the data ID key from the name used in the
            # substitution string.  Use that and the substition
            # type to come up with the pattern to use in the regex.
            name = match.group("name")
            if name == "patch":
                pattern = r"\d+,\d+"
            elif match.group("type") in "id":  # integers
                pattern = r"0*\d+"
            else:
                pattern = ".+"
            # Create a new named groups for the first occurence of a key
            # within an element.
            if name not in self.keys:
                if previousKeys and name in previousKeys:
                    # Key is new to this part of the template, but it appeared
                    # in some previous part of the template.  We'll format the
                    # original template with the data ID from that previous
                    # step later.
                    start, stop = match.span()
                    self.regex.addSubstitutionTerm(self.template[start:stop])
                else:
                    # Key is new; expect to extract a data ID value from it.
                    self.regex.addRegexTerm(r"(?P<%s>%s)" % (name, pattern))
                    self.keys[name] = allKeys[name]
            else:
                # Require a match with the last group for a second
                # occurrence.
                self.regex.addRegexTerm(r"(?P=<%s>)" % name)
            # Remember the end of this match
            last = match.end()
        # Append anything remaining after the last substitution string.
        self.regex.addRegexTerm(re.escape(self.template[last:]))
        # If there are no substitutions, join and compile into a single regex
        # now.
        self.regex = self.regex.simplify()
 

Member Function Documentation

__str__()

def lsst.obs.base.gen2to3.repoWalker.parser.PathElementParser.__str__

(

self

)

Definition at line 189 of file parser.py.

    def __str__(self):
        return f"{type(self).__name__}({self.regex})"
 

parse()

Optional[dict] lsst.obs.base.gen2to3.repoWalker.parser.PathElementParser.parse	(		self,
		str	name,
		dict	lastDataId,
		*Optional[Log]	log = None
	)

Parse the path element.

Parameters
----------
name : `str`
    The path name to parse.
lastDataId : `dict`
    The cumulative Gen2 data ID obtaining by calling `parse` on parsers
    for parent directories of the same path.
log : `Log`, optional
    Log to use to report warnings and debug information.

Returns
-------
dataId : `dict` or `None`
    Gen2 data ID that combines key-value pairs obtained from this path
    with those from ``lastDataId``.  `None` if ``name`` is not matched
    by this parser.  If the keys extracted are inconsistent with those
    in ``lastDataID``, a warning is sent to ``log`` and `None` is
    returned.

Definition at line 192 of file parser.py.

    def parse(self, name: str, lastDataId: dict, *, log: Optional[Log] = None) -> Optional[dict]:
        """Parse the path element.
 
        Parameters
        ----------
        name : `str`
            The path name to parse.
        lastDataId : `dict`
            The cumulative Gen2 data ID obtaining by calling `parse` on parsers
            for parent directories of the same path.
        log : `Log`, optional
            Log to use to report warnings and debug information.
 
        Returns
        -------
        dataId : `dict` or `None`
            Gen2 data ID that combines key-value pairs obtained from this path
            with those from ``lastDataId``.  `None` if ``name`` is not matched
            by this parser.  If the keys extracted are inconsistent with those
            in ``lastDataID``, a warning is sent to ``log`` and `None` is
            returned.
        """
        m = self.regex.format(lastDataId).fullmatch(name)
        if m is None:
            return None
        newDataId = {k: v(m.group(k)) for k, v in self.keys.items()}
        for commonKey in lastDataId.keys() & newDataId.keys():
            if newDataId[commonKey] != lastDataId[commonKey]:
                if log is not None:
                    log.warn("Inconsistent value %s=%r when parsing %r with %r.",
                             commonKey, newDataId[commonKey], name, lastDataId)
                return None
        newDataId.update(lastDataId)
        return newDataId
 

Member Data Documentation

keys

lsst.obs.base.gen2to3.repoWalker.parser.PathElementParser.keys

Definition at line 137 of file parser.py.

regex

lsst.obs.base.gen2to3.repoWalker.parser.PathElementParser.regex

Definition at line 141 of file parser.py.

template

lsst.obs.base.gen2to3.repoWalker.parser.PathElementParser.template

Definition at line 136 of file parser.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/0.2.1/Linux64/obs_base/21.0.0-27-gbbd0d29+ae871e0f33/python/lsst/obs/base/gen2to3/repoWalker/parser.py