lsst.obs.base.gen2to3.translators.TranslatorFactory Class Reference

Public Member Functions
def	__init__ (self, Optional[Log] log=None)
def	__str__ (self)
def	addRule (self, KeyHandler handler, Optional[str] instrument=None, Optional[str] datasetTypeName=None, Iterable[str] gen2keys=(), bool consume=True)
def	addGenericInstrumentRules (self, str instrumentName, str calibFilterType="physical_filter", str detectorKey="ccd", str exposureKey="visit")
def	makeMatching (self, str datasetTypeName, Dict[str, type] gen2keys, Optional[str] instrument=None, Optional[BaseSkyMap] skyMap=None, Optional[str] skyMapName=None)

Public Attributes
	log

Detailed Description

A class that manages a system of rules for translating Gen2 data IDs
to Gen3 data IDs, and uses these to construct translators for particular
dataset types.

Parameters
----------
log : `lsst.log.Log`, optional
    A logger for diagnostic messages.

Definition at line 234 of file translators.py.

Constructor & Destructor Documentation

__init__()

def lsst.obs.base.gen2to3.translators.TranslatorFactory.__init__	(		self,
		Optional[Log]	log = None
	)

Definition at line 244 of file translators.py.

    def __init__(self, log: Optional[Log] = None):
        # The rules used to match KeyHandlers when constructing a Translator.
        self._rules: Dict[
            Optional[str],  # instrument name (or None to match any)
            Dict[
                Optional[str],  # dataset type name (or None to match any)
                # gen2keys, handler, consume
                List[Tuple[FrozenSet[str], KeyHandler, bool]]
            ]
        ] = {
            None: {
                None: []
            }
        }
        self._addDefaultRules()
        if log is None:
            log = Log.getLogger("obs.base.gen2to3.TranslatorFactory")
        self.log = log
 

Member Function Documentation

__str__()

def lsst.obs.base.gen2to3.translators.TranslatorFactory.__str__

(

self

)

Definition at line 263 of file translators.py.

    def __str__(self):
        lines = []
        for instrumentName, nested in self._rules.items():
            if instrumentName is None:
                instrumentName = "[any instrument]"
            for datasetTypeName, rules in nested.items():
                if datasetTypeName is None:
                    datasetTypeName = "[any dataset type]"
                lines.append(f"{instrumentName} + {datasetTypeName}:")
                for gen2keys, handler, consume in rules:
                    consumed = " (consumed)" if consume else ""
                    lines.append(f"   {gen2keys}{consumed}: {handler}")
        return "\n".join(lines)
 

addGenericInstrumentRules()

def lsst.obs.base.gen2to3.translators.TranslatorFactory.addGenericInstrumentRules	(		self,
		str	instrumentName,
		str	calibFilterType = "physical_filter",
		str	detectorKey = "ccd",
		str	exposureKey = "visit"
	)

Add translation rules that depend on some properties of the
instrument but are otherwise generic.

Parameters
----------
instrument : `str`
    The short (dimension) name of the instrument that conversion is
    going to be run on.
calibFilterType : `str`, optional
    One of ``physical_filter`` or ``band``, indicating which
    of those the gen2 calibRegistry uses as the ``filter`` key.
detectorKey : `str`, optional
    The gen2 key used to identify what in gen3 is `detector`.
exposureKey : `str`, optional
    The gen2 key used to identify what in gen3 is `exposure`.

Definition at line 358 of file translators.py.

                                  exposureKey: str = "visit"):
        """Add translation rules that depend on some properties of the
        instrument but are otherwise generic.
 
        Parameters
        ----------
        instrument : `str`
            The short (dimension) name of the instrument that conversion is
            going to be run on.
        calibFilterType : `str`, optional
            One of ``physical_filter`` or ``band``, indicating which
            of those the gen2 calibRegistry uses as the ``filter`` key.
        detectorKey : `str`, optional
            The gen2 key used to identify what in gen3 is `detector`.
        exposureKey : `str`, optional
            The gen2 key used to identify what in gen3 is `exposure`.
        """
        # Add instrument to Gen3 data ID if Gen2 contains exposureKey,
        # detectorKey, "visit", or "calibDate".  (Multiple rules may match, so
        # we'll actually set instrument in the same dict more than once).
        self.addRule(ConstantKeyHandler("instrument", instrumentName),
                     instrument=instrumentName, gen2keys=(exposureKey,), consume=False)
        self.addRule(ConstantKeyHandler("instrument", instrumentName),
                     instrument=instrumentName, gen2keys=(detectorKey,), consume=False)
        self.addRule(ConstantKeyHandler("instrument", instrumentName),
                     instrument=instrumentName, gen2keys=("calibDate",), consume=False)
        self.addRule(ConstantKeyHandler("instrument", instrumentName),
                     instrument=instrumentName, gen2keys=("visit",), consume=False)
 
        # Copy Gen2 exposureKey to Gen3 'exposure' for raw only.  Also consume
        # filter, since that's implied by 'exposure' in Gen3.
        self.addRule(CopyKeyHandler("exposure", exposureKey),
                     instrument=instrumentName, datasetTypeName="raw", gen2keys=(exposureKey,),
                     consume=(exposureKey, "filter"))
 
        # Copy Gen2 'visit' to Gen3 'visit' otherwise.  Also consume filter.
        self.addRule(CopyKeyHandler("visit"), instrument=instrumentName, gen2keys=("visit",),
                     consume=("visit", "filter"))
 
        # Copy Gen2 'ccd' to Gen3 'detector;
        self.addRule(CopyKeyHandler("detector", detectorKey),
                     instrument=instrumentName,
                     gen2keys=(detectorKey,))
 
        # Add instrument for transmission curve datasets (transmission_sensor
        # is already handled by the above rules).
        self.addRule(ConstantKeyHandler("instrument", instrumentName),
                     instrument=instrumentName, datasetTypeName="transmission_optics")
        self.addRule(ConstantKeyHandler("instrument", instrumentName),
                     instrument=instrumentName, datasetTypeName="transmission_atmosphere")
        self.addRule(ConstantKeyHandler("instrument", instrumentName),
                     instrument=instrumentName, datasetTypeName="transmission_filter")
        self.addRule(CopyKeyHandler("physical_filter", "filter"),
                     instrument=instrumentName, datasetTypeName="transmission_filter")
 
        # Add calibration mapping for filter dependent types
        for calibType in ('flat', 'sky', 'fringe'):
            self.addRule(CopyKeyHandler(calibFilterType, "filter"),
                         instrument=instrumentName, datasetTypeName=calibType)
 

addRule()

def lsst.obs.base.gen2to3.translators.TranslatorFactory.addRule	(		self,
		KeyHandler	handler,
		Optional[str]	instrument = None,
		Optional[str]	datasetTypeName = None,
		Iterable[str]	gen2keys = (),
		bool	consume = True
	)

Add a KeyHandler and an associated matching rule.

Parameters
----------
handler : `KeyHandler`
    A KeyHandler instance to add to a Translator when this rule
    matches.
instrument : `str`
    Gen3 instrument name the Gen2 repository must be associated with
    for this rule to match, or None to match any instrument.
datasetTypeName : `str`
    Name of the DatasetType this rule matches, or None to match any
    DatasetType.
gen2Keys : sequence
    Sequence of Gen2 data ID keys that must all be present for this
    rule to match.
consume : `bool` or `tuple`
    If True (default), remove all entries in gen2keys from the set of
    keys being matched to in order to prevent less-specific handlers
    from matching them.
    May also be a `tuple` listing only the keys to consume.

Definition at line 277 of file translators.py.

                consume: bool = True):
        """Add a KeyHandler and an associated matching rule.
 
        Parameters
        ----------
        handler : `KeyHandler`
            A KeyHandler instance to add to a Translator when this rule
            matches.
        instrument : `str`
            Gen3 instrument name the Gen2 repository must be associated with
            for this rule to match, or None to match any instrument.
        datasetTypeName : `str`
            Name of the DatasetType this rule matches, or None to match any
            DatasetType.
        gen2Keys : sequence
            Sequence of Gen2 data ID keys that must all be present for this
            rule to match.
        consume : `bool` or `tuple`
            If True (default), remove all entries in gen2keys from the set of
            keys being matched to in order to prevent less-specific handlers
            from matching them.
            May also be a `tuple` listing only the keys to consume.
        """
        # Ensure consume is always a frozenset, so we can process it uniformly
        # from here on.
        if consume is True:
            consume = frozenset(gen2keys)
        elif consume:
            consume = frozenset(consume)
        else:
            consume = frozenset()
        # find the rules for this instrument, or if we haven't seen it before,
        # add a nested dictionary that matches any DatasetType name and then
        # append this rule.
        rulesForInstrument = self._rules.setdefault(instrument, {None: []})
        rulesForInstrumentAndDatasetType = rulesForInstrument.setdefault(datasetTypeName, [])
        rulesForInstrumentAndDatasetType.append((frozenset(gen2keys), handler, consume))
 

makeMatching()

def lsst.obs.base.gen2to3.translators.TranslatorFactory.makeMatching	(		self,
		str	datasetTypeName,
		Dict[str, type]	gen2keys,
		Optional[str]	instrument = None,
		Optional[BaseSkyMap]	skyMap = None,
		Optional[str]	skyMapName = None
	)

Construct a Translator appropriate for instances of the given
dataset.

Parameters
----------
datasetTypeName : `str`
    Name of the dataset type.
gen2keys: `dict`
    Keys of a Gen2 data ID for this dataset.
instrument: `str`, optional
    Name of the Gen3 instrument dimension for translated data IDs.
skyMap: `~lsst.skymap.BaseSkyMap`, optional
    The skymap instance that defines any tract/patch data IDs.
    `~lsst.skymap.BaseSkyMap` instances.
skyMapName : `str`, optional
    Gen3 SkyMap Dimension name to be associated with any tract or patch
    Dimensions.

Returns
-------
translator : `Translator`
    A translator whose translate() method can be used to transform Gen2
    data IDs to Gen3 dataIds.

Definition at line 421 of file translators.py.

                     skyMap: Optional[BaseSkyMap] = None, skyMapName: Optional[str] = None):
        """Construct a Translator appropriate for instances of the given
        dataset.
 
        Parameters
        ----------
        datasetTypeName : `str`
            Name of the dataset type.
        gen2keys: `dict`
            Keys of a Gen2 data ID for this dataset.
        instrument: `str`, optional
            Name of the Gen3 instrument dimension for translated data IDs.
        skyMap: `~lsst.skymap.BaseSkyMap`, optional
            The skymap instance that defines any tract/patch data IDs.
            `~lsst.skymap.BaseSkyMap` instances.
        skyMapName : `str`, optional
            Gen3 SkyMap Dimension name to be associated with any tract or patch
            Dimensions.
 
        Returns
        -------
        translator : `Translator`
            A translator whose translate() method can be used to transform Gen2
            data IDs to Gen3 dataIds.
        """
        if instrument is not None:
            rulesForInstrument = self._rules.get(instrument, {None: []})
        else:
            rulesForInstrument = {None: []}
        rulesForAnyInstrument = self._rules[None]
        candidateRules = itertools.chain(
            rulesForInstrument.get(datasetTypeName, []),     # this instrument, this DatasetType
            rulesForInstrument[None],                         # this instrument, any DatasetType
            rulesForAnyInstrument.get(datasetTypeName, []),  # any instrument, this DatasetType
            rulesForAnyInstrument[None],                      # any instrument, any DatasetType
        )
        matchedHandlers = []
        targetKeys = set(gen2keys)
        self.log.debug("Constructing data ID translator for %s with Gen2 keys %s...",
                       datasetTypeName, gen2keys)
        for ruleKeys, ruleHandlers, consume in candidateRules:
            if ruleKeys.issubset(targetKeys):
                matchedHandlers.append(ruleHandlers)
                targetKeys -= consume
        self.log.debug("...matched %d handlers: %s, with %s unmatched.",
                       len(matchedHandlers), matchedHandlers, targetKeys)
        return Translator(matchedHandlers, skyMap=skyMap, skyMapName=skyMapName,
                          datasetTypeName=datasetTypeName, log=self.log)
 
 

Member Data Documentation

log

lsst.obs.base.gen2to3.translators.TranslatorFactory.log

Definition at line 261 of file translators.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/0.2.1/Linux64/obs_base/21.0.0-27-gbbd0d29+ae871e0f33/python/lsst/obs/base/gen2to3/translators.py