lsst.pipe.base.makeLightWeightButler Namespace Reference

Functions
None	buildLightweightButler (Butler butler, QuantumGraph graph, str outputLocation, str run, *bool clobber=False, Optional[Callable[[Butler], Butler]] butlerModifier=None, Optional[Iterable[str]] collections=None)

Variables
	DataSetTypeMap = Mapping[DatasetType, Set[DataCoordinate]]

Function Documentation

buildLightweightButler()

None lsst.pipe.base.makeLightWeightButler.buildLightweightButler	(	Butler	butler,
		QuantumGraph	graph,
		str	outputLocation,
		str	run,
		*bool	clobber = False,
		Optional[Callable[[Butler], Butler]]	butlerModifier = None,
		Optional[Iterable[str]]	collections = None
	)

buildLightweightButler is a function that is responsible for exporting
input `QuantumGraphs` into a new minimal `~lsst.daf.butler.Butler` which
only contains datasets specified by the `QuantumGraph`. These datasets are
both those that already exist in the input `~lsst.daf.butler.Butler`, and
those that are expected to be produced during the execution of the
`QuantumGraph`.

Parameters
----------
butler : `lsst.daf.butler.Bulter`
    This is the existing `~lsst.daf.butler.Butler` instance from which
    existing datasets will be exported. This should be the
    `~lsst.daf.butler.Butler` which was used to create any `QuantumGraphs`
    that will be converted with this object.
graph : `QuantumGraph`
    Graph containing nodes that are to be exported into a lightweight
    butler
outputLocation : `str`
    Location at which the lightweight butler is to be exported
run : `str`
    The run collection that the exported datasets are to be placed in.
clobber : `bool`, Optional
    By default a butler will not be created if a file or directory
    already exists at the output location. If this is set to `True`
    what is at the location will be deleted prior to running the
    export. Defaults to `False`
butlerModifier : `~typing.Callable`, Optional
    If supplied this should be a callable that accepts a
    `~lsst.daf.butler.Butler`, and returns an instantiated
    `~lsst.daf.butler.Butler`. This callable may be used to make any
    modifications to the `~lsst.daf.butler.Butler` desired. This
    will be called after importing all datasets that exist in the input
    `~lsst.daf.butler.Butler` but prior to inserting Datasets expected
    to be produced. Examples of what this method could do include
    things such as creating collections/runs/ etc.
collections : `~typing.Iterable` of `str`, Optional
    An iterable of collection names that will be exported from the input
    `~lsst.daf.butler.Butler` when creating the lightweight butler. If not
    supplied the `~lsst.daf.butler.Butler`\ 's `~lsst.daf.butler.Registry`
    default collections will be used.

Raises
------
FileExistsError
    Raise if something exists in the filesystem at the specified output
    location and clobber is `False`

Definition at line 180 of file makeLightWeightButler.py.

                           ) -> None:
    r"""buildLightweightButler is a function that is responsible for exporting
    input `QuantumGraphs` into a new minimal `~lsst.daf.butler.Butler` which
    only contains datasets specified by the `QuantumGraph`. These datasets are
    both those that already exist in the input `~lsst.daf.butler.Butler`, and
    those that are expected to be produced during the execution of the
    `QuantumGraph`.
 
    Parameters
    ----------
    butler : `lsst.daf.butler.Bulter`
        This is the existing `~lsst.daf.butler.Butler` instance from which
        existing datasets will be exported. This should be the
        `~lsst.daf.butler.Butler` which was used to create any `QuantumGraphs`
        that will be converted with this object.
    graph : `QuantumGraph`
        Graph containing nodes that are to be exported into a lightweight
        butler
    outputLocation : `str`
        Location at which the lightweight butler is to be exported
    run : `str`
        The run collection that the exported datasets are to be placed in.
    clobber : `bool`, Optional
        By default a butler will not be created if a file or directory
        already exists at the output location. If this is set to `True`
        what is at the location will be deleted prior to running the
        export. Defaults to `False`
    butlerModifier : `~typing.Callable`, Optional
        If supplied this should be a callable that accepts a
        `~lsst.daf.butler.Butler`, and returns an instantiated
        `~lsst.daf.butler.Butler`. This callable may be used to make any
        modifications to the `~lsst.daf.butler.Butler` desired. This
        will be called after importing all datasets that exist in the input
        `~lsst.daf.butler.Butler` but prior to inserting Datasets expected
        to be produced. Examples of what this method could do include
        things such as creating collections/runs/ etc.
    collections : `~typing.Iterable` of `str`, Optional
        An iterable of collection names that will be exported from the input
        `~lsst.daf.butler.Butler` when creating the lightweight butler. If not
        supplied the `~lsst.daf.butler.Butler`\ 's `~lsst.daf.butler.Registry`
        default collections will be used.
 
    Raises
    ------
    FileExistsError
        Raise if something exists in the filesystem at the specified output
        location and clobber is `False`
    """
    # Do this first to Fail Fast if the output exists
    if (dirExists := os.path.exists(outputLocation)) and not clobber:
        raise FileExistsError("Cannot create a butler at specified location, location exists")
 
    exports, inserts = _accumulate(graph)
    yamlBuffer = _export(butler, collections, exports)
 
    newButler = _setupNewButler(butler, outputLocation, dirExists)
 
    newButler = _import(yamlBuffer, newButler, inserts, run, butlerModifier)
    newButler._config.dumpToUri(f"{outputLocation}/butler.yaml")

Variable Documentation

DataSetTypeMap

lsst.pipe.base.makeLightWeightButler.DataSetTypeMap = Mapping[DatasetType, Set[DataCoordinate]]

Definition at line 40 of file makeLightWeightButler.py.