lsst.pipe.base.executionButlerBuilder Namespace Reference

Functions
Butler	buildExecutionButler (Butler butler, QuantumGraph graph, Union[str, ButlerURI] outputLocation, str run, *bool clobber=False, Optional[Callable[[Butler], Butler]] butlerModifier=None, Optional[Iterable[str]] collections=None)

Variables
	DataSetTypeMap = Mapping[DatasetType, Set[DataCoordinate]]

Function Documentation

buildExecutionButler()

Butler lsst.pipe.base.executionButlerBuilder.buildExecutionButler	(	Butler	butler,
		QuantumGraph	graph,
		Union[str, ButlerURI]	outputLocation,
		str	run,
		*bool	clobber = False,
		Optional[Callable[[Butler], Butler]]	butlerModifier = None,
		Optional[Iterable[str]]	collections = None
	)

buildExecutionButler is a function that is responsible for exporting
input `QuantumGraphs` into a new minimal `~lsst.daf.butler.Butler` which
only contains datasets specified by the `QuantumGraph`. These datasets are
both those that already exist in the input `~lsst.daf.butler.Butler`, and
those that are expected to be produced during the execution of the
`QuantumGraph`.

Parameters
----------
butler : `lsst.daf.butler.Bulter`
    This is the existing `~lsst.daf.butler.Butler` instance from which
    existing datasets will be exported. This should be the
    `~lsst.daf.butler.Butler` which was used to create any `QuantumGraphs`
    that will be converted with this object.
graph : `QuantumGraph`
    Graph containing nodes that are to be exported into an execution
    butler
outputLocation : `str` or `~lsst.daf.butler.ButlerURI`
    URI Location at which the execution butler is to be exported. May be
    specified as a string or a ButlerURI instance.
run : `str` optional
    The run collection that the exported datasets are to be placed in. If
    None, the default value in registry.defaults will be used.
clobber : `bool`, Optional
    By default a butler will not be created if a file or directory
    already exists at the output location. If this is set to `True`
    what is at the location will be deleted prior to running the
    export. Defaults to `False`
butlerModifier : `~typing.Callable`, Optional
    If supplied this should be a callable that accepts a
    `~lsst.daf.butler.Butler`, and returns an instantiated
    `~lsst.daf.butler.Butler`. This callable may be used to make any
    modifications to the `~lsst.daf.butler.Butler` desired. This
    will be called after importing all datasets that exist in the input
    `~lsst.daf.butler.Butler` but prior to inserting Datasets expected
    to be produced. Examples of what this method could do include
    things such as creating collections/runs/ etc.
collections : `~typing.Iterable` of `str`, Optional
    An iterable of collection names that will be exported from the input
    `~lsst.daf.butler.Butler` when creating the execution butler. If not
    supplied the `~lsst.daf.butler.Butler`\ 's `~lsst.daf.butler.Registry`
    default collections will be used.

Returns
-------
executionButler : `lsst.daf.butler.Butler`
    An instance of the newly created execution butler

Raises
------
FileExistsError
    Raised if something exists in the filesystem at the specified output
    location and clobber is `False`
NotADirectoryError
    Raised if specified output URI does not correspond to a directory

Definition at line 183 of file executionButlerBuilder.py.

                         ) -> Butler:
    r"""buildExecutionButler is a function that is responsible for exporting
    input `QuantumGraphs` into a new minimal `~lsst.daf.butler.Butler` which
    only contains datasets specified by the `QuantumGraph`. These datasets are
    both those that already exist in the input `~lsst.daf.butler.Butler`, and
    those that are expected to be produced during the execution of the
    `QuantumGraph`.
 
    Parameters
    ----------
    butler : `lsst.daf.butler.Bulter`
        This is the existing `~lsst.daf.butler.Butler` instance from which
        existing datasets will be exported. This should be the
        `~lsst.daf.butler.Butler` which was used to create any `QuantumGraphs`
        that will be converted with this object.
    graph : `QuantumGraph`
        Graph containing nodes that are to be exported into an execution
        butler
    outputLocation : `str` or `~lsst.daf.butler.ButlerURI`
        URI Location at which the execution butler is to be exported. May be
        specified as a string or a ButlerURI instance.
    run : `str` optional
        The run collection that the exported datasets are to be placed in. If
        None, the default value in registry.defaults will be used.
    clobber : `bool`, Optional
        By default a butler will not be created if a file or directory
        already exists at the output location. If this is set to `True`
        what is at the location will be deleted prior to running the
        export. Defaults to `False`
    butlerModifier : `~typing.Callable`, Optional
        If supplied this should be a callable that accepts a
        `~lsst.daf.butler.Butler`, and returns an instantiated
        `~lsst.daf.butler.Butler`. This callable may be used to make any
        modifications to the `~lsst.daf.butler.Butler` desired. This
        will be called after importing all datasets that exist in the input
        `~lsst.daf.butler.Butler` but prior to inserting Datasets expected
        to be produced. Examples of what this method could do include
        things such as creating collections/runs/ etc.
    collections : `~typing.Iterable` of `str`, Optional
        An iterable of collection names that will be exported from the input
        `~lsst.daf.butler.Butler` when creating the execution butler. If not
        supplied the `~lsst.daf.butler.Butler`\ 's `~lsst.daf.butler.Registry`
        default collections will be used.
 
    Returns
    -------
    executionButler : `lsst.daf.butler.Butler`
        An instance of the newly created execution butler
 
    Raises
    ------
    FileExistsError
        Raised if something exists in the filesystem at the specified output
        location and clobber is `False`
    NotADirectoryError
        Raised if specified output URI does not correspond to a directory
    """
    outputLocation = ButlerURI(outputLocation)
 
    # Do this first to Fail Fast if the output exists
    if (dirExists := outputLocation.exists()) and not clobber:
        raise FileExistsError("Cannot create a butler at specified location, location exists")
    if not outputLocation.isdir():
        raise NotADirectoryError("The specified output URI does not appear to correspond to a directory")
 
    exports, inserts = _accumulate(graph)
    yamlBuffer = _export(butler, collections, exports)
 
    newButler = _setupNewButler(butler, outputLocation, dirExists)
 
    return _import(yamlBuffer, newButler, inserts, run, butlerModifier)

Variable Documentation

DataSetTypeMap

lsst.pipe.base.executionButlerBuilder.DataSetTypeMap = Mapping[DatasetType, Set[DataCoordinate]]

Definition at line 38 of file executionButlerBuilder.py.