Inheritance diagram for lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask:

Public Member Functions
def	__init__ (self, butler=None, schema=None, refObjLoader=None, reuse=tuple(), **kwargs)

def	__reduce__ (self)

def	batchWallTime (cls, time, parsedCmd, numCpus)
	Return walltime request for batch job. More...

def	runDataRef (self, patchRefList)
	Run multiband processing on coadds. More...

def	runDetection (self, cache, patchRef)
	Run detection on a patch. More...

def	runMergeDetections (self, cache, dataIdList)
	Run detection merging on a patch. More...

def	runDeblendMerged (self, cache, dataIdList)

def	runMeasurements (self, cache, dataId)

def	runMergeMeasurements (self, cache, dataIdList)
	Run measurement merging on a patch. More...

def	runForcedPhot (self, cache, dataId)
	Run forced photometry on a patch for a single filter. More...

def	writeMetadata (self, dataRef)

def	parseAndRun (cls, args, *kwargs)

def	parseAndRun (cls, args=None, config=None, log=None, doReturnResults=False)

def	parseAndSubmit (cls, args=None, **kwargs)

def	batchCommand (cls, args)
	Return command to run CmdLineTask. More...

def	logOperation (self, operation, catch=False, trace=True)
	Provide a context manager for logging an operation. More...

def	applyOverrides (cls, config)

def	writeConfig (self, butler, clobber=False, doBackup=True)

def	writeSchemas (self, butler, clobber=False, doBackup=True)

def	writePackageVersions (self, butler, clobber=False, doBackup=True, dataset="packages")

def	emptyMetadata (self)

def	getSchemaCatalogs (self)

def	getAllSchemaCatalogs (self)

def	getFullMetadata (self)

def	getFullName (self)

def	getName (self)

def	getTaskDict (self)

def	makeSubtask (self, name, **keyArgs)

def	timer (self, name, logLevel=Log.DEBUG)

def	makeField (cls, doc)

Public Attributes
	butler

	reuse

	measurementInput

	deblenderOutput

	coaddType

	metadata

	log

	config

Static Public Attributes
	ConfigClass = MultiBandDriverConfig

	RunnerClass = MultiBandDriverTaskRunner

bool	canMultiprocess = True

Detailed Description

Multi-node driver for multiband processing

Definition at line 95 of file multiBandDriver.py.

Constructor & Destructor Documentation

◆ init()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.__init__	(		self,
			butler = `None`,
			schema = `None`,
			refObjLoader = `None`,
			reuse = `tuple()`,
		**	kwargs
	)

Parameters

[in]	butler	the butler can be used to retrieve schema or passed to the refObjLoader constructor in case it is needed.
[in]	schema	the schema of the source detection catalog used as input.
[in]	refObjLoader	an instance of LoadReferenceObjectsTasks that supplies an external reference catalog. May be None if the butler argument is provided or all steps requiring a reference catalog are disabled.

Definition at line 101 of file multiBandDriver.py.

     def __init__(self, butler=None, schema=None, refObjLoader=None, reuse=tuple(), **kwargs):
         """!
         @param[in] butler: the butler can be used to retrieve schema or passed to the refObjLoader constructor
             in case it is needed.
         @param[in] schema: the schema of the source detection catalog used as input.
         @param[in] refObjLoader: an instance of LoadReferenceObjectsTasks that supplies an external reference
             catalog.  May be None if the butler argument is provided or all steps requiring a reference
             catalog are disabled.
         """
         BatchPoolTask.__init__(self, **kwargs)
         if schema is None:
             assert butler is not None, "Butler not provided"
             schema = butler.get(self.config.coaddName +
                                 "Coadd_det_schema", immediate=True).schema
         self.butler = butler
         self.reuse = tuple(reuse)
         self.makeSubtask("detectCoaddSources")
         self.makeSubtask("mergeCoaddDetections", schema=schema)
         if self.config.measureCoaddSources.inputCatalog.startswith("deblended"):
             # Ensure that the output from deblendCoaddSources matches the input to measureCoaddSources
             self.measurementInput = self.config.measureCoaddSources.inputCatalog
             self.deblenderOutput = []
             self.deblenderOutput.append("deblendedFlux")
             if self.measurementInput not in self.deblenderOutput:
                 err = "Measurement input '{0}' is not in the list of deblender output catalogs '{1}'"
                 raise ValueError(err.format(self.measurementInput, self.deblenderOutput))
  
             self.makeSubtask("deblendCoaddSources",
                              schema=afwTable.Schema(self.mergeCoaddDetections.schema),
                              peakSchema=afwTable.Schema(self.mergeCoaddDetections.merged.getPeakSchema()),
                              butler=butler)
             measureInputSchema = afwTable.Schema(self.deblendCoaddSources.schema)
         else:
             measureInputSchema = afwTable.Schema(self.mergeCoaddDetections.schema)
         self.makeSubtask("measureCoaddSources", schema=measureInputSchema,
                          peakSchema=afwTable.Schema(
                              self.mergeCoaddDetections.merged.getPeakSchema()),
                          refObjLoader=refObjLoader, butler=butler)
         self.makeSubtask("mergeCoaddMeasurements", schema=afwTable.Schema(
             self.measureCoaddSources.schema))
         self.makeSubtask("forcedPhotCoadd", refSchema=afwTable.Schema(
             self.mergeCoaddMeasurements.schema))
         if self.config.hasFakes:
             self.coaddType = "fakes_" + self.config.coaddName
         else:
             self.coaddType = self.config.coaddName
  

Member Function Documentation

◆ reduce()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.__reduce__ ( self )

Pickler

Reimplemented from lsst.pipe.base.task.Task.

Definition at line 148 of file multiBandDriver.py.

     def __reduce__(self):
         """Pickler"""
         return unpickle, (self.__class__, [], dict(config=self.config, name=self._name,
                                                    parentTask=self._parentTask, log=self.log,
                                                    butler=self.butler, reuse=self.reuse))
  

◆ applyOverrides()

def lsst.pipe.base.cmdLineTask.CmdLineTask.applyOverrides	(	cls,
		config
	)

inherited

A hook to allow a task to change the values of its config *after*
the camera-specific overrides are loaded but before any command-line
overrides are applied.

Parameters
----------
config : instance of task's ``ConfigClass``
    Task configuration.

Notes
-----
This is necessary in some cases because the camera-specific overrides
may retarget subtasks, wiping out changes made in
ConfigClass.setDefaults. See LSST Trac ticket #2282 for more
discussion.

.. warning::

   This is called by CmdLineTask.parseAndRun; other ways of
   constructing a config will not apply these overrides.

Reimplemented in lsst.pipe.drivers.constructCalibs.FringeTask, lsst.pipe.drivers.constructCalibs.FlatTask, lsst.pipe.drivers.constructCalibs.DarkTask, and lsst.pipe.drivers.constructCalibs.BiasTask.

Definition at line 587 of file cmdLineTask.py.

     def applyOverrides(cls, config):
         """A hook to allow a task to change the values of its config *after*
         the camera-specific overrides are loaded but before any command-line
         overrides are applied.
  
         Parameters
         ----------
         config : instance of task's ``ConfigClass``
             Task configuration.
  
         Notes
         -----
         This is necessary in some cases because the camera-specific overrides
         may retarget subtasks, wiping out changes made in
         ConfigClass.setDefaults. See LSST Trac ticket #2282 for more
         discussion.
  
         .. warning::
  
            This is called by CmdLineTask.parseAndRun; other ways of
            constructing a config will not apply these overrides.
         """
         pass
  

◆ batchCommand()

def lsst.ctrl.pool.parallel.BatchCmdLineTask.batchCommand	(	cls,
		args
	)

inherited

Return command to run CmdLineTask.

    @param cls: Class
    @param args: Parsed batch job arguments (from BatchArgumentParser)

Definition at line 476 of file parallel.py.

     def batchCommand(cls, args):
         """!Return command to run CmdLineTask
  
         @param cls: Class
         @param args: Parsed batch job arguments (from BatchArgumentParser)
         """
         job = args.job if args.job is not None else "job"
         module = cls.__module__
         script = ("import os; os.umask(%#05o); " +
                   "import lsst.base; lsst.base.disableImplicitThreading(); " +
                   "import lsst.ctrl.pool.log; lsst.ctrl.pool.log.jobLog(\"%s\"); ") % (UMASK, job)
  
         if args.batchStats:
             script += ("import lsst.ctrl.pool.parallel; import atexit; " +
                        "atexit.register(lsst.ctrl.pool.parallel.printProcessStats); ")
  
         script += "import %s; %s.%s.parseAndRun();" % (module, module, cls.__name__)
  
         profilePre = "import cProfile; import os; cProfile.run(\"\"\""
         profilePost = "\"\"\", filename=\"profile-" + job + "-%s-%d.dat\" % (os.uname()[1], os.getpid()))"
  
         return ("python -c '" + (profilePre if args.batchProfile else "") + script +
                 (profilePost if args.batchProfile else "") + "' " + shCommandFromArgs(args.leftover) +
                 " --noExit")
  

◆ batchWallTime()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.batchWallTime	(	cls,
		time,
		parsedCmd,
		numCpus
	)

Return walltime request for batch job.

    @param time: Requested time per iteration
    @param parsedCmd: Results of argument parsing
    @param numCores: Number of cores

Reimplemented from lsst.ctrl.pool.parallel.BatchCmdLineTask.

Definition at line 165 of file multiBandDriver.py.

     def batchWallTime(cls, time, parsedCmd, numCpus):
         """!Return walltime request for batch job
  
         @param time: Requested time per iteration
         @param parsedCmd: Results of argument parsing
         @param numCores: Number of cores
         """
         numTargets = 0
         for refList in parsedCmd.id.refList:
             numTargets += len(refList)
         return time*numTargets/float(numCpus)
  

◆ emptyMetadata()

def lsst.pipe.base.task.Task.emptyMetadata ( self )

inherited

Empty (clear) the metadata for this Task and all sub-Tasks.

Definition at line 166 of file task.py.

     def emptyMetadata(self):
         """Empty (clear) the metadata for this Task and all sub-Tasks.
         """
         for subtask in self._taskDict.values():
             subtask.metadata = dafBase.PropertyList()
  

◆ getAllSchemaCatalogs()

def lsst.pipe.base.task.Task.getAllSchemaCatalogs ( self )

inherited

Get schema catalogs for all tasks in the hierarchy, combining the
results into a single dict.

Returns
-------
schemacatalogs : `dict`
    Keys are butler dataset type, values are a empty catalog (an
    instance of the appropriate `lsst.afw.table` Catalog type) for all
    tasks in the hierarchy, from the top-level task down
    through all subtasks.

Notes
-----
This method may be called on any task in the hierarchy; it will return
the same answer, regardless.

The default implementation should always suffice. If your subtask uses
schemas the override `Task.getSchemaCatalogs`, not this method.

Definition at line 204 of file task.py.

     def getAllSchemaCatalogs(self):
         """Get schema catalogs for all tasks in the hierarchy, combining the
         results into a single dict.
  
         Returns
         -------
         schemacatalogs : `dict`
             Keys are butler dataset type, values are a empty catalog (an
             instance of the appropriate `lsst.afw.table` Catalog type) for all
             tasks in the hierarchy, from the top-level task down
             through all subtasks.
  
         Notes
         -----
         This method may be called on any task in the hierarchy; it will return
         the same answer, regardless.
  
         The default implementation should always suffice. If your subtask uses
         schemas the override `Task.getSchemaCatalogs`, not this method.
         """
         schemaDict = self.getSchemaCatalogs()
         for subtask in self._taskDict.values():
             schemaDict.update(subtask.getSchemaCatalogs())
         return schemaDict
  

◆ getFullMetadata()

def lsst.pipe.base.task.Task.getFullMetadata ( self )

inherited

Get metadata for all tasks.

Returns
-------
metadata : `lsst.daf.base.PropertySet`
    The `~lsst.daf.base.PropertySet` keys are the full task name.
    Values are metadata for the top-level task and all subtasks,
    sub-subtasks, etc.

Notes
-----
The returned metadata includes timing information (if
``@timer.timeMethod`` is used) and any metadata set by the task. The
name of each item consists of the full task name with ``.`` replaced
by ``:``, followed by ``.`` and the name of the item, e.g.::

    topLevelTaskName:subtaskName:subsubtaskName.itemName

using ``:`` in the full task name disambiguates the rare situation
that a task has a subtask and a metadata item with the same name.

Definition at line 229 of file task.py.

     def getFullMetadata(self):
         """Get metadata for all tasks.
  
         Returns
         -------
         metadata : `lsst.daf.base.PropertySet`
             The `~lsst.daf.base.PropertySet` keys are the full task name.
             Values are metadata for the top-level task and all subtasks,
             sub-subtasks, etc.
  
         Notes
         -----
         The returned metadata includes timing information (if
         ``@timer.timeMethod`` is used) and any metadata set by the task. The
         name of each item consists of the full task name with ``.`` replaced
         by ``:``, followed by ``.`` and the name of the item, e.g.::
  
             topLevelTaskName:subtaskName:subsubtaskName.itemName
  
         using ``:`` in the full task name disambiguates the rare situation
         that a task has a subtask and a metadata item with the same name.
         """
         fullMetadata = dafBase.PropertySet()
         for fullName, task in self.getTaskDict().items():
             fullMetadata.set(fullName.replace(".", ":"), task.metadata)
         return fullMetadata
  

◆ getFullName()

def lsst.pipe.base.task.Task.getFullName ( self )

inherited

Get the task name as a hierarchical name including parent task
names.

Returns
-------
fullName : `str`
    The full name consists of the name of the parent task and each
    subtask separated by periods. For example:

    - The full name of top-level task "top" is simply "top".
    - The full name of subtask "sub" of top-level task "top" is
      "top.sub".
    - The full name of subtask "sub2" of subtask "sub" of top-level
      task "top" is "top.sub.sub2".

Definition at line 256 of file task.py.

     def getFullName(self):
         """Get the task name as a hierarchical name including parent task
         names.
  
         Returns
         -------
         fullName : `str`
             The full name consists of the name of the parent task and each
             subtask separated by periods. For example:
  
             - The full name of top-level task "top" is simply "top".
             - The full name of subtask "sub" of top-level task "top" is
               "top.sub".
             - The full name of subtask "sub2" of subtask "sub" of top-level
               task "top" is "top.sub.sub2".
         """
         return self._fullName
  

◆ getName()

def lsst.pipe.base.task.Task.getName ( self )

inherited

Get the name of the task.

Returns
-------
taskName : `str`
    Name of the task.

See also
--------
getFullName

Definition at line 274 of file task.py.

     def getName(self):
         """Get the name of the task.
  
         Returns
         -------
         taskName : `str`
             Name of the task.
  
         See also
         --------
         getFullName
         """
         return self._name
  

◆ getSchemaCatalogs()

def lsst.pipe.base.task.Task.getSchemaCatalogs ( self )

inherited

Get the schemas generated by this task.

Returns
-------
schemaCatalogs : `dict`
    Keys are butler dataset type, values are an empty catalog (an
    instance of the appropriate `lsst.afw.table` Catalog type) for
    this task.

Notes
-----

.. warning::

   Subclasses that use schemas must override this method. The default
   implementation returns an empty dict.

This method may be called at any time after the Task is constructed,
which means that all task schemas should be computed at construction
time, *not* when data is actually processed. This reflects the
philosophy that the schema should not depend on the data.

Returning catalogs rather than just schemas allows us to save e.g.
slots for SourceCatalog as well.

See also
--------
Task.getAllSchemaCatalogs

Definition at line 172 of file task.py.

     def getSchemaCatalogs(self):
         """Get the schemas generated by this task.
  
         Returns
         -------
         schemaCatalogs : `dict`
             Keys are butler dataset type, values are an empty catalog (an
             instance of the appropriate `lsst.afw.table` Catalog type) for
             this task.
  
         Notes
         -----
  
         .. warning::
  
            Subclasses that use schemas must override this method. The default
            implementation returns an empty dict.
  
         This method may be called at any time after the Task is constructed,
         which means that all task schemas should be computed at construction
         time, *not* when data is actually processed. This reflects the
         philosophy that the schema should not depend on the data.
  
         Returning catalogs rather than just schemas allows us to save e.g.
         slots for SourceCatalog as well.
  
         See also
         --------
         Task.getAllSchemaCatalogs
         """
         return {}
  

◆ getTaskDict()

def lsst.pipe.base.task.Task.getTaskDict ( self )

inherited

Get a dictionary of all tasks as a shallow copy.

Returns
-------
taskDict : `dict`
    Dictionary containing full task name: task object for the top-level
    task and all subtasks, sub-subtasks, etc.

Definition at line 288 of file task.py.

     def getTaskDict(self):
         """Get a dictionary of all tasks as a shallow copy.
  
         Returns
         -------
         taskDict : `dict`
             Dictionary containing full task name: task object for the top-level
             task and all subtasks, sub-subtasks, etc.
         """
         return self._taskDict.copy()
  

◆ logOperation()

def lsst.ctrl.pool.parallel.BatchCmdLineTask.logOperation	(	self,
		operation,
		catch = `False`,
		trace = `True`
	)

inherited

Provide a context manager for logging an operation.

    @param operation: description of operation (string)
    @param catch: Catch all exceptions?
    @param trace: Log a traceback of caught exception?

    Note that if 'catch' is True, all exceptions are swallowed, but there may
    be other side-effects such as undefined variables.

Definition at line 502 of file parallel.py.

     def logOperation(self, operation, catch=False, trace=True):
         """!Provide a context manager for logging an operation
  
         @param operation: description of operation (string)
         @param catch: Catch all exceptions?
         @param trace: Log a traceback of caught exception?
  
         Note that if 'catch' is True, all exceptions are swallowed, but there may
         be other side-effects such as undefined variables.
         """
         self.log.info("%s: Start %s" % (NODE, operation))
         try:
             yield
         except Exception:
             if catch:
                 cls, e, _ = sys.exc_info()
                 self.log.warn("%s: Caught %s while %s: %s" % (NODE, cls.__name__, operation, e))
                 if trace:
                     self.log.info("%s: Traceback:\n%s" % (NODE, traceback.format_exc()))
                 return
             raise
         finally:
             self.log.info("%s: Finished %s" % (NODE, operation))
  
  

◆ makeField()

def lsst.pipe.base.task.Task.makeField	(	cls,
		doc
	)

inherited

Make a `lsst.pex.config.ConfigurableField` for this task.

Parameters
----------
doc : `str`
    Help text for the field.

Returns
-------
configurableField : `lsst.pex.config.ConfigurableField`
    A `~ConfigurableField` for this task.

Examples
--------
Provides a convenient way to specify this task is a subtask of another
task.

Here is an example of use:

.. code-block:: python

    class OtherTaskConfig(lsst.pex.config.Config):
        aSubtask = ATaskClass.makeField("brief description of task")

Definition at line 359 of file task.py.

     def makeField(cls, doc):
         """Make a `lsst.pex.config.ConfigurableField` for this task.
  
         Parameters
         ----------
         doc : `str`
             Help text for the field.
  
         Returns
         -------
         configurableField : `lsst.pex.config.ConfigurableField`
             A `~ConfigurableField` for this task.
  
         Examples
         --------
         Provides a convenient way to specify this task is a subtask of another
         task.
  
         Here is an example of use:
  
         .. code-block:: python
  
             class OtherTaskConfig(lsst.pex.config.Config):
                 aSubtask = ATaskClass.makeField("brief description of task")
         """
         return ConfigurableField(doc=doc, target=cls)
  

◆ makeSubtask()

def lsst.pipe.base.task.Task.makeSubtask	(		self,
			name,
		**	keyArgs
	)

inherited

Create a subtask as a new instance as the ``name`` attribute of this
task.

Parameters
----------
name : `str`
    Brief name of the subtask.
keyArgs
    Extra keyword arguments used to construct the task. The following
    arguments are automatically provided and cannot be overridden:

    - "config".
    - "parentTask".

Notes
-----
The subtask must be defined by ``Task.config.name``, an instance of
`~lsst.pex.config.ConfigurableField` or
`~lsst.pex.config.RegistryField`.

Definition at line 299 of file task.py.

     def makeSubtask(self, name, **keyArgs):
         """Create a subtask as a new instance as the ``name`` attribute of this
         task.
  
         Parameters
         ----------
         name : `str`
             Brief name of the subtask.
         keyArgs
             Extra keyword arguments used to construct the task. The following
             arguments are automatically provided and cannot be overridden:
  
             - "config".
             - "parentTask".
  
         Notes
         -----
         The subtask must be defined by ``Task.config.name``, an instance of
         `~lsst.pex.config.ConfigurableField` or
         `~lsst.pex.config.RegistryField`.
         """
         taskField = getattr(self.config, name, None)
         if taskField is None:
             raise KeyError(f"{self.getFullName()}'s config does not have field {name!r}")
         subtask = taskField.apply(name=name, parentTask=self, **keyArgs)
         setattr(self, name, subtask)
  

◆ parseAndRun() [1/2]

def lsst.ctrl.pool.parallel.BatchPoolTask.parseAndRun	(		cls,
		*	args,
		**	kwargs
	)

inherited

Run with a MPI process pool

Definition at line 534 of file parallel.py.

     def parseAndRun(cls, *args, **kwargs):
         """Run with a MPI process pool"""
         pool = startPool()
         super(BatchPoolTask, cls).parseAndRun(*args, **kwargs)
         pool.exit()
  
  

◆ parseAndRun() [2/2]

def lsst.pipe.base.cmdLineTask.CmdLineTask.parseAndRun	(	cls,
		args = `None`,
		config = `None`,
		log = `None`,
		doReturnResults = `False`
	)

inherited

Parse an argument list and run the command.

Parameters
----------
args : `list`, optional
    List of command-line arguments; if `None` use `sys.argv`.
config : `lsst.pex.config.Config`-type, optional
    Config for task. If `None` use `Task.ConfigClass`.
log : `lsst.log.Log`-type, optional
    Log. If `None` use the default log.
doReturnResults : `bool`, optional
    If `True`, return the results of this task. Default is `False`.
    This is only intended for unit tests and similar use. It can
    easily exhaust memory (if the task returns enough data and you
    call it enough times) and it will fail when using multiprocessing
    if the returned data cannot be pickled.

Returns
-------
struct : `lsst.pipe.base.Struct`
    Fields are:

    ``argumentParser``
        the argument parser (`lsst.pipe.base.ArgumentParser`).
    ``parsedCmd``
        the parsed command returned by the argument parser's
        `~lsst.pipe.base.ArgumentParser.parse_args` method
        (`argparse.Namespace`).
    ``taskRunner``
        the task runner used to run the task (an instance of
        `Task.RunnerClass`).
    ``resultList``
        results returned by the task runner's ``run`` method, one entry
        per invocation (`list`). This will typically be a list of
        `Struct`, each containing at least an ``exitStatus`` integer
        (0 or 1); see `Task.RunnerClass` (`TaskRunner` by default) for
        more details.

Notes
-----
Calling this method with no arguments specified is the standard way to
run a command-line task from the command-line. For an example see
``pipe_tasks`` ``bin/makeSkyMap.py`` or almost any other file in that
directory.

If one or more of the dataIds fails then this routine will exit (with
a status giving the number of failed dataIds) rather than returning
this struct;  this behaviour can be overridden by specifying the
``--noExit`` command-line option.

Definition at line 612 of file cmdLineTask.py.

     def parseAndRun(cls, args=None, config=None, log=None, doReturnResults=False):
         """Parse an argument list and run the command.
  
         Parameters
         ----------
         args : `list`, optional
             List of command-line arguments; if `None` use `sys.argv`.
         config : `lsst.pex.config.Config`-type, optional
             Config for task. If `None` use `Task.ConfigClass`.
         log : `lsst.log.Log`-type, optional
             Log. If `None` use the default log.
         doReturnResults : `bool`, optional
             If `True`, return the results of this task. Default is `False`.
             This is only intended for unit tests and similar use. It can
             easily exhaust memory (if the task returns enough data and you
             call it enough times) and it will fail when using multiprocessing
             if the returned data cannot be pickled.
  
         Returns
         -------
         struct : `lsst.pipe.base.Struct`
             Fields are:
  
             ``argumentParser``
                 the argument parser (`lsst.pipe.base.ArgumentParser`).
             ``parsedCmd``
                 the parsed command returned by the argument parser's
                 `~lsst.pipe.base.ArgumentParser.parse_args` method
                 (`argparse.Namespace`).
             ``taskRunner``
                 the task runner used to run the task (an instance of
                 `Task.RunnerClass`).
             ``resultList``
                 results returned by the task runner's ``run`` method, one entry
                 per invocation (`list`). This will typically be a list of
                 `Struct`, each containing at least an ``exitStatus`` integer
                 (0 or 1); see `Task.RunnerClass` (`TaskRunner` by default) for
                 more details.
  
         Notes
         -----
         Calling this method with no arguments specified is the standard way to
         run a command-line task from the command-line. For an example see
         ``pipe_tasks`` ``bin/makeSkyMap.py`` or almost any other file in that
         directory.
  
         If one or more of the dataIds fails then this routine will exit (with
         a status giving the number of failed dataIds) rather than returning
         this struct;  this behaviour can be overridden by specifying the
         ``--noExit`` command-line option.
         """
         if args is None:
             commandAsStr = " ".join(sys.argv)
             args = sys.argv[1:]
         else:
             commandAsStr = "{}{}".format(lsst.utils.get_caller_name(skip=1), tuple(args))
  
         argumentParser = cls._makeArgumentParser()
         if config is None:
             config = cls.ConfigClass()
         parsedCmd = argumentParser.parse_args(config=config, args=args, log=log, override=cls.applyOverrides)
         # print this message after parsing the command so the log is fully
         # configured
         parsedCmd.log.info("Running: %s", commandAsStr)
  
         taskRunner = cls.RunnerClass(TaskClass=cls, parsedCmd=parsedCmd, doReturnResults=doReturnResults)
         resultList = taskRunner.run(parsedCmd)
  
         try:
             nFailed = sum(((res.exitStatus != 0) for res in resultList))
         except (TypeError, AttributeError) as e:
             # NOTE: TypeError if resultList is None, AttributeError if it
             # doesn't have exitStatus.
             parsedCmd.log.warn("Unable to retrieve exit status (%s); assuming success", e)
             nFailed = 0
  
         if nFailed > 0:
             if parsedCmd.noExit:
                 parsedCmd.log.error("%d dataRefs failed; not exiting as --noExit was set", nFailed)
             else:
                 sys.exit(nFailed)
  
         return Struct(
             argumentParser=argumentParser,
             parsedCmd=parsedCmd,
             taskRunner=taskRunner,
             resultList=resultList,
         )
  

◆ parseAndSubmit()

def lsst.ctrl.pool.parallel.BatchCmdLineTask.parseAndSubmit	(		cls,
			args = `None`,
		**	kwargs
	)

inherited

Definition at line 435 of file parallel.py.

     def parseAndSubmit(cls, args=None, **kwargs):
         taskParser = cls._makeArgumentParser(doBatch=True, add_help=False)
         batchParser = BatchArgumentParser(parent=taskParser)
         batchArgs = batchParser.parse_args(config=cls.ConfigClass(), args=args, override=cls.applyOverrides,
                                            **kwargs)
  
         if not cls.RunnerClass(cls, batchArgs.parent).precall(batchArgs.parent):  # Write config, schema
             taskParser.error("Error in task preparation")
  
         setBatchType(batchArgs.batch)
  
         if batchArgs.batch is None:     # don't use a batch system
             sys.argv = [sys.argv[0]] + batchArgs.leftover  # Remove all batch arguments
  
             return cls.parseAndRun()
         else:
             if batchArgs.walltime > 0:
                 walltime = batchArgs.walltime
             else:
                 numCores = batchArgs.cores if batchArgs.cores > 0 else batchArgs.nodes*batchArgs.procs
                 walltime = cls.batchWallTime(batchArgs.time, batchArgs.parent, numCores)
  
             command = cls.batchCommand(batchArgs)
             batchArgs.batch.run(command, walltime=walltime)
  

◆ runDataRef()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.runDataRef	(	self,
		patchRefList
	)

Run multiband processing on coadds.

    Only the master node runs this method.

    No real MPI communication (scatter/gather) takes place: all I/O goes
    through the disk. We want the intermediate stages on disk, and the
    component Tasks are implemented around this, so we just follow suit.

    @param patchRefList:  Data references to run measurement

Definition at line 178 of file multiBandDriver.py.

     def runDataRef(self, patchRefList):
         """!Run multiband processing on coadds
  
         Only the master node runs this method.
  
         No real MPI communication (scatter/gather) takes place: all I/O goes
         through the disk. We want the intermediate stages on disk, and the
         component Tasks are implemented around this, so we just follow suit.
  
         @param patchRefList:  Data references to run measurement
         """
         for patchRef in patchRefList:
             if patchRef:
                 butler = patchRef.getButler()
                 break
         else:
             raise RuntimeError("No valid patches")
         pool = Pool("all")
         pool.cacheClear()
         pool.storeSet(butler=butler)
         # MultiBand measurements require that the detection stage be completed
         # before measurements can be made.
         #
         # The configuration for coaddDriver.py allows detection to be turned
         # of in the event that fake objects are to be added during the
         # detection process.  This allows the long co-addition process to be
         # run once, and multiple different MultiBand reruns (with different
         # fake objects) to exist from the same base co-addition.
         #
         # However, we only re-run detection if doDetection is explicitly True
         # here (this should always be the opposite of coaddDriver.doDetection);
         # otherwise we have no way to tell reliably whether any detections
         # present in an input repo are safe to use.
         if self.config.doDetection:
             detectionList = []
             for patchRef in patchRefList:
                 if ("detectCoaddSources" in self.reuse and
                         patchRef.datasetExists(self.coaddType + "Coadd_calexp", write=True)):
                     self.log.info("Skipping detectCoaddSources for %s; output already exists." %
                                   patchRef.dataId)
                     continue
                 if not patchRef.datasetExists(self.coaddType + "Coadd"):
                     self.log.debug("Not processing %s; required input %sCoadd missing." %
                                    (patchRef.dataId, self.config.coaddName))
                     continue
                 detectionList.append(patchRef)
  
             pool.map(self.runDetection, detectionList)
  
         patchRefList = [patchRef for patchRef in patchRefList if
                         patchRef.datasetExists(self.coaddType + "Coadd_calexp") and
                         patchRef.datasetExists(self.config.coaddName + "Coadd_det",
                                                write=self.config.doDetection)]
         dataIdList = [patchRef.dataId for patchRef in patchRefList]
  
         # Group by patch
         patches = {}
         tract = None
         for patchRef in patchRefList:
             dataId = patchRef.dataId
             if tract is None:
                 tract = dataId["tract"]
             else:
                 assert tract == dataId["tract"]
  
             patch = dataId["patch"]
             if patch not in patches:
                 patches[patch] = []
             patches[patch].append(dataId)
  
         pool.map(self.runMergeDetections, patches.values())
  
         # Deblend merged detections, and test for reprocessing
         #
         # The reprocessing allows us to have multiple attempts at deblending large footprints. Large
         # footprints can suck up a lot of memory in the deblender, which means that when we process on a
         # cluster, we want to refuse to deblend them (they're flagged "deblend.parent-too-big"). But since
         # they may have astronomically interesting data, we want the ability to go back and reprocess them
         # with a more permissive configuration when we have more memory or processing time.
         #
         # self.runDeblendMerged will return whether there are any footprints in that image that required
         # reprocessing.  We need to convert that list of booleans into a dict mapping the patchId (x,y) to
         # a boolean. That tells us whether the merge measurement and forced photometry need to be re-run on
         # a particular patch.
         #
         # This determination of which patches need to be reprocessed exists only in memory (the measurements
         # have been written, clobbering the old ones), so if there was an exception we would lose this
         # information, leaving things in an inconsistent state (measurements, merged measurements and
         # forced photometry old). To attempt to preserve this status, we touch a file (dataset named
         # "deepCoadd_multibandReprocessing") --- if this file exists, we need to re-run the measurements,
         # merge and forced photometry.
         #
         # This is, hopefully, a temporary workaround until we can improve the
         # deblender.
         try:
             reprocessed = pool.map(self.runDeblendMerged, patches.values())
         finally:
             if self.config.reprocessing:
                 patchReprocessing = {}
                 for dataId, reprocess in zip(dataIdList, reprocessed):
                     patchId = dataId["patch"]
                     patchReprocessing[patchId] = patchReprocessing.get(
                         patchId, False) or reprocess
                 # Persist the determination, to make error recover easier
                 reprocessDataset = self.config.coaddName + "Coadd_multibandReprocessing"
                 for patchId in patchReprocessing:
                     if not patchReprocessing[patchId]:
                         continue
                     dataId = dict(tract=tract, patch=patchId)
                     if patchReprocessing[patchId]:
                         filename = butler.get(
                             reprocessDataset + "_filename", dataId)[0]
                         open(filename, 'a').close()  # Touch file
                     elif butler.datasetExists(reprocessDataset, dataId):
                         # We must have failed at some point while reprocessing
                         # and we're starting over
                         patchReprocessing[patchId] = True
  
         # Only process patches that have been identifiedz as needing it
         pool.map(self.runMeasurements, [dataId1 for dataId1 in dataIdList if not self.config.reprocessing or
                                         patchReprocessing[dataId1["patch"]]])
         pool.map(self.runMergeMeasurements, [idList for patchId, idList in patches.items() if
                                              not self.config.reprocessing or patchReprocessing[patchId]])
         pool.map(self.runForcedPhot, [dataId1 for dataId1 in dataIdList if not self.config.reprocessing or
                                       patchReprocessing[dataId1["patch"]]])
  
         # Remove persisted reprocessing determination
         if self.config.reprocessing:
             for patchId in patchReprocessing:
                 if not patchReprocessing[patchId]:
                     continue
                 dataId = dict(tract=tract, patch=patchId)
                 filename = butler.get(
                     reprocessDataset + "_filename", dataId)[0]
                 os.unlink(filename)
  

◆ runDeblendMerged()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.runDeblendMerged	(	self,
		cache,
		dataIdList
	)

Run the deblender on a list of dataId's

Only slave nodes execute this method.

Parameters
----------
cache: Pool cache
    Pool cache with butler.
dataIdList: list
    Data identifier for patch in each band.

Returns
-------
result: bool
    whether the patch requires reprocessing.

Definition at line 349 of file multiBandDriver.py.

     def runDeblendMerged(self, cache, dataIdList):
         """Run the deblender on a list of dataId's
  
         Only slave nodes execute this method.
  
         Parameters
         ----------
         cache: Pool cache
             Pool cache with butler.
         dataIdList: list
             Data identifier for patch in each band.
  
         Returns
         -------
         result: bool
             whether the patch requires reprocessing.
         """
         with self.logOperation("deblending %s" % (dataIdList,)):
             dataRefList = [getDataRef(cache.butler, dataId, self.coaddType + "Coadd_calexp") for
                            dataId in dataIdList]
             reprocessing = False  # Does this patch require reprocessing?
             if ("deblendCoaddSources" in self.reuse and
                 all([dataRef.datasetExists(self.config.coaddName + "Coadd_" + self.measurementInput,
                                            write=True) for dataRef in dataRefList])):
                 if not self.config.reprocessing:
                     self.log.info("Skipping deblendCoaddSources for %s; output already exists" % dataIdList)
                     return False
  
                 # Footprints are the same every band, therefore we can check just one
                 catalog = dataRefList[0].get(self.config.coaddName + "Coadd_" + self.measurementInput)
                 bigFlag = catalog["deblend_parentTooBig"]
                 # Footprints marked too large by the previous deblender run
                 numOldBig = bigFlag.sum()
                 if numOldBig == 0:
                     self.log.info("No large footprints in %s" % (dataRefList[0].dataId))
                     return False
  
                 # This if-statement can be removed after DM-15662
                 if self.config.deblendCoaddSources.simultaneous:
                     deblender = self.deblendCoaddSources.multiBandDeblend
                 else:
                     deblender = self.deblendCoaddSources.singleBandDeblend
  
                 # isLargeFootprint() can potentially return False for a source that is marked
                 # too big in the catalog, because of "new"/different deblender configs.
                 # numNewBig is the number of footprints that *will* be too big if reprocessed
                 numNewBig = sum((deblender.isLargeFootprint(src.getFootprint()) for
                                  src in catalog[bigFlag]))
                 if numNewBig == numOldBig:
                     self.log.info("All %d formerly large footprints continue to be large in %s" %
                                   (numOldBig, dataRefList[0].dataId,))
                     return False
                 self.log.info("Found %d large footprints to be reprocessed in %s" %
                               (numOldBig - numNewBig, [dataRef.dataId for dataRef in dataRefList]))
                 reprocessing = True
  
             self.deblendCoaddSources.runDataRef(dataRefList)
             return reprocessing
  

◆ runDetection()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.runDetection	(	self,
		cache,
		patchRef
	)

Run detection on a patch.

   Only slave nodes execute this method.

   @param cache: Pool cache, containing butler
   @param patchRef: Patch on which to do detection

Definition at line 314 of file multiBandDriver.py.

     def runDetection(self, cache, patchRef):
         """! Run detection on a patch
  
         Only slave nodes execute this method.
  
         @param cache: Pool cache, containing butler
         @param patchRef: Patch on which to do detection
         """
         with self.logOperation("do detections on {}".format(patchRef.dataId)):
             idFactory = self.detectCoaddSources.makeIdFactory(patchRef)
             coadd = patchRef.get(self.coaddType + "Coadd", immediate=True)
             expId = int(patchRef.get(self.config.coaddName + "CoaddId"))
             self.detectCoaddSources.emptyMetadata()
             detResults = self.detectCoaddSources.run(coadd, idFactory, expId=expId)
             self.detectCoaddSources.write(detResults, patchRef)
             self.detectCoaddSources.writeMetadata(patchRef)
  

◆ runForcedPhot()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.runForcedPhot	(	self,
		cache,
		dataId
	)

Run forced photometry on a patch for a single filter.

    Only slave nodes execute this method.

    @param cache: Pool cache, with butler
    @param dataId: Data identifier for patch

Definition at line 448 of file multiBandDriver.py.

     def runForcedPhot(self, cache, dataId):
         """!Run forced photometry on a patch for a single filter
  
         Only slave nodes execute this method.
  
         @param cache: Pool cache, with butler
         @param dataId: Data identifier for patch
         """
         with self.logOperation("forced photometry on %s" % (dataId,)):
             dataRef = getDataRef(cache.butler, dataId,
                                  self.coaddType + "Coadd_calexp")
             if ("forcedPhotCoadd" in self.reuse and
                 not self.config.reprocessing and
                     dataRef.datasetExists(self.config.coaddName + "Coadd_forced_src", write=True)):
                 self.log.info("Skipping forcedPhotCoadd for %s; output already exists" % dataId)
                 return
             self.forcedPhotCoadd.runDataRef(dataRef)
  

◆ runMeasurements()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.runMeasurements	(	self,
		cache,
		dataId
	)

Run measurement on a patch for a single filter

Only slave nodes execute this method.

Parameters
----------
cache: Pool cache
    Pool cache, with butler
dataId: dataRef
    Data identifier for patch

Definition at line 408 of file multiBandDriver.py.

     def runMeasurements(self, cache, dataId):
         """Run measurement on a patch for a single filter
  
         Only slave nodes execute this method.
  
         Parameters
         ----------
         cache: Pool cache
             Pool cache, with butler
         dataId: dataRef
             Data identifier for patch
         """
         with self.logOperation("measurements on %s" % (dataId,)):
             dataRef = getDataRef(cache.butler, dataId, self.coaddType + "Coadd_calexp")
             if ("measureCoaddSources" in self.reuse and
                 not self.config.reprocessing and
                     dataRef.datasetExists(self.config.coaddName + "Coadd_meas", write=True)):
                 self.log.info("Skipping measuretCoaddSources for %s; output already exists" % dataId)
                 return
             self.measureCoaddSources.runDataRef(dataRef)
  

◆ runMergeDetections()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.runMergeDetections	(	self,
		cache,
		dataIdList
	)

Run detection merging on a patch.

    Only slave nodes execute this method.

    @param cache: Pool cache, containing butler
    @param dataIdList: List of data identifiers for the patch in different filters

Definition at line 331 of file multiBandDriver.py.

     def runMergeDetections(self, cache, dataIdList):
         """!Run detection merging on a patch
  
         Only slave nodes execute this method.
  
         @param cache: Pool cache, containing butler
         @param dataIdList: List of data identifiers for the patch in different filters
         """
         with self.logOperation("merge detections from %s" % (dataIdList,)):
             dataRefList = [getDataRef(cache.butler, dataId, self.coaddType + "Coadd_calexp") for
                            dataId in dataIdList]
             if ("mergeCoaddDetections" in self.reuse and
                     dataRefList[0].datasetExists(self.config.coaddName + "Coadd_mergeDet", write=True)):
                 self.log.info("Skipping mergeCoaddDetections for %s; output already exists." %
                               dataRefList[0].dataId)
                 return
             self.mergeCoaddDetections.runDataRef(dataRefList)
  

◆ runMergeMeasurements()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.runMergeMeasurements	(	self,
		cache,
		dataIdList
	)

Run measurement merging on a patch.

    Only slave nodes execute this method.

    @param cache: Pool cache, containing butler
    @param dataIdList: List of data identifiers for the patch in different filters

Definition at line 429 of file multiBandDriver.py.

     def runMergeMeasurements(self, cache, dataIdList):
         """!Run measurement merging on a patch
  
         Only slave nodes execute this method.
  
         @param cache: Pool cache, containing butler
         @param dataIdList: List of data identifiers for the patch in different filters
         """
         with self.logOperation("merge measurements from %s" % (dataIdList,)):
             dataRefList = [getDataRef(cache.butler, dataId, self.coaddType + "Coadd_calexp") for
                            dataId in dataIdList]
             if ("mergeCoaddMeasurements" in self.reuse and
                 not self.config.reprocessing and
                     dataRefList[0].datasetExists(self.config.coaddName + "Coadd_ref", write=True)):
                 self.log.info("Skipping mergeCoaddMeasurements for %s; output already exists" %
                               dataRefList[0].dataId)
                 return
             self.mergeCoaddMeasurements.runDataRef(dataRefList)
  

◆ timer()

def lsst.pipe.base.task.Task.timer	(	self,
		name,
		logLevel = `Log.DEBUG`
	)

inherited

Context manager to log performance data for an arbitrary block of
code.

Parameters
----------
name : `str`
    Name of code being timed; data will be logged using item name:
    ``Start`` and ``End``.
logLevel
    A `lsst.log` level constant.

Examples
--------
Creating a timer context:

.. code-block:: python

    with self.timer("someCodeToTime"):
        pass  # code to time

See also
--------
timer.logInfo

Definition at line 327 of file task.py.

     def timer(self, name, logLevel=Log.DEBUG):
         """Context manager to log performance data for an arbitrary block of
         code.
  
         Parameters
         ----------
         name : `str`
             Name of code being timed; data will be logged using item name:
             ``Start`` and ``End``.
         logLevel
             A `lsst.log` level constant.
  
         Examples
         --------
         Creating a timer context:
  
         .. code-block:: python
  
             with self.timer("someCodeToTime"):
                 pass  # code to time
  
         See also
         --------
         timer.logInfo
         """
         logInfo(obj=self, prefix=name + "Start", logLevel=logLevel)
         try:
             yield
         finally:
             logInfo(obj=self, prefix=name + "End", logLevel=logLevel)
  

◆ writeConfig()

def lsst.pipe.base.cmdLineTask.CmdLineTask.writeConfig	(	self,
		butler,
		clobber = `False`,
		doBackup = `True`
	)

inherited

Write the configuration used for processing the data, or check that
an existing one is equal to the new one if present.

Parameters
----------
butler : `lsst.daf.persistence.Butler`
    Data butler used to write the config. The config is written to
    dataset type `CmdLineTask._getConfigName`.
clobber : `bool`, optional
    A boolean flag that controls what happens if a config already has
    been saved:

    - `True`: overwrite or rename the existing config, depending on
      ``doBackup``.
    - `False`: raise `TaskError` if this config does not match the
      existing config.
doBackup : `bool`, optional
    Set to `True` to backup the config files if clobbering.

Definition at line 727 of file cmdLineTask.py.

     def writeConfig(self, butler, clobber=False, doBackup=True):
         """Write the configuration used for processing the data, or check that
         an existing one is equal to the new one if present.
  
         Parameters
         ----------
         butler : `lsst.daf.persistence.Butler`
             Data butler used to write the config. The config is written to
             dataset type `CmdLineTask._getConfigName`.
         clobber : `bool`, optional
             A boolean flag that controls what happens if a config already has
             been saved:
  
             - `True`: overwrite or rename the existing config, depending on
               ``doBackup``.
             - `False`: raise `TaskError` if this config does not match the
               existing config.
         doBackup : `bool`, optional
             Set to `True` to backup the config files if clobbering.
         """
         configName = self._getConfigName()
         if configName is None:
             return
         if clobber:
             butler.put(self.config, configName, doBackup=doBackup)
         elif butler.datasetExists(configName, write=True):
             # this may be subject to a race condition; see #2789
             try:
                 oldConfig = butler.get(configName, immediate=True)
             except Exception as exc:
                 raise type(exc)(f"Unable to read stored config file {configName} (exc); "
                                 "consider using --clobber-config")
  
             def logConfigMismatch(msg):
                 self.log.fatal("Comparing configuration: %s", msg)
  
             if not self.config.compare(oldConfig, shortcut=False, output=logConfigMismatch):
                 raise TaskError(
                     f"Config does not match existing task config {configName!r} on disk; "
                     "tasks configurations must be consistent within the same output repo "
                     "(override with --clobber-config)")
         else:
             butler.put(self.config, configName)
  

◆ writeMetadata()

def lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.writeMetadata	(	self,
		dataRef
	)

We don't collect any metadata, so skip

Reimplemented from lsst.pipe.base.cmdLineTask.CmdLineTask.

Definition at line 466 of file multiBandDriver.py.

     def writeMetadata(self, dataRef):
         """We don't collect any metadata, so skip"""
         pass

◆ writePackageVersions()

def lsst.pipe.base.cmdLineTask.CmdLineTask.writePackageVersions	(	self,
		butler,
		clobber = `False`,
		doBackup = `True`,
		dataset = `"packages"`
	)

inherited

Compare and write package versions.

Parameters
----------
butler : `lsst.daf.persistence.Butler`
    Data butler used to read/write the package versions.
clobber : `bool`, optional
    A boolean flag that controls what happens if versions already have
    been saved:

    - `True`: overwrite or rename the existing version info, depending
      on ``doBackup``.
    - `False`: raise `TaskError` if this version info does not match
      the existing.
doBackup : `bool`, optional
    If `True` and clobbering, old package version files are backed up.
dataset : `str`, optional
    Name of dataset to read/write.

Raises
------
TaskError
    Raised if there is a version mismatch with current and persisted
    lists of package versions.

Notes
-----
Note that this operation is subject to a race condition.

Definition at line 829 of file cmdLineTask.py.

     def writePackageVersions(self, butler, clobber=False, doBackup=True, dataset="packages"):
         """Compare and write package versions.
  
         Parameters
         ----------
         butler : `lsst.daf.persistence.Butler`
             Data butler used to read/write the package versions.
         clobber : `bool`, optional
             A boolean flag that controls what happens if versions already have
             been saved:
  
             - `True`: overwrite or rename the existing version info, depending
               on ``doBackup``.
             - `False`: raise `TaskError` if this version info does not match
               the existing.
         doBackup : `bool`, optional
             If `True` and clobbering, old package version files are backed up.
         dataset : `str`, optional
             Name of dataset to read/write.
  
         Raises
         ------
         TaskError
             Raised if there is a version mismatch with current and persisted
             lists of package versions.
  
         Notes
         -----
         Note that this operation is subject to a race condition.
         """
         packages = Packages.fromSystem()
  
         if clobber:
             return butler.put(packages, dataset, doBackup=doBackup)
         if not butler.datasetExists(dataset, write=True):
             return butler.put(packages, dataset)
  
         try:
             old = butler.get(dataset, immediate=True)
         except Exception as exc:
             raise type(exc)(f"Unable to read stored version dataset {dataset} ({exc}); "
                             "consider using --clobber-versions or --no-versions")
         # Note that because we can only detect python modules that have been
         # imported, the stored list of products may be more or less complete
         # than what we have now.  What's important is that the products that
         # are in common have the same version.
         diff = packages.difference(old)
         if diff:
             versions_str = "; ".join(f"{pkg}: {diff[pkg][1]} vs {diff[pkg][0]}" for pkg in diff)
             raise TaskError(
                 f"Version mismatch ({versions_str}); consider using --clobber-versions or --no-versions")
         # Update the old set of packages in case we have more packages that
         # haven't been persisted.
         extra = packages.extra(old)
         if extra:
             old.update(packages)
             butler.put(old, dataset, doBackup=doBackup)
  

◆ writeSchemas()

def lsst.pipe.base.cmdLineTask.CmdLineTask.writeSchemas	(	self,
		butler,
		clobber = `False`,
		doBackup = `True`
	)

inherited

Write the schemas returned by
`lsst.pipe.base.Task.getAllSchemaCatalogs`.

Parameters
----------
butler : `lsst.daf.persistence.Butler`
    Data butler used to write the schema. Each schema is written to the
    dataset type specified as the key in the dict returned by
    `~lsst.pipe.base.Task.getAllSchemaCatalogs`.
clobber : `bool`, optional
    A boolean flag that controls what happens if a schema already has
    been saved:

    - `True`: overwrite or rename the existing schema, depending on
      ``doBackup``.
    - `False`: raise `TaskError` if this schema does not match the
      existing schema.
doBackup : `bool`, optional
    Set to `True` to backup the schema files if clobbering.

Notes
-----
If ``clobber`` is `False` and an existing schema does not match a
current schema, then some schemas may have been saved successfully
and others may not, and there is no easy way to tell which is which.

Definition at line 771 of file cmdLineTask.py.

     def writeSchemas(self, butler, clobber=False, doBackup=True):
         """Write the schemas returned by
         `lsst.pipe.base.Task.getAllSchemaCatalogs`.
  
         Parameters
         ----------
         butler : `lsst.daf.persistence.Butler`
             Data butler used to write the schema. Each schema is written to the
             dataset type specified as the key in the dict returned by
             `~lsst.pipe.base.Task.getAllSchemaCatalogs`.
         clobber : `bool`, optional
             A boolean flag that controls what happens if a schema already has
             been saved:
  
             - `True`: overwrite or rename the existing schema, depending on
               ``doBackup``.
             - `False`: raise `TaskError` if this schema does not match the
               existing schema.
         doBackup : `bool`, optional
             Set to `True` to backup the schema files if clobbering.
  
         Notes
         -----
         If ``clobber`` is `False` and an existing schema does not match a
         current schema, then some schemas may have been saved successfully
         and others may not, and there is no easy way to tell which is which.
         """
         for dataset, catalog in self.getAllSchemaCatalogs().items():
             schemaDataset = dataset + "_schema"
             if clobber:
                 butler.put(catalog, schemaDataset, doBackup=doBackup)
             elif butler.datasetExists(schemaDataset, write=True):
                 oldSchema = butler.get(schemaDataset, immediate=True).getSchema()
                 if not oldSchema.compare(catalog.getSchema(), afwTable.Schema.IDENTICAL):
                     raise TaskError(
                         f"New schema does not match schema {dataset!r} on disk; "
                         "schemas must be consistent within the same output repo "
                         "(override with --clobber-config)")
             else:
                 butler.put(catalog, schemaDataset)
  

Member Data Documentation

◆ butler

lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.butler

Definition at line 115 of file multiBandDriver.py.

◆ canMultiprocess

bool lsst.pipe.base.cmdLineTask.CmdLineTask.canMultiprocess = True

staticinherited

Definition at line 584 of file cmdLineTask.py.

◆ coaddType

lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.coaddType

Definition at line 144 of file multiBandDriver.py.

◆ config

lsst.pipe.base.task.Task.config

inherited

Definition at line 162 of file task.py.

◆ ConfigClass

lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.ConfigClass = MultiBandDriverConfig

static

Definition at line 97 of file multiBandDriver.py.

◆ deblenderOutput

lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.deblenderOutput

Definition at line 122 of file multiBandDriver.py.

◆ log

lsst.pipe.base.task.Task.log

inherited

Definition at line 161 of file task.py.

◆ measurementInput

lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.measurementInput

Definition at line 121 of file multiBandDriver.py.

◆ metadata

lsst.pipe.base.task.Task.metadata

inherited

Definition at line 134 of file task.py.

◆ reuse

lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.reuse

Definition at line 116 of file multiBandDriver.py.

◆ RunnerClass

lsst.pipe.drivers.multiBandDriver.MultiBandDriverTask.RunnerClass = MultiBandDriverTaskRunner

static

Definition at line 99 of file multiBandDriver.py.

The documentation for this class was generated from the following file:

/j/snowflake/release/lsstsw/stack/0.4.3/Linux64/pipe_drivers/21.0.0-5-gd00fb1e+05fce91b99/python/lsst/pipe/drivers/multiBandDriver.py

Public Member Functions

Public Attributes

Static Public Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ __reduce__()

◆ applyOverrides()

◆ batchCommand()

◆ batchWallTime()

◆ emptyMetadata()

◆ getAllSchemaCatalogs()

◆ getFullMetadata()

◆ getFullName()

◆ getName()

◆ getSchemaCatalogs()

◆ getTaskDict()

◆ logOperation()

◆ makeField()

◆ makeSubtask()

◆ parseAndRun() [1/2]

◆ parseAndRun() [2/2]

◆ parseAndSubmit()

◆ runDataRef()

◆ runDeblendMerged()

◆ runDetection()

◆ runForcedPhot()

◆ runMeasurements()

◆ runMergeDetections()

◆ runMergeMeasurements()

◆ timer()

◆ writeConfig()

◆ writeMetadata()

◆ writePackageVersions()

◆ writeSchemas()

Member Data Documentation

◆ butler

◆ canMultiprocess

◆ coaddType

◆ config

◆ ConfigClass

◆ deblenderOutput

◆ log

◆ measurementInput

◆ metadata

◆ reuse

◆ RunnerClass

◆ init()

◆ reduce()