lsst.pipe.tasks.postprocess.PostprocessAnalysis Class Reference

Inheritance diagram for lsst.pipe.tasks.postprocess.PostprocessAnalysis:

Public Member Functions
	__init__ (self, handles, functors, filt=None, flags=None, refFlags=None, forcedFlags=None)
	defaultFuncs (self)
	func (self)
	noDupCols (self)
	df (self)
	compute (self, dropna=False, pool=None)

Public Attributes
	handles
	functors
	filt
	flags
	forcedFlags
	refFlags

Protected Attributes
	_df

Static Protected Attributes
list	_defaultRefFlags = []
tuple	_defaultFuncs = ()

Detailed Description

Calculate columns from DataFrames or handles storing DataFrames.

This object manages and organizes an arbitrary set of computations
on a catalog.  The catalog is defined by a
`DeferredDatasetHandle` or `InMemoryDatasetHandle` object
(or list thereof), such as a ``deepCoadd_obj`` dataset, and the
computations are defined by a collection of
`~lsst.pipe.tasks.functors.Functor` objects (or, equivalently, a
``CompositeFunctor``).

After the object is initialized, accessing the ``.df`` attribute (which
holds the `pandas.DataFrame` containing the results of the calculations)
triggers computation of said dataframe.

One of the conveniences of using this object is the ability to define a
desired common filter for all functors.  This enables the same functor
collection to be passed to several different `PostprocessAnalysis` objects
without having to change the original functor collection, since the ``filt``
keyword argument of this object triggers an overwrite of the ``filt``
property for all functors in the collection.

This object also allows a list of refFlags to be passed, and defines a set
of default refFlags that are always included even if not requested.

If a list of DataFrames or Handles is passed, rather than a single one,
then the calculations will be mapped over all the input catalogs.  In
principle, it should be straightforward to parallelize this activity, but
initial tests have failed (see TODO in code comments).

Parameters
----------
handles : `~lsst.daf.butler.DeferredDatasetHandle` or
          `~lsst.pipe.base.InMemoryDatasetHandle` or
          list of these.
    Source catalog(s) for computation.
functors : `list`, `dict`, or `~lsst.pipe.tasks.functors.CompositeFunctor`
    Computations to do (functors that act on ``handles``).
    If a dict, the output
    DataFrame will have columns keyed accordingly.
    If a list, the column keys will come from the
    ``.shortname`` attribute of each functor.

filt : `str`, optional
    Filter in which to calculate.  If provided,
    this will overwrite any existing ``.filt`` attribute
    of the provided functors.

flags : `list`, optional
    List of flags (per-band) to include in output table.
    Taken from the ``meas`` dataset if applied to a multilevel Object Table.

refFlags : `list`, optional
    List of refFlags (only reference band) to include in output table.

forcedFlags : `list`, optional
    List of flags (per-band) to include in output table.
    Taken from the ``forced_src`` dataset if applied to a
    multilevel Object Table. Intended for flags from measurement plugins
    only run during multi-band forced-photometry.

Definition at line 441 of file postprocess.py.

Constructor & Destructor Documentation

__init__()

lsst.pipe.tasks.postprocess.PostprocessAnalysis.__init__	(		self,
			handles,
			functors,
			filt = None,
			flags = None,
			refFlags = None,
			forcedFlags = None )

Definition at line 505 of file postprocess.py.

    def __init__(self, handles, functors, filt=None, flags=None, refFlags=None, forcedFlags=None):
        self.handles = handles
        self.functors = functors
 
        self.filt = filt
        self.flags = list(flags) if flags is not None else []
        self.forcedFlags = list(forcedFlags) if forcedFlags is not None else []
        self.refFlags = list(self._defaultRefFlags)
        if refFlags is not None:
            self.refFlags += list(refFlags)
 
        self._df = None
 

Member Function Documentation

compute()

lsst.pipe.tasks.postprocess.PostprocessAnalysis.compute	(		self,
			dropna = False,
			pool = None )

Definition at line 550 of file postprocess.py.

    def compute(self, dropna=False, pool=None):
        # map over multiple handles
        if type(self.handles) in (list, tuple):
            if pool is None:
                dflist = [self.func(handle, dropna=dropna) for handle in self.handles]
            else:
                # TODO: Figure out why this doesn't work (pyarrow pickling
                # issues?)
                dflist = pool.map(functools.partial(self.func, dropna=dropna), self.handles)
            self._df = pd.concat(dflist)
        else:
            self._df = self.func(self.handles, dropna=dropna)
 
        return self._df
 
 

defaultFuncs()

lsst.pipe.tasks.postprocess.PostprocessAnalysis.defaultFuncs

(

self

)

Definition at line 519 of file postprocess.py.

    def defaultFuncs(self):
        funcs = dict(self._defaultFuncs)
        return funcs
 

df()

lsst.pipe.tasks.postprocess.PostprocessAnalysis.df

(

self

)

Definition at line 545 of file postprocess.py.

    def df(self):
        if self._df is None:
            self.compute()
        return self._df
 

func()

lsst.pipe.tasks.postprocess.PostprocessAnalysis.func

(

self

)

Definition at line 524 of file postprocess.py.

    def func(self):
        additionalFuncs = self.defaultFuncs
        additionalFuncs.update({flag: Column(flag, dataset='forced_src') for flag in self.forcedFlags})
        additionalFuncs.update({flag: Column(flag, dataset='ref') for flag in self.refFlags})
        additionalFuncs.update({flag: Column(flag, dataset='meas') for flag in self.flags})
 
        if isinstance(self.functors, CompositeFunctor):
            func = self.functors
        else:
            func = CompositeFunctor(self.functors)
 
        func.funcDict.update(additionalFuncs)
        func.filt = self.filt
 
        return func
 

noDupCols()

lsst.pipe.tasks.postprocess.PostprocessAnalysis.noDupCols

(

self

)

Definition at line 541 of file postprocess.py.

    def noDupCols(self):
        return [name for name, func in self.func.funcDict.items() if func.noDup or func.dataset == 'ref']
 

Member Data Documentation

_defaultFuncs

tuple lsst.pipe.tasks.postprocess.PostprocessAnalysis._defaultFuncs = ()

staticprotected

Definition at line 503 of file postprocess.py.

_defaultRefFlags

list lsst.pipe.tasks.postprocess.PostprocessAnalysis._defaultRefFlags = []

staticprotected

Definition at line 502 of file postprocess.py.

_df

lsst.pipe.tasks.postprocess.PostprocessAnalysis._df

protected

Definition at line 516 of file postprocess.py.

filt

lsst.pipe.tasks.postprocess.PostprocessAnalysis.filt

Definition at line 509 of file postprocess.py.

flags

lsst.pipe.tasks.postprocess.PostprocessAnalysis.flags

Definition at line 510 of file postprocess.py.

forcedFlags

lsst.pipe.tasks.postprocess.PostprocessAnalysis.forcedFlags

Definition at line 511 of file postprocess.py.

functors

lsst.pipe.tasks.postprocess.PostprocessAnalysis.functors

Definition at line 507 of file postprocess.py.

handles

lsst.pipe.tasks.postprocess.PostprocessAnalysis.handles

Definition at line 506 of file postprocess.py.

refFlags

lsst.pipe.tasks.postprocess.PostprocessAnalysis.refFlags

Definition at line 512 of file postprocess.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-8.0.0/Linux64/pipe_tasks/g2a9a014e59+6f9be1b9cd/python/lsst/pipe/tasks/postprocess.py