Inheritance diagram for lsst.pipe.tasks.postprocess.PostprocessAnalysis:

Public Member Functions
def	__init__ (self, parq, functors, filt=None, flags=None, refFlags=None)

def	defaultFuncs (self)

def	func (self)

def	noDupCols (self)

def	df (self)

def	compute (self, dropna=False, pool=None)

Public Attributes
	parq

	functors

	filt

	flags

	refFlags

Detailed Description

Calculate columns from ParquetTable

This object manages and organizes an arbitrary set of computations
on a catalog.  The catalog is defined by a
`lsst.pipe.tasks.parquetTable.ParquetTable` object (or list thereof), such as a
`deepCoadd_obj` dataset, and the computations are defined by a collection
of `lsst.pipe.tasks.functor.Functor` objects (or, equivalently,
a `CompositeFunctor`).

After the object is initialized, accessing the `.df` attribute (which
holds the `pandas.DataFrame` containing the results of the calculations) triggers
computation of said dataframe.

One of the conveniences of using this object is the ability to define a desired common
filter for all functors.  This enables the same functor collection to be passed to
several different `PostprocessAnalysis` objects without having to change the original
functor collection, since the `filt` keyword argument of this object triggers an
overwrite of the `filt` property for all functors in the collection.

This object also allows a list of refFlags to be passed, and defines a set of default
refFlags that are always included even if not requested.

If a list of `ParquetTable` object is passed, rather than a single one, then the
calculations will be mapped over all the input catalogs.  In principle, it should
be straightforward to parallelize this activity, but initial tests have failed
(see TODO in code comments).

Parameters
----------
parq : `lsst.pipe.tasks.ParquetTable` (or list of such)
    Source catalog(s) for computation

functors : `list`, `dict`, or `lsst.pipe.tasks.functors.CompositeFunctor`
    Computations to do (functors that act on `parq`).
    If a dict, the output
    DataFrame will have columns keyed accordingly.
    If a list, the column keys will come from the
    `.shortname` attribute of each functor.

filt : `str` (optional)
    Filter in which to calculate.  If provided,
    this will overwrite any existing `.filt` attribute
    of the provided functors.

flags : `list` (optional)
    List of flags (per-band) to include in output table.

refFlags : `list` (optional)
    List of refFlags (only reference band) to include in output table.

Definition at line 342 of file postprocess.py.

Constructor & Destructor Documentation

◆ init()

def lsst.pipe.tasks.postprocess.PostprocessAnalysis.__init__	(	self,
		parq,
		functors,
		filt = `None`,
		flags = `None`,
		refFlags = `None`
	)

Definition at line 399 of file postprocess.py.

     def __init__(self, parq, functors, filt=None, flags=None, refFlags=None):
         self.parq = parq
         self.functors = functors
  
         self.filt = filt
         self.flags = list(flags) if flags is not None else []
         self.refFlags = list(self._defaultRefFlags)
         if refFlags is not None:
             self.refFlags += list(refFlags)
  
         self._df = None
  

Member Function Documentation

◆ compute()

def lsst.pipe.tasks.postprocess.PostprocessAnalysis.compute	(	self,
		dropna = `False`,
		pool = `None`
	)

Definition at line 442 of file postprocess.py.

     def compute(self, dropna=False, pool=None):
         # map over multiple parquet tables
         if type(self.parq) in (list, tuple):
             if pool is None:
                 dflist = [self.func(parq, dropna=dropna) for parq in self.parq]
             else:
                 # TODO: Figure out why this doesn't work (pyarrow pickling issues?)
                 dflist = pool.map(functools.partial(self.func, dropna=dropna), self.parq)
             self._df = pd.concat(dflist)
         else:
             self._df = self.func(self.parq, dropna=dropna)
  
         return self._df
  
  

◆ defaultFuncs()

def lsst.pipe.tasks.postprocess.PostprocessAnalysis.defaultFuncs ( self )

Definition at line 412 of file postprocess.py.

     def defaultFuncs(self):
         funcs = dict(self._defaultFuncs)
         return funcs
  

◆ df()

def lsst.pipe.tasks.postprocess.PostprocessAnalysis.df ( self )

Definition at line 437 of file postprocess.py.

     def df(self):
         if self._df is None:
             self.compute()
         return self._df
  

◆ func()

def lsst.pipe.tasks.postprocess.PostprocessAnalysis.func ( self )

Definition at line 417 of file postprocess.py.

     def func(self):
         additionalFuncs = self.defaultFuncs
         additionalFuncs.update({flag: Column(flag, dataset='ref') for flag in self.refFlags})
         additionalFuncs.update({flag: Column(flag, dataset='meas') for flag in self.flags})
  
         if isinstance(self.functors, CompositeFunctor):
             func = self.functors
         else:
             func = CompositeFunctor(self.functors)
  
         func.funcDict.update(additionalFuncs)
         func.filt = self.filt
  
         return func
  

◆ noDupCols()

def lsst.pipe.tasks.postprocess.PostprocessAnalysis.noDupCols ( self )

Definition at line 433 of file postprocess.py.

     def noDupCols(self):
         return [name for name, func in self.func.funcDict.items() if func.noDup or func.dataset == 'ref']
  

Member Data Documentation

◆ filt

lsst.pipe.tasks.postprocess.PostprocessAnalysis.filt

Definition at line 403 of file postprocess.py.

◆ flags

lsst.pipe.tasks.postprocess.PostprocessAnalysis.flags

Definition at line 404 of file postprocess.py.

◆ functors

lsst.pipe.tasks.postprocess.PostprocessAnalysis.functors

Definition at line 401 of file postprocess.py.

◆ parq

lsst.pipe.tasks.postprocess.PostprocessAnalysis.parq

Definition at line 400 of file postprocess.py.

◆ refFlags

lsst.pipe.tasks.postprocess.PostprocessAnalysis.refFlags

Definition at line 405 of file postprocess.py.

The documentation for this class was generated from the following file:

/j/snowflake/release/lsstsw/stack/0.4.3/Linux64/pipe_tasks/21.0.0-61-ge77b8116+a869d0a5a5/python/lsst/pipe/tasks/postprocess.py

Public Member Functions

Public Attributes