LSST Applications  21.0.0-172-gfb10e10a+18fedfabac,22.0.0+297cba6710,22.0.0+80564b0ff1,22.0.0+8d77f4f51a,22.0.0+a28f4c53b1,22.0.0+dcf3732eb2,22.0.1-1-g7d6de66+2a20fdde0d,22.0.1-1-g8e32f31+297cba6710,22.0.1-1-geca5380+7fa3b7d9b6,22.0.1-12-g44dc1dc+2a20fdde0d,22.0.1-15-g6a90155+515f58c32b,22.0.1-16-g9282f48+790f5f2caa,22.0.1-2-g92698f7+dcf3732eb2,22.0.1-2-ga9b0f51+7fa3b7d9b6,22.0.1-2-gd1925c9+bf4f0e694f,22.0.1-24-g1ad7a390+a9625a72a8,22.0.1-25-g5bf6245+3ad8ecd50b,22.0.1-25-gb120d7b+8b5510f75f,22.0.1-27-g97737f7+2a20fdde0d,22.0.1-32-gf62ce7b1+aa4237961e,22.0.1-4-g0b3f228+2a20fdde0d,22.0.1-4-g243d05b+871c1b8305,22.0.1-4-g3a563be+32dcf1063f,22.0.1-4-g44f2e3d+9e4ab0f4fa,22.0.1-42-gca6935d93+ba5e5ca3eb,22.0.1-5-g15c806e+85460ae5f3,22.0.1-5-g58711c4+611d128589,22.0.1-5-g75bb458+99c117b92f,22.0.1-6-g1c63a23+7fa3b7d9b6,22.0.1-6-g50866e6+84ff5a128b,22.0.1-6-g8d3140d+720564cf76,22.0.1-6-gd805d02+cc5644f571,22.0.1-8-ge5750ce+85460ae5f3,master-g6e05de7fdc+babf819c66,master-g99da0e417a+8d77f4f51a,w.2021.48
LSST Data Management Base Package
Public Member Functions | Public Attributes | List of all members
lsst.pipe.tasks.functors.Functor Class Reference
Inheritance diagram for lsst.pipe.tasks.functors.Functor:
lsst.pipe.tasks.functors.Color lsst.pipe.tasks.functors.Column lsst.pipe.tasks.functors.CompositeFunctor lsst.pipe.tasks.functors.CustomFunctor lsst.pipe.tasks.functors.DeconvolvedMoments lsst.pipe.tasks.functors.E1 lsst.pipe.tasks.functors.E2 lsst.pipe.tasks.functors.HsmFwhm lsst.pipe.tasks.functors.HsmTraceSize lsst.pipe.tasks.functors.HtmIndex20 lsst.pipe.tasks.functors.Index lsst.pipe.tasks.functors.Labeller lsst.pipe.tasks.functors.LocalPhotometry lsst.pipe.tasks.functors.LocalWcs lsst.pipe.tasks.functors.Mag lsst.pipe.tasks.functors.MagDiff lsst.pipe.tasks.functors.Photometry lsst.pipe.tasks.functors.PsfHsmTraceSizeDiff lsst.pipe.tasks.functors.PsfSdssTraceSizeDiff lsst.pipe.tasks.functors.RadiusFromQuadrupole lsst.pipe.tasks.functors.Ratio lsst.pipe.tasks.functors.ReferenceBand lsst.pipe.tasks.functors.SdssTraceSize

Public Member Functions

def __init__ (self, filt=None, dataset=None, noDup=None)
def noDup (self)
def columns (self)
def multilevelColumns (self, data, columnIndex=None, returnTuple=False)
def __call__ (self, data, dropna=False)
def difference (self, data1, data2, **kwargs)
def fail (self, df)
def name (self)
def shortname (self)

Public Attributes


Detailed Description

Define and execute a calculation on a ParquetTable

The `__call__` method accepts either a `ParquetTable` object or a
`DeferredDatasetHandle`, and returns the
result of the calculation as a single column.  Each functor defines what
columns are needed for the calculation, and only these columns are read
from the `ParquetTable`.

The action of  `__call__` consists of two steps: first, loading the
necessary columns from disk into memory as a `pandas.DataFrame` object;
and second, performing the computation on this dataframe and returning the

To define a new `Functor`, a subclass must define a `_func` method,
that takes a `pandas.DataFrame` and returns result in a `pandas.Series`.
In addition, it must define the following attributes

* `_columns`: The columns necessary to perform the calculation
* `name`: A name appropriate for a figure axis label
* `shortname`: A name appropriate for use as a dictionary key

On initialization, a `Functor` should declare what band (`filt` kwarg)
and dataset (e.g. `'ref'`, `'meas'`, `'forced_src'`) it is intended to be
applied to. This enables the `_get_data` method to extract the proper
columns from the parquet file. If not specified, the dataset will fall back
on the `_defaultDataset`attribute. If band is not specified and `dataset`
is anything other than `'ref'`, then an error will be raised when trying to
perform the calculation.

Originally, `Functor` was set up to expect
datasets formatted like the `deepCoadd_obj` dataset; that is, a
dataframe with a multi-level column index, with the levels of the
column index being `band`, `dataset`, and `column`.
It has since been generalized to apply to dataframes without mutli-level
indices and multi-level indices with just `dataset` and `column` levels.
In addition, the `_get_data` method that reads
the dataframe from the `ParquetTable` will return a dataframe with column
index levels defined by the `_dfLevels` attribute; by default, this is

The `_dfLevels` attributes should generally not need to
be changed, unless `_func` needs columns from multiple filters or datasets
to do the calculation.
An example of this is the `lsst.pipe.tasks.functors.Color` functor, for
which `_dfLevels = ('band', 'column')`, and `_func` expects the dataframe
it gets to have those levels in the column index.

filt : str
    Filter upon which to do the calculation

dataset : str
    Dataset upon which to do the calculation
    (e.g., 'ref', 'meas', 'forced_src').

Definition at line 78 of file

Constructor & Destructor Documentation

◆ __init__()

def lsst.pipe.tasks.functors.Functor.__init__ (   self,
  filt = None,
  dataset = None,
  noDup = None 

Definition at line 142 of file

142  def __init__(self, filt=None, dataset=None, noDup=None):
143  self.filt = filt
144  self.dataset = dataset if dataset is not None else self._defaultDataset
145  self._noDup = noDup

Member Function Documentation

◆ __call__()

def lsst.pipe.tasks.functors.Functor.__call__ (   self,
  dropna = False 

Definition at line 340 of file

340  def __call__(self, data, dropna=False):
341  try:
342  df = self._get_data(data)
343  vals = self._func(df)
344  except Exception:
345  vals =
346  if dropna:
347  vals = self._dropna(vals)
349  return vals

◆ columns()

def lsst.pipe.tasks.functors.Functor.columns (   self)

◆ difference()

def lsst.pipe.tasks.functors.Functor.difference (   self,
**  kwargs 
Computes difference between functor called on two different ParquetTable objects

Definition at line 351 of file

351  def difference(self, data1, data2, **kwargs):
352  """Computes difference between functor called on two different ParquetTable objects
353  """
354  return self(data1, **kwargs) - self(data2, **kwargs)

◆ fail()

def (   self,

Definition at line 356 of file

356  def fail(self, df):
357  return pd.Series(np.full(len(df), np.nan), index=df.index)

◆ multilevelColumns()

def lsst.pipe.tasks.functors.Functor.multilevelColumns (   self,
  columnIndex = None,
  returnTuple = False 
Returns columns needed by functor from multilevel dataset

To access tables with multilevel column structure, the `MultilevelParquetTable`
or `DeferredDatasetHandle` need to be passed either a list of tuples or a

data : `MultilevelParquetTable` or `DeferredDatasetHandle`

columnIndex (optional): pandas `Index` object
    either passed or read in from `DeferredDatasetHandle`.

`returnTuple` : bool
    If true, then return a list of tuples rather than the column dictionary
    specification.  This is set to `True` by `CompositeFunctor` in order to be able to
    combine columns from the various component functors.

Definition at line 229 of file

229  def multilevelColumns(self, data, columnIndex=None, returnTuple=False):
230  """Returns columns needed by functor from multilevel dataset
232  To access tables with multilevel column structure, the `MultilevelParquetTable`
233  or `DeferredDatasetHandle` need to be passed either a list of tuples or a
234  dictionary.
236  Parameters
237  ----------
238  data : `MultilevelParquetTable` or `DeferredDatasetHandle`
240  columnIndex (optional): pandas `Index` object
241  either passed or read in from `DeferredDatasetHandle`.
243  `returnTuple` : bool
244  If true, then return a list of tuples rather than the column dictionary
245  specification. This is set to `True` by `CompositeFunctor` in order to be able to
246  combine columns from the various component functors.
248  """
249  if isinstance(data, DeferredDatasetHandle) and columnIndex is None:
250  columnIndex = data.get(component="columns")
252  # Confirm that the dataset has the column levels the functor is expecting it to have.
253  columnLevels = self._get_data_columnLevels(data, columnIndex)
255  columnDict = {'column': self.columns,
256  'dataset': self.dataset}
257  if self.filt is None:
258  columnLevelNames = self._get_data_columnLevelNames(data, columnIndex)
259  if "band" in columnLevels:
260  if self.dataset == "ref":
261  columnDict["band"] = columnLevelNames["band"][0]
262  else:
263  raise ValueError(f"'filt' not set for functor {}"
264  f"(dataset {self.dataset}) "
265  "and ParquetTable "
266  "contains multiple filters in column index. "
267  "Set 'filt' or set 'dataset' to 'ref'.")
268  else:
269  columnDict['band'] = self.filt
271  if isinstance(data, MultilevelParquetTable):
272  return data._colsFromDict(columnDict)
273  elif isinstance(data, DeferredDatasetHandle):
274  if returnTuple:
275  return self._colsFromDict(columnDict, columnIndex=columnIndex)
276  else:
277  return columnDict

◆ name()

def (   self)

◆ noDup()

def lsst.pipe.tasks.functors.Functor.noDup (   self)

Definition at line 148 of file

148  def noDup(self):
149  if self._noDup is not None:
150  return self._noDup
151  else:
152  return self._defaultNoDup

◆ shortname()

def lsst.pipe.tasks.functors.Functor.shortname (   self)
Short name of functor (suitable for column name/dict key)

Reimplemented in lsst.pipe.tasks.functors.Color, and lsst.pipe.tasks.functors.MagDiff.

Definition at line 366 of file

366  def shortname(self):
367  """Short name of functor (suitable for column name/dict key)
368  """
369  return

Member Data Documentation

◆ dataset


Definition at line 144 of file

◆ filt


Definition at line 143 of file

The documentation for this class was generated from the following file: