Inheritance diagram for lsst.pipe.tasks.postprocess.TransformCatalogBaseTask:

Public Member Functions
	outputDataset (self)

	inputDataset (self)

	ConfigClass (self)

	__init__ (self, args, *kwargs)

	runQuantum (self, butlerQC, inputRefs, outputRefs)

	run (self, handle, funcs=None, dataId=None, band=None)

	getFunctors (self)

	getAnalysis (self, handles, funcs=None, band=None)

	transform (self, band, handles, funcs, dataId)

Public Attributes
	funcs

Protected Member Functions
	_DefaultName (self)

Detailed Description

Base class for transforming/standardizing a catalog by applying functors
that convert units and apply calibrations.

The purpose of this task is to perform a set of computations on an input
``DeferredDatasetHandle`` or ``InMemoryDatasetHandle`` that holds a
``DataFrame`` dataset (such as ``deepCoadd_obj``), and write the results to
a new dataset (which needs to be declared in an ``outputDataset``
attribute).

The calculations to be performed are defined in a YAML file that specifies
a set of functors to be computed, provided as a ``--functorFile`` config
parameter. An example of such a YAML file is the following:

    funcs:
        sourceId:
            functor: Index
        x:
            functor: Column
            args: slot_Centroid_x
        y:
            functor: Column
            args: slot_Centroid_y
        psfFlux:
            functor: LocalNanojansky
            args:
                - slot_PsfFlux_instFlux
                - slot_PsfFlux_instFluxErr
                - base_LocalPhotoCalib
                - base_LocalPhotoCalibErr
        psfFluxErr:
            functor: LocalNanojanskyErr
            args:
                - slot_PsfFlux_instFlux
                - slot_PsfFlux_instFluxErr
                - base_LocalPhotoCalib
                - base_LocalPhotoCalibErr
    flags:
        - detect_isPrimary

The names for each entry under "func" will become the names of columns in
the output dataset.  All the functors referenced are defined in
`~lsst.pipe.tasks.functors`.  Positional arguments to be passed to each
functor are in the `args` list, and any additional entries for each column
other than "functor" or "args" (e.g., ``'filt'``, ``'dataset'``) are
treated as keyword arguments to be passed to the functor initialization.

The "flags" entry is the default shortcut for `Column` functors.
All columns listed under "flags" will be copied to the output table
untransformed. They can be of any datatype.
In the special case of transforming a multi-level oject table with
band and dataset indices (deepCoadd_obj), these will be taked from the
`meas` dataset and exploded out per band.

There are two special shortcuts that only apply when transforming
multi-level Object (deepCoadd_obj) tables:
 -  The "refFlags" entry is shortcut for `Column` functor
    taken from the `'ref'` dataset if transforming an ObjectTable.
 -  The "forcedFlags" entry is shortcut for `Column` functors.
    taken from the ``forced_src`` dataset if transforming an ObjectTable.
    These are expanded out per band.


This task uses the `lsst.pipe.tasks.postprocess.PostprocessAnalysis` object
to organize and excecute the calculations.

Definition at line 606 of file postprocess.py.

Constructor & Destructor Documentation

◆ init()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.__init__	(		self,
		*	args,
		**	kwargs )

Definition at line 688 of file postprocess.py.

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        if self.config.functorFile:
            self.log.info('Loading tranform functor definitions from %s',
                          self.config.functorFile)
            self.funcs = CompositeFunctor.from_file(self.config.functorFile)
            self.funcs.update(dict(PostprocessAnalysis._defaultFuncs))
        else:
            self.funcs = None
 

Member Function Documentation

◆ _DefaultName()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask._DefaultName ( self )

protected

Definition at line 673 of file postprocess.py.

    def _DefaultName(self):
        raise NotImplementedError('Subclass must define "_DefaultName" attribute')
 

◆ ConfigClass()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.ConfigClass ( self )

Definition at line 685 of file postprocess.py.

    def ConfigClass(self):
        raise NotImplementedError('Subclass must define "ConfigClass" attribute')
 

◆ getAnalysis()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.getAnalysis	(	self,
		handles,
		funcs = None,
		band = None )

Definition at line 741 of file postprocess.py.

    def getAnalysis(self, handles, funcs=None, band=None):
        if funcs is None:
            funcs = self.funcs
        analysis = PostprocessAnalysis(handles, funcs, filt=band)
        return analysis
 

◆ getFunctors()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.getFunctors ( self )

Definition at line 738 of file postprocess.py.

    def getFunctors(self):
        return self.funcs
 

◆ inputDataset()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.inputDataset ( self )

Definition at line 681 of file postprocess.py.

    def inputDataset(self):
        raise NotImplementedError('Subclass must define "inputDataset" attribute')
 

◆ outputDataset()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.outputDataset ( self )

Definition at line 677 of file postprocess.py.

    def outputDataset(self):
        raise NotImplementedError('Subclass must define "outputDataset" attribute')
 

◆ run()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.run	(	self,
		handle,
		funcs = None,
		dataId = None,
		band = None )

Do postprocessing calculations

Takes a ``DeferredDatasetHandle`` or ``InMemoryDatasetHandle`` or
``DataFrame`` object and dataId,
returns a dataframe with results of postprocessing calculations.

Parameters
----------
handles : `~lsst.daf.butler.DeferredDatasetHandle` or
          `~lsst.pipe.base.InMemoryDatasetHandle` or
          `~pandas.DataFrame`, or list of these.
    DataFrames from which calculations are done.
funcs : `~lsst.pipe.tasks.functors.Functor`
    Functors to apply to the table's columns
dataId : dict, optional
    Used to add a `patchId` column to the output dataframe.
band : `str`, optional
    Filter band that is being processed.

Returns
-------
df : `pandas.DataFrame`

Definition at line 708 of file postprocess.py.

    def run(self, handle, funcs=None, dataId=None, band=None):
        """Do postprocessing calculations
 
        Takes a ``DeferredDatasetHandle`` or ``InMemoryDatasetHandle`` or
        ``DataFrame`` object and dataId,
        returns a dataframe with results of postprocessing calculations.
 
        Parameters
        ----------
        handles : `~lsst.daf.butler.DeferredDatasetHandle` or
                  `~lsst.pipe.base.InMemoryDatasetHandle` or
                  `~pandas.DataFrame`, or list of these.
            DataFrames from which calculations are done.
        funcs : `~lsst.pipe.tasks.functors.Functor`
            Functors to apply to the table's columns
        dataId : dict, optional
            Used to add a `patchId` column to the output dataframe.
        band : `str`, optional
            Filter band that is being processed.
 
        Returns
        -------
        df : `pandas.DataFrame`
        """
        self.log.info("Transforming/standardizing the source table dataId: %s", dataId)
 
        df = self.transform(band, handle, funcs, dataId).df
        self.log.info("Made a table of %d columns and %d rows", len(df.columns), len(df))
        return df
 

◆ runQuantum()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.runQuantum	(	self,
		butlerQC,
		inputRefs,
		outputRefs )

Definition at line 698 of file postprocess.py.

    def runQuantum(self, butlerQC, inputRefs, outputRefs):
        inputs = butlerQC.get(inputRefs)
        if self.funcs is None:
            raise ValueError("config.functorFile is None. "
                             "Must be a valid path to yaml in order to run Task as a PipelineTask.")
        result = self.run(handle=inputs['inputCatalog'], funcs=self.funcs,
                          dataId=dict(outputRefs.outputCatalog.dataId.mapping))
        outputs = pipeBase.Struct(outputCatalog=result)
        butlerQC.put(outputs, outputRefs)
 

◆ transform()

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.transform	(	self,
		band,
		handles,
		funcs,
		dataId )

Definition at line 747 of file postprocess.py.

    def transform(self, band, handles, funcs, dataId):
        analysis = self.getAnalysis(handles, funcs=funcs, band=band)
        df = analysis.df
        if dataId and self.config.columnsFromDataId:
            for key in self.config.columnsFromDataId:
                if key in dataId:
                    df[key] = dataId[key]
                else:
                    raise ValueError(f"'{key}' in config.columnsFromDataId not found in dataId: {dataId}")
 
        if self.config.primaryKey:
            if df.index.name != self.config.primaryKey and self.config.primaryKey in df:
                df.reset_index(inplace=True, drop=True)
                df.set_index(self.config.primaryKey, inplace=True)
 
        return pipeBase.Struct(
            df=df,
            analysis=analysis
        )
 
 

Member Data Documentation

◆ funcs

lsst.pipe.tasks.postprocess.TransformCatalogBaseTask.funcs

Definition at line 693 of file postprocess.py.

The documentation for this class was generated from the following file:

/j/snowflake/release/lsstsw/stack/lsst-scipipe-8.0.0/Linux64/pipe_tasks/g95921f966b+d83dc58ecd/python/lsst/pipe/tasks/postprocess.py

Public Member Functions

Public Attributes

Protected Member Functions