Inheritance diagram for lsst.pipe.tasks.functors.HtmIndex20:

Public Member Functions
	__init__ (self, ra, dec, **kwargs)

	name (self)

	noDup (self)

	columns (self)

	multilevelColumns (self, data, columnIndex=None, returnTuple=False)

	__call__ (self, data, dropna=False)

	difference (self, data1, data2, **kwargs)

	fail (self, df)

	shortname (self)

Public Attributes
	pixelator = sphgeom.HtmPixelization(self.htmLevel)

	ra = ra

	dec = dec

	filt = filt

str	dataset = dataset if dataset is not None else self._defaultDataset

	log = logging.getLogger(type(self).__name__)

	name

Static Public Attributes
int	htmLevel = 20

Protected Member Functions
	_func (self, df)

	_get_data_columnLevels (self, data, columnIndex=None)

	_get_data_columnLevelNames (self, data, columnIndex=None)

	_colsFromDict (self, colDict, columnIndex=None)

	_get_columnIndex (self, data)

	_get_data (self, data)

	_setLevels (self, df)

	_dropna (self, vals)

Protected Attributes
list	_columns = [self.ra, self.dec]

	_noDup = noDup

Static Protected Attributes
bool	_radians = True

str	_defaultDataset = 'ref'

tuple	_dfLevels = ('column',)

bool	_defaultNoDup = False

Detailed Description

Compute the level 20 HtmIndex for the catalog.

Notes
-----
This functor was implemented to satisfy requirements of old APDB interface
which required the ``pixelId`` column in DiaObject with HTM20 index.
The APDB interface had migrated to not need that information, but we keep
this class in case it may be useful for something else.

Definition at line 772 of file functors.py.

Constructor & Destructor Documentation

◆ init()

lsst.pipe.tasks.functors.HtmIndex20.__init__	(		self,
			ra,
			dec,
		**	kwargs )

Definition at line 786 of file functors.py.

    def __init__(self, ra, dec, **kwargs):
        self.pixelator = sphgeom.HtmPixelization(self.htmLevel)
        self.ra = ra
        self.dec = dec
        self._columns = [self.ra, self.dec]
        super().__init__(**kwargs)
 

Member Function Documentation

◆ call()

lsst.pipe.tasks.functors.Functor.__call__	(	self,
		data,
		dropna = False )

inherited

Definition at line 348 of file functors.py.

    def __call__(self, data, dropna=False):
        df = self._get_data(data)
        try:
            vals = self._func(df)
        except Exception as e:
            self.log.error("Exception in %s call: %s: %s", self.name, type(e).__name__, e)
            vals = self.fail(df)
        if dropna:
            vals = self._dropna(vals)
 
        return vals
 

◆ _colsFromDict()

lsst.pipe.tasks.functors.Functor._colsFromDict	(	self,
		colDict,
		columnIndex = None )

protectedinherited

Converts dictionary column specficiation to a list of columns.

Definition at line 218 of file functors.py.

    def _colsFromDict(self, colDict, columnIndex=None):
        """Converts dictionary column specficiation to a list of columns."""
        new_colDict = {}
        columnLevels = self._get_data_columnLevels(None, columnIndex=columnIndex)
 
        for i, lev in enumerate(columnLevels):
            if lev in colDict:
                if isinstance(colDict[lev], str):
                    new_colDict[lev] = [colDict[lev]]
                else:
                    new_colDict[lev] = colDict[lev]
            else:
                new_colDict[lev] = columnIndex.levels[i]
 
        levelCols = [new_colDict[lev] for lev in columnLevels]
        cols = list(product(*levelCols))
        colsAvailable = [col for col in cols if col in columnIndex]
        return colsAvailable
 

◆ _dropna()

lsst.pipe.tasks.functors.Functor._dropna	(		self,
			vals )

protectedinherited

Definition at line 345 of file functors.py.

    def _dropna(self, vals):
        return vals.dropna()
 

◆ _func()

lsst.pipe.tasks.functors.HtmIndex20._func	(		self,
			df )

protected

Reimplemented from lsst.pipe.tasks.functors.Functor.

Definition at line 793 of file functors.py.

    def _func(self, df):
 
        def computePixel(row):
            if self._radians:
                sphPoint = geom.SpherePoint(row[self.ra],
                                            row[self.dec],
                                            geom.radians)
            else:
                sphPoint = geom.SpherePoint(row[self.ra],
                                            row[self.dec],
                                            geom.degrees)
            return self.pixelator.index(sphPoint.getVector())
 
        return df.apply(computePixel, axis=1, result_type='reduce').astype('int64')
 
 

◆ _get_columnIndex()

lsst.pipe.tasks.functors.Functor._get_columnIndex	(		self,
			data )

protectedinherited

Return columnIndex.

Definition at line 294 of file functors.py.

    def _get_columnIndex(self, data):
        """Return columnIndex."""
 
        if isinstance(data, (DeferredDatasetHandle, InMemoryDatasetHandle)):
            return data.get(component="columns")
        else:
            return None
 

◆ _get_data()

lsst.pipe.tasks.functors.Functor._get_data	(		self,
			data )

protectedinherited

Retrieve DataFrame necessary for calculation.

The data argument can be a `~pandas.DataFrame`, a
`~lsst.daf.butler.DeferredDatasetHandle`, or
an `~lsst.pipe.base.InMemoryDatasetHandle`.

Returns a DataFrame upon which `self._func` can act.

Definition at line 302 of file functors.py.

    def _get_data(self, data):
        """Retrieve DataFrame necessary for calculation.
 
        The data argument can be a `~pandas.DataFrame`, a
        `~lsst.daf.butler.DeferredDatasetHandle`, or
        an `~lsst.pipe.base.InMemoryDatasetHandle`.
 
        Returns a DataFrame upon which `self._func` can act.
        """
        # We wrap a DataFrame in a handle here to take advantage of the
        # DataFrame delegate DataFrame column wrangling abilities.
        if isinstance(data, pd.DataFrame):
            _data = InMemoryDatasetHandle(data, storageClass="DataFrame")
        elif isinstance(data, (DeferredDatasetHandle, InMemoryDatasetHandle)):
            _data = data
        else:
            raise RuntimeError(f"Unexpected type provided for data. Got {get_full_type_name(data)}.")
 
        # First thing to do: check to see if the data source has a multilevel
        # column index or not.
        columnIndex = self._get_columnIndex(_data)
        is_multiLevel = isinstance(columnIndex, pd.MultiIndex)
 
        # Get proper columns specification for this functor.
        if is_multiLevel:
            columns = self.multilevelColumns(_data, columnIndex=columnIndex)
        else:
            columns = self.columns
 
        # Load in-memory DataFrame with appropriate columns the gen3 way.
        df = _data.get(parameters={"columns": columns})
 
        # Drop unnecessary column levels.
        if is_multiLevel:
            df = self._setLevels(df)
 
        return df
 

◆ _get_data_columnLevelNames()

lsst.pipe.tasks.functors.Functor._get_data_columnLevelNames	(	self,
		data,
		columnIndex = None )

protectedinherited

Gets the content of each of the column levels for a multilevel
table.

Definition at line 204 of file functors.py.

    def _get_data_columnLevelNames(self, data, columnIndex=None):
        """Gets the content of each of the column levels for a multilevel
        table.
        """
        if columnIndex is None:
            columnIndex = data.get(component="columns")
 
        columnLevels = columnIndex.names
        columnLevelNames = {
            level: list(np.unique(np.array([c for c in columnIndex])[:, i]))
            for i, level in enumerate(columnLevels)
        }
        return columnLevelNames
 

◆ _get_data_columnLevels()

lsst.pipe.tasks.functors.Functor._get_data_columnLevels	(	self,
		data,
		columnIndex = None )

protectedinherited

Gets the names of the column index levels.

This should only be called in the context of a multilevel table.

Parameters
----------
data : various
    The data to be read, can be a
    `~lsst.daf.butler.DeferredDatasetHandle` or
    `~lsst.pipe.base.InMemoryDatasetHandle`.
columnIndex (optional): pandas `~pandas.Index` object
    If not passed, then it is read from the
    `~lsst.daf.butler.DeferredDatasetHandle`
    for `~lsst.pipe.base.InMemoryDatasetHandle`.

Definition at line 184 of file functors.py.

    def _get_data_columnLevels(self, data, columnIndex=None):
        """Gets the names of the column index levels.
 
        This should only be called in the context of a multilevel table.
 
        Parameters
        ----------
        data : various
            The data to be read, can be a
            `~lsst.daf.butler.DeferredDatasetHandle` or
            `~lsst.pipe.base.InMemoryDatasetHandle`.
        columnIndex (optional): pandas `~pandas.Index` object
            If not passed, then it is read from the
            `~lsst.daf.butler.DeferredDatasetHandle`
            for `~lsst.pipe.base.InMemoryDatasetHandle`.
        """
        if columnIndex is None:
            columnIndex = data.get(component="columns")
        return columnIndex.names
 

◆ _setLevels()

lsst.pipe.tasks.functors.Functor._setLevels	(		self,
			df )

protectedinherited

Definition at line 340 of file functors.py.

    def _setLevels(self, df):
        levelsToDrop = [n for n in df.columns.names if n not in self._dfLevels]
        df.columns = df.columns.droplevel(levelsToDrop)
        return df
 

◆ columns()

lsst.pipe.tasks.functors.Functor.columns ( self )

inherited

Columns required to perform calculation.

Definition at line 178 of file functors.py.

    def columns(self):
        """Columns required to perform calculation."""
        if not hasattr(self, '_columns'):
            raise NotImplementedError('Must define columns property or _columns attribute')
        return self._columns
 

◆ difference()

lsst.pipe.tasks.functors.Functor.difference	(		self,
			data1,
			data2,
		**	kwargs )

inherited

Computes difference between functor called on two different
DataFrame/Handle objects.

Definition at line 360 of file functors.py.

    def difference(self, data1, data2, **kwargs):
        """Computes difference between functor called on two different
        DataFrame/Handle objects.
        """
        return self(data1, **kwargs) - self(data2, **kwargs)
 

◆ fail()

lsst.pipe.tasks.functors.Functor.fail	(		self,
			df )

inherited

Definition at line 366 of file functors.py.

    def fail(self, df):
        return pd.Series(np.full(len(df), np.nan), index=df.index)
 

◆ multilevelColumns()

lsst.pipe.tasks.functors.Functor.multilevelColumns	(	self,
		data,
		columnIndex = None,
		returnTuple = False )

inherited

Returns columns needed by functor from multilevel dataset.

To access tables with multilevel column structure, the
`~lsst.daf.butler.DeferredDatasetHandle` or
`~lsst.pipe.base.InMemoryDatasetHandle` needs to be passed
either a list of tuples or a dictionary.

Parameters
----------
data : various
    The data as either `~lsst.daf.butler.DeferredDatasetHandle`, or
    `~lsst.pipe.base.InMemoryDatasetHandle`.
columnIndex (optional): pandas `~pandas.Index` object
    Either passed or read in from
    `~lsst.daf.butler.DeferredDatasetHandle`.
`returnTuple` : `bool`
    If true, then return a list of tuples rather than the column
    dictionary specification.
    This is set to `True` by `CompositeFunctor` in order to be able to
    combine columns from the various component functors.

Reimplemented in lsst.pipe.tasks.functors.Color, and lsst.pipe.tasks.functors.CompositeFunctor.

Definition at line 237 of file functors.py.

    def multilevelColumns(self, data, columnIndex=None, returnTuple=False):
        """Returns columns needed by functor from multilevel dataset.
 
        To access tables with multilevel column structure, the
        `~lsst.daf.butler.DeferredDatasetHandle` or
        `~lsst.pipe.base.InMemoryDatasetHandle` needs to be passed
        either a list of tuples or a dictionary.
 
        Parameters
        ----------
        data : various
            The data as either `~lsst.daf.butler.DeferredDatasetHandle`, or
            `~lsst.pipe.base.InMemoryDatasetHandle`.
        columnIndex (optional): pandas `~pandas.Index` object
            Either passed or read in from
            `~lsst.daf.butler.DeferredDatasetHandle`.
        `returnTuple` : `bool`
            If true, then return a list of tuples rather than the column
            dictionary specification.
            This is set to `True` by `CompositeFunctor` in order to be able to
            combine columns from the various component functors.
 
        """
        if not isinstance(data, (DeferredDatasetHandle, InMemoryDatasetHandle)):
            raise RuntimeError(f"Unexpected data type. Got {get_full_type_name(data)}.")
 
        if columnIndex is None:
            columnIndex = data.get(component="columns")
 
        # Confirm that the dataset has the column levels the functor is
        # expecting it to have.
        columnLevels = self._get_data_columnLevels(data, columnIndex)
 
        columnDict = {'column': self.columns,
                      'dataset': self.dataset}
        if self.filt is None:
            columnLevelNames = self._get_data_columnLevelNames(data, columnIndex)
            if "band" in columnLevels:
                if self.dataset == "ref":
                    columnDict["band"] = columnLevelNames["band"][0]
                else:
                    raise ValueError(f"'filt' not set for functor {self.name}"
                                     f"(dataset {self.dataset}) "
                                     "and DataFrame "
                                     "contains multiple filters in column index. "
                                     "Set 'filt' or set 'dataset' to 'ref'.")
        else:
            columnDict['band'] = self.filt
 
        if returnTuple:
            return self._colsFromDict(columnDict, columnIndex=columnIndex)
        else:
            return columnDict
 

◆ name()

lsst.pipe.tasks.functors.Functor.name ( self )

inherited

Full name of functor (suitable for figure labels).

Definition at line 370 of file functors.py.

    def name(self):
        """Full name of functor (suitable for figure labels)."""
        return NotImplementedError
 

◆ noDup()

lsst.pipe.tasks.functors.Functor.noDup ( self )

inherited

Do not explode by band if used on object table.

Definition at line 170 of file functors.py.

    def noDup(self):
        """Do not explode by band if used on object table."""
        if self._noDup is not None:
            return self._noDup
        else:
            return self._defaultNoDup
 

◆ shortname()

lsst.pipe.tasks.functors.Functor.shortname ( self )

inherited

Short name of functor (suitable for column name/dict key).

Reimplemented in lsst.pipe.tasks.functors.Color, and lsst.pipe.tasks.functors.MagDiff.

Definition at line 375 of file functors.py.

    def shortname(self):
        """Short name of functor (suitable for column name/dict key)."""
        return self.name
 
 

Member Data Documentation

◆ _columns

list lsst.pipe.tasks.functors.HtmIndex20._columns = [self.ra, self.dec]

protected

Definition at line 790 of file functors.py.

◆ _defaultDataset

str lsst.pipe.tasks.functors.Functor._defaultDataset = 'ref'

staticprotectedinherited

Definition at line 159 of file functors.py.

◆ _defaultNoDup

bool lsst.pipe.tasks.functors.Functor._defaultNoDup = False

staticprotectedinherited

Definition at line 161 of file functors.py.

◆ _dfLevels

tuple lsst.pipe.tasks.functors.Functor._dfLevels = ('column',)

staticprotectedinherited

Definition at line 160 of file functors.py.

◆ _noDup

lsst.pipe.tasks.functors.Functor._noDup = noDup

protectedinherited

Definition at line 166 of file functors.py.

◆ _radians

bool lsst.pipe.tasks.functors.HtmIndex20._radians = True

staticprotected

Definition at line 784 of file functors.py.

◆ dataset

str lsst.pipe.tasks.functors.Functor.dataset = dataset if dataset is not None else self._defaultDataset

inherited

Definition at line 165 of file functors.py.

◆ dec

lsst.pipe.tasks.functors.HtmIndex20.dec = dec

Definition at line 789 of file functors.py.

◆ filt

lsst.pipe.tasks.functors.Functor.filt = filt

inherited

Reimplemented in lsst.pipe.tasks.functors.Color, lsst.pipe.tasks.functors.Color, lsst.pipe.tasks.functors.CompositeFunctor, and lsst.pipe.tasks.functors.CompositeFunctor.

Definition at line 164 of file functors.py.

◆ htmLevel

int lsst.pipe.tasks.functors.HtmIndex20.htmLevel = 20

static

Definition at line 783 of file functors.py.

◆ log

lsst.pipe.tasks.functors.Functor.log = logging.getLogger(type(self).__name__)

inherited

Definition at line 167 of file functors.py.

◆ name

lsst.pipe.tasks.functors.Functor.name

inherited

Definition at line 353 of file functors.py.

◆ pixelator

lsst.pipe.tasks.functors.HtmIndex20.pixelator = sphgeom.HtmPixelization(self.htmLevel)

Definition at line 787 of file functors.py.

◆ ra

lsst.pipe.tasks.functors.HtmIndex20.ra = ra

Definition at line 788 of file functors.py.

The documentation for this class was generated from the following file:

/j/snowflake/release/lsstsw/stack/lsst-scipipe-12.1.0/Linux/pipe_tasks/g99af87f6a8+b0f4ad7b8d/python/lsst/pipe/tasks/functors.py

Public Member Functions

Public Attributes

Static Public Attributes

Protected Member Functions

Protected Attributes

Static Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ __call__()

◆ _colsFromDict()

◆ _dropna()

◆ _func()

◆ _get_columnIndex()

◆ _get_data()

◆ _get_data_columnLevelNames()

◆ _get_data_columnLevels()

◆ _setLevels()

◆ columns()

◆ difference()

◆ fail()

◆ multilevelColumns()

◆ name()

◆ noDup()

◆ shortname()

Member Data Documentation

◆ _columns

◆ _defaultDataset

◆ _defaultNoDup

◆ _dfLevels

◆ _noDup

◆ _radians

◆ dataset

◆ dec

◆ filt

◆ htmLevel

◆ log

◆ name

◆ pixelator

◆ ra

◆ init()

◆ call()