LSST Applications  21.0.0-172-gfb10e10a+18fedfabac,22.0.0+297cba6710,22.0.0+80564b0ff1,22.0.0+8d77f4f51a,22.0.0+a28f4c53b1,22.0.0+dcf3732eb2,22.0.1-1-g7d6de66+2a20fdde0d,22.0.1-1-g8e32f31+297cba6710,22.0.1-1-geca5380+7fa3b7d9b6,22.0.1-12-g44dc1dc+2a20fdde0d,22.0.1-15-g6a90155+515f58c32b,22.0.1-16-g9282f48+790f5f2caa,22.0.1-2-g92698f7+dcf3732eb2,22.0.1-2-ga9b0f51+7fa3b7d9b6,22.0.1-2-gd1925c9+bf4f0e694f,22.0.1-24-g1ad7a390+a9625a72a8,22.0.1-25-g5bf6245+3ad8ecd50b,22.0.1-25-gb120d7b+8b5510f75f,22.0.1-27-g97737f7+2a20fdde0d,22.0.1-32-gf62ce7b1+aa4237961e,22.0.1-4-g0b3f228+2a20fdde0d,22.0.1-4-g243d05b+871c1b8305,22.0.1-4-g3a563be+32dcf1063f,22.0.1-4-g44f2e3d+9e4ab0f4fa,22.0.1-42-gca6935d93+ba5e5ca3eb,22.0.1-5-g15c806e+85460ae5f3,22.0.1-5-g58711c4+611d128589,22.0.1-5-g75bb458+99c117b92f,22.0.1-6-g1c63a23+7fa3b7d9b6,22.0.1-6-g50866e6+84ff5a128b,22.0.1-6-g8d3140d+720564cf76,22.0.1-6-gd805d02+cc5644f571,22.0.1-8-ge5750ce+85460ae5f3,master-g6e05de7fdc+babf819c66,master-g99da0e417a+8d77f4f51a,w.2021.48
LSST Data Management Base Package
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.meas.base.diaCalculation.DiaObjectCalculationTask Class Reference
Inheritance diagram for lsst.meas.base.diaCalculation.DiaObjectCalculationTask:
lsst.meas.base.catalogCalculation.CatalogCalculationTask

Public Member Functions

def __init__ (self, plugMetadata=None, **kwargs)
 
def initializePlugins (self)
 
def run (self, diaObjectCat, diaSourceCat, updatedDiaObjectIds, filterNames)
 
def callCompute (self, diaObjectCat, diaSourceCat, updatedDiaObjectIds, filterNames)
 
def run (self, measCat)
 
def callCompute (self, catalog)
 

Public Attributes

 plugMetadata
 
 plugins
 
 outputCols
 
 executionDict
 
 schema
 

Static Public Attributes

 ConfigClass = DiaObjectCalculationConfig
 

Detailed Description

Run plugins which operate on a catalog of DIA sources.

This task facilitates running plugins which will operate on a source
catalog. These plugins may do things such as classifying an object based
on source record entries inserted during a measurement task.

This task differs from CatalogCaculationTask in the following ways:

-No multi mode is available for plugins. All plugins are assumed to run
 in single mode.

-Input and output catalog types are assumed to be `pandas.DataFrames` with
 columns following those used in the Apdb.

-No schema argument is passed to the plugins. Each plugin specifies
 output columns and required inputs.

Parameters
----------
plugMetaData : `lsst.daf.base.PropertyList` or `None`
    Will be modified in-place to contain metadata about the plugins being
    run. If `None`, an empty `~lsst.daf.base.PropertyList` will be
    created.
**kwargs
    Additional arguments passed to the superclass constructor.

Notes
-----
Plugins may either take an entire catalog to work on at a time, or work on
individual records.

Definition at line 172 of file diaCalculation.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.meas.base.diaCalculation.DiaObjectCalculationTask.__init__ (   self,
  plugMetadata = None,
**  kwargs 
)

Definition at line 207 of file diaCalculation.py.

207  def __init__(self, plugMetadata=None, **kwargs):
208  lsst.pipe.base.Task.__init__(self, **kwargs)
209  if plugMetadata is None:
210  plugMetadata = lsst.daf.base.PropertyList()
211  self.plugMetadata = plugMetadata
212  self.plugins = PluginMap()
213  self.outputCols = []
214 
215  self.initializePlugins()
216 
Class for storing ordered metadata with comments.
Definition: PropertyList.h:68

Member Function Documentation

◆ callCompute() [1/2]

def lsst.meas.base.catalogCalculation.CatalogCalculationTask.callCompute (   self,
  catalog 
)
inherited
Run each of the plugins on the catalog.

Parameters
----------
catalog : `lsst.afw.table.SourceCatalog`
    The catalog on which the plugins will operate.

Definition at line 241 of file catalogCalculation.py.

241  def callCompute(self, catalog):
242  """Run each of the plugins on the catalog.
243 
244  Parameters
245  ----------
246  catalog : `lsst.afw.table.SourceCatalog`
247  The catalog on which the plugins will operate.
248  """
249  for runlevel in sorted(self.executionDict):
250  # Run all of the plugins which take a whole catalog first
251  for plug in self.executionDict[runlevel].multi:
252  with CCContext(plug, catalog, self.log):
253  plug.calculate(catalog)
254  # Run all the plugins which take single catalog entries
255  for measRecord in catalog:
256  for plug in self.executionDict[runlevel].single:
257  with CCContext(plug, measRecord, self.log):
258  plug.calculate(measRecord)

◆ callCompute() [2/2]

def lsst.meas.base.diaCalculation.DiaObjectCalculationTask.callCompute (   self,
  diaObjectCat,
  diaSourceCat,
  updatedDiaObjectIds,
  filterNames 
)
Run each of the plugins on the catalog.

For catalog column names see the lsst.cat schema definitions for the
DiaObject and DiaSource tables (http://github.com/lsst/cat).

Parameters
----------
diaObjectCat : `pandas.DataFrame`
    DiaObjects to update values of and append new objects to. DataFrame
    should be indexed on "diaObjectId"
diaSourceCat : `pandas.DataFrame`
    DiaSources associated with the DiaObjects in diaObjectCat.
    DataFrame must be indexed on
    ["diaObjectId", "filterName", "diaSourceId"]`
updatedDiaObjectIds : `numpy.ndarray`
    Integer ids of the DiaObjects to update and create.
filterNames : `list` of `str`
    List of string names of filters to be being processed.

Returns
-------
returnStruct : `lsst.pipe.base.Struct`
    Struct containing:

    ``diaObjectCat``
        Full set of DiaObjects including both un-updated and
        updated/new DiaObjects (`pandas.DataFrame`).
    ``updatedDiaObjects``
        Catalog of DiaObjects  that were updated or created by this
        task (`pandas.DataFrame`).

Raises
------
KeyError
    Raises if `pandas.DataFrame` indexing is not properly set.

Definition at line 346 of file diaCalculation.py.

350  filterNames):
351  """Run each of the plugins on the catalog.
352 
353  For catalog column names see the lsst.cat schema definitions for the
354  DiaObject and DiaSource tables (http://github.com/lsst/cat).
355 
356  Parameters
357  ----------
358  diaObjectCat : `pandas.DataFrame`
359  DiaObjects to update values of and append new objects to. DataFrame
360  should be indexed on "diaObjectId"
361  diaSourceCat : `pandas.DataFrame`
362  DiaSources associated with the DiaObjects in diaObjectCat.
363  DataFrame must be indexed on
364  ["diaObjectId", "filterName", "diaSourceId"]`
365  updatedDiaObjectIds : `numpy.ndarray`
366  Integer ids of the DiaObjects to update and create.
367  filterNames : `list` of `str`
368  List of string names of filters to be being processed.
369 
370  Returns
371  -------
372  returnStruct : `lsst.pipe.base.Struct`
373  Struct containing:
374 
375  ``diaObjectCat``
376  Full set of DiaObjects including both un-updated and
377  updated/new DiaObjects (`pandas.DataFrame`).
378  ``updatedDiaObjects``
379  Catalog of DiaObjects that were updated or created by this
380  task (`pandas.DataFrame`).
381 
382  Raises
383  ------
384  KeyError
385  Raises if `pandas.DataFrame` indexing is not properly set.
386  """
387  # DiaObjects will be updated in place.
388  diaObjectsToUpdate = diaObjectCat.loc[updatedDiaObjectIds, :]
389  self.log.info("Calculating summary stats for %i DiaObjects",
390  len(diaObjectsToUpdate))
391 
392  updatingDiaSources = diaSourceCat.loc[updatedDiaObjectIds, :]
393  diaSourcesGB = updatingDiaSources.groupby(level=0)
394  for runlevel in sorted(self.executionDict):
395  for plug in self.executionDict[runlevel].single:
396  if plug.needsFilter:
397  continue
398  for updatedDiaObjectId in updatedDiaObjectIds:
399 
400  # Sub-select diaSources associated with this diaObject.
401  objDiaSources = updatingDiaSources.loc[updatedDiaObjectId]
402 
403  # Sub-select on diaSources observed in the current filter.
404  with CCContext(plug, updatedDiaObjectId, self.log):
405  # We feed the catalog we need to update and the id
406  # so as to get a few into the catalog and not a copy.
407  # This updates the values in the catalog.
408  plug.calculate(diaObjects=diaObjectsToUpdate,
409  diaObjectId=updatedDiaObjectId,
410  diaSources=objDiaSources,
411  filterDiaSources=None,
412  filterName=None)
413  for plug in self.executionDict[runlevel].multi:
414  if plug.needsFilter:
415  continue
416  with CCContext(plug, diaObjectsToUpdate, self.log):
417  plug.calculate(diaObjects=diaObjectsToUpdate,
418  diaSources=diaSourcesGB,
419  filterDiaSources=None,
420  filterName=None)
421 
422  for filterName in filterNames:
423  try:
424  updatingFilterDiaSources = updatingDiaSources.loc[
425  (slice(None), filterName), :
426  ]
427  except KeyError:
428  self.log.warning("No DiaSource data with fitler=%s. "
429  "Continuing...", filterName)
430  continue
431  # Level=0 here groups by diaObjectId.
432  filterDiaSourcesGB = updatingFilterDiaSources.groupby(level=0)
433 
434  for runlevel in sorted(self.executionDict):
435  for plug in self.executionDict[runlevel].single:
436  if not plug.needsFilter:
437  continue
438  for updatedDiaObjectId in updatedDiaObjectIds:
439 
440  # Sub-select diaSources associated with this diaObject.
441  objDiaSources = updatingDiaSources.loc[updatedDiaObjectId]
442 
443  # Sub-select on diaSources observed in the current filter.
444  try:
445  filterObjDiaSources = objDiaSources.loc[filterName]
446  except KeyError:
447  self.log.warning(
448  "DiaObjectId={updatedDiaObjectId} has no "
449  "DiaSources for filter=%s. "
450  "Continuing...", filterName)
451  with CCContext(plug, updatedDiaObjectId, self.log):
452  # We feed the catalog we need to update and the id
453  # so as to get a few into the catalog and not a copy.
454  # This updates the values in the catalog.
455  plug.calculate(diaObjects=diaObjectsToUpdate,
456  diaObjectId=updatedDiaObjectId,
457  diaSources=objDiaSources,
458  filterDiaSources=filterObjDiaSources,
459  filterName=filterName)
460  for plug in self.executionDict[runlevel].multi:
461  if not plug.needsFilter:
462  continue
463  with CCContext(plug, diaObjectsToUpdate, self.log):
464  plug.calculate(diaObjects=diaObjectsToUpdate,
465  diaSources=diaSourcesGB,
466  filterDiaSources=filterDiaSourcesGB,
467  filterName=filterName)
468  # Need to store the newly updated diaObjects directly as the editing
469  # a view into diaObjectsToUpdate does not update the values of
470  # diaObjectCat.
471  diaObjectCat.loc[updatedDiaObjectIds, :] = diaObjectsToUpdate
472  return lsst.pipe.base.Struct(
473  diaObjectCat=diaObjectCat,
474  updatedDiaObjects=diaObjectsToUpdate)
475 

◆ initializePlugins()

def lsst.meas.base.diaCalculation.DiaObjectCalculationTask.initializePlugins (   self)
Initialize the plugins according to the configuration.

Reimplemented from lsst.meas.base.catalogCalculation.CatalogCalculationTask.

Definition at line 217 of file diaCalculation.py.

217  def initializePlugins(self):
218  """Initialize the plugins according to the configuration.
219  """
220 
221  pluginType = namedtuple('pluginType', 'single multi')
222  self.executionDict = {}
223  # Read the properties for each plugin. Allocate a dictionary entry for
224  # each run level. Verify that the plugins are above the minimum run
225  # level for an catalogCalculation plugin. For each run level, the
226  # plugins are sorted into either single record, or multi record groups
227  # to later be run appropriately
228  for executionOrder, name, config, PluginClass in sorted(self.config.plugins.apply()):
229  if executionOrder not in self.executionDict:
230  self.executionDict[executionOrder] = pluginType(single=[], multi=[])
231  if PluginClass.getExecutionOrder() >= BasePlugin.DEFAULT_CATALOGCALCULATION:
232  plug = PluginClass(config, name, metadata=self.plugMetadata)
233 
234  self._validatePluginCols(plug)
235 
236  self.plugins[name] = plug
237  if plug.plugType == 'single':
238  self.executionDict[executionOrder].single.append(plug)
239  elif plug.plugType == 'multi':
240  self.executionDict[executionOrder].multi.append(plug)
241  else:
242  errorTuple = (PluginClass, PluginClass.getExecutionOrder(),
243  BasePlugin.DEFAULT_CATALOGCALCULATION)
244  raise ValueError("{} has an execution order less than the minimum for an catalogCalculation "
245  "plugin. Value {} : Minimum {}".format(*errorTuple))
246 
def format(config, name=None, writeSourceLine=True, prefix="", verbose=False)
Definition: history.py:174

◆ run() [1/2]

def lsst.meas.base.diaCalculation.DiaObjectCalculationTask.run (   self,
  diaObjectCat,
  diaSourceCat,
  updatedDiaObjectIds,
  filterNames 
)
The entry point for the DIA catalog calculation task.

Run method both updates the values in the diaObjectCat and appends
newly created DiaObjects to the catalog. For catalog column names
see the lsst.cat schema definitions for the DiaObject and DiaSource
tables (http://github.com/lsst/cat).

Parameters
----------
diaObjectCat : `pandas.DataFrame`
    DiaObjects to update values of and append new objects to. DataFrame
    should be indexed on "diaObjectId"
diaSourceCat : `pandas.DataFrame`
    DiaSources associated with the DiaObjects in diaObjectCat.
    DataFrame should be indexed on
    `["diaObjectId", "filterName", "diaSourceId"]`
updatedDiaObjectIds : `numpy.ndarray`
    Integer ids of the DiaObjects to update and create.
filterNames : `list` of `str`
    List of string names of filters to be being processed.

Returns
-------
returnStruct : `lsst.pipe.base.Struct`
    Struct containing:

    ``diaObjectCat``
        Full set of DiaObjects including both un-updated and
        updated/new DiaObjects (`pandas.DataFrame`).
    ``updatedDiaObjects``
        Catalog of DiaObjects  that were updated or created by this
        task (`pandas.DataFrame`).

Definition at line 278 of file diaCalculation.py.

282  filterNames):
283  """The entry point for the DIA catalog calculation task.
284 
285  Run method both updates the values in the diaObjectCat and appends
286  newly created DiaObjects to the catalog. For catalog column names
287  see the lsst.cat schema definitions for the DiaObject and DiaSource
288  tables (http://github.com/lsst/cat).
289 
290  Parameters
291  ----------
292  diaObjectCat : `pandas.DataFrame`
293  DiaObjects to update values of and append new objects to. DataFrame
294  should be indexed on "diaObjectId"
295  diaSourceCat : `pandas.DataFrame`
296  DiaSources associated with the DiaObjects in diaObjectCat.
297  DataFrame should be indexed on
298  `["diaObjectId", "filterName", "diaSourceId"]`
299  updatedDiaObjectIds : `numpy.ndarray`
300  Integer ids of the DiaObjects to update and create.
301  filterNames : `list` of `str`
302  List of string names of filters to be being processed.
303 
304  Returns
305  -------
306  returnStruct : `lsst.pipe.base.Struct`
307  Struct containing:
308 
309  ``diaObjectCat``
310  Full set of DiaObjects including both un-updated and
311  updated/new DiaObjects (`pandas.DataFrame`).
312  ``updatedDiaObjects``
313  Catalog of DiaObjects that were updated or created by this
314  task (`pandas.DataFrame`).
315  """
316  if diaObjectCat.index.name is None:
317  diaObjectCat.set_index("diaObjectId", inplace=True, drop=False)
318  elif diaObjectCat.index.name != "diaObjectId":
319  self.log.warning(
320  "Input diaObjectCat is indexed on column(s) incompatible with "
321  "this task. Should be indexed on 'diaObjectId'. Trying to set "
322  "index regardless")
323  diaObjectCat.set_index("diaObjectId", inplace=True, drop=False)
324 
325  # ``names`` by default is FrozenList([None]) hence we access the first
326  # element and test for None.
327  if diaSourceCat.index.names[0] is None:
328  diaSourceCat.set_index(
329  ["diaObjectId", "filterName", "diaSourceId"],
330  inplace=True,
331  drop=False)
332  elif (diaSourceCat.index.names
333  != ["diaObjectId", "filterName", "diaSourceId"]):
334  diaSourceCat.reset_index(inplace=True)
335  diaSourceCat.set_index(
336  ["diaObjectId", "filterName", "diaSourceId"],
337  inplace=True,
338  drop=False)
339 
340  return self.callCompute(diaObjectCat,
341  diaSourceCat,
342  updatedDiaObjectIds,
343  filterNames)
344 

◆ run() [2/2]

def lsst.meas.base.catalogCalculation.CatalogCalculationTask.run (   self,
  measCat 
)
inherited
The entry point for the catalog calculation task.

Parameters
----------
meascat : `lsst.afw.table.SourceCatalog`
    Catalog for measurement.

Definition at line 231 of file catalogCalculation.py.

231  def run(self, measCat):
232  """The entry point for the catalog calculation task.
233 
234  Parameters
235  ----------
236  meascat : `lsst.afw.table.SourceCatalog`
237  Catalog for measurement.
238  """
239  self.callCompute(measCat)
240 
def run(self, coaddExposures, bbox, wcs)
Definition: getTemplate.py:603

Member Data Documentation

◆ ConfigClass

lsst.meas.base.diaCalculation.DiaObjectCalculationTask.ConfigClass = DiaObjectCalculationConfig
static

Definition at line 204 of file diaCalculation.py.

◆ executionDict

lsst.meas.base.diaCalculation.DiaObjectCalculationTask.executionDict

Definition at line 222 of file diaCalculation.py.

◆ outputCols

lsst.meas.base.diaCalculation.DiaObjectCalculationTask.outputCols

Definition at line 213 of file diaCalculation.py.

◆ plugins

lsst.meas.base.diaCalculation.DiaObjectCalculationTask.plugins

Definition at line 212 of file diaCalculation.py.

◆ plugMetadata

lsst.meas.base.diaCalculation.DiaObjectCalculationTask.plugMetadata

Definition at line 211 of file diaCalculation.py.

◆ schema

lsst.meas.base.catalogCalculation.CatalogCalculationTask.schema
inherited

Definition at line 196 of file catalogCalculation.py.


The documentation for this class was generated from the following file: