LSST Applications  21.0.0-172-gfb10e10a+18fedfabac,22.0.0+297cba6710,22.0.0+80564b0ff1,22.0.0+8d77f4f51a,22.0.0+a28f4c53b1,22.0.0+dcf3732eb2,22.0.1-1-g7d6de66+2a20fdde0d,22.0.1-1-g8e32f31+297cba6710,22.0.1-1-geca5380+7fa3b7d9b6,22.0.1-12-g44dc1dc+2a20fdde0d,22.0.1-15-g6a90155+515f58c32b,22.0.1-16-g9282f48+790f5f2caa,22.0.1-2-g92698f7+dcf3732eb2,22.0.1-2-ga9b0f51+7fa3b7d9b6,22.0.1-2-gd1925c9+bf4f0e694f,22.0.1-24-g1ad7a390+a9625a72a8,22.0.1-25-g5bf6245+3ad8ecd50b,22.0.1-25-gb120d7b+8b5510f75f,22.0.1-27-g97737f7+2a20fdde0d,22.0.1-32-gf62ce7b1+aa4237961e,22.0.1-4-g0b3f228+2a20fdde0d,22.0.1-4-g243d05b+871c1b8305,22.0.1-4-g3a563be+32dcf1063f,22.0.1-4-g44f2e3d+9e4ab0f4fa,22.0.1-42-gca6935d93+ba5e5ca3eb,22.0.1-5-g15c806e+85460ae5f3,22.0.1-5-g58711c4+611d128589,22.0.1-5-g75bb458+99c117b92f,22.0.1-6-g1c63a23+7fa3b7d9b6,22.0.1-6-g50866e6+84ff5a128b,22.0.1-6-g8d3140d+720564cf76,22.0.1-6-gd805d02+cc5644f571,22.0.1-8-ge5750ce+85460ae5f3,master-g6e05de7fdc+babf819c66,master-g99da0e417a+8d77f4f51a,w.2021.48
LSST Data Management Base Package
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.meas.algorithms.ingestIndexReferenceTask.ConvertReferenceCatalogBase Class Reference
Inheritance diagram for lsst.meas.algorithms.ingestIndexReferenceTask.ConvertReferenceCatalogBase:
lsst.meas.algorithms.convertReferenceCatalog.ConvertReferenceCatalogTask lsst.meas.algorithms.ingestIndexReferenceTask.IngestIndexedReferenceTask

Public Member Functions

def __init__ (self, *args, **kwargs)
 
def run (self, inputFiles)
 
def makeSchema (self, dtype)
 

Public Attributes

 indexer
 

Static Public Attributes

bool canMultiprocess = False
 
 ConfigClass = ConvertReferenceCatalogConfig
 

Detailed Description

Base class for producing and loading indexed reference catalogs,
shared between gen2 and gen3.

This implements an indexing scheme based on hierarchical triangular
mesh (HTM). The term index really means breaking the catalog into
localized chunks called shards.  In this case each shard contains
the entries from the catalog in a single HTM trixel

For producing catalogs this task makes the following assumptions
about the input catalogs:
- RA, Dec are in decimal degrees.
- Epoch is available in a column, in a format supported by astropy.time.Time.
- There are no off-diagonal covariance terms, such as covariance
  between RA and Dec, or between PM RA and PM Dec. Support for such
 covariance would have to be added to to the config, including consideration
 of the units in the input catalog.

Definition at line 305 of file ingestIndexReferenceTask.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.meas.algorithms.ingestIndexReferenceTask.ConvertReferenceCatalogBase.__init__ (   self,
args,
**  kwargs 
)

Reimplemented in lsst.meas.algorithms.convertReferenceCatalog.ConvertReferenceCatalogTask.

Definition at line 326 of file ingestIndexReferenceTask.py.

326  def __init__(self, *args, **kwargs):
327  super().__init__(*args, **kwargs)
328  self.indexer = IndexerRegistry[self.config.dataset_config.indexer.name](
329  self.config.dataset_config.indexer.active)
330  self.makeSubtask('file_reader')
331 

Member Function Documentation

◆ makeSchema()

def lsst.meas.algorithms.ingestIndexReferenceTask.ConvertReferenceCatalogBase.makeSchema (   self,
  dtype 
)
Make the schema to use in constructing the persisted catalogs.

Parameters
----------
dtype : `numpy.dtype`
    Data type describing each entry in ``config.extra_col_names``
    for the catalogs being ingested.

Returns
-------
schemaAndKeyMap : `tuple` of (`lsst.afw.table.Schema`, `dict`)
    A tuple containing two items:
    - The schema for the output source catalog.
    - A map of catalog keys to use in filling the record

Definition at line 400 of file ingestIndexReferenceTask.py.

400  def makeSchema(self, dtype):
401  """Make the schema to use in constructing the persisted catalogs.
402 
403  Parameters
404  ----------
405  dtype : `numpy.dtype`
406  Data type describing each entry in ``config.extra_col_names``
407  for the catalogs being ingested.
408 
409  Returns
410  -------
411  schemaAndKeyMap : `tuple` of (`lsst.afw.table.Schema`, `dict`)
412  A tuple containing two items:
413  - The schema for the output source catalog.
414  - A map of catalog keys to use in filling the record
415  """
416  # make a schema with the standard fields
417  schema = LoadReferenceObjectsTask.makeMinimalSchema(
418  filterNameList=self.config.mag_column_list,
419  addCentroid=False,
420  addIsPhotometric=bool(self.config.is_photometric_name),
421  addIsResolved=bool(self.config.is_resolved_name),
422  addIsVariable=bool(self.config.is_variable_name),
423  coordErrDim=2 if bool(self.config.ra_err_name) else 0,
424  addProperMotion=2 if bool(self.config.pm_ra_name) else 0,
425  properMotionErrDim=2 if bool(self.config.pm_ra_err_name) else 0,
426  addParallax=bool(self.config.parallax_name),
427  )
428  keysToSkip = set(("id", "centroid_x", "centroid_y", "hasCentroid"))
429  key_map = {fieldName: schema[fieldName].asKey() for fieldName in schema.getOrderedNames()
430  if fieldName not in keysToSkip}
431 
432  def addField(name):
433  if dtype[name].kind == 'U':
434  # dealing with a string like thing. Need to get type and size.
435  at_size = dtype[name].itemsize
436  return schema.addField(name, type=str, size=at_size)
437  else:
438  at_type = dtype[name].type
439  return schema.addField(name, at_type)
440 
441  for col in self.config.extra_col_names:
442  key_map[col] = addField(col)
443  return schema, key_map
444 
daf::base::PropertySet * set
Definition: fits.cc:912

◆ run()

def lsst.meas.algorithms.ingestIndexReferenceTask.ConvertReferenceCatalogBase.run (   self,
  inputFiles 
)
Index a set of files comprising a reference catalog.

Outputs are persisted in the butler repository.

Parameters
----------
inputFiles : `list`
    A list of file paths to read.

Definition at line 332 of file ingestIndexReferenceTask.py.

332  def run(self, inputFiles):
333  """Index a set of files comprising a reference catalog.
334 
335  Outputs are persisted in the butler repository.
336 
337  Parameters
338  ----------
339  inputFiles : `list`
340  A list of file paths to read.
341  """
342  self._preRun()
343  schema, key_map = self._saveMasterSchema(inputFiles[0])
344  # create an HTM we can interrogate about pixel ids
345  htm = lsst.sphgeom.HtmPixelization(self.indexer.htm.get_depth())
346  filenames = self._getButlerFilenames(htm)
347  worker = self.config.manager.target(filenames,
348  self.config,
349  self.file_reader,
350  self.indexer,
351  schema,
352  key_map,
353  htm.universe()[0],
354  addRefCatMetadata,
355  self.log)
356  result = worker.run(inputFiles)
357 
358  self._persistConfig()
359  self._postRun(result)
360 
HtmPixelization provides HTM indexing of points and regions.
def run(self, coaddExposures, bbox, wcs)
Definition: getTemplate.py:603

Member Data Documentation

◆ canMultiprocess

bool lsst.meas.algorithms.ingestIndexReferenceTask.ConvertReferenceCatalogBase.canMultiprocess = False
static

Definition at line 323 of file ingestIndexReferenceTask.py.

◆ ConfigClass

lsst.meas.algorithms.ingestIndexReferenceTask.ConvertReferenceCatalogBase.ConfigClass = ConvertReferenceCatalogConfig
static

Definition at line 324 of file ingestIndexReferenceTask.py.

◆ indexer

lsst.meas.algorithms.ingestIndexReferenceTask.ConvertReferenceCatalogBase.indexer

Definition at line 328 of file ingestIndexReferenceTask.py.


The documentation for this class was generated from the following file: