lsst.meas.algorithms.ingestIndexManager.IngestGaiaManager Class Reference

Inheritance diagram for lsst.meas.algorithms.ingestIndexManager.IngestGaiaManager:

Public Member Functions
def	run (self, inputFiles)
def	getCatalog (self, pixelId, schema, nNewElements)

Static Public Member Functions
def	computeCoord (row, ra_name, dec_name)

Public Attributes
	filenames
	config
	file_reader
	indexer
	schema
	key_map
	htmRange
	addRefCatMetadata
	log
	coord_err_unit
	nInputFiles

Detailed Description

Special-case ingest manager to deal with Gaia fluxes.

Definition at line 405 of file ingestIndexManager.py.

Member Function Documentation

computeCoord()

def lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.computeCoord (   row,   ra_name,   dec_name  )

staticinherited

Create an ICRS coord. from a row of a catalog being ingested.

Parameters
----------
row : `numpy.ndarray`
    Row from catalog being ingested.
ra_name : `str`
    Name of RA key in catalog being ingested.
dec_name : `str`
    Name of Dec key in catalog being ingested.

Returns
-------
coord : `lsst.geom.SpherePoint`
    ICRS coordinate.

Definition at line 235 of file ingestIndexManager.py.

    def computeCoord(row, ra_name, dec_name):
        """Create an ICRS coord. from a row of a catalog being ingested.
 
        Parameters
        ----------
        row : `numpy.ndarray`
            Row from catalog being ingested.
        ra_name : `str`
            Name of RA key in catalog being ingested.
        dec_name : `str`
            Name of Dec key in catalog being ingested.
 
        Returns
        -------
        coord : `lsst.geom.SpherePoint`
            ICRS coordinate.
        """
        return lsst.geom.SpherePoint(row[ra_name], row[dec_name], lsst.geom.degrees)
 

getCatalog()

def lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.getCatalog (   self,   pixelId,   schema,   nNewElements  )

inherited

Get a catalog from disk or create it if it doesn't exist.

Parameters
----------
pixelId : `dict`
    Identifier for catalog to retrieve
schema : `lsst.afw.table.Schema`
    Schema to use in catalog creation it does not exist.
nNewElements : `int`
    The number of new elements that will be added to the catalog,
    so space can be preallocated.

Returns
-------
catalog : `lsst.afw.table.SimpleCatalog`
    The new or read-and-resized catalog specified by `dataId`.

Definition at line 206 of file ingestIndexManager.py.

    def getCatalog(self, pixelId, schema, nNewElements):
        """Get a catalog from disk or create it if it doesn't exist.
 
        Parameters
        ----------
        pixelId : `dict`
            Identifier for catalog to retrieve
        schema : `lsst.afw.table.Schema`
            Schema to use in catalog creation it does not exist.
        nNewElements : `int`
            The number of new elements that will be added to the catalog,
            so space can be preallocated.
 
        Returns
        -------
        catalog : `lsst.afw.table.SimpleCatalog`
            The new or read-and-resized catalog specified by `dataId`.
        """
        # This is safe, because we lock on this file before getCatalog is called.
        if os.path.isfile(self.filenames[pixelId]):
            catalog = afwTable.SimpleCatalog.readFits(self.filenames[pixelId])
            catalog.resize(len(catalog) + nNewElements)
            return catalog.copy(deep=True)  # ensure contiguity, so that column-assignment works
        catalog = afwTable.SimpleCatalog(schema)
        catalog.resize(nNewElements)
        self.addRefCatMetadata(catalog)
        return catalog
 

run()

def lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.run (   self,   inputFiles  )

inherited

Index a set of input files from a reference catalog, and write the
output to the appropriate filenames, in parallel.

Parameters
----------
inputFiles : `list`
    A list of file paths to read data from.

Definition at line 88 of file ingestIndexManager.py.

    def run(self, inputFiles):
        """Index a set of input files from a reference catalog, and write the
        output to the appropriate filenames, in parallel.
 
        Parameters
        ----------
        inputFiles : `list`
            A list of file paths to read data from.
        """
        global COUNTER, FILE_PROGRESS
        self.nInputFiles = len(inputFiles)
 
        with multiprocessing.Manager() as manager:
            COUNTER.value = 0
            FILE_PROGRESS.value = 0
            fileLocks = manager.dict()
            self.log.info("Creating %s file locks.", self.htmRange[1] - self.htmRange[0])
            for i in range(self.htmRange[0], self.htmRange[1]):
                fileLocks[i] = manager.Lock()
            self.log.info("File locks created.")
            with multiprocessing.Pool(self.config.n_processes) as pool:
                pool.starmap(self._ingestOneFile, zip(inputFiles, itertools.repeat(fileLocks)))
 

Member Data Documentation

addRefCatMetadata

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.addRefCatMetadata

inherited

Definition at line 82 of file ingestIndexManager.py.

config

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.config

inherited

Definition at line 76 of file ingestIndexManager.py.

coord_err_unit

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.coord_err_unit

inherited

Definition at line 86 of file ingestIndexManager.py.

file_reader

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.file_reader

inherited

Definition at line 77 of file ingestIndexManager.py.

filenames

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.filenames

inherited

Definition at line 75 of file ingestIndexManager.py.

htmRange

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.htmRange

inherited

Definition at line 81 of file ingestIndexManager.py.

indexer

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.indexer

inherited

Definition at line 78 of file ingestIndexManager.py.

key_map

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.key_map

inherited

Definition at line 80 of file ingestIndexManager.py.

log

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.log

inherited

Definition at line 83 of file ingestIndexManager.py.

nInputFiles

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.nInputFiles

inherited

Definition at line 98 of file ingestIndexManager.py.

schema

lsst.meas.algorithms.ingestIndexManager.IngestIndexManager.schema

inherited

Definition at line 79 of file ingestIndexManager.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-0.7.0/Linux64/meas_algorithms/22.0.1-21-g532228a4+2ac85e833c/python/lsst/meas/algorithms/ingestIndexManager.py