lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask Class Reference

Inheritance diagram for lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask:

Public Member Functions
	__init__ (self, initInputs=None, **kwargs)
	runQuantum (self, butlerQC, inputRefs, outputRefs)
	fgcmMakeAllStarObservations (self, groupedHandles, visitCat, sourceSchema, camera, calibFluxApertureRadius=None)

Public Attributes
	sourceSchema

Static Public Attributes
	ConfigClass = FgcmBuildStarsTableConfig
bool	canMultiprocess = False

Protected Member Functions
	_groupHandles (self, sourceTableHandleDict, visitSummaryHandleDict)
	_get_sourceTable_visit_columns (self, inColumns)

Static Protected Attributes
str	_DefaultName = "fgcmBuildStarsTable"

Detailed Description

Build stars for the FGCM global calibration, using sourceTable_visit catalogs.

Definition at line 209 of file fgcmBuildStarsTable.py.

Constructor & Destructor Documentation

__init__()

lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask.__init__	(		self,
			initInputs = None,
		**	kwargs )

Reimplemented from lsst.fgcmcal.fgcmBuildStarsBase.FgcmBuildStarsBaseTask.

Definition at line 218 of file fgcmBuildStarsTable.py.

    def __init__(self, initInputs=None, **kwargs):
        super().__init__(initInputs=initInputs, **kwargs)
        if initInputs is not None:
            self.sourceSchema = initInputs["sourceSchema"].schema
 

Member Function Documentation

_get_sourceTable_visit_columns()

lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask._get_sourceTable_visit_columns ( self, inColumns )

protected

Get the sourceTable_visit columns from the config.

Parameters
----------
inColumns : `list`
    List of columns available in the sourceTable_visit

Returns
-------
columns : `list`
    List of columns to read from sourceTable_visit.

Definition at line 460 of file fgcmBuildStarsTable.py.

    def _get_sourceTable_visit_columns(self, inColumns):
        """
        Get the sourceTable_visit columns from the config.
 
        Parameters
        ----------
        inColumns : `list`
            List of columns available in the sourceTable_visit
 
        Returns
        -------
        columns : `list`
            List of columns to read from sourceTable_visit.
        """
        # Some names are hard-coded in the parquet table.
        columns = ['visit', 'detector',
                   'ra', 'dec', 'x', 'y', self.config.psfCandidateName,
                   self.config.instFluxField, self.config.instFluxField + 'Err',
                   self.config.apertureInnerInstFluxField, self.config.apertureInnerInstFluxField + 'Err',
                   self.config.apertureOuterInstFluxField, self.config.apertureOuterInstFluxField + 'Err']
        if self.sourceSelector.config.doFlags:
            columns.extend(self.sourceSelector.config.flags.bad)
        if self.sourceSelector.config.doUnresolved:
            columns.append(self.sourceSelector.config.unresolved.name)
        if self.sourceSelector.config.doIsolated:
            columns.append(self.sourceSelector.config.isolated.parentName)
            columns.append(self.sourceSelector.config.isolated.nChildName)
        if self.sourceSelector.config.doRequirePrimary:
            columns.append(self.sourceSelector.config.requirePrimary.primaryColName)
        if self.config.doSubtractLocalBackground:
            columns.append(self.config.localBackgroundFluxField)
 
        return columns

_groupHandles()

lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask._groupHandles ( self, sourceTableHandleDict, visitSummaryHandleDict )

protected

Group sourceTable and visitSummary handles.

Parameters
----------
sourceTableHandleDict : `dict` [`int`, `str`]
    Dict of source tables, keyed by visit.
visitSummaryHandleDict : `dict` [int, `str`]
    Dict of visit summary catalogs, keyed by visit.

Returns
-------
groupedHandles : `dict` [`int`, `list`]
    Dictionary with sorted visit keys, and `list`s with
    `lsst.daf.butler.DeferredDataSetHandle`.  The first
    item in the list will be the visitSummary ref, and
    the second will be the source table ref.

Definition at line 293 of file fgcmBuildStarsTable.py.

    def _groupHandles(self, sourceTableHandleDict, visitSummaryHandleDict):
        """Group sourceTable and visitSummary handles.
 
        Parameters
        ----------
        sourceTableHandleDict : `dict` [`int`, `str`]
            Dict of source tables, keyed by visit.
        visitSummaryHandleDict : `dict` [int, `str`]
            Dict of visit summary catalogs, keyed by visit.
 
        Returns
        -------
        groupedHandles : `dict` [`int`, `list`]
            Dictionary with sorted visit keys, and `list`s with
            `lsst.daf.butler.DeferredDataSetHandle`.  The first
            item in the list will be the visitSummary ref, and
            the second will be the source table ref.
        """
        groupedHandles = collections.defaultdict(list)
        visits = sorted(sourceTableHandleDict.keys())
 
        for visit in visits:
            groupedHandles[visit] = [visitSummaryHandleDict[visit],
                                     sourceTableHandleDict[visit]]
 
        return groupedHandles
 

fgcmMakeAllStarObservations()

lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask.fgcmMakeAllStarObservations	(		self,
			groupedHandles,
			visitCat,
			sourceSchema,
			camera,
			calibFluxApertureRadius = None )

Compile all good star observations from visits in visitCat.

Parameters
----------
groupedHandles : `dict` [`list` [`lsst.daf.butler.DeferredDatasetHandle`]]
    Dataset handles, grouped by visit.
visitCat : `afw.table.BaseCatalog`
    Catalog with visit data for FGCM
sourceSchema : `lsst.afw.table.Schema`
    Schema for the input src catalogs.
camera : `lsst.afw.cameraGeom.Camera`
calibFluxApertureRadius : `float`, optional
    Aperture radius for calibration flux.
inStarObsCat : `afw.table.BaseCatalog`
    Input observation catalog.  If this is incomplete, observations
    will be appended from when it was cut off.

Returns
-------
fgcmStarObservations : `afw.table.BaseCatalog`
    Full catalog of good observations.

Raises
------
RuntimeError: Raised if doSubtractLocalBackground is True and
   calibFluxApertureRadius is not set.

Reimplemented from lsst.fgcmcal.fgcmBuildStarsBase.FgcmBuildStarsBaseTask.

Definition at line 320 of file fgcmBuildStarsTable.py.

                                    calibFluxApertureRadius=None):
        startTime = time.time()
 
        if self.config.doSubtractLocalBackground and calibFluxApertureRadius is None:
            raise RuntimeError("Must set calibFluxApertureRadius if doSubtractLocalBackground is True.")
 
        # To get the correct output schema, we use the legacy code.
        # We are not actually using this mapper, except to grab the outputSchema
        sourceMapper = self._makeSourceMapper(sourceSchema)
        outputSchema = sourceMapper.getOutputSchema()
 
        # Construct mapping from ccd number to index
        ccdMapping = {}
        for ccdIndex, detector in enumerate(camera):
            ccdMapping[detector.getId()] = ccdIndex
 
        approxPixelAreaFields = computeApproxPixelAreaFields(camera)
 
        fullCatalog = afwTable.BaseCatalog(outputSchema)
 
        visitKey = outputSchema['visit'].asKey()
        ccdKey = outputSchema['ccd'].asKey()
        instMagKey = outputSchema['instMag'].asKey()
        instMagErrKey = outputSchema['instMagErr'].asKey()
        deltaMagAperKey = outputSchema['deltaMagAper'].asKey()
 
        # Prepare local background if desired
        if self.config.doSubtractLocalBackground:
            localBackgroundArea = np.pi*calibFluxApertureRadius**2.
 
        columns = None
 
        k = 2.5/np.log(10.)
 
        for counter, visit in enumerate(visitCat):
            expTime = visit['exptime']
 
            handle = groupedHandles[visit['visit']][-1]
 
            if columns is None:
                inColumns = handle.get(component='columns')
                columns = self._get_sourceTable_visit_columns(inColumns)
            df = handle.get(parameters={'columns': columns})
 
            goodSrc = self.sourceSelector.selectSources(df)
 
            # Need to add a selection based on the local background correction
            # if necessary
            if self.config.doSubtractLocalBackground:
                localBackground = localBackgroundArea*df[self.config.localBackgroundFluxField].values
                use, = np.where((goodSrc.selected)
                                & ((df[self.config.instFluxField].values - localBackground) > 0.0))
            else:
                use, = np.where(goodSrc.selected)
 
            tempCat = afwTable.BaseCatalog(fullCatalog.schema)
            tempCat.resize(use.size)
 
            tempCat['ra'][:] = np.deg2rad(df['ra'].values[use])
            tempCat['dec'][:] = np.deg2rad(df['dec'].values[use])
            tempCat['x'][:] = df['x'].values[use]
            tempCat['y'][:] = df['y'].values[use]
            # The "visit" name in the parquet table is hard-coded.
            tempCat[visitKey][:] = df['visit'].values[use]
            tempCat[ccdKey][:] = df['detector'].values[use]
            tempCat['psf_candidate'] = df[self.config.psfCandidateName].values[use]
 
            with warnings.catch_warnings():
                # Ignore warnings, we will filter infinites and nans below
                warnings.simplefilter("ignore")
 
                instMagInner = -2.5*np.log10(df[self.config.apertureInnerInstFluxField].values[use])
                instMagErrInner = k*(df[self.config.apertureInnerInstFluxField + 'Err'].values[use]
                                     / df[self.config.apertureInnerInstFluxField].values[use])
                instMagOuter = -2.5*np.log10(df[self.config.apertureOuterInstFluxField].values[use])
                instMagErrOuter = k*(df[self.config.apertureOuterInstFluxField + 'Err'].values[use]
                                     / df[self.config.apertureOuterInstFluxField].values[use])
                tempCat[deltaMagAperKey][:] = instMagInner - instMagOuter
                # Set bad values to illegal values for fgcm.
                tempCat[deltaMagAperKey][~np.isfinite(tempCat[deltaMagAperKey][:])] = 99.0
 
            if self.config.doSubtractLocalBackground:
                # At the moment we only adjust the flux and not the flux
                # error by the background because the error on
                # base_LocalBackground_instFlux is the rms error in the
                # background annulus, not the error on the mean in the
                # background estimate (which is much smaller, by sqrt(n)
                # pixels used to estimate the background, which we do not
                # have access to in this task).  In the default settings,
                # the annulus is sufficiently large such that these
                # additional errors are are negligibly small (much less
                # than a mmag in quadrature).
 
                # This is the difference between the mag with local background correction
                # and the mag without local background correction.
                tempCat['deltaMagBkg'] = (-2.5*np.log10(df[self.config.instFluxField].values[use]
                                                        - localBackground[use]) -
                                          -2.5*np.log10(df[self.config.instFluxField].values[use]))
            else:
                tempCat['deltaMagBkg'][:] = 0.0
 
            # Need to loop over ccds here
            for detector in camera:
                ccdId = detector.getId()
                # used index for all observations with a given ccd
                use2 = (tempCat[ccdKey] == ccdId)
                tempCat['jacobian'][use2] = approxPixelAreaFields[ccdId].evaluate(tempCat['x'][use2],
                                                                                  tempCat['y'][use2])
                scaledInstFlux = (df[self.config.instFluxField].values[use[use2]]
                                  * visit['scaling'][ccdMapping[ccdId]])
                tempCat[instMagKey][use2] = (-2.5*np.log10(scaledInstFlux) + 2.5*np.log10(expTime))
 
            # Compute instMagErr from instFluxErr/instFlux, any scaling
            # will cancel out.
            tempCat[instMagErrKey][:] = k*(df[self.config.instFluxField + 'Err'].values[use]
                                           / df[self.config.instFluxField].values[use])
 
            # Apply the jacobian if configured
            if self.config.doApplyWcsJacobian:
                tempCat[instMagKey][:] -= 2.5*np.log10(tempCat['jacobian'][:])
 
            fullCatalog.extend(tempCat)
 
            deltaOk = (np.isfinite(instMagInner) & np.isfinite(instMagErrInner)
                       & np.isfinite(instMagOuter) & np.isfinite(instMagErrOuter))
 
            visit['deltaAper'] = np.median(instMagInner[deltaOk] - instMagOuter[deltaOk])
            visit['sources_read'] = True
 
            self.log.info("  Found %d good stars in visit %d (deltaAper = %0.3f)",
                          use.size, visit['visit'], visit['deltaAper'])
 
        self.log.info("Found all good star observations in %.2f s" %
                      (time.time() - startTime))
 
        return fullCatalog
 

runQuantum()

lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask.runQuantum	(		self,
			butlerQC,
			inputRefs,
			outputRefs )

Definition at line 223 of file fgcmBuildStarsTable.py.

    def runQuantum(self, butlerQC, inputRefs, outputRefs):
        inputRefDict = butlerQC.get(inputRefs)
 
        sourceTableHandles = inputRefDict['sourceTable_visit']
 
        self.log.info("Running with %d sourceTable_visit handles",
                      len(sourceTableHandles))
 
        sourceTableHandleDict = {sourceTableHandle.dataId['visit']: sourceTableHandle for
                                 sourceTableHandle in sourceTableHandles}
 
        if self.config.doReferenceMatches:
            # Get the LUT handle
            lutHandle = inputRefDict['fgcmLookUpTable']
 
            # Prepare the reference catalog loader
            refConfig = LoadReferenceObjectsConfig()
            refConfig.filterMap = self.config.fgcmLoadReferenceCatalog.filterMap
            refObjLoader = ReferenceObjectLoader(dataIds=[ref.datasetRef.dataId
                                                          for ref in inputRefs.refCat],
                                                 refCats=butlerQC.get(inputRefs.refCat),
                                                 name=self.config.connections.refCat,
                                                 log=self.log,
                                                 config=refConfig)
            self.makeSubtask('fgcmLoadReferenceCatalog',
                             refObjLoader=refObjLoader,
                             refCatName=self.config.connections.refCat)
        else:
            lutHandle = None
 
        # Compute aperture radius if necessary.  This is useful to do now before
        # any heave lifting has happened (fail early).
        calibFluxApertureRadius = None
        if self.config.doSubtractLocalBackground:
            try:
                calibFluxApertureRadius = computeApertureRadiusFromName(self.config.instFluxField)
            except RuntimeError as e:
                raise RuntimeError("Could not determine aperture radius from %s. "
                                   "Cannot use doSubtractLocalBackground." %
                                   (self.config.instFluxField)) from e
 
        visitSummaryHandles = inputRefDict['visitSummary']
        visitSummaryHandleDict = {visitSummaryHandle.dataId['visit']: visitSummaryHandle for
                                  visitSummaryHandle in visitSummaryHandles}
 
        camera = inputRefDict['camera']
        groupedHandles = self._groupHandles(sourceTableHandleDict,
                                            visitSummaryHandleDict)
 
        visitCat = self.fgcmMakeVisitCatalog(camera, groupedHandles)
 
        rad = calibFluxApertureRadius
        fgcmStarObservationCat = self.fgcmMakeAllStarObservations(groupedHandles,
                                                                  visitCat,
                                                                  self.sourceSchema,
                                                                  camera,
                                                                  calibFluxApertureRadius=rad)
 
        butlerQC.put(visitCat, outputRefs.fgcmVisitCatalog)
        butlerQC.put(fgcmStarObservationCat, outputRefs.fgcmStarObservations)
 
        fgcmStarIdCat, fgcmStarIndicesCat, fgcmRefCat = self.fgcmMatchStars(visitCat,
                                                                            fgcmStarObservationCat,
                                                                            lutHandle=lutHandle)
 
        butlerQC.put(fgcmStarIdCat, outputRefs.fgcmStarIds)
        butlerQC.put(fgcmStarIndicesCat, outputRefs.fgcmStarIndices)
        if fgcmRefCat is not None:
            butlerQC.put(fgcmRefCat, outputRefs.fgcmReferenceStars)
 

Member Data Documentation

_DefaultName

str lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask._DefaultName = "fgcmBuildStarsTable"

staticprotected

Definition at line 214 of file fgcmBuildStarsTable.py.

canMultiprocess

bool lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask.canMultiprocess = False

static

Definition at line 216 of file fgcmBuildStarsTable.py.

ConfigClass

lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask.ConfigClass = FgcmBuildStarsTableConfig

static

Definition at line 213 of file fgcmBuildStarsTable.py.

sourceSchema

lsst.fgcmcal.fgcmBuildStarsTable.FgcmBuildStarsTableTask.sourceSchema

Definition at line 221 of file fgcmBuildStarsTable.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-8.0.0/Linux64/fgcmcal/g487adcacf7+71499e7cba/python/lsst/fgcmcal/fgcmBuildStarsTable.py