lsst.pipe.tasks.processCcdWithFakes Namespace Reference

Classes
class	docstring
class	ProcessCcdWithFakesConnections

Variables
	exposure : `lsst.afw.image.exposure.exposure.ExposureF`
	skyMap : `lsst.skymap.SkyMap`
	wcs : `lsst.afw.geom.SkyWcs`, optional
	photoCalib : `lsst.afw.image.photoCalib.PhotoCalib`, optional
	icSourceCat : `lsst.afw.table.SourceCatalog`, optional
	sfdSourceCat : `lsst.afw.table.SourceCatalog`, optional
	externalSkyWcsGlobalCatalog
	externalSkyWcsTractCatalog : `lsst.afw.table.ExposureCatalog`, optional
	externalPhotoCalibGlobalCatalog
	externalPhotoCalibTractCatalog
	idGenerator : `lsst.meas.base.IdGenerator`, optional
	resultStruct : `lsst.pipe.base.struct.Struct`
	radius
	fakeCats : `list` of `lsst.daf.butler.DeferredDatasetHandle`
	combinedFakeCat : `pandas.DataFrame`
	fakeCat : `pandas.core.frame.DataFrame`
	movingFakeCat : `pandas.DataFrame`
	calibCat : `lsst.afw.table.SourceCatalog`
	sourceCat : `lsst.afw.table.SourceCatalog`
	fieldsToCopy : `lsst.pex.config.listField.List`
	newCat : `lsst.afw.table.SourceCatalog`
	band : `str`
	rngSeed : `int`
	dataFrame : `pandas.DataFrame`

Detailed Description

Insert fake sources into calexps

Variable Documentation

band

lsst.pipe.tasks.processCcdWithFakes.band : `str`

Definition at line 755 of file processCcdWithFakes.py.

calibCat

lsst.pipe.tasks.processCcdWithFakes.calibCat : `lsst.afw.table.SourceCatalog`

selected = exposure.getInfo().getVisitInfo().getId() == fakeCat["visit"]

return fakeCat[selected]

def copyCalibrationFields(self, calibCat, sourceCat, fieldsToCopy):

Definition at line 538 of file processCcdWithFakes.py.

combinedFakeCat

lsst.pipe.tasks.processCcdWithFakes.combinedFakeCat : `pandas.DataFrame`

Definition at line 495 of file processCcdWithFakes.py.

dataFrame

lsst.pipe.tasks.processCcdWithFakes.dataFrame : `pandas.DataFrame`

Definition at line 766 of file processCcdWithFakes.py.

exposure

lsst.pipe.tasks.processCcdWithFakes.exposure : `lsst.afw.image.exposure.exposure.ExposureF`

Definition at line 406 of file processCcdWithFakes.py.

externalPhotoCalibGlobalCatalog

lsst.pipe.tasks.processCcdWithFakes.externalPhotoCalibGlobalCatalog

Initial value:

 : `lsst.afw.table.ExposureCatalog`, \
                optional

Definition at line 425 of file processCcdWithFakes.py.

externalPhotoCalibTractCatalog

lsst.pipe.tasks.processCcdWithFakes.externalPhotoCalibTractCatalog

Initial value:

 : `lsst.afw.table.ExposureCatalog`, \
                optional

Definition at line 428 of file processCcdWithFakes.py.

externalSkyWcsGlobalCatalog

lsst.pipe.tasks.processCcdWithFakes.externalSkyWcsGlobalCatalog

Initial value:

 : `lsst.afw.table.ExposureCatalog`, \
                optional

Definition at line 420 of file processCcdWithFakes.py.

externalSkyWcsTractCatalog

lsst.pipe.tasks.processCcdWithFakes.externalSkyWcsTractCatalog : `lsst.afw.table.ExposureCatalog`, optional

Definition at line 423 of file processCcdWithFakes.py.

fakeCat

lsst.pipe.tasks.processCcdWithFakes.fakeCat : `pandas.core.frame.DataFrame`

if len(fakeCats) == 1:
    return fakeCats[0].get()
outputCat = []
for fakeCatRef in fakeCats:
    cat = fakeCatRef.get()
    tractId = fakeCatRef.dataId["tract"]
    # Make sure all data is within the inner part of the tract.
    outputCat.append(cat[
        skyMap.findTractIdArray(cat[self.config.insertFakes.ra_col],
                                cat[self.config.insertFakes.dec_col],
                                degrees=False)
        == tractId])

return pd.concat(outputCat)

def getVisitMatchedFakeCat(self, fakeCat, exposure):

# Make a new SourceCatalog with the data from sourceCat so that we can add the new columns to it
sourceSchemaMapper = afwTable.SchemaMapper(sourceCat.schema)
sourceSchemaMapper.addMinimalSchema(sourceCat.schema, True)

calibSchemaMapper = afwTable.SchemaMapper(calibCat.schema, sourceCat.schema)

# Add the desired columns from the option fieldsToCopy
missingFieldNames = []
for fieldName in fieldsToCopy:
    if fieldName in calibCat.schema:
        schemaItem = calibCat.schema.find(fieldName)
        calibSchemaMapper.editOutputSchema().addField(schemaItem.getField())
        schema = calibSchemaMapper.editOutputSchema()
        calibSchemaMapper.addMapping(schemaItem.getKey(), schema.find(fieldName).getField())
    else:
        missingFieldNames.append(fieldName)
if missingFieldNames:
    raise RuntimeError(f"calibCat is missing fields {missingFieldNames} specified in "
                       "fieldsToCopy")

if "calib_detected" not in calibSchemaMapper.getOutputSchema():
    self.calibSourceKey = calibSchemaMapper.addOutputField(afwTable.Field["Flag"]("calib_detected",
                                                           "Source was detected as an icSource"))
else:
    self.calibSourceKey = None

schema = calibSchemaMapper.getOutputSchema()
newCat = afwTable.SourceCatalog(schema)
newCat.reserve(len(sourceCat))
newCat.extend(sourceCat, sourceSchemaMapper)

# Set the aliases so it doesn't complain.
for k, v in sourceCat.schema.getAliasMap().items():
    newCat.schema.getAliasMap().set(k, v)

select = newCat["deblend_nChild"] == 0
matches = afwTable.matchXy(newCat[select], calibCat, self.config.matchRadiusPix)
# Check that no sourceCat sources are listed twice (we already know
# that each match has a unique calibCat source ID, due to using
# that ID as the key in bestMatches)
numMatches = len(matches)
numUniqueSources = len(set(m[1].getId() for m in matches))
if numUniqueSources != numMatches:
    self.log.warning("%d calibCat sources matched only %d sourceCat sources", numMatches,
                     numUniqueSources)

self.log.info("Copying flags from calibCat to sourceCat for %s sources", numMatches)

# For each match: set the calibSourceKey flag and copy the desired
# fields
for src, calibSrc, d in matches:
    if self.calibSourceKey:
        src.setFlag(self.calibSourceKey, True)
    # src.assign copies the footprint from calibSrc, which we don't want
    # (DM-407)
    # so set calibSrc's footprint to src's footprint before src.assign,
    # then restore it
    calibSrcFootprint = calibSrc.getFootprint()
    try:
        calibSrc.setFootprint(src.getFootprint())
        src.assign(calibSrc, calibSchemaMapper)
    finally:
        calibSrc.setFootprint(calibSrcFootprint)

return newCat


class ProcessCcdWithVariableFakesConnections(ProcessCcdWithFakesConnections):
ccdVisitFakeMagnitudes = cT.Output(
doc="Catalog of fakes with magnitudes scattered for this ccdVisit.",
name="{fakesType}ccdVisitFakeMagnitudes",
storageClass="DataFrame",
dimensions=("instrument", "visit", "detector"),
)


class ProcessCcdWithVariableFakesConfig(ProcessCcdWithFakesConfig,
                                pipelineConnections=ProcessCcdWithVariableFakesConnections):
scatterSize = pexConfig.RangeField(
dtype=float,
default=0.4,
min=0,
max=100,
doc="Amount of scatter to add to the visit magnitude for variable "
    "sources."
)


class ProcessCcdWithVariableFakesTask(ProcessCcdWithFakesTask):

_DefaultName = "processCcdWithVariableFakes"
ConfigClass = ProcessCcdWithVariableFakesConfig

def run(self, fakeCats, exposure, skyMap, wcs=None, photoCalib=None,
        icSourceCat=None, sfdSourceCat=None, idGenerator=None):

fakeCat = self.composeFakeCat(fakeCats, skyMap)

if wcs is None:
    wcs = exposure.getWcs()

if photoCalib is None:
    photoCalib = exposure.getPhotoCalib()

if idGenerator is None:
    idGenerator = IdGenerator()

band = exposure.getFilter().bandLabel
ccdVisitMagnitudes = self.addVariability(
    fakeCat,
    band,
    exposure,
    photoCalib,
    idGenerator.catalog_id,
)

self.insertFakes.run(fakeCat, exposure, wcs, photoCalib)

# detect, deblend and measure sources
returnedStruct = self.calibrate.run(exposure, idGenerator=idGenerator)
sourceCat = returnedStruct.sourceCat

sourceCat = self.copyCalibrationFields(sfdSourceCat, sourceCat, self.config.srcFieldsToCopy)

resultStruct = pipeBase.Struct(outputExposure=exposure,
                               outputCat=sourceCat,
                               ccdVisitFakeMagnitudes=ccdVisitMagnitudes)
return resultStruct

def addVariability(self, fakeCat, band, exposure, photoCalib, rngSeed):

Definition at line 519 of file processCcdWithFakes.py.

fakeCats

lsst.pipe.tasks.processCcdWithFakes.fakeCats : `list` of `lsst.daf.butler.DeferredDatasetHandle`

fakeCat = self.composeFakeCat(fakeCats, skyMap)

if wcs is None:
    wcs = exposure.getWcs()

if photoCalib is None:
    photoCalib = exposure.getPhotoCalib()

if self.config.doMatchVisit:
    fakeCat = self.getVisitMatchedFakeCat(fakeCat, exposure)

self.insertFakes.run(fakeCat, exposure, wcs, photoCalib)

# detect, deblend and measure sources
if idGenerator is None:
    idGenerator = IdGenerator()
returnedStruct = self.calibrate.run(exposure, idGenerator=idGenerator)
sourceCat = returnedStruct.sourceCat

sourceCat = self.copyCalibrationFields(sfdSourceCat, sourceCat, self.config.srcFieldsToCopy)

resultStruct = pipeBase.Struct(outputExposure=exposure, outputCat=sourceCat)
return resultStruct

def composeFakeCat(self, fakeCats, skyMap):

Definition at line 488 of file processCcdWithFakes.py.

fieldsToCopy

lsst.pipe.tasks.processCcdWithFakes.fieldsToCopy : `lsst.pex.config.listField.List`

Definition at line 542 of file processCcdWithFakes.py.

icSourceCat

lsst.pipe.tasks.processCcdWithFakes.icSourceCat : `lsst.afw.table.SourceCatalog`, optional

Definition at line 414 of file processCcdWithFakes.py.

idGenerator

lsst.pipe.tasks.processCcdWithFakes.idGenerator : `lsst.meas.base.IdGenerator`, optional

Definition at line 431 of file processCcdWithFakes.py.

movingFakeCat

lsst.pipe.tasks.processCcdWithFakes.movingFakeCat : `pandas.DataFrame`

Definition at line 526 of file processCcdWithFakes.py.

newCat

lsst.pipe.tasks.processCcdWithFakes.newCat : `lsst.afw.table.SourceCatalog`

Definition at line 547 of file processCcdWithFakes.py.

photoCalib

lsst.pipe.tasks.processCcdWithFakes.photoCalib : `lsst.afw.image.photoCalib.PhotoCalib`, optional

Definition at line 412 of file processCcdWithFakes.py.

radius

lsst.pipe.tasks.processCcdWithFakes.radius

Definition at line 445 of file processCcdWithFakes.py.

resultStruct

lsst.pipe.tasks.processCcdWithFakes.resultStruct : `lsst.pipe.base.struct.Struct`

Definition at line 436 of file processCcdWithFakes.py.

rngSeed

lsst.pipe.tasks.processCcdWithFakes.rngSeed : `int`

Definition at line 761 of file processCcdWithFakes.py.

sfdSourceCat

lsst.pipe.tasks.processCcdWithFakes.sfdSourceCat : `lsst.afw.table.SourceCatalog`, optional

Definition at line 417 of file processCcdWithFakes.py.

skyMap

lsst.pipe.tasks.processCcdWithFakes.skyMap : `lsst.skymap.SkyMap`

Definition at line 408 of file processCcdWithFakes.py.

sourceCat

lsst.pipe.tasks.processCcdWithFakes.sourceCat : `lsst.afw.table.SourceCatalog`

Definition at line 540 of file processCcdWithFakes.py.

wcs

lsst.pipe.tasks.processCcdWithFakes.wcs : `lsst.afw.geom.SkyWcs`, optional

Definition at line 410 of file processCcdWithFakes.py.