LSST Applications  21.0.0-172-gfb10e10a+18fedfabac,22.0.0+297cba6710,22.0.0+80564b0ff1,22.0.0+8d77f4f51a,22.0.0+a28f4c53b1,22.0.0+dcf3732eb2,22.0.1-1-g7d6de66+2a20fdde0d,22.0.1-1-g8e32f31+297cba6710,22.0.1-1-geca5380+7fa3b7d9b6,22.0.1-12-g44dc1dc+2a20fdde0d,22.0.1-15-g6a90155+515f58c32b,22.0.1-16-g9282f48+790f5f2caa,22.0.1-2-g92698f7+dcf3732eb2,22.0.1-2-ga9b0f51+7fa3b7d9b6,22.0.1-2-gd1925c9+bf4f0e694f,22.0.1-24-g1ad7a390+a9625a72a8,22.0.1-25-g5bf6245+3ad8ecd50b,22.0.1-25-gb120d7b+8b5510f75f,22.0.1-27-g97737f7+2a20fdde0d,22.0.1-32-gf62ce7b1+aa4237961e,22.0.1-4-g0b3f228+2a20fdde0d,22.0.1-4-g243d05b+871c1b8305,22.0.1-4-g3a563be+32dcf1063f,22.0.1-4-g44f2e3d+9e4ab0f4fa,22.0.1-42-gca6935d93+ba5e5ca3eb,22.0.1-5-g15c806e+85460ae5f3,22.0.1-5-g58711c4+611d128589,22.0.1-5-g75bb458+99c117b92f,22.0.1-6-g1c63a23+7fa3b7d9b6,22.0.1-6-g50866e6+84ff5a128b,22.0.1-6-g8d3140d+720564cf76,22.0.1-6-gd805d02+cc5644f571,22.0.1-8-ge5750ce+85460ae5f3,master-g6e05de7fdc+babf819c66,master-g99da0e417a+8d77f4f51a,w.2021.48
LSST Data Management Base Package
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask Class Reference
Inheritance diagram for lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask:
lsst.fgcmcal.fgcmCalibrateTractBase.FgcmCalibrateTractBaseTask

Public Member Functions

def __init__ (self, initInputs=None, **kwargs)
 
def runQuantum (self, butlerQC, inputRefs, outputRefs)
 
def runDataRef (self, butler, dataRefs)
 
def run (self, dataRefDict, tract, buildStarsRefObjLoader=None, returnCatalogs=True, butler=None)
 

Public Attributes

 sourceSchema
 

Static Public Attributes

 ConfigClass = FgcmCalibrateTractTableConfig
 
 RunnerClass = FgcmCalibrateTractRunner
 
bool canMultiprocess = False
 

Detailed Description

Calibrate a single tract using fgcmcal, using sourceTable_visit (parquet)
input catalogs.

Definition at line 166 of file fgcmCalibrateTractTable.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.__init__ (   self,
  initInputs = None,
**  kwargs 
)

Definition at line 177 of file fgcmCalibrateTractTable.py.

177  def __init__(self, initInputs=None, **kwargs):
178  super().__init__(initInputs=initInputs, **kwargs)
179  if initInputs is not None:
180  self.sourceSchema = initInputs["sourceSchema"].schema
181 

Member Function Documentation

◆ run()

def lsst.fgcmcal.fgcmCalibrateTractBase.FgcmCalibrateTractBaseTask.run (   self,
  dataRefDict,
  tract,
  buildStarsRefObjLoader = None,
  returnCatalogs = True,
  butler = None 
)
inherited
Run the calibrations for a single tract with fgcm.

Parameters
----------
dataRefDict : `dict`
    All dataRefs are `lsst.daf.persistence.ButlerDataRef` (gen2) or
    `lsst.daf.butler.DeferredDatasetHandle` (gen3)
    dataRef dictionary with the following keys.  Note that all
    keys need not be set based on config parameters.

    ``"camera"``
        Camera object (`lsst.afw.cameraGeom.Camera`)
    ``"source_catalogs"``
        `list` of dataRefs for input source catalogs.
    ``"sourceSchema"``
        Schema for the source catalogs.
    ``"fgcmLookUpTable"``
        dataRef for the FGCM look-up table.
    ``"calexps"``
        `list` of dataRefs for the input calexps (Gen3 only)
    ``"fgcmPhotoCalibs"``
        `dict` of output photoCalib dataRefs.  Key is
        (tract, visit, detector). (Gen3 only)
        Present if doZeropointOutput is True.
    ``"fgcmTransmissionAtmospheres"``
        `dict` of output atmosphere transmission dataRefs.
        Key is (tract, visit). (Gen3 only)
        Present if doAtmosphereOutput is True.
tract : `int`
    Tract number
buildStarsRefObjLoader : `lsst.meas.algorithms.ReferenceObjectLoader`, optional
    Reference object loader object for fgcmBuildStars.
returnCatalogs : `bool`, optional
    Return photoCalibs as per-visit exposure catalogs.
butler : `lsst.daf.persistence.Butler`, optional
    Gen2 butler used for reference star outputs

Returns
-------
outstruct : `lsst.pipe.base.Struct`
    Output structure with keys:

    offsets : `np.ndarray`
        Final reference offsets, per band.
    repeatability : `np.ndarray`
        Raw fgcm repeatability for bright stars, per band.
    atmospheres : `generator` [(`int`, `lsst.afw.image.TransmissionCurve`)]
        Generator that returns (visit, transmissionCurve) tuples.
    photoCalibs : `generator` [(`int`, `int`, `str`, `lsst.afw.image.PhotoCalib`)]
        Generator that returns (visit, ccd, filtername, photoCalib) tuples.
        (returned if returnCatalogs is False).
    photoCalibCatalogs : `generator` [(`int`, `lsst.afw.table.ExposureCatalog`)]
        Generator that returns (visit, exposureCatalog) tuples.
        (returned if returnCatalogs is True).

Definition at line 265 of file fgcmCalibrateTractBase.py.

266  buildStarsRefObjLoader=None, returnCatalogs=True, butler=None):
267  """Run the calibrations for a single tract with fgcm.
268 
269  Parameters
270  ----------
271  dataRefDict : `dict`
272  All dataRefs are `lsst.daf.persistence.ButlerDataRef` (gen2) or
273  `lsst.daf.butler.DeferredDatasetHandle` (gen3)
274  dataRef dictionary with the following keys. Note that all
275  keys need not be set based on config parameters.
276 
277  ``"camera"``
278  Camera object (`lsst.afw.cameraGeom.Camera`)
279  ``"source_catalogs"``
280  `list` of dataRefs for input source catalogs.
281  ``"sourceSchema"``
282  Schema for the source catalogs.
283  ``"fgcmLookUpTable"``
284  dataRef for the FGCM look-up table.
285  ``"calexps"``
286  `list` of dataRefs for the input calexps (Gen3 only)
287  ``"fgcmPhotoCalibs"``
288  `dict` of output photoCalib dataRefs. Key is
289  (tract, visit, detector). (Gen3 only)
290  Present if doZeropointOutput is True.
291  ``"fgcmTransmissionAtmospheres"``
292  `dict` of output atmosphere transmission dataRefs.
293  Key is (tract, visit). (Gen3 only)
294  Present if doAtmosphereOutput is True.
295  tract : `int`
296  Tract number
297  buildStarsRefObjLoader : `lsst.meas.algorithms.ReferenceObjectLoader`, optional
298  Reference object loader object for fgcmBuildStars.
299  returnCatalogs : `bool`, optional
300  Return photoCalibs as per-visit exposure catalogs.
301  butler : `lsst.daf.persistence.Butler`, optional
302  Gen2 butler used for reference star outputs
303 
304  Returns
305  -------
306  outstruct : `lsst.pipe.base.Struct`
307  Output structure with keys:
308 
309  offsets : `np.ndarray`
310  Final reference offsets, per band.
311  repeatability : `np.ndarray`
312  Raw fgcm repeatability for bright stars, per band.
313  atmospheres : `generator` [(`int`, `lsst.afw.image.TransmissionCurve`)]
314  Generator that returns (visit, transmissionCurve) tuples.
315  photoCalibs : `generator` [(`int`, `int`, `str`, `lsst.afw.image.PhotoCalib`)]
316  Generator that returns (visit, ccd, filtername, photoCalib) tuples.
317  (returned if returnCatalogs is False).
318  photoCalibCatalogs : `generator` [(`int`, `lsst.afw.table.ExposureCatalog`)]
319  Generator that returns (visit, exposureCatalog) tuples.
320  (returned if returnCatalogs is True).
321  """
322  self.log.info("Running on tract %d", (tract))
323 
324  # Compute the aperture radius if necessary. This is useful to do now before
325  # any heavy lifting has happened (fail early).
326  calibFluxApertureRadius = None
327  if self.config.fgcmBuildStars.doSubtractLocalBackground:
328  try:
329  field = self.config.fgcmBuildStars.instFluxField
330  calibFluxApertureRadius = computeApertureRadiusFromDataRef(dataRefDict['source_catalogs'][0],
331  field)
332  except RuntimeError:
333  raise RuntimeError("Could not determine aperture radius from %s. "
334  "Cannot use doSubtractLocalBackground." %
335  (field))
336 
337  # Run the build stars tasks
338 
339  # Note that we will need visitCat at the end of the procedure for the outputs
340  if isinstance(butler, dafPersist.Butler):
341  # Gen2
342  groupedDataRefs = self.fgcmBuildStars._findAndGroupDataRefsGen2(butler, dataRefDict['camera'],
343  dataRefDict['source_catalogs'])
344  else:
345  # Gen3
346  groupedDataRefs = self.fgcmBuildStars._groupDataRefs(dataRefDict['sourceTableDataRefDict'],
347  dataRefDict['visitSummaryDataRefDict'])
348  visitCat = self.fgcmBuildStars.fgcmMakeVisitCatalog(dataRefDict['camera'], groupedDataRefs)
349  rad = calibFluxApertureRadius
350  fgcmStarObservationCat = self.fgcmBuildStars.fgcmMakeAllStarObservations(groupedDataRefs,
351  visitCat,
352  dataRefDict['sourceSchema'],
353  dataRefDict['camera'],
354  calibFluxApertureRadius=rad)
355 
356  if self.fgcmBuildStars.config.doReferenceMatches:
357  lutDataRef = dataRefDict['fgcmLookUpTable']
358  if buildStarsRefObjLoader is not None:
359  self.fgcmBuildStars.makeSubtask("fgcmLoadReferenceCatalog",
360  refObjLoader=buildStarsRefObjLoader)
361  else:
362  self.fgcmBuildStars.makeSubtask("fgcmLoadReferenceCatalog", butler=butler)
363  else:
364  lutDataRef = None
365 
366  fgcmStarIdCat, fgcmStarIndicesCat, fgcmRefCat = \
367  self.fgcmBuildStars.fgcmMatchStars(visitCat,
368  fgcmStarObservationCat,
369  lutDataRef=lutDataRef)
370 
371  # Load the LUT
372  lutCat = dataRefDict['fgcmLookUpTable'].get()
373  fgcmLut, lutIndexVals, lutStd = translateFgcmLut(lutCat,
374  dict(self.config.fgcmFitCycle.physicalFilterMap))
375  del lutCat
376 
377  # Translate the visit catalog into fgcm format
378  fgcmExpInfo = translateVisitCatalog(visitCat)
379 
380  configDict = makeConfigDict(self.config.fgcmFitCycle, self.log, dataRefDict['camera'],
381  self.config.fgcmFitCycle.maxIterBeforeFinalCycle,
382  True, False, lutIndexVals[0]['FILTERNAMES'],
383  tract=tract)
384 
385  focalPlaneProjector = FocalPlaneProjector(dataRefDict['camera'],
386  self.config.fgcmFitCycle.defaultCameraOrientation)
387 
388  # Set up the fit cycle task
389 
390  noFitsDict = {'lutIndex': lutIndexVals,
391  'lutStd': lutStd,
392  'expInfo': fgcmExpInfo,
393  'focalPlaneProjector': focalPlaneProjector}
394 
395  fgcmFitCycle = fgcm.FgcmFitCycle(configDict, useFits=False,
396  noFitsDict=noFitsDict, noOutput=True)
397 
398  # We determine the conversion from the native units (typically radians) to
399  # degrees for the first star. This allows us to treat coord_ra/coord_dec as
400  # numpy arrays rather than Angles, which would we approximately 600x slower.
401  conv = fgcmStarObservationCat[0]['ra'].asDegrees() / float(fgcmStarObservationCat[0]['ra'])
402 
403  # To load the stars, we need an initial parameter object
404  fgcmPars = fgcm.FgcmParameters.newParsWithArrays(fgcmFitCycle.fgcmConfig,
405  fgcmLut,
406  fgcmExpInfo)
407 
408  # Match star observations to visits
409  # Only those star observations that match visits from fgcmExpInfo['VISIT'] will
410  # actually be transferred into fgcm using the indexing below.
411 
412  obsIndex = fgcmStarIndicesCat['obsIndex']
413  visitIndex = np.searchsorted(fgcmExpInfo['VISIT'],
414  fgcmStarObservationCat['visit'][obsIndex])
415 
416  refMag, refMagErr = extractReferenceMags(fgcmRefCat,
417  self.config.fgcmFitCycle.bands,
418  self.config.fgcmFitCycle.physicalFilterMap)
419  refId = fgcmRefCat['fgcm_id'][:]
420 
421  fgcmStars = fgcm.FgcmStars(fgcmFitCycle.fgcmConfig)
422  fgcmStars.loadStars(fgcmPars,
423  fgcmStarObservationCat['visit'][obsIndex],
424  fgcmStarObservationCat['ccd'][obsIndex],
425  fgcmStarObservationCat['ra'][obsIndex] * conv,
426  fgcmStarObservationCat['dec'][obsIndex] * conv,
427  fgcmStarObservationCat['instMag'][obsIndex],
428  fgcmStarObservationCat['instMagErr'][obsIndex],
429  fgcmExpInfo['FILTERNAME'][visitIndex],
430  fgcmStarIdCat['fgcm_id'][:],
431  fgcmStarIdCat['ra'][:],
432  fgcmStarIdCat['dec'][:],
433  fgcmStarIdCat['obsArrIndex'][:],
434  fgcmStarIdCat['nObs'][:],
435  obsX=fgcmStarObservationCat['x'][obsIndex],
436  obsY=fgcmStarObservationCat['y'][obsIndex],
437  obsDeltaMagBkg=fgcmStarObservationCat['deltaMagBkg'][obsIndex],
438  psfCandidate=fgcmStarObservationCat['psf_candidate'][obsIndex],
439  refID=refId,
440  refMag=refMag,
441  refMagErr=refMagErr,
442  flagID=None,
443  flagFlag=None,
444  computeNobs=True)
445 
446  # Clear out some memory
447  del fgcmStarIdCat
448  del fgcmStarIndicesCat
449  del fgcmRefCat
450 
451  fgcmFitCycle.setLUT(fgcmLut)
452  fgcmFitCycle.setStars(fgcmStars, fgcmPars)
453 
454  converged = False
455  cycleNumber = 0
456 
457  previousReservedRawRepeatability = np.zeros(fgcmPars.nBands) + 1000.0
458  previousParInfo = None
459  previousParams = None
460  previousSuperStar = None
461 
462  while (not converged and cycleNumber < self.config.maxFitCycles):
463 
464  fgcmFitCycle.fgcmConfig.updateCycleNumber(cycleNumber)
465 
466  if cycleNumber > 0:
467  # Use parameters from previous cycle
468  fgcmPars = fgcm.FgcmParameters.loadParsWithArrays(fgcmFitCycle.fgcmConfig,
469  fgcmExpInfo,
470  previousParInfo,
471  previousParams,
472  previousSuperStar)
473  # We need to reset the star magnitudes and errors for the next
474  # cycle
475  fgcmFitCycle.fgcmStars.reloadStarMagnitudes(fgcmStarObservationCat['instMag'][obsIndex],
476  fgcmStarObservationCat['instMagErr'][obsIndex])
477  fgcmFitCycle.initialCycle = False
478 
479  fgcmFitCycle.setPars(fgcmPars)
480  fgcmFitCycle.finishSetup()
481 
482  fgcmFitCycle.run()
483 
484  # Grab the parameters for the next cycle
485  previousParInfo, previousParams = fgcmFitCycle.fgcmPars.parsToArrays()
486  previousSuperStar = fgcmFitCycle.fgcmPars.parSuperStarFlat.copy()
487 
488  self.log.info("Raw repeatability after cycle number %d is:" % (cycleNumber))
489  for i, band in enumerate(fgcmFitCycle.fgcmPars.bands):
490  if not fgcmFitCycle.fgcmPars.hasExposuresInBand[i]:
491  continue
492  rep = fgcmFitCycle.fgcmPars.compReservedRawRepeatability[i] * 1000.0
493  self.log.info(" Band %s, repeatability: %.2f mmag" % (band, rep))
494 
495  # Check for convergence
496  if np.alltrue((previousReservedRawRepeatability
497  - fgcmFitCycle.fgcmPars.compReservedRawRepeatability)
498  < self.config.convergenceTolerance):
499  self.log.info("Raw repeatability has converged after cycle number %d." % (cycleNumber))
500  converged = True
501  else:
502  fgcmFitCycle.fgcmConfig.expGrayPhotometricCut[:] = fgcmFitCycle.updatedPhotometricCut
503  fgcmFitCycle.fgcmConfig.expGrayHighCut[:] = fgcmFitCycle.updatedHighCut
504  fgcmFitCycle.fgcmConfig.precomputeSuperStarInitialCycle = False
505  fgcmFitCycle.fgcmConfig.freezeStdAtmosphere = False
506  previousReservedRawRepeatability[:] = fgcmFitCycle.fgcmPars.compReservedRawRepeatability
507  self.log.info("Setting exposure gray photometricity cuts to:")
508  for i, band in enumerate(fgcmFitCycle.fgcmPars.bands):
509  if not fgcmFitCycle.fgcmPars.hasExposuresInBand[i]:
510  continue
511  cut = fgcmFitCycle.updatedPhotometricCut[i] * 1000.0
512  self.log.info(" Band %s, photometricity cut: %.2f mmag" % (band, cut))
513 
514  cycleNumber += 1
515 
516  # Log warning if not converged
517  if not converged:
518  self.log.warning("Maximum number of fit cycles exceeded (%d) without convergence.", cycleNumber)
519 
520  # Do final clean-up iteration
521  fgcmFitCycle.fgcmConfig.freezeStdAtmosphere = False
522  fgcmFitCycle.fgcmConfig.resetParameters = False
523  fgcmFitCycle.fgcmConfig.maxIter = 0
524  fgcmFitCycle.fgcmConfig.outputZeropoints = True
525  fgcmFitCycle.fgcmConfig.outputStandards = True
526  fgcmFitCycle.fgcmConfig.doPlots = self.config.doDebuggingPlots
527  fgcmFitCycle.fgcmConfig.updateCycleNumber(cycleNumber)
528  fgcmFitCycle.initialCycle = False
529 
530  fgcmPars = fgcm.FgcmParameters.loadParsWithArrays(fgcmFitCycle.fgcmConfig,
531  fgcmExpInfo,
532  previousParInfo,
533  previousParams,
534  previousSuperStar)
535  fgcmFitCycle.fgcmStars.reloadStarMagnitudes(fgcmStarObservationCat['instMag'][obsIndex],
536  fgcmStarObservationCat['instMagErr'][obsIndex])
537  fgcmFitCycle.setPars(fgcmPars)
538  fgcmFitCycle.finishSetup()
539 
540  self.log.info("Running final clean-up fit cycle...")
541  fgcmFitCycle.run()
542 
543  self.log.info("Raw repeatability after clean-up cycle is:")
544  for i, band in enumerate(fgcmFitCycle.fgcmPars.bands):
545  if not fgcmFitCycle.fgcmPars.hasExposuresInBand[i]:
546  continue
547  rep = fgcmFitCycle.fgcmPars.compReservedRawRepeatability[i] * 1000.0
548  self.log.info(" Band %s, repeatability: %.2f mmag" % (band, rep))
549 
550  # Do the outputs. Need to keep track of tract.
551 
552  superStarChebSize = fgcmFitCycle.fgcmZpts.zpStruct['FGCM_FZPT_SSTAR_CHEB'].shape[1]
553  zptChebSize = fgcmFitCycle.fgcmZpts.zpStruct['FGCM_FZPT_CHEB'].shape[1]
554 
555  zptSchema = makeZptSchema(superStarChebSize, zptChebSize)
556  zptCat = makeZptCat(zptSchema, fgcmFitCycle.fgcmZpts.zpStruct)
557 
558  atmSchema = makeAtmSchema()
559  atmCat = makeAtmCat(atmSchema, fgcmFitCycle.fgcmZpts.atmStruct)
560 
561  stdStruct, goodBands = fgcmFitCycle.fgcmStars.retrieveStdStarCatalog(fgcmFitCycle.fgcmPars)
562  stdSchema = makeStdSchema(len(goodBands))
563  stdCat = makeStdCat(stdSchema, stdStruct, goodBands)
564 
565  outStruct = self.fgcmOutputProducts.generateTractOutputProducts(dataRefDict,
566  tract,
567  visitCat,
568  zptCat, atmCat, stdCat,
569  self.config.fgcmBuildStars,
570  returnCatalogs=returnCatalogs,
571  butler=butler)
572 
573  outStruct.repeatability = fgcmFitCycle.fgcmPars.compReservedRawRepeatability
574 
575  fgcmFitCycle.freeSharedMemory()
576 
577  return outStruct
def extractReferenceMags(refStars, bands, filterMap)
Definition: utilities.py:887
def computeApertureRadiusFromDataRef(dataRef, fluxField)
Definition: utilities.py:812
def makeStdSchema(nBands)
Definition: utilities.py:742
def makeAtmCat(atmSchema, atmStruct)
Definition: utilities.py:709
def makeConfigDict(config, log, camera, maxIter, resetFitParameters, outputZeropoints, lutFilterNames, tract=None)
Definition: utilities.py:52
def translateFgcmLut(lutCat, physicalFilterMap)
Definition: utilities.py:220
def makeZptCat(zptSchema, zpStruct)
Definition: utilities.py:630
def makeStdCat(stdSchema, stdStruct, goodBands)
Definition: utilities.py:774
def makeZptSchema(superStarChebyshevSize, zptChebyshevSize)
Definition: utilities.py:544
def translateVisitCatalog(visitCat)
Definition: utilities.py:350

◆ runDataRef()

def lsst.fgcmcal.fgcmCalibrateTractBase.FgcmCalibrateTractBaseTask.runDataRef (   self,
  butler,
  dataRefs 
)
inherited
Run full FGCM calibration on a single tract, including building star list,
fitting multiple cycles, and making outputs.

Parameters
----------
butler:  `lsst.daf.persistence.Butler`
dataRefs: `list` of `lsst.daf.persistence.ButlerDataRef`
   Data references for the input visits.
   These may be either per-ccd "src" or per-visit"sourceTable_visit"
   references.

Raises
------
RuntimeError: Raised if `config.fgcmBuildStars.doReferenceMatches` is
   not True, or if fgcmLookUpTable is not available, or if
   doSubtractLocalBackground is True and aperture radius cannot be
   determined.

Definition at line 196 of file fgcmCalibrateTractBase.py.

196  def runDataRef(self, butler, dataRefs):
197  """
198  Run full FGCM calibration on a single tract, including building star list,
199  fitting multiple cycles, and making outputs.
200 
201  Parameters
202  ----------
203  butler: `lsst.daf.persistence.Butler`
204  dataRefs: `list` of `lsst.daf.persistence.ButlerDataRef`
205  Data references for the input visits.
206  These may be either per-ccd "src" or per-visit"sourceTable_visit"
207  references.
208 
209  Raises
210  ------
211  RuntimeError: Raised if `config.fgcmBuildStars.doReferenceMatches` is
212  not True, or if fgcmLookUpTable is not available, or if
213  doSubtractLocalBackground is True and aperture radius cannot be
214  determined.
215  """
216  datasetType = dataRefs[0].butlerSubset.datasetType
217  self.log.info("Running with %d %s dataRefs" % (len(dataRefs), datasetType))
218 
219  if not butler.datasetExists('fgcmLookUpTable'):
220  raise RuntimeError("Must run FgcmCalibrateTract with an fgcmLookUpTable")
221 
222  if not self.config.fgcmBuildStars.doReferenceMatches:
223  raise RuntimeError("Must run FgcmCalibrateTract with fgcmBuildStars.doReferenceMatches")
224  if isinstance(self.config.fgcmBuildStars, FgcmBuildStarsConfig):
225  if self.config.fgcmBuildStars.checkAllCcds:
226  raise RuntimeError("Cannot run FgcmCalibrateTract with "
227  "fgcmBuildStars.checkAllCcds set to True")
228 
229  tract = int(dataRefs[0].dataId['tract'])
230  camera = butler.get('camera')
231 
232  dataRefDict = {}
233  dataRefDict['camera'] = camera
234  dataRefDict['source_catalogs'] = dataRefs
235  dataRefDict['sourceSchema'] = butler.get('src_schema', immediate=True).schema
236  dataRefDict['fgcmLookUpTable'] = butler.dataRef('fgcmLookUpTable')
237 
238  struct = self.run(dataRefDict, tract, butler=butler, returnCatalogs=False)
239 
240  visitDataRefName = self.config.fgcmBuildStars.visitDataRefName
241  ccdDataRefName = self.config.fgcmBuildStars.ccdDataRefName
242 
243  if struct.photoCalibs is not None:
244  self.log.info("Outputting photoCalib files.")
245 
246  for visit, detector, physicalFilter, photoCalib in struct.photoCalibs:
247  butler.put(photoCalib, 'fgcm_tract_photoCalib',
248  dataId={visitDataRefName: visit,
249  ccdDataRefName: detector,
250  'filter': physicalFilter,
251  'tract': tract})
252 
253  self.log.info("Done outputting photoCalib files.")
254 
255  if struct.atmospheres is not None:
256  self.log.info("Outputting atmosphere files.")
257  for visit, atm in struct.atmospheres:
258  butler.put(atm, "transmission_atmosphere_fgcm_tract",
259  dataId={visitDataRefName: visit,
260  'tract': tract})
261  self.log.info("Done outputting atmosphere transmissions.")
262 
263  return pipeBase.Struct(repeatability=struct.repeatability)
264 

◆ runQuantum()

def lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.runQuantum (   self,
  butlerQC,
  inputRefs,
  outputRefs 
)

Definition at line 182 of file fgcmCalibrateTractTable.py.

182  def runQuantum(self, butlerQC, inputRefs, outputRefs):
183  dataRefDict = butlerQC.get(inputRefs)
184 
185  self.log.info("Running with %d sourceTable_visit dataRefs", (len(dataRefDict['source_catalogs'])))
186 
187  # Run the build stars tasks
188  tract = butlerQC.quantum.dataId['tract']
189 
190  dataRefDict['sourceSchema'] = self.sourceSchema
191 
192  sourceTableRefs = dataRefDict['source_catalogs']
193  sourceTableDataRefDict = {sourceTableRef.dataId['visit']: sourceTableRef for
194  sourceTableRef in sourceTableRefs}
195 
196  visitSummaryRefs = dataRefDict['visitSummary']
197  visitSummaryDataRefDict = {visitSummaryRef.dataId['visit']: visitSummaryRef for
198  visitSummaryRef in visitSummaryRefs}
199 
200  dataRefDict['sourceTableDataRefDict'] = sourceTableDataRefDict
201  dataRefDict['visitSummaryDataRefDict'] = visitSummaryDataRefDict
202 
203  # And the outputs
204  if self.config.fgcmOutputProducts.doZeropointOutput:
205  photoCalibRefDict = {photoCalibRef.dataId.byName()['visit']:
206  photoCalibRef for photoCalibRef in outputRefs.fgcmPhotoCalib}
207  dataRefDict['fgcmPhotoCalibs'] = photoCalibRefDict
208 
209  if self.config.fgcmOutputProducts.doAtmosphereOutput:
210  atmRefDict = {atmRef.dataId.byName()['visit']: atmRef for
211  atmRef in outputRefs.fgcmTransmissionAtmosphere}
212  dataRefDict['fgcmTransmissionAtmospheres'] = atmRefDict
213 
214  if self.config.fgcmBuildStars.doReferenceMatches:
215  refConfig = self.config.fgcmBuildStars.fgcmLoadReferenceCatalog.refObjLoader
216  loader = ReferenceObjectLoader(dataIds=[ref.datasetRef.dataId
217  for ref in inputRefs.refCat],
218  refCats=butlerQC.get(inputRefs.refCat),
219  config=refConfig,
220  log=self.log)
221  buildStarsRefObjLoader = loader
222  else:
223  buildStarsRefObjLoader = None
224 
225  if self.config.fgcmOutputProducts.doReferenceCalibration:
226  refConfig = self.config.fgcmOutputProducts.refObjLoader
227  loader = ReferenceObjectLoader(dataIds=[ref.datasetRef.dataId
228  for ref in inputRefs.refCat],
229  refCats=butlerQC.get(inputRefs.refCat),
230  config=refConfig,
231  log=self.log)
232  self.fgcmOutputProducts.refObjLoader = loader
233 
234  struct = self.run(dataRefDict, tract,
235  buildStarsRefObjLoader=buildStarsRefObjLoader)
236 
237  if struct.photoCalibCatalogs is not None:
238  self.log.info("Outputting photoCalib catalogs.")
239  for visit, expCatalog in struct.photoCalibCatalogs:
240  butlerQC.put(expCatalog, photoCalibRefDict[visit])
241  self.log.info("Done outputting photoCalib catalogs.")
242 
243  if struct.atmospheres is not None:
244  self.log.info("Outputting atmosphere transmission files.")
245  for visit, atm in struct.atmospheres:
246  butlerQC.put(atm, atmRefDict[visit])
247  self.log.info("Done outputting atmosphere files.")
248 
249  # Turn raw repeatability into simple catalog for persistence
250  schema = afwTable.Schema()
251  schema.addField('rawRepeatability', type=np.float64,
252  doc="Per-band raw repeatability in FGCM calibration.")
253  repeatabilityCat = afwTable.BaseCatalog(schema)
254  repeatabilityCat.resize(len(struct.repeatability))
255  repeatabilityCat['rawRepeatability'][:] = struct.repeatability
256 
257  butlerQC.put(repeatabilityCat, outputRefs.fgcmRepeatability)
258 
259  return
260 
Defines the fields and offsets for a table.
Definition: Schema.h:51

Member Data Documentation

◆ canMultiprocess

bool lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.canMultiprocess = False
static

Definition at line 175 of file fgcmCalibrateTractTable.py.

◆ ConfigClass

lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.ConfigClass = FgcmCalibrateTractTableConfig
static

Definition at line 171 of file fgcmCalibrateTractTable.py.

◆ RunnerClass

lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.RunnerClass = FgcmCalibrateTractRunner
static

Definition at line 172 of file fgcmCalibrateTractTable.py.

◆ sourceSchema

lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.sourceSchema

Definition at line 180 of file fgcmCalibrateTractTable.py.


The documentation for this class was generated from the following file: