LSST Applications  21.0.0-147-g0e635eb1+1acddb5be5,22.0.0+052faf71bd,22.0.0+1ea9a8b2b2,22.0.0+6312710a6c,22.0.0+729191ecac,22.0.0+7589c3a021,22.0.0+9f079a9461,22.0.1-1-g7d6de66+b8044ec9de,22.0.1-1-g87000a6+536b1ee016,22.0.1-1-g8e32f31+6312710a6c,22.0.1-10-gd060f87+016f7cdc03,22.0.1-12-g9c3108e+df145f6f68,22.0.1-16-g314fa6d+c825727ab8,22.0.1-19-g93a5c75+d23f2fb6d8,22.0.1-19-gb93eaa13+aab3ef7709,22.0.1-2-g8ef0a89+b8044ec9de,22.0.1-2-g92698f7+9f079a9461,22.0.1-2-ga9b0f51+052faf71bd,22.0.1-2-gac51dbf+052faf71bd,22.0.1-2-gb66926d+6312710a6c,22.0.1-2-gcb770ba+09e3807989,22.0.1-20-g32debb5+b8044ec9de,22.0.1-23-gc2439a9a+fb0756638e,22.0.1-3-g496fd5d+09117f784f,22.0.1-3-g59f966b+1e6ba2c031,22.0.1-3-g849a1b8+f8b568069f,22.0.1-3-gaaec9c0+c5c846a8b1,22.0.1-32-g5ddfab5d3+60ce4897b0,22.0.1-4-g037fbe1+64e601228d,22.0.1-4-g8623105+b8044ec9de,22.0.1-5-g096abc9+d18c45d440,22.0.1-5-g15c806e+57f5c03693,22.0.1-7-gba73697+57f5c03693,master-g6e05de7fdc+c1283a92b8,master-g72cdda8301+729191ecac,w.2021.39
LSST Data Management Base Package
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask Class Reference
Inheritance diagram for lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask:
lsst.fgcmcal.fgcmCalibrateTractBase.FgcmCalibrateTractBaseTask

Public Member Functions

def __init__ (self, initInputs=None, **kwargs)
 
def runQuantum (self, butlerQC, inputRefs, outputRefs)
 
def runDataRef (self, butler, dataRefs)
 
def run (self, dataRefDict, tract, buildStarsRefObjLoader=None, returnCatalogs=True, butler=None)
 

Public Attributes

 sourceSchema
 

Static Public Attributes

 ConfigClass = FgcmCalibrateTractTableConfig
 
 RunnerClass = FgcmCalibrateTractRunner
 
bool canMultiprocess = False
 

Detailed Description

Calibrate a single tract using fgcmcal, using sourceTable_visit (parquet)
input catalogs.

Definition at line 166 of file fgcmCalibrateTractTable.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.__init__ (   self,
  initInputs = None,
**  kwargs 
)

Definition at line 177 of file fgcmCalibrateTractTable.py.

177  def __init__(self, initInputs=None, **kwargs):
178  super().__init__(initInputs=initInputs, **kwargs)
179  if initInputs is not None:
180  self.sourceSchema = initInputs["sourceSchema"].schema
181 

Member Function Documentation

◆ run()

def lsst.fgcmcal.fgcmCalibrateTractBase.FgcmCalibrateTractBaseTask.run (   self,
  dataRefDict,
  tract,
  buildStarsRefObjLoader = None,
  returnCatalogs = True,
  butler = None 
)
inherited
Run the calibrations for a single tract with fgcm.

Parameters
----------
dataRefDict : `dict`
    All dataRefs are `lsst.daf.persistence.ButlerDataRef` (gen2) or
    `lsst.daf.butler.DeferredDatasetHandle` (gen3)
    dataRef dictionary with the following keys.  Note that all
    keys need not be set based on config parameters.

    ``"camera"``
        Camera object (`lsst.afw.cameraGeom.Camera`)
    ``"source_catalogs"``
        `list` of dataRefs for input source catalogs.
    ``"sourceSchema"``
        Schema for the source catalogs.
    ``"fgcmLookUpTable"``
        dataRef for the FGCM look-up table.
    ``"calexps"``
        `list` of dataRefs for the input calexps (Gen3 only)
    ``"fgcmPhotoCalibs"``
        `dict` of output photoCalib dataRefs.  Key is
        (tract, visit, detector). (Gen3 only)
        Present if doZeropointOutput is True.
    ``"fgcmTransmissionAtmospheres"``
        `dict` of output atmosphere transmission dataRefs.
        Key is (tract, visit). (Gen3 only)
        Present if doAtmosphereOutput is True.
tract : `int`
    Tract number
buildStarsRefObjLoader : `lsst.meas.algorithms.ReferenceObjectLoader`, optional
    Reference object loader object for fgcmBuildStars.
returnCatalogs : `bool`, optional
    Return photoCalibs as per-visit exposure catalogs.
butler : `lsst.daf.persistence.Butler`, optional
    Gen2 butler used for reference star outputs

Returns
-------
outstruct : `lsst.pipe.base.Struct`
    Output structure with keys:

    offsets : `np.ndarray`
        Final reference offsets, per band.
    repeatability : `np.ndarray`
        Raw fgcm repeatability for bright stars, per band.
    atmospheres : `generator` [(`int`, `lsst.afw.image.TransmissionCurve`)]
        Generator that returns (visit, transmissionCurve) tuples.
    photoCalibs : `generator` [(`int`, `int`, `str`, `lsst.afw.image.PhotoCalib`)]
        Generator that returns (visit, ccd, filtername, photoCalib) tuples.
        (returned if returnCatalogs is False).
    photoCalibCatalogs : `generator` [(`int`, `lsst.afw.table.ExposureCatalog`)]
        Generator that returns (visit, exposureCatalog) tuples.
        (returned if returnCatalogs is True).

Definition at line 263 of file fgcmCalibrateTractBase.py.

264  buildStarsRefObjLoader=None, returnCatalogs=True, butler=None):
265  """Run the calibrations for a single tract with fgcm.
266 
267  Parameters
268  ----------
269  dataRefDict : `dict`
270  All dataRefs are `lsst.daf.persistence.ButlerDataRef` (gen2) or
271  `lsst.daf.butler.DeferredDatasetHandle` (gen3)
272  dataRef dictionary with the following keys. Note that all
273  keys need not be set based on config parameters.
274 
275  ``"camera"``
276  Camera object (`lsst.afw.cameraGeom.Camera`)
277  ``"source_catalogs"``
278  `list` of dataRefs for input source catalogs.
279  ``"sourceSchema"``
280  Schema for the source catalogs.
281  ``"fgcmLookUpTable"``
282  dataRef for the FGCM look-up table.
283  ``"calexps"``
284  `list` of dataRefs for the input calexps (Gen3 only)
285  ``"fgcmPhotoCalibs"``
286  `dict` of output photoCalib dataRefs. Key is
287  (tract, visit, detector). (Gen3 only)
288  Present if doZeropointOutput is True.
289  ``"fgcmTransmissionAtmospheres"``
290  `dict` of output atmosphere transmission dataRefs.
291  Key is (tract, visit). (Gen3 only)
292  Present if doAtmosphereOutput is True.
293  tract : `int`
294  Tract number
295  buildStarsRefObjLoader : `lsst.meas.algorithms.ReferenceObjectLoader`, optional
296  Reference object loader object for fgcmBuildStars.
297  returnCatalogs : `bool`, optional
298  Return photoCalibs as per-visit exposure catalogs.
299  butler : `lsst.daf.persistence.Butler`, optional
300  Gen2 butler used for reference star outputs
301 
302  Returns
303  -------
304  outstruct : `lsst.pipe.base.Struct`
305  Output structure with keys:
306 
307  offsets : `np.ndarray`
308  Final reference offsets, per band.
309  repeatability : `np.ndarray`
310  Raw fgcm repeatability for bright stars, per band.
311  atmospheres : `generator` [(`int`, `lsst.afw.image.TransmissionCurve`)]
312  Generator that returns (visit, transmissionCurve) tuples.
313  photoCalibs : `generator` [(`int`, `int`, `str`, `lsst.afw.image.PhotoCalib`)]
314  Generator that returns (visit, ccd, filtername, photoCalib) tuples.
315  (returned if returnCatalogs is False).
316  photoCalibCatalogs : `generator` [(`int`, `lsst.afw.table.ExposureCatalog`)]
317  Generator that returns (visit, exposureCatalog) tuples.
318  (returned if returnCatalogs is True).
319  """
320  self.log.info("Running on tract %d", (tract))
321 
322  # Compute the aperture radius if necessary. This is useful to do now before
323  # any heavy lifting has happened (fail early).
324  calibFluxApertureRadius = None
325  if self.config.fgcmBuildStars.doSubtractLocalBackground:
326  try:
327  field = self.config.fgcmBuildStars.instFluxField
328  calibFluxApertureRadius = computeApertureRadiusFromDataRef(dataRefDict['source_catalogs'][0],
329  field)
330  except RuntimeError:
331  raise RuntimeError("Could not determine aperture radius from %s. "
332  "Cannot use doSubtractLocalBackground." %
333  (field))
334 
335  # Run the build stars tasks
336 
337  # Note that we will need visitCat at the end of the procedure for the outputs
338  if isinstance(butler, dafPersist.Butler):
339  # Gen2
340  groupedDataRefs = self.fgcmBuildStars._findAndGroupDataRefsGen2(butler, dataRefDict['camera'],
341  dataRefDict['source_catalogs'])
342  else:
343  # Gen3
344  groupedDataRefs = self.fgcmBuildStars._groupDataRefs(dataRefDict['sourceTableDataRefDict'],
345  dataRefDict['visitSummaryDataRefDict'])
346  visitCat = self.fgcmBuildStars.fgcmMakeVisitCatalog(dataRefDict['camera'], groupedDataRefs)
347  rad = calibFluxApertureRadius
348  fgcmStarObservationCat = self.fgcmBuildStars.fgcmMakeAllStarObservations(groupedDataRefs,
349  visitCat,
350  dataRefDict['sourceSchema'],
351  dataRefDict['camera'],
352  calibFluxApertureRadius=rad)
353 
354  if self.fgcmBuildStars.config.doReferenceMatches:
355  lutDataRef = dataRefDict['fgcmLookUpTable']
356  if buildStarsRefObjLoader is not None:
357  self.fgcmBuildStars.makeSubtask("fgcmLoadReferenceCatalog",
358  refObjLoader=buildStarsRefObjLoader)
359  else:
360  self.fgcmBuildStars.makeSubtask("fgcmLoadReferenceCatalog", butler=butler)
361  else:
362  lutDataRef = None
363 
364  fgcmStarIdCat, fgcmStarIndicesCat, fgcmRefCat = \
365  self.fgcmBuildStars.fgcmMatchStars(visitCat,
366  fgcmStarObservationCat,
367  lutDataRef=lutDataRef)
368 
369  # Load the LUT
370  lutCat = dataRefDict['fgcmLookUpTable'].get()
371  fgcmLut, lutIndexVals, lutStd = translateFgcmLut(lutCat,
372  dict(self.config.fgcmFitCycle.physicalFilterMap))
373  del lutCat
374 
375  # Translate the visit catalog into fgcm format
376  fgcmExpInfo = translateVisitCatalog(visitCat)
377 
378  configDict = makeConfigDict(self.config.fgcmFitCycle, self.log, dataRefDict['camera'],
379  self.config.fgcmFitCycle.maxIterBeforeFinalCycle,
380  True, False, lutIndexVals[0]['FILTERNAMES'],
381  tract=tract)
382 
383  # Use the first orientation.
384  # TODO: DM-21215 will generalize to arbitrary camera orientations
385  ccdOffsets = computeCcdOffsets(dataRefDict['camera'], fgcmExpInfo['TELROT'][0])
386 
387  # Set up the fit cycle task
388 
389  noFitsDict = {'lutIndex': lutIndexVals,
390  'lutStd': lutStd,
391  'expInfo': fgcmExpInfo,
392  'ccdOffsets': ccdOffsets}
393 
394  fgcmFitCycle = fgcm.FgcmFitCycle(configDict, useFits=False,
395  noFitsDict=noFitsDict, noOutput=True)
396 
397  # We determine the conversion from the native units (typically radians) to
398  # degrees for the first star. This allows us to treat coord_ra/coord_dec as
399  # numpy arrays rather than Angles, which would we approximately 600x slower.
400  conv = fgcmStarObservationCat[0]['ra'].asDegrees() / float(fgcmStarObservationCat[0]['ra'])
401 
402  # To load the stars, we need an initial parameter object
403  fgcmPars = fgcm.FgcmParameters.newParsWithArrays(fgcmFitCycle.fgcmConfig,
404  fgcmLut,
405  fgcmExpInfo)
406 
407  # Match star observations to visits
408  # Only those star observations that match visits from fgcmExpInfo['VISIT'] will
409  # actually be transferred into fgcm using the indexing below.
410 
411  obsIndex = fgcmStarIndicesCat['obsIndex']
412  visitIndex = np.searchsorted(fgcmExpInfo['VISIT'],
413  fgcmStarObservationCat['visit'][obsIndex])
414 
415  refMag, refMagErr = extractReferenceMags(fgcmRefCat,
416  self.config.fgcmFitCycle.bands,
417  self.config.fgcmFitCycle.physicalFilterMap)
418  refId = fgcmRefCat['fgcm_id'][:]
419 
420  fgcmStars = fgcm.FgcmStars(fgcmFitCycle.fgcmConfig)
421  fgcmStars.loadStars(fgcmPars,
422  fgcmStarObservationCat['visit'][obsIndex],
423  fgcmStarObservationCat['ccd'][obsIndex],
424  fgcmStarObservationCat['ra'][obsIndex] * conv,
425  fgcmStarObservationCat['dec'][obsIndex] * conv,
426  fgcmStarObservationCat['instMag'][obsIndex],
427  fgcmStarObservationCat['instMagErr'][obsIndex],
428  fgcmExpInfo['FILTERNAME'][visitIndex],
429  fgcmStarIdCat['fgcm_id'][:],
430  fgcmStarIdCat['ra'][:],
431  fgcmStarIdCat['dec'][:],
432  fgcmStarIdCat['obsArrIndex'][:],
433  fgcmStarIdCat['nObs'][:],
434  obsX=fgcmStarObservationCat['x'][obsIndex],
435  obsY=fgcmStarObservationCat['y'][obsIndex],
436  obsDeltaMagBkg=fgcmStarObservationCat['deltaMagBkg'][obsIndex],
437  psfCandidate=fgcmStarObservationCat['psf_candidate'][obsIndex],
438  refID=refId,
439  refMag=refMag,
440  refMagErr=refMagErr,
441  flagID=None,
442  flagFlag=None,
443  computeNobs=True)
444 
445  # Clear out some memory
446  del fgcmStarIdCat
447  del fgcmStarIndicesCat
448  del fgcmRefCat
449 
450  fgcmFitCycle.setLUT(fgcmLut)
451  fgcmFitCycle.setStars(fgcmStars, fgcmPars)
452 
453  converged = False
454  cycleNumber = 0
455 
456  previousReservedRawRepeatability = np.zeros(fgcmPars.nBands) + 1000.0
457  previousParInfo = None
458  previousParams = None
459  previousSuperStar = None
460 
461  while (not converged and cycleNumber < self.config.maxFitCycles):
462 
463  fgcmFitCycle.fgcmConfig.updateCycleNumber(cycleNumber)
464 
465  if cycleNumber > 0:
466  # Use parameters from previous cycle
467  fgcmPars = fgcm.FgcmParameters.loadParsWithArrays(fgcmFitCycle.fgcmConfig,
468  fgcmExpInfo,
469  previousParInfo,
470  previousParams,
471  previousSuperStar)
472  # We need to reset the star magnitudes and errors for the next
473  # cycle
474  fgcmFitCycle.fgcmStars.reloadStarMagnitudes(fgcmStarObservationCat['instMag'][obsIndex],
475  fgcmStarObservationCat['instMagErr'][obsIndex])
476  fgcmFitCycle.initialCycle = False
477 
478  fgcmFitCycle.setPars(fgcmPars)
479  fgcmFitCycle.finishSetup()
480 
481  fgcmFitCycle.run()
482 
483  # Grab the parameters for the next cycle
484  previousParInfo, previousParams = fgcmFitCycle.fgcmPars.parsToArrays()
485  previousSuperStar = fgcmFitCycle.fgcmPars.parSuperStarFlat.copy()
486 
487  self.log.info("Raw repeatability after cycle number %d is:" % (cycleNumber))
488  for i, band in enumerate(fgcmFitCycle.fgcmPars.bands):
489  if not fgcmFitCycle.fgcmPars.hasExposuresInBand[i]:
490  continue
491  rep = fgcmFitCycle.fgcmPars.compReservedRawRepeatability[i] * 1000.0
492  self.log.info(" Band %s, repeatability: %.2f mmag" % (band, rep))
493 
494  # Check for convergence
495  if np.alltrue((previousReservedRawRepeatability
496  - fgcmFitCycle.fgcmPars.compReservedRawRepeatability)
497  < self.config.convergenceTolerance):
498  self.log.info("Raw repeatability has converged after cycle number %d." % (cycleNumber))
499  converged = True
500  else:
501  fgcmFitCycle.fgcmConfig.expGrayPhotometricCut[:] = fgcmFitCycle.updatedPhotometricCut
502  fgcmFitCycle.fgcmConfig.expGrayHighCut[:] = fgcmFitCycle.updatedHighCut
503  fgcmFitCycle.fgcmConfig.precomputeSuperStarInitialCycle = False
504  fgcmFitCycle.fgcmConfig.freezeStdAtmosphere = False
505  previousReservedRawRepeatability[:] = fgcmFitCycle.fgcmPars.compReservedRawRepeatability
506  self.log.info("Setting exposure gray photometricity cuts to:")
507  for i, band in enumerate(fgcmFitCycle.fgcmPars.bands):
508  if not fgcmFitCycle.fgcmPars.hasExposuresInBand[i]:
509  continue
510  cut = fgcmFitCycle.updatedPhotometricCut[i] * 1000.0
511  self.log.info(" Band %s, photometricity cut: %.2f mmag" % (band, cut))
512 
513  cycleNumber += 1
514 
515  # Log warning if not converged
516  if not converged:
517  self.log.warning("Maximum number of fit cycles exceeded (%d) without convergence.", cycleNumber)
518 
519  # Do final clean-up iteration
520  fgcmFitCycle.fgcmConfig.freezeStdAtmosphere = False
521  fgcmFitCycle.fgcmConfig.resetParameters = False
522  fgcmFitCycle.fgcmConfig.maxIter = 0
523  fgcmFitCycle.fgcmConfig.outputZeropoints = True
524  fgcmFitCycle.fgcmConfig.outputStandards = True
525  fgcmFitCycle.fgcmConfig.doPlots = self.config.doDebuggingPlots
526  fgcmFitCycle.fgcmConfig.updateCycleNumber(cycleNumber)
527  fgcmFitCycle.initialCycle = False
528 
529  fgcmPars = fgcm.FgcmParameters.loadParsWithArrays(fgcmFitCycle.fgcmConfig,
530  fgcmExpInfo,
531  previousParInfo,
532  previousParams,
533  previousSuperStar)
534  fgcmFitCycle.fgcmStars.reloadStarMagnitudes(fgcmStarObservationCat['instMag'][obsIndex],
535  fgcmStarObservationCat['instMagErr'][obsIndex])
536  fgcmFitCycle.setPars(fgcmPars)
537  fgcmFitCycle.finishSetup()
538 
539  self.log.info("Running final clean-up fit cycle...")
540  fgcmFitCycle.run()
541 
542  self.log.info("Raw repeatability after clean-up cycle is:")
543  for i, band in enumerate(fgcmFitCycle.fgcmPars.bands):
544  if not fgcmFitCycle.fgcmPars.hasExposuresInBand[i]:
545  continue
546  rep = fgcmFitCycle.fgcmPars.compReservedRawRepeatability[i] * 1000.0
547  self.log.info(" Band %s, repeatability: %.2f mmag" % (band, rep))
548 
549  # Do the outputs. Need to keep track of tract.
550 
551  superStarChebSize = fgcmFitCycle.fgcmZpts.zpStruct['FGCM_FZPT_SSTAR_CHEB'].shape[1]
552  zptChebSize = fgcmFitCycle.fgcmZpts.zpStruct['FGCM_FZPT_CHEB'].shape[1]
553 
554  zptSchema = makeZptSchema(superStarChebSize, zptChebSize)
555  zptCat = makeZptCat(zptSchema, fgcmFitCycle.fgcmZpts.zpStruct)
556 
557  atmSchema = makeAtmSchema()
558  atmCat = makeAtmCat(atmSchema, fgcmFitCycle.fgcmZpts.atmStruct)
559 
560  stdStruct, goodBands = fgcmFitCycle.fgcmStars.retrieveStdStarCatalog(fgcmFitCycle.fgcmPars)
561  stdSchema = makeStdSchema(len(goodBands))
562  stdCat = makeStdCat(stdSchema, stdStruct, goodBands)
563 
564  outStruct = self.fgcmOutputProducts.generateTractOutputProducts(dataRefDict,
565  tract,
566  visitCat,
567  zptCat, atmCat, stdCat,
568  self.config.fgcmBuildStars,
569  returnCatalogs=returnCatalogs,
570  butler=butler)
571 
572  outStruct.repeatability = fgcmFitCycle.fgcmPars.compReservedRawRepeatability
573 
574  fgcmFitCycle.freeSharedMemory()
575 
576  return outStruct
def extractReferenceMags(refStars, bands, filterMap)
Definition: utilities.py:883
def computeApertureRadiusFromDataRef(dataRef, fluxField)
Definition: utilities.py:808
def makeStdSchema(nBands)
Definition: utilities.py:738
def makeAtmCat(atmSchema, atmStruct)
Definition: utilities.py:705
def makeConfigDict(config, log, camera, maxIter, resetFitParameters, outputZeropoints, lutFilterNames, tract=None)
Definition: utilities.py:51
def translateFgcmLut(lutCat, physicalFilterMap)
Definition: utilities.py:218
def makeZptCat(zptSchema, zpStruct)
Definition: utilities.py:626
def makeStdCat(stdSchema, stdStruct, goodBands)
Definition: utilities.py:770
def makeZptSchema(superStarChebyshevSize, zptChebyshevSize)
Definition: utilities.py:540
def computeCcdOffsets(camera, defaultOrientation)
Definition: utilities.py:399
def translateVisitCatalog(visitCat)
Definition: utilities.py:348

◆ runDataRef()

def lsst.fgcmcal.fgcmCalibrateTractBase.FgcmCalibrateTractBaseTask.runDataRef (   self,
  butler,
  dataRefs 
)
inherited
Run full FGCM calibration on a single tract, including building star list,
fitting multiple cycles, and making outputs.

Parameters
----------
butler:  `lsst.daf.persistence.Butler`
dataRefs: `list` of `lsst.daf.persistence.ButlerDataRef`
   Data references for the input visits.
   These may be either per-ccd "src" or per-visit"sourceTable_visit"
   references.

Raises
------
RuntimeError: Raised if `config.fgcmBuildStars.doReferenceMatches` is
   not True, or if fgcmLookUpTable is not available, or if
   doSubtractLocalBackground is True and aperture radius cannot be
   determined.

Definition at line 194 of file fgcmCalibrateTractBase.py.

194  def runDataRef(self, butler, dataRefs):
195  """
196  Run full FGCM calibration on a single tract, including building star list,
197  fitting multiple cycles, and making outputs.
198 
199  Parameters
200  ----------
201  butler: `lsst.daf.persistence.Butler`
202  dataRefs: `list` of `lsst.daf.persistence.ButlerDataRef`
203  Data references for the input visits.
204  These may be either per-ccd "src" or per-visit"sourceTable_visit"
205  references.
206 
207  Raises
208  ------
209  RuntimeError: Raised if `config.fgcmBuildStars.doReferenceMatches` is
210  not True, or if fgcmLookUpTable is not available, or if
211  doSubtractLocalBackground is True and aperture radius cannot be
212  determined.
213  """
214  datasetType = dataRefs[0].butlerSubset.datasetType
215  self.log.info("Running with %d %s dataRefs" % (len(dataRefs), datasetType))
216 
217  if not butler.datasetExists('fgcmLookUpTable'):
218  raise RuntimeError("Must run FgcmCalibrateTract with an fgcmLookUpTable")
219 
220  if not self.config.fgcmBuildStars.doReferenceMatches:
221  raise RuntimeError("Must run FgcmCalibrateTract with fgcmBuildStars.doReferenceMatches")
222  if isinstance(self.config.fgcmBuildStars, FgcmBuildStarsConfig):
223  if self.config.fgcmBuildStars.checkAllCcds:
224  raise RuntimeError("Cannot run FgcmCalibrateTract with "
225  "fgcmBuildStars.checkAllCcds set to True")
226 
227  tract = int(dataRefs[0].dataId['tract'])
228  camera = butler.get('camera')
229 
230  dataRefDict = {}
231  dataRefDict['camera'] = camera
232  dataRefDict['source_catalogs'] = dataRefs
233  dataRefDict['sourceSchema'] = butler.get('src_schema', immediate=True).schema
234  dataRefDict['fgcmLookUpTable'] = butler.dataRef('fgcmLookUpTable')
235 
236  struct = self.run(dataRefDict, tract, butler=butler, returnCatalogs=False)
237 
238  visitDataRefName = self.config.fgcmBuildStars.visitDataRefName
239  ccdDataRefName = self.config.fgcmBuildStars.ccdDataRefName
240 
241  if struct.photoCalibs is not None:
242  self.log.info("Outputting photoCalib files.")
243 
244  for visit, detector, physicalFilter, photoCalib in struct.photoCalibs:
245  butler.put(photoCalib, 'fgcm_tract_photoCalib',
246  dataId={visitDataRefName: visit,
247  ccdDataRefName: detector,
248  'filter': physicalFilter,
249  'tract': tract})
250 
251  self.log.info("Done outputting photoCalib files.")
252 
253  if struct.atmospheres is not None:
254  self.log.info("Outputting atmosphere files.")
255  for visit, atm in struct.atmospheres:
256  butler.put(atm, "transmission_atmosphere_fgcm_tract",
257  dataId={visitDataRefName: visit,
258  'tract': tract})
259  self.log.info("Done outputting atmosphere transmissions.")
260 
261  return pipeBase.Struct(repeatability=struct.repeatability)
262 

◆ runQuantum()

def lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.runQuantum (   self,
  butlerQC,
  inputRefs,
  outputRefs 
)

Definition at line 182 of file fgcmCalibrateTractTable.py.

182  def runQuantum(self, butlerQC, inputRefs, outputRefs):
183  dataRefDict = butlerQC.get(inputRefs)
184 
185  self.log.info("Running with %d sourceTable_visit dataRefs", (len(dataRefDict['source_catalogs'])))
186 
187  # Run the build stars tasks
188  tract = butlerQC.quantum.dataId['tract']
189 
190  dataRefDict['sourceSchema'] = self.sourceSchema
191 
192  sourceTableRefs = dataRefDict['source_catalogs']
193  sourceTableDataRefDict = {sourceTableRef.dataId['visit']: sourceTableRef for
194  sourceTableRef in sourceTableRefs}
195 
196  visitSummaryRefs = dataRefDict['visitSummary']
197  visitSummaryDataRefDict = {visitSummaryRef.dataId['visit']: visitSummaryRef for
198  visitSummaryRef in visitSummaryRefs}
199 
200  dataRefDict['sourceTableDataRefDict'] = sourceTableDataRefDict
201  dataRefDict['visitSummaryDataRefDict'] = visitSummaryDataRefDict
202 
203  # And the outputs
204  if self.config.fgcmOutputProducts.doZeropointOutput:
205  photoCalibRefDict = {photoCalibRef.dataId.byName()['visit']:
206  photoCalibRef for photoCalibRef in outputRefs.fgcmPhotoCalib}
207  dataRefDict['fgcmPhotoCalibs'] = photoCalibRefDict
208 
209  if self.config.fgcmOutputProducts.doAtmosphereOutput:
210  atmRefDict = {atmRef.dataId.byName()['visit']: atmRef for
211  atmRef in outputRefs.fgcmTransmissionAtmosphere}
212  dataRefDict['fgcmTransmissionAtmospheres'] = atmRefDict
213 
214  if self.config.fgcmBuildStars.doReferenceMatches:
215  refConfig = self.config.fgcmBuildStars.fgcmLoadReferenceCatalog.refObjLoader
216  loader = ReferenceObjectLoader(dataIds=[ref.datasetRef.dataId
217  for ref in inputRefs.refCat],
218  refCats=butlerQC.get(inputRefs.refCat),
219  config=refConfig,
220  log=self.log)
221  buildStarsRefObjLoader = loader
222  else:
223  buildStarsRefObjLoader = None
224 
225  if self.config.fgcmOutputProducts.doReferenceCalibration:
226  refConfig = self.config.fgcmOutputProducts.refObjLoader
227  loader = ReferenceObjectLoader(dataIds=[ref.datasetRef.dataId
228  for ref in inputRefs.refCat],
229  refCats=butlerQC.get(inputRefs.refCat),
230  config=refConfig,
231  log=self.log)
232  self.fgcmOutputProducts.refObjLoader = loader
233 
234  struct = self.run(dataRefDict, tract,
235  buildStarsRefObjLoader=buildStarsRefObjLoader)
236 
237  if struct.photoCalibCatalogs is not None:
238  self.log.info("Outputting photoCalib catalogs.")
239  for visit, expCatalog in struct.photoCalibCatalogs:
240  butlerQC.put(expCatalog, photoCalibRefDict[visit])
241  self.log.info("Done outputting photoCalib catalogs.")
242 
243  if struct.atmospheres is not None:
244  self.log.info("Outputting atmosphere transmission files.")
245  for visit, atm in struct.atmospheres:
246  butlerQC.put(atm, atmRefDict[visit])
247  self.log.info("Done outputting atmosphere files.")
248 
249  # Turn raw repeatability into simple catalog for persistence
250  schema = afwTable.Schema()
251  schema.addField('rawRepeatability', type=np.float64,
252  doc="Per-band raw repeatability in FGCM calibration.")
253  repeatabilityCat = afwTable.BaseCatalog(schema)
254  repeatabilityCat.resize(len(struct.repeatability))
255  repeatabilityCat['rawRepeatability'][:] = struct.repeatability
256 
257  butlerQC.put(repeatabilityCat, outputRefs.fgcmRepeatability)
258 
259  return
260 
Defines the fields and offsets for a table.
Definition: Schema.h:51

Member Data Documentation

◆ canMultiprocess

bool lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.canMultiprocess = False
static

Definition at line 175 of file fgcmCalibrateTractTable.py.

◆ ConfigClass

lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.ConfigClass = FgcmCalibrateTractTableConfig
static

Definition at line 171 of file fgcmCalibrateTractTable.py.

◆ RunnerClass

lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.RunnerClass = FgcmCalibrateTractRunner
static

Definition at line 172 of file fgcmCalibrateTractTable.py.

◆ sourceSchema

lsst.fgcmcal.fgcmCalibrateTractTable.FgcmCalibrateTractTableTask.sourceSchema

Definition at line 180 of file fgcmCalibrateTractTable.py.


The documentation for this class was generated from the following file: