LSSTApplications  10.0+286,10.0+36,10.0+46,10.0-2-g4f67435,10.1+152,10.1+37,11.0,11.0+1,11.0-1-g47edd16,11.0-1-g60db491,11.0-1-g7418c06,11.0-2-g04d2804,11.0-2-g68503cd,11.0-2-g818369d,11.0-2-gb8b8ce7
LSSTDataManagementBasePackage
Public Member Functions | Public Attributes | Static Public Attributes | Static Private Attributes | List of all members
lsst.pipe.tasks.photoCal.PhotoCalTask Class Reference

Calculate the zero point of an exposure given a lsst.afw.table.ReferenceMatchVector. More...

Inheritance diagram for lsst.pipe.tasks.photoCal.PhotoCalTask:

Public Member Functions

def __init__
 Create the photometric calibration task. More...
 
def getSourceKeys
 Return a struct containing the source catalog keys for fields used by PhotoCalTask. More...
 
def selectMatches
 Select reference/source matches according the criteria specified in the config. More...
 
def extractMagArrays
 Extract magnitude and magnitude error arrays from the given matches. More...
 
def run
 Do photometric calibration - select matches to use and (possibly iteratively) compute the zero point. More...
 
def getZeroPoint
 Flux calibration code, returning (ZeroPoint, Distribution Width, Number of stars) More...
 

Public Attributes

 scatterPlot
 
 fig
 
 outputField
 

Static Public Attributes

 ConfigClass = PhotoCalConfig
 

Static Private Attributes

string _DefaultName = "photoCal"
 

Detailed Description

Calculate the zero point of an exposure given a lsst.afw.table.ReferenceMatchVector.

Contents

Description

Calculate the zero point of an exposure given a lsst.afw.table.ReferenceMatchVector.

Calculate an Exposure's zero-point given a set of flux measurements of stars matched to an input catalogue. The type of flux to use is specified by PhotoCalConfig.fluxField.

The algorithm clips outliers iteratively, with parameters set in the configuration.

Note
This task can adds fields to the schema, so any code calling this task must ensure that these columns are indeed present in the input match list; see A complete example of using PhotoCalTask

Task initialisation

Create the photometric calibration task. See PhotoCalTask.init for documentation

Inputs/Outputs to the run method

Do photometric calibration - select matches to use and (possibly iteratively) compute the zero point.

Parameters
[in]exposureExposure upon which the sources in the matches were detected.
[in]matchesInput lsst.afw.table.ReferenceMatchVector (i.e. a list of lsst.afw.table.Match with first being of type lsst.afw.table.SimpleRecord and second type lsst.afw.table.SourceRecord — the reference object and matched object respectively). (will not be modified except to set the outputField if requested.).
Returns
Struct of:
  • calib ----— lsst::afw::image::Calib object containing the zero point
  • arrays ---— Magnitude arrays returned be PhotoCalTask.extractMagArrays
  • matches --— Final ReferenceMatchVector, as returned by PhotoCalTask.selectMatches.

The exposure is only used to provide the name of the filter being calibrated (it may also be used to generate debugging plots).

The reference objects:

The measured sources:

Exceptions
RuntimeErrorwith the following strings:
`sources' schema does not contain the calibration object flag "XXX"`
The constructor added fields to the schema that aren't in the Sources
No input matches
The input match vector is empty
All matches eliminated by source flags
The flags specified by badFlags in the config eliminated all candidate objects
No sources remain in match list after reference catalog cuts
The reference catalogue has a column "photometric", but no matched objects have it set
No sources remaining in match list after magnitude limit cuts
All surviving matches are either too faint in the catalogue or have negative or NaN flux
No reference stars are available
No matches survive all the checks

Configuration parameters

See PhotoCalConfig

Debug variables

The command line task interface supports a flag -d to import debug.py from your PYTHONPATH; see Using lsstDebug to control debugging output for more about debug.py files.

The available variables in PhotoCalTask are:

display
If True enable other debug outputs
displaySources
If True, display the exposure on ds9's frame 1 and overlay the source catalogue:
red x
Bad objects
blue +
Matched objects deemed unsuitable for photometric calibration. Additional information is:
  • a cyan o for galaxies
  • a magenta o for variables
magenta *
Objects that failed the flux cut
green o
Objects used in the photometric calibration
scatterPlot
Make a scatter plot of flux v. reference magnitude as a function of reference magnitude.
  • good objects in blue
  • rejected objects in red (if scatterPlot is 2 or more, prompt to continue after each iteration)

A complete example of using PhotoCalTask

This code is in examples/photoCalTask.py, and can be run as e.g.

1 examples/photoCalTask.py

Import the tasks (there are some other standard imports; read the file for details)

We need to create both our tasks before processing any data as the task constructors can add extra columns to the schema which we get from the input catalogue, scrCat:

Astrometry first:

(that filterMap line is because our test code doesn't use a filter that the reference catalogue recognises, so we tell it to use the r band)

Then photometry:

If the schema has indeed changed we need to add the new columns to the source table (yes; this should be easier!)

We're now ready to process the data (we could loop over multiple exposures/catalogues using the same task objects):

We can then unpack and use the results:

To investigate the Debug variables, put something like

1 import lsstDebug
2 def DebugInfo(name):
3  di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively
4  if name.endswith(".PhotoCal"):
5  di.display = 1
6 
7  return di
8 
9 lsstDebug.Info = DebugInfo

into your debug.py file and run photoCalTask.py with the –debug flag.

Definition at line 142 of file photoCal.py.

Constructor & Destructor Documentation

def lsst.pipe.tasks.photoCal.PhotoCalTask.__init__ (   self,
  schema,
  kwds 
)

Create the photometric calibration task.

See PhotoCalTask.init for documentation

Definition at line 270 of file photoCal.py.

270 
271  def __init__(self, schema, **kwds):
272  """!Create the photometric calibration task. See PhotoCalTask.init for documentation
273  """
274  pipeBase.Task.__init__(self, **kwds)
275  self.scatterPlot = None
276  self.fig = None
277  if self.config.doWriteOutput:
278  self.outputField = schema.addField("photocal_photometricStandard", type="Flag",
279  doc="set if source was used in photometric calibration")
280  else:
281  self.outputField = None
lsst.pipe.tasks.photoCal.PhotoCalTask.__init__
def __init__
Create the photometric calibration task.
Definition: photoCal.py:270

Member Function Documentation

def lsst.pipe.tasks.photoCal.PhotoCalTask.extractMagArrays (   self,
  matches,
  filterName,
  sourceKeys 
)

Extract magnitude and magnitude error arrays from the given matches.

Parameters
[in]matchesReference/source matches, a lsst::afw::table::ReferenceMatchVector
[in]filterNameName of filter being calibrated
[in]sourceKeysStruct of source catalog keys, as returned by getSourceKeys()
Returns
Struct containing srcMag, refMag, srcMagErr, refMagErr, and magErr numpy arrays where magErr is an error in the magnitude; the error in srcMag - refMag If nonzero, config.magErrFloor will be added to magErr only (not srcMagErr or refMagErr), as magErr is what is later used to determine the zero point. Struct also contains refFluxFieldList: a list of field names of the reference catalog used for fluxes (1 or 2 strings)
Note
These magnitude arrays are the inputs to the photometric calibration, some may have been discarded by clipping while estimating the calibration (https://jira.lsstcorp.org/browse/DM-813)

Definition at line 419 of file photoCal.py.

419 
420  def extractMagArrays(self, matches, filterName, sourceKeys):
421  """!Extract magnitude and magnitude error arrays from the given matches.
422 
423  \param[in] matches Reference/source matches, a \link lsst::afw::table::ReferenceMatchVector\endlink
424  \param[in] filterName Name of filter being calibrated
425  \param[in] sourceKeys Struct of source catalog keys, as returned by getSourceKeys()
426 
427  \return Struct containing srcMag, refMag, srcMagErr, refMagErr, and magErr numpy arrays
428  where magErr is an error in the magnitude; the error in srcMag - refMag
429  If nonzero, config.magErrFloor will be added to magErr *only* (not srcMagErr or refMagErr), as
430  magErr is what is later used to determine the zero point.
431  Struct also contains refFluxFieldList: a list of field names of the reference catalog used for fluxes
432  (1 or 2 strings)
433  \note These magnitude arrays are the \em inputs to the photometric calibration, some may have been
434  discarded by clipping while estimating the calibration (https://jira.lsstcorp.org/browse/DM-813)
435  """
436  srcFluxArr = np.array([m.second.get(sourceKeys.flux) for m in matches])
437  srcFluxErrArr = np.array([m.second.get(sourceKeys.fluxErr) for m in matches])
438  if not np.all(np.isfinite(srcFluxErrArr)):
439  # this is an unpleasant hack; see DM-2308 requesting a better solution
440  self.log.warn("Source catalog does not have flux uncertainties; using sqrt(flux).")
441  srcFluxErrArr = np.sqrt(srcFluxArr)
442 
443  # convert source flux from DN to an estimate of Jy
444  JanskysPerABFlux = 3631.0
445  srcFluxArr = srcFluxArr * JanskysPerABFlux
446  srcFluxErrArr = srcFluxErrArr * JanskysPerABFlux
447 
448  if not matches:
449  raise RuntimeError("No reference stars are available")
450  refSchema = matches[0].first.schema
451 
452  applyColorTerms = self.config.applyColorTerms
453  applyCTReason = "config.applyColorTerms is %s" % (self.config.applyColorTerms,)
454  if self.config.applyColorTerms is None:
455  # apply color terms if color term data is available and photoCatName specified
456  ctDataAvail = len(self.config.colorterms.data) > 0
457  photoCatSpecified = self.config.photoCatName is not None
458  applyCTReason += " and data %s available" % ("is" if ctDataAvail else "is not")
459  applyCTReason += " and photoRefCat %s None" % ("is not" if photoCatSpecified else "is")
460  applyColorTerms = ctDataAvail and photoCatSpecified
461 
462  if applyColorTerms:
463  self.log.info("Applying color terms for filterName=%r, config.photoCatName=%s because %s" %
464  (filterName, self.config.photoCatName, applyCTReason))
465  ct = self.config.colorterms.getColorterm(
466  filterName=filterName, photoCatName=self.config.photoCatName, doRaise=True)
467  else:
468  self.log.info("Not applying color terms because %s" % (applyCTReason,))
469  ct = None
470 
471  if ct: # we have a color term to worry about
472  fluxFieldList = [getRefFluxField(refSchema, filt) for filt in (ct.primary, ct.secondary)]
473  missingFluxFieldList = []
474  for fluxField in fluxFieldList:
475  try:
476  refSchema.find(fluxField).key
477  except KeyError:
478  missingFluxFieldList.append(fluxField)
479 
480  if missingFluxFieldList:
481  self.log.warn("Source catalog does not have fluxes for %s; ignoring color terms" %
482  " ".join(missingFluxFieldList))
483  ct = None
484 
485  if not ct:
486  fluxFieldList = [getRefFluxField(refSchema, filterName)]
487 
488  refFluxArrList = [] # list of ref arrays, one per flux field
489  refFluxErrArrList = [] # list of ref flux arrays, one per flux field
490  for fluxField in fluxFieldList:
491  fluxKey = refSchema.find(fluxField).key
492  refFluxArr = np.array([m.first.get(fluxKey) for m in matches])
493  try:
494  fluxErrKey = refSchema.find(fluxField + "Sigma").key
495  refFluxErrArr = np.array([m.first.get(fluxErrKey) for m in matches])
496  except KeyError:
497  # Reference catalogue may not have flux uncertainties; HACK
498  self.log.warn("Reference catalog does not have flux uncertainties for %s; using sqrt(flux)."
499  % fluxField)
500  refFluxErrArr = np.sqrt(refFluxArr)
501 
502  refFluxArrList.append(refFluxArr)
503  refFluxErrArrList.append(refFluxErrArr)
504 
505  if ct: # we have a color term to worry about
506  refMagArr1 = np.array([abMagFromFlux(rf1) for rf1 in refFluxArrList[0]]) # primary
507  refMagArr2 = np.array([abMagFromFlux(rf2) for rf2 in refFluxArrList[1]]) # secondary
508 
509  refMagArr = ct.transformMags(refMagArr1, refMagArr2)
510  refFluxErrArr = ct.propagateFluxErrors(refFluxErrArrList[0], refFluxErrArrList[1])
511  else:
512  refMagArr = np.array([abMagFromFlux(rf) for rf in refFluxArrList[0]])
513 
514  srcMagArr = np.array([abMagFromFlux(sf) for sf in srcFluxArr])
515 
516  # Fitting with error bars in both axes is hard
517  # for now ignore reference flux error, but ticket DM-2308 is a request for a better solution
518  magErrArr = np.array([abMagErrFromFluxErr(fe, sf) for fe, sf in izip(srcFluxErrArr, srcFluxArr)])
519  if self.config.magErrFloor != 0.0:
520  magErrArr = (magErrArr**2 + self.config.magErrFloor**2)**0.5
521 
522  srcMagErrArr = np.array([abMagErrFromFluxErr(sfe, sf) for sfe, sf in izip(srcFluxErrArr, srcFluxArr)])
523  refMagErrArr = np.array([abMagErrFromFluxErr(rfe, rf) for rfe, rf in izip(refFluxErrArr, refFluxArr)])
524 
525  return pipeBase.Struct(
526  srcMag = srcMagArr,
527  refMag = refMagArr,
528  magErr = magErrArr,
529  srcMagErr = srcMagErrArr,
530  refMagErr = refMagErrArr,
531  refFluxFieldList = fluxFieldList,
532  )
lsst::afw::image::abMagErrFromFluxErr
double abMagErrFromFluxErr(double fluxErr, double flux)
Compute AB magnitude error from flux and flux error in Janskys.
Definition: Calib.h:64
lsst::afw::image::abMagFromFlux
double abMagFromFlux(double flux)
Compute AB magnitude from flux in Janskys.
Definition: Calib.h:59
lsst.pipe.tasks.photoCal.PhotoCalTask.extractMagArrays
def extractMagArrays
Extract magnitude and magnitude error arrays from the given matches.
Definition: photoCal.py:419
lsst::meas::algorithms.loadReferenceObjects.getRefFluxField
def getRefFluxField
Get name of flux field in schema.
Definition: loadReferenceObjects.py:14
def lsst.pipe.tasks.photoCal.PhotoCalTask.getSourceKeys (   self,
  schema 
)

Return a struct containing the source catalog keys for fields used by PhotoCalTask.

Returned fields include:

Definition at line 282 of file photoCal.py.

282 
283  def getSourceKeys(self, schema):
284  """!Return a struct containing the source catalog keys for fields used by PhotoCalTask.
285 
286  Returned fields include:
287  - flux
288  - fluxErr
289  - goodFlags: a list of keys for field names in self.config.goodFlags
290  - badFlags: a list of keys for field names in self.config.badFlags
291  """
292  goodFlags = [schema.find(name).key for name in self.config.goodFlags]
293  flux = schema.find(self.config.fluxField).key
294  fluxErr = schema.find(self.config.fluxField + "Sigma").key
295  badFlags = [schema.find(name).key for name in self.config.badFlags]
296 
297  return pipeBase.Struct(flux=flux, fluxErr=fluxErr, goodFlags=goodFlags, badFlags=badFlags)
lsst.pipe.tasks.photoCal.PhotoCalTask.getSourceKeys
def getSourceKeys
Return a struct containing the source catalog keys for fields used by PhotoCalTask.
Definition: photoCal.py:282
def lsst.pipe.tasks.photoCal.PhotoCalTask.getZeroPoint (   self,
  src,
  ref,
  srcErr = None,
  zp0 = None 
)

Flux calibration code, returning (ZeroPoint, Distribution Width, Number of stars)

We perform nIter iterations of a simple sigma-clipping algorithm with a couple of twists:

  1. We use the median/interquartile range to estimate the position to clip around, and the "sigma" to use.
  2. We never allow sigma to go above a critical value sigmaMax — if we do, a sufficiently large estimate will prevent the clipping from ever taking effect.
  3. Rather than start with the median we start with a crude mode. This means that a set of magnitude residuals with a tight core and asymmetrical outliers will start in the core. We use the width of this core to set our maximum sigma (see 2.)

Definition at line 640 of file photoCal.py.

640 
641  def getZeroPoint(self, src, ref, srcErr=None, zp0=None):
642  """!Flux calibration code, returning (ZeroPoint, Distribution Width, Number of stars)
643 
644  We perform nIter iterations of a simple sigma-clipping algorithm with a couple of twists:
645  1. We use the median/interquartile range to estimate the position to clip around, and the
646  "sigma" to use.
647  2. We never allow sigma to go _above_ a critical value sigmaMax --- if we do, a sufficiently
648  large estimate will prevent the clipping from ever taking effect.
649  3. Rather than start with the median we start with a crude mode. This means that a set of magnitude
650  residuals with a tight core and asymmetrical outliers will start in the core. We use the width of
651  this core to set our maximum sigma (see 2.)
652  """
653  sigmaMax = self.config.sigmaMax
654 
655  dmag = ref - src
656 
657  indArr = np.argsort(dmag)
658  dmag = dmag[indArr]
659 
660  if srcErr is not None:
661  dmagErr = srcErr[indArr]
662  else:
663  dmagErr = np.ones(len(dmag))
664 
665  # need to remove nan elements to avoid errors in stats calculation with numpy
666  ind_noNan = np.array([i for i in range(len(dmag))
667  if (not np.isnan(dmag[i]) and not np.isnan(dmagErr[i])) ])
668  dmag = dmag[ind_noNan]
669  dmagErr = dmagErr[ind_noNan]
670 
671  IQ_TO_STDEV = 0.741301109252802; # 1 sigma in units of interquartile (assume Gaussian)
672 
673  npt = len(dmag)
674  ngood = npt
675  good = None # set at end of first iteration
676  for i in range(self.config.nIter):
677  if i > 0:
678  npt = sum(good)
679 
680  center = None
681  if i == 0:
682  #
683  # Start by finding the mode
684  #
685  nhist = 20
686  try:
687  hist, edges = np.histogram(dmag, nhist, new=True)
688  except TypeError:
689  hist, edges = np.histogram(dmag, nhist) # they removed new=True around numpy 1.5
690  imode = np.arange(nhist)[np.where(hist == hist.max())]
691 
692  if imode[-1] - imode[0] + 1 == len(imode): # Multiple modes, but all contiguous
693  if zp0:
694  center = zp0
695  else:
696  center = 0.5*(edges[imode[0]] + edges[imode[-1] + 1])
697 
698  peak = sum(hist[imode])/len(imode) # peak height
699 
700  # Estimate FWHM of mode
701  j = imode[0]
702  while j >= 0 and hist[j] > 0.5*peak:
703  j -= 1
704  j = max(j, 0)
705  q1 = dmag[sum(hist[range(j)])]
706 
707  j = imode[-1]
708  while j < nhist and hist[j] > 0.5*peak:
709  j += 1
710  j = min(j, nhist - 1)
711  j = min(sum(hist[range(j)]), npt - 1)
712  q3 = dmag[j]
713 
714  if q1 == q3:
715  q1 = dmag[int(0.25*npt)]
716  q3 = dmag[int(0.75*npt)]
717 
718  sig = (q3 - q1)/2.3 # estimate of standard deviation (based on FWHM; 2.358 for Gaussian)
719 
720  if sigmaMax is None:
721  sigmaMax = 2*sig # upper bound on st. dev. for clipping. multiplier is a heuristic
722 
723  self.log.logdebug("Photo calibration histogram: center = %.2f, sig = %.2f"
724  % (center, sig))
725 
726  else:
727  if sigmaMax is None:
728  sigmaMax = dmag[-1] - dmag[0]
729 
730  center = np.median(dmag)
731  q1 = dmag[int(0.25*npt)]
732  q3 = dmag[int(0.75*npt)]
733  sig = (q3 - q1)/2.3 # estimate of standard deviation (based on FWHM; 2.358 for Gaussian)
734 
735  if center is None: # usually equivalent to (i > 0)
736  gdmag = dmag[good]
737  if self.config.useMedian:
738  center = np.median(gdmag)
739  else:
740  gdmagErr = dmagErr[good]
741  center = np.average(gdmag, weights=gdmagErr)
742 
743  q3 = gdmag[min(int(0.75*npt + 0.5), npt - 1)]
744  q1 = gdmag[min(int(0.25*npt + 0.5), npt - 1)]
745 
746  sig = IQ_TO_STDEV*(q3 - q1) # estimate of standard deviation
747 
748  good = abs(dmag - center) < self.config.nSigma*min(sig, sigmaMax) # don't clip too softly
749 
750  #-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
751  if self.scatterPlot:
752  try:
753  self.fig.clf()
754 
755  axes = self.fig.add_axes((0.1, 0.1, 0.85, 0.80));
756 
757  axes.plot(ref[good], dmag[good] - center, "b+")
758  axes.errorbar(ref[good], dmag[good] - center, yerr=dmagErr[good],
759  linestyle='', color='b')
760 
761  bad = np.logical_not(good)
762  if len(ref[bad]) > 0:
763  axes.plot(ref[bad], dmag[bad] - center, "r+")
764  axes.errorbar(ref[bad], dmag[bad] - center, yerr=dmagErr[bad],
765  linestyle='', color='r')
766 
767  axes.plot((-100, 100), (0, 0), "g-")
768  for x in (-1, 1):
769  axes.plot((-100, 100), x*0.05*np.ones(2), "g--")
770 
771  axes.set_ylim(-1.1, 1.1)
772  axes.set_xlim(24, 13)
773  axes.set_xlabel("Reference")
774  axes.set_ylabel("Reference - Instrumental")
775 
776  self.fig.show()
777 
778  if self.scatterPlot > 1:
779  reply = None
780  while i == 0 or reply != "c":
781  try:
782  reply = raw_input("Next iteration? [ynhpc] ")
783  except EOFError:
784  reply = "n"
785 
786  if reply == "h":
787  print >> sys.stderr, "Options: c[ontinue] h[elp] n[o] p[db] y[es]"
788  continue
789 
790  if reply in ("", "c", "n", "p", "y"):
791  break
792  else:
793  print >> sys.stderr, "Unrecognised response: %s" % reply
794 
795  if reply == "n":
796  break
797  elif reply == "p":
798  import pdb; pdb.set_trace()
799  except Exception, e:
800  print >> sys.stderr, "Error plotting in PhotoCal.getZeroPoint: %s" % e
801 
802  #-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
803 
804  old_ngood = ngood
805  ngood = sum(good)
806  if ngood == 0:
807  msg = "PhotoCal.getZeroPoint: no good stars remain"
808 
809  if i == 0: # failed the first time round -- probably all fell in one bin
810  center = np.average(dmag, weights=dmagErr)
811  msg += " on first iteration; using average of all calibration stars"
812 
813  self.log.log(self.log.WARN, msg)
814 
815  return pipeBase.Struct(
816  zp = center,
817  sigma = sig,
818  ngood = len(dmag)
819  )
820  elif ngood == old_ngood:
821  break
822 
823  if False:
824  ref = ref[good]
825  dmag = dmag[good]
826  dmagErr = dmagErr[good]
827 
828  dmag = dmag[good]
829  dmagErr = dmagErr[good]
830  zp, weightSum = np.average(dmag, weights=1/dmagErr**2, returned=True)
831  sigma = np.sqrt(1.0/weightSum)
832  return pipeBase.Struct(
833  zp = zp,
834  sigma = sigma,
835  ngood = len(dmag),
836  )
ndarray::sum
boost::enable_if< typename ExpressionTraits< Scalar >::IsScalar, Scalar >::type sum(Scalar const &scalar)
Definition: operators.h:1250
lsst.pipe.tasks.photoCal.PhotoCalTask.getZeroPoint
def getZeroPoint
Flux calibration code, returning (ZeroPoint, Distribution Width, Number of stars) ...
Definition: photoCal.py:640
def lsst.pipe.tasks.photoCal.PhotoCalTask.run (   self,
  exposure,
  matches 
)

Do photometric calibration - select matches to use and (possibly iteratively) compute the zero point.

Parameters
[in]exposureExposure upon which the sources in the matches were detected.
[in]matchesInput lsst.afw.table.ReferenceMatchVector (i.e. a list of lsst.afw.table.Match with first being of type lsst.afw.table.SimpleRecord and second type lsst.afw.table.SourceRecord — the reference object and matched object respectively). (will not be modified except to set the outputField if requested.).
Returns
Struct of:
  • calib ----— lsst::afw::image::Calib object containing the zero point
  • arrays ---— Magnitude arrays returned be PhotoCalTask.extractMagArrays
  • matches --— Final ReferenceMatchVector, as returned by PhotoCalTask.selectMatches.

The exposure is only used to provide the name of the filter being calibrated (it may also be used to generate debugging plots).

The reference objects:

The measured sources:

Exceptions
RuntimeErrorwith the following strings:
`sources' schema does not contain the calibration object flag "XXX"`
The constructor added fields to the schema that aren't in the Sources
No input matches
The input match vector is empty
All matches eliminated by source flags
The flags specified by badFlags in the config eliminated all candidate objects
No sources remain in match list after reference catalog cuts
The reference catalogue has a column "photometric", but no matched objects have it set
No sources remaining in match list after magnitude limit cuts
All surviving matches are either too faint in the catalogue or have negative or NaN flux
No reference stars are available
No matches survive all the checks

Definition at line 534 of file photoCal.py.

534 
535  def run(self, exposure, matches):
536  """!Do photometric calibration - select matches to use and (possibly iteratively) compute
537  the zero point.
538 
539  \param[in] exposure Exposure upon which the sources in the matches were detected.
540  \param[in] matches Input lsst.afw.table.ReferenceMatchVector
541  (\em i.e. a list of lsst.afw.table.Match with
542  \c first being of type lsst.afw.table.SimpleRecord and \c second type lsst.afw.table.SourceRecord ---
543  the reference object and matched object respectively).
544  (will not be modified except to set the outputField if requested.).
545 
546  \return Struct of:
547  - calib ------- \link lsst::afw::image::Calib\endlink object containing the zero point
548  - arrays ------ Magnitude arrays returned be PhotoCalTask.extractMagArrays
549  - matches ----- Final ReferenceMatchVector, as returned by PhotoCalTask.selectMatches.
550 
551  The exposure is only used to provide the name of the filter being calibrated (it may also be
552  used to generate debugging plots).
553 
554  The reference objects:
555  - Must include a field \c photometric; True for objects which should be considered as
556  photometric standards
557  - Must include a field \c flux; the flux used to impose a magnitude limit and also to calibrate
558  the data to (unless a color term is specified, in which case ColorTerm.primary is used;
559  See https://jira.lsstcorp.org/browse/DM-933)
560  - May include a field \c stargal; if present, True means that the object is a star
561  - May include a field \c var; if present, True means that the object is variable
562 
563  The measured sources:
564  - Must include PhotoCalConfig.fluxField; the flux measurement to be used for calibration
565 
566  \throws RuntimeError with the following strings:
567 
568  <DL>
569  <DT> `sources' schema does not contain the calibration object flag "XXX"`
570  <DD> The constructor added fields to the schema that aren't in the Sources
571  <DT> No input matches
572  <DD> The input match vector is empty
573  <DT> All matches eliminated by source flags
574  <DD> The flags specified by \c badFlags in the config eliminated all candidate objects
575  <DT> No sources remain in match list after reference catalog cuts
576  <DD> The reference catalogue has a column "photometric", but no matched objects have it set
577  <DT> No sources remaining in match list after magnitude limit cuts
578  <DD> All surviving matches are either too faint in the catalogue or have negative or \c NaN flux
579  <DT> No reference stars are available
580  <DD> No matches survive all the checks
581  </DL>
582  """
583  import lsstDebug
584 
585  display = lsstDebug.Info(__name__).display
586  displaySources = display and lsstDebug.Info(__name__).displaySources
587  self.scatterPlot = display and lsstDebug.Info(__name__).scatterPlot
588 
589  if self.scatterPlot:
590  from matplotlib import pyplot
591  try:
592  self.fig.clf()
593  except:
594  self.fig = pyplot.figure()
595 
596  if displaySources:
597  frame = 1
598  ds9.mtv(exposure, frame=frame, title="photocal")
599  else:
600  frame = None
601 
602  filterName = exposure.getFilter().getName()
603  sourceKeys = self.getSourceKeys(matches[0].second.schema)
604  matches = self.selectMatches(matches=matches, sourceKeys=sourceKeys, filterName=filterName,
605  frame=frame)
606  arrays = self.extractMagArrays(matches=matches, filterName=filterName, sourceKeys=sourceKeys)
607 
608  if matches and self.outputField:
609  try:
610  # matches[].second is a measured source, wherein we wish to set outputField.
611  # Check that the field is present in the Sources schema.
612  matches[0].second.getSchema().find(self.outputField)
613  except:
614  raise RuntimeError("sources' schema does not contain the used-in-calibration flag \"%s\"" %
615  self.config.outputField)
616 
617  # Fit for zeropoint. We can run the code more than once, so as to
618  # give good stars that got clipped by a bad first guess a second
619  # chance.
620 
621  calib = Calib()
622  zp = None # initial guess
623  r = self.getZeroPoint(arrays.srcMag, arrays.refMag, arrays.magErr, zp0=zp)
624  zp = r.zp
625  self.log.info("Magnitude zero point: %f +/- %f from %d stars" % (r.zp, r.sigma, r.ngood))
626 
627  flux0 = 10**(0.4*r.zp) # Flux of mag=0 star
628  flux0err = 0.4*math.log(10)*flux0*r.sigma # Error in flux0
629 
630  calib.setFluxMag0(flux0, flux0err)
631 
632  return pipeBase.Struct(
633  calib = calib,
634  arrays = arrays,
635  matches = matches,
636  zp = r.zp,
637  sigma = r.sigma,
638  ngood = r.ngood,
639  )
lsst.pipe.tasks.photoCal.PhotoCalTask.run
def run
Do photometric calibration - select matches to use and (possibly iteratively) compute the zero point...
Definition: photoCal.py:534
lsst.pipe.tasks.photoCal.PhotoCalTask.extractMagArrays
def extractMagArrays
Extract magnitude and magnitude error arrays from the given matches.
Definition: photoCal.py:419
lsst.pipe.tasks.photoCal.PhotoCalTask.getZeroPoint
def getZeroPoint
Flux calibration code, returning (ZeroPoint, Distribution Width, Number of stars) ...
Definition: photoCal.py:640
lsst.pipe.tasks.photoCal.PhotoCalTask.getSourceKeys
def getSourceKeys
Return a struct containing the source catalog keys for fields used by PhotoCalTask.
Definition: photoCal.py:282
lsst.pipe.tasks.photoCal.PhotoCalTask.selectMatches
def selectMatches
Select reference/source matches according the criteria specified in the config.
Definition: photoCal.py:299
def lsst.pipe.tasks.photoCal.PhotoCalTask.selectMatches (   self,
  matches,
  sourceKeys,
  filterName,
  frame = None 
)

Select reference/source matches according the criteria specified in the config.

Parameters
[in]matchesReferenceMatchVector (not modified)
[in]sourceKeysStruct of source catalog keys, as returned by getSourceKeys()
[in]filterNamename of camera filter; used to obtain the reference flux field
[in]frameds9 frame number to use for debugging display if frame is non-None, display information about trimmed objects on that ds9 frame:
  • Bad: red x
  • Unsuitable objects: blue + (and a cyan o if a galaxy)
  • Failed flux cut: magenta *
Returns
a lsst.afw.table.ReferenceMatchVector that contains only the selected matches. If a schema was passed during task construction, a flag field will be set on sources in the selected matches.
Exceptions
ValueErrorThere are no valid matches.

Definition at line 299 of file photoCal.py.

299 
300  def selectMatches(self, matches, sourceKeys, filterName, frame=None):
301  """!Select reference/source matches according the criteria specified in the config.
302 
303  \param[in] matches ReferenceMatchVector (not modified)
304  \param[in] sourceKeys Struct of source catalog keys, as returned by getSourceKeys()
305  \param[in] filterName name of camera filter; used to obtain the reference flux field
306  \param[in] frame ds9 frame number to use for debugging display
307  if frame is non-None, display information about trimmed objects on that ds9 frame:
308  - Bad: red x
309  - Unsuitable objects: blue + (and a cyan o if a galaxy)
310  - Failed flux cut: magenta *
311 
312  \return a \link lsst.afw.table.ReferenceMatchVector\endlink that contains only the selected matches.
313  If a schema was passed during task construction, a flag field will be set on sources
314  in the selected matches.
315 
316  \throws ValueError There are no valid matches.
317  """
318 
319  self.log.logdebug("Number of input matches: %d" % (len(matches)))
320  if len(matches) == 0:
321  raise ValueError("No input matches")
322 
323  # Only use stars for which the flags indicate the photometry is good.
324  afterFlagCutInd = [i for i, m in enumerate(matches) if checkSourceFlags(m.second, sourceKeys)]
325  afterFlagCut = [matches[i] for i in afterFlagCutInd]
326  self.log.logdebug("Number of matches after source flag cuts: %d" % (len(afterFlagCut)))
327 
328  if len(afterFlagCut) != len(matches):
329  if frame is not None:
330  with ds9.Buffering():
331  for i, m in enumerate(matches):
332  if i not in afterFlagCutInd:
333  x, y = m.second.getCentroid()
334  ds9.dot("x", x, y, size=4, frame=frame, ctype=ds9.RED)
335 
336  matches = afterFlagCut
337 
338  if len(matches) == 0:
339  raise ValueError("All matches eliminated by source flags")
340 
341  refSchema = matches[0].first.schema
342  try:
343  photometricKey = refSchema.find("photometric").key
344  try:
345  resolvedKey = refSchema.find("resolved").key
346  except:
347  resolvedKey = None
348 
349  try:
350  variableKey = refSchema.find("variable").key
351  except:
352  variableKey = None
353  except:
354  self.log.warn("No 'photometric' flag key found in reference schema.")
355  photometricKey = None
356 
357  if photometricKey is not None:
358  afterRefCutInd = [i for i, m in enumerate(matches) if m.first.get(photometricKey)]
359  afterRefCut = [matches[i] for i in afterRefCutInd]
360 
361  if len(afterRefCut) != len(matches):
362  if frame is not None:
363  with ds9.Buffering():
364  for i, m in enumerate(matches):
365  if i not in afterRefCutInd:
366  x, y = m.second.getCentroid()
367  ds9.dot("+", x, y, size=4, frame=frame, ctype=ds9.BLUE)
368 
369  if resolvedKey and m.first.get(resolvedKey):
370  ds9.dot("o", x, y, size=6, frame=frame, ctype=ds9.CYAN)
371  if variableKey and m.first.get(variableKey):
372  ds9.dot("o", x, y, size=6, frame=frame, ctype=ds9.MAGENTA)
373 
374  matches = afterRefCut
375 
376  self.log.logdebug("Number of matches after reference catalog cuts: %d" % (len(matches)))
377  if len(matches) == 0:
378  raise RuntimeError("No sources remain in match list after reference catalog cuts.")
379  fluxName = getRefFluxField(refSchema, filterName)
380  fluxKey = refSchema.find(fluxName).key
381  if self.config.magLimit is not None:
382  fluxLimit = fluxFromABMag(self.config.magLimit)
383 
384  afterMagCutInd = [i for i, m in enumerate(matches) if (m.first.get(fluxKey) > fluxLimit
385  and m.second.getPsfFlux() > 0.0)]
386  else:
387  afterMagCutInd = [i for i, m in enumerate(matches) if m.second.getPsfFlux() > 0.0]
388 
389  afterMagCut = [matches[i] for i in afterMagCutInd]
390 
391  if len(afterMagCut) != len(matches):
392  if frame is not None:
393  with ds9.Buffering():
394  for i, m in enumerate(matches):
395  if i not in afterMagCutInd:
396  x, y = m.second.getCentroid()
397  ds9.dot("*", x, y, size=4, frame=frame, ctype=ds9.MAGENTA)
398 
399  matches = afterMagCut
400 
401  self.log.logdebug("Number of matches after magnitude limit cuts: %d" % (len(matches)))
402 
403  if len(matches) == 0:
404  raise RuntimeError("No sources remaining in match list after magnitude limit cuts.")
405 
406  if frame is not None:
407  with ds9.Buffering():
408  for m in matches:
409  x, y = m.second.getCentroid()
410  ds9.dot("o", x, y, size=4, frame=frame, ctype=ds9.GREEN)
411 
412  result = afwTable.ReferenceMatchVector()
413  for m in matches:
414  if self.outputField is not None:
415  m.second.set(self.outputField, True)
416  result.append(m)
417  return result
lsst::afw::table::ReferenceMatchVector
std::vector< ReferenceMatch > ReferenceMatchVector
Definition: fwd.h:97
lsst.pipe.tasks.photoCal.checkSourceFlags
def checkSourceFlags
Return True if the given source has all good flags set and none of the bad flags set.
Definition: photoCal.py:40
lsst::afw::image::fluxFromABMag
double fluxFromABMag(double mag)
Compute flux in Janskys from AB magnitude.
Definition: Calib.h:69
lsst::meas::algorithms.loadReferenceObjects.getRefFluxField
def getRefFluxField
Get name of flux field in schema.
Definition: loadReferenceObjects.py:14
lsst.pipe.tasks.photoCal.PhotoCalTask.selectMatches
def selectMatches
Select reference/source matches according the criteria specified in the config.
Definition: photoCal.py:299

Member Data Documentation

string lsst.pipe.tasks.photoCal.PhotoCalTask._DefaultName = "photoCal"
staticprivate

Definition at line 268 of file photoCal.py.

lsst.pipe.tasks.photoCal.PhotoCalTask.ConfigClass = PhotoCalConfig
static

Definition at line 267 of file photoCal.py.

lsst.pipe.tasks.photoCal.PhotoCalTask.fig

Definition at line 275 of file photoCal.py.

lsst.pipe.tasks.photoCal.PhotoCalTask.outputField

Definition at line 277 of file photoCal.py.

lsst.pipe.tasks.photoCal.PhotoCalTask.scatterPlot

Definition at line 274 of file photoCal.py.

The documentation for this class was generated from the following file:
Generated on Thu Sep 24 2015 02:29:43 for LSSTApplications by   doxygen 1.8.5