LSSTApplications  10.0+286,10.0+36,10.0+46,10.0-2-g4f67435,10.1+152,10.1+37,11.0,11.0+1,11.0-1-g47edd16,11.0-1-g60db491,11.0-1-g7418c06,11.0-2-g04d2804,11.0-2-g68503cd,11.0-2-g818369d,11.0-2-gb8b8ce7
LSSTDataManagementBasePackage
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.pipe.tasks.processImage.ProcessImageTask Class Reference
Inheritance diagram for lsst.pipe.tasks.processImage.ProcessImageTask:

Public Member Functions

def __init__
 
def makeIdFactory
 
def getExposureId
 
def process
 
def matchSources
 
def propagateCalibFlags
 
def getSchemaCatalogs
 
def writeBackgrounds
 
def restoreBackgrounds
 

Public Attributes

 schemaMapper
 
 calibSourceKey
 
 schema
 
 algMetadata
 

Static Public Attributes

 ConfigClass = ProcessImageConfig
 
string dataPrefix = ""
 

Detailed Description

An abstract base class for tasks do simple calibration, detection, deblending, and measurement
on individual images.

Other command-line Process***Tasks (such as ProcessCcdTask and ProcessCoaddTask) rely on
ProcessImageTask for their main algorithmic code, and only need to add pre- and post- processing
and serialization.

Subclasses are responsible for meeting the requirements of CmdLineTask.

Definition at line 91 of file processImage.py.

Constructor & Destructor Documentation

def lsst.pipe.tasks.processImage.ProcessImageTask.__init__ (   self,
  kwargs 
)

Definition at line 104 of file processImage.py.

104 
105  def __init__(self, **kwargs):
106  pipeBase.CmdLineTask.__init__(self, **kwargs)
107 
108  self.makeSubtask("calibrate")
109 
110  # Setup our schema by starting with fields we want to propagate from icSrc.
111  calibSchema = self.calibrate.schema
112  self.schemaMapper = afwTable.SchemaMapper(calibSchema)
113  self.schemaMapper.addMinimalSchema(afwTable.SourceTable.makeMinimalSchema(), False)
114 
115  # Add fields needed to identify stars used in the calibration step
116  self.calibSourceKey = self.schemaMapper.addOutputField(
117  afwTable.Field["Flag"]("calib_detected", "Source was detected as an icSrc")
118  )
119 
120  for key in self.calibrate.getCalibKeys():
121  self.schemaMapper.addMapping(key)
122  self.schema = self.schemaMapper.getOutputSchema()
123  self.algMetadata = dafBase.PropertyList()
124  if self.config.doDetection:
125  self.makeSubtask("detection", schema=self.schema)
126  if self.config.doDeblend:
127  self.makeSubtask("deblend", schema=self.schema)
128  if self.config.doMeasurement:
129  self.makeSubtask("measurement", schema=self.schema, algMetadata=self.algMetadata)
lsst::daf::base::PropertyList
Class for storing ordered metadata with comments.
Definition: PropertyList.h:81
lsst::afw::table::SchemaMapper
A mapping between the keys of two Schemas, used to copy data between them.
Definition: SchemaMapper.h:19
lsst::afw::table::Field
A description of a field in a table.
Definition: Field.h:22

Member Function Documentation

def lsst.pipe.tasks.processImage.ProcessImageTask.getExposureId (   self,
  sensorRef 
)

Definition at line 133 of file processImage.py.

133 
134  def getExposureId(self, sensorRef):
135  raise NotImplementedError()
def lsst.pipe.tasks.processImage.ProcessImageTask.getSchemaCatalogs (   self) 
Return a dict of empty catalogs for each catalog dataset produced by this task.

Definition at line 292 of file processImage.py.

292 
293  def getSchemaCatalogs(self):
294  """Return a dict of empty catalogs for each catalog dataset produced by this task."""
295  src = afwTable.SourceCatalog(self.schema)
296  src.getTable().setMetadata(self.algMetadata)
297  d = {self.dataPrefix + "src": src}
298  icSrc = None
299  try:
300  icSrc = afwTable.SourceCatalog(self.calibrate.schema)
301  icSrc.getTable().setMetadata(self.calibrate.algMetadata)
302  except AttributeError:
303  pass
304  if icSrc is not None:
305  d[self.dataPrefix + "icSrc"] = icSrc
306  return d
lsst::afw::table::SortedCatalogT
Custom catalog class for record/table subclasses that are guaranteed to have an ID, and should generally be sorted by that ID.
Definition: fwd.h:55
def lsst.pipe.tasks.processImage.ProcessImageTask.makeIdFactory (   self,
  sensorRef 
)

Definition at line 130 of file processImage.py.

130 
131  def makeIdFactory(self, sensorRef):
132  raise NotImplementedError()
def lsst.pipe.tasks.processImage.ProcessImageTask.matchSources (   self,
  exposure,
  sources 
) 
Match the sources to the reference object loaded by the calibrate task

Return two items:
- matches  list of reference object/source matches (an lsst.afw.table.ReferenceMatchVector)
- matchMeta  metadata about the field (an lsst.daf.base.PropertyList)

Definition at line 237 of file processImage.py.

237 
238  def matchSources(self, exposure, sources):
239  """Match the sources to the reference object loaded by the calibrate task
240 
241  Return two items:
242  - matches list of reference object/source matches (an lsst.afw.table.ReferenceMatchVector)
243  - matchMeta metadata about the field (an lsst.daf.base.PropertyList)
244  """
245  try:
246  astrometry = self.calibrate.astrometry
247  except Exception:
248  self.log.warn("Failed to find an astrometry solver in the calibrate task")
249  return None, None
250 
251  astromRet = astrometry.loadAndMatch(exposure=exposure, sourceCat=sources)
252  return astromRet.matches, astromRet.matchMeta
def lsst.pipe.tasks.processImage.ProcessImageTask.process (   self,
  dataRef,
  inputExposure,
  enableWriteSources = True 
) 
Process an Image

@param dataRef: data reference that corresponds to the input image
@param inputExposure:  exposure to process
@param enableWriteSources: if True then writing sources is allowed.
    Set False if you need to postprocess sources before writing them.

@return pipe_base Struct containing these fields:
- postIsrExposure: exposure after ISR performed if calib.doIsr or config.doCalibrate, else None
- exposure: calibrated exposure (calexp): as computed if config.doCalibrate,
    else as upersisted and updated if config.doDetection, else None
- calib: object returned by calibration process if config.doCalibrate, else None
- sources: detected source if config.doPhotometry, else None

Definition at line 137 of file processImage.py.

137 
138  def process(self, dataRef, inputExposure, enableWriteSources=True):
139  """Process an Image
140 
141  @param dataRef: data reference that corresponds to the input image
142  @param inputExposure: exposure to process
143  @param enableWriteSources: if True then writing sources is allowed.
144  Set False if you need to postprocess sources before writing them.
145 
146  @return pipe_base Struct containing these fields:
147  - postIsrExposure: exposure after ISR performed if calib.doIsr or config.doCalibrate, else None
148  - exposure: calibrated exposure (calexp): as computed if config.doCalibrate,
149  else as upersisted and updated if config.doDetection, else None
150  - calib: object returned by calibration process if config.doCalibrate, else None
151  - sources: detected source if config.doPhotometry, else None
152  """
153  idFactory = self.makeIdFactory(dataRef)
154 
155  # initialize outputs
156  calExposure = inputExposure
157  calib = None
158  sources = None
159  backgrounds = afwMath.BackgroundList()
160  if self.config.doCalibrate:
161  calib = self.calibrate.run(inputExposure, idFactory=idFactory)
162  calExposure = calib.exposure
163  if self.config.doWriteCalibrate:
164  dataRef.put(calib.sources, self.dataPrefix + "icSrc")
165  if calib.matches is not None and self.config.doWriteCalibrateMatches:
166  normalizedMatches = afwTable.packMatches(calib.matches)
167  normalizedMatches.table.setMetadata(calib.matchMeta)
168  dataRef.put(normalizedMatches, self.dataPrefix + "icMatch")
169  try:
170  for bg in calib.backgrounds:
171  backgrounds.append(bg)
172  except TypeError:
173  backgrounds.append(calib.backgrounds)
174  except AttributeError:
175  self.log.warn("The calibration task did not return any backgrounds. " +
176  "Any background subtracted in the calibration process cannot be persisted.")
177  else:
178  calib = None
179 
180  if self.config.doDetection:
181  table = afwTable.SourceTable.make(self.schema, idFactory)
182  table.setMetadata(self.algMetadata)
183  detections = self.detection.run(table, calExposure)
184  sources = detections.sources
185  fpSets = detections.fpSets
186  if fpSets.background:
187  backgrounds.append(fpSets.background)
188 
189  if self.config.doDeblend:
190  self.deblend.run(calExposure, sources, calExposure.getPsf())
191 
192  if self.config.doMeasurement:
193  self.measurement.run(calExposure, sources, exposureId=self.getExposureId(dataRef))
194 
195  if self.config.doWriteCalibrate:
196  # wait until after detection and measurement, since detection sets detected mask bits
197  # and both require a background subtracted exposure;
198  # note that this overwrites an existing calexp if doCalibrate false
199 
200  if calExposure is None:
201  self.log.warn("calibrated exposure is None; cannot save it")
202  else:
203  if self.config.persistBackgroundModel:
204  self.writeBackgrounds(dataRef, backgrounds)
205  else:
206  self.restoreBackgrounds(calExposure, backgrounds)
207  dataRef.put(calExposure, self.dataPrefix + "calexp")
208 
209  if calib is not None:
210  self.propagateCalibFlags(calib.sources, sources)
211 
212  if sources is not None and self.config.doWriteSources:
213  sourceWriteFlags = (0 if self.config.doWriteHeavyFootprintsInSources
214  else afwTable.SOURCE_IO_NO_HEAVY_FOOTPRINTS)
215  if enableWriteSources:
216  dataRef.put(sources, self.dataPrefix + 'src', flags=sourceWriteFlags)
217 
218  if self.config.doMeasurement and self.config.doWriteSourceMatches:
219  self.log.info("Matching src to reference catalogue" % (dataRef.dataId))
220  srcMatches, srcMatchMeta = self.matchSources(calExposure, sources)
221 
222  normalizedSrcMatches = afwTable.packMatches(srcMatches)
223  normalizedSrcMatches.table.setMetadata(srcMatchMeta)
224  dataRef.put(normalizedSrcMatches, self.dataPrefix + "srcMatch")
225  else:
226  srcMatches = None; srcMatchMeta = None
227 
228  return pipeBase.Struct(
229  inputExposure = inputExposure,
230  exposure = calExposure,
231  calib = calib,
232  sources = sources,
233  matches = srcMatches,
234  matchMeta = srcMatchMeta,
235  backgrounds = backgrounds,
236  )
lsst::afw::table::packMatches
BaseCatalog packMatches(std::vector< Match< Record1, Record2 > > const &matches)
Return a table representation of a MatchVector that can be used to persist it.
def lsst.pipe.tasks.processImage.ProcessImageTask.propagateCalibFlags (   self,
  icSources,
  sources,
  matchRadius = 1 
) 
Match the icSources and sources, and propagate Interesting Flags (e.g. PSF star) to the sources

Definition at line 253 of file processImage.py.

253 
254  def propagateCalibFlags(self, icSources, sources, matchRadius=1):
255  """Match the icSources and sources, and propagate Interesting Flags (e.g. PSF star) to the sources
256  """
257  self.log.info("Matching icSource and Source catalogs to propagate flags.")
258  if icSources is None or sources is None:
259  return
260 
261  closest = False # return all matched objects
262  matched = afwTable.matchRaDec(icSources, sources, matchRadius*afwGeom.arcseconds, closest)
263  if self.config.doDeblend:
264  matched = [m for m in matched if m[1].get("deblend_nChild") == 0] # if deblended, keep children
265  #
266  # Because we had to allow multiple matches to handle parents, we now need to
267  # prune to the best matches
268  #
269  bestMatches = {}
270  for m0, m1, d in matched:
271  id0 = m0.getId()
272  if bestMatches.has_key(id0):
273  if d > bestMatches[id0][2]:
274  continue
275 
276  bestMatches[id0] = (m0, m1, d)
277 
278  matched = bestMatches.values()
279  #
280  # Check that we got it right
281  #
282  if len(set(m[0].getId() for m in matched)) != len(matched):
283  self.log.warn("At least one icSource is matched to more than one Source")
284  #
285  # Copy over the desired flags
286  #
287  for ics, s, d in matched:
288  s.setFlag(self.calibSourceKey, True)
289  s.assign(ics, self.schemaMapper)
290 
291  return
lsst::afw::table::matchRaDec
std::vector< Match< typename Cat::Record, typename Cat::Record > > matchRaDec(Cat const &cat, Angle radius, bool symmetric=true)
def lsst.pipe.tasks.processImage.ProcessImageTask.restoreBackgrounds (   self,
  exp,
  backgrounds 
) 
Add backgrounds back in to an exposure

@param exp: Exposure to which to add backgrounds
@param backgrounds: List of background models

Definition at line 317 of file processImage.py.

317 
318  def restoreBackgrounds(self, exp, backgrounds):
319  """Add backgrounds back in to an exposure
320 
321  @param exp: Exposure to which to add backgrounds
322  @param backgrounds: List of background models
323  """
324  mi = exp.getMaskedImage()
325  mi += backgrounds.getImage()
def lsst.pipe.tasks.processImage.ProcessImageTask.writeBackgrounds (   self,
  dataRef,
  backgrounds 
) 
Backgrounds are persisted via the butler

@param dataRef: Data reference
@param backgrounds: List of background models

Definition at line 307 of file processImage.py.

307 
308  def writeBackgrounds(self, dataRef, backgrounds):
309  """Backgrounds are persisted via the butler
310 
311  @param dataRef: Data reference
312  @param backgrounds: List of background models
313  """
314  self.log.warn("Persisting background models")
315 
316  dataRef.put(backgrounds, self.dataPrefix+"calexpBackground")

Member Data Documentation

lsst.pipe.tasks.processImage.ProcessImageTask.algMetadata

Definition at line 122 of file processImage.py.

lsst.pipe.tasks.processImage.ProcessImageTask.calibSourceKey

Definition at line 115 of file processImage.py.

lsst.pipe.tasks.processImage.ProcessImageTask.ConfigClass = ProcessImageConfig
static

Definition at line 101 of file processImage.py.

string lsst.pipe.tasks.processImage.ProcessImageTask.dataPrefix = ""
static

Definition at line 102 of file processImage.py.

lsst.pipe.tasks.processImage.ProcessImageTask.schema

Definition at line 121 of file processImage.py.

lsst.pipe.tasks.processImage.ProcessImageTask.schemaMapper

Definition at line 111 of file processImage.py.

The documentation for this class was generated from the following file:
Generated on Thu Sep 24 2015 02:29:43 for LSSTApplications by   doxygen 1.8.5