LSSTApplications  19.0.0-10-g920eed2,19.0.0-11-g48a0200+2,19.0.0-18-gfc4e62b+10,19.0.0-2-g3b2f90d+2,19.0.0-2-gd671419+5,19.0.0-20-g5a5a17ab+8,19.0.0-21-g2644856+10,19.0.0-22-gc5dc5b1+5,19.0.0-23-gdc29a50+2,19.0.0-24-g923e380+10,19.0.0-25-g6c8df7140,19.0.0-27-g567f04d+6,19.0.0-3-g2b32d65+5,19.0.0-3-g8227491+9,19.0.0-3-g9c54d0d+9,19.0.0-3-gca68e65+5,19.0.0-3-gcfc5f51+5,19.0.0-3-ge110943+8,19.0.0-3-ge74d124,19.0.0-3-gfe04aa6+10,19.0.0-4-g06f5963+5,19.0.0-4-g3d16501+10,19.0.0-4-g4a9c019+5,19.0.0-4-g5a8b323,19.0.0-4-g66397f0+1,19.0.0-4-g8278b9b+1,19.0.0-4-g8557e14,19.0.0-4-g8964aba+10,19.0.0-4-ge404a01+9,19.0.0-5-g40f3a5a,19.0.0-5-g4db63b3,19.0.0-5-gfb03ce7+10,19.0.0-6-gbaebbfb+9,19.0.0-60-gafafd468+10,19.0.0-64-gf672fef+7,19.0.0-7-g039c0b5+8,19.0.0-7-gbea9075+4,19.0.0-7-gc567de5+10,19.0.0-8-g3a3ce09+5,19.0.0-9-g463f923+9,w.2020.21
LSSTDataManagementBasePackage
transformMeasurement.py
Go to the documentation of this file.
1 #
2 # LSST Data Management System
3 # Copyright 2008-2015 AURA/LSST.
4 #
5 # This product includes software developed by the
6 # LSST Project (http://www.lsst.org/).
7 #
8 # This program is free software: you can redistribute it and/or modify
9 # it under the terms of the GNU General Public License as published by
10 # the Free Software Foundation, either version 3 of the License, or
11 # (at your option) any later version.
12 #
13 # This program is distributed in the hope that it will be useful,
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 # GNU General Public License for more details.
17 #
18 # You should have received a copy of the LSST License Statement and
19 # the GNU General Public License along with this program. If not,
20 # see <http://www.lsstcorp.org/LegalNotices/>.
21 #
22 """
23 Tasks for transforming raw measurement outputs to calibrated quantities.
24 """
25 import lsst.afw.table as afwTable
26 import lsst.pex.config as pexConfig
27 import lsst.pipe.base as pipeBase
28 
29 
30 def makeContiguous(catalog):
31  """!Return a version of the input catalog which is contiguous in memory."""
32  if not catalog.isContiguous():
33  return catalog.copy(deep=True)
34  else:
35  return catalog
36 
37 
38 class TransformConfig(pexConfig.Config):
39  """!Configuration for TransformTask."""
40  copyFields = pexConfig.ListField(
41  dtype=str,
42  doc="Fields to copy from input to output catalog without transformation",
43  default=('id', 'coord_ra', 'coord_dec')
44  )
45 
46 
52 
53 
54 class TransformTask(pipeBase.Task):
55  r"""!
56  \anchor TransformTask_
57 
58  \brief Transform a SourceCatalog containing raw measurements to calibrated form.
59 
60  \section pipe_tasks_transform_Contents Contents
61 
62  - \ref pipe_tasks_transform_purpose
63  - \ref pipe_tasks_transform_initialize
64  - \ref pipe_tasks_transform_invoke
65 
66  \section pipe_tasks_transform_purpose Description
67 
68  Given a set of measurement algorithms with their associated configuration,
69  the table of source measurements they have produced, and information about
70  an associated WCS and calibration, transform the raw measurement output to
71  a calibrated form.
72 
73  Transformations are defined on a per-measurement-plugin basis. In
74  addition, a configurable set of fields may be simply copied from the input
75  to the output catalog.
76 
77  This task operates on an input SourceCatalog and returns a BaseCatalog
78  containing the transformed results. It requires that the caller supply
79  information on the configuration of the measurement task which produced
80  the input data as well as the world coordinate system and calibration
81  under which the transformation will take place. It provides no
82  functionality for reading or writing data from a Butler: rather,
83  per-dataset-type command line tasks are provided to obtain the appropriate
84  information from a Butler (or elsewhere) and then delegate to this task.
85 
86  \section pipe_tasks_transform_initialize Task initialization
87 
88  \copydoc \_\_init\_\_
89 
90  \section pipe_tasks_transform_invoke Task invocation
91 
92  \copydoc run
93  """
94  ConfigClass = TransformConfig
95  _DefaultName = "transform"
96 
97  def __init__(self, measConfig, inputSchema, outputDataset, *args, **kwargs):
98  """!Initialize TransformTask.
99 
100  @param[in] measConfig Configuration for the measurement task which
101  produced the measurments being transformed.
102  @param[in] inputSchema The schema of the input catalog.
103  @param[in] outputDataset The butler dataset type of the output catalog.
104  @param[in] *args Passed through to pipeBase.Task.__init__()
105  @param[in] *kwargs Passed through to pipeBase.Task.__init__()
106  """
107  pipeBase.Task.__init__(self, *args, **kwargs)
108 
109  # This task can be used to generate multiple different output dataset types. We
110  # need to be able to specify the output type together with its schema.
111  self.outputDataset = outputDataset
112 
113  # Define a mapper and add the basic fields to be copied.
114  self.mapper = afwTable.SchemaMapper(inputSchema)
115  for field in self.config.copyFields:
116  self.mapper.addMapping(inputSchema.find(field).key)
117 
118  # Build a list of all transforms that will be applied to the input. We
119  # will iterate over this in run().
120  self.transforms = []
121  for name in measConfig.plugins.names:
122  config = measConfig.plugins.get(name)
123  transformClass = measConfig.plugins.registry.get(name).PluginClass.getTransformClass()
124  self.transforms.append(transformClass(config, name, self.mapper))
125 
126  def getSchemaCatalogs(self):
127  """!Return a dict containing an empty catalog representative of this task's output."""
128  transformedSrc = afwTable.BaseCatalog(self.mapper.getOutputSchema())
129  return {self.outputDataset: transformedSrc}
130 
131  def run(self, inputCat, wcs, photoCalib):
132  """!Transform raw source measurements to calibrated quantities.
133 
134  @param[in] inputCat SourceCatalog of sources to transform.
135  @param[in] wcs The world coordinate system under which transformations will take place.
136  @param[in] photoCalib The calibration under which transformations will take place.
137 
138  @return A BaseCatalog containing the transformed measurements.
139  """
140  outputCat = afwTable.BaseCatalog(self.mapper.getOutputSchema())
141  outputCat.extend(inputCat, mapper=self.mapper)
142 
143  # Transforms may use a ColumnView on the input and output catalogs,
144  # which requires that the data be contiguous in memory.
145  inputCat = makeContiguous(inputCat)
146  outputCat = makeContiguous(outputCat)
147 
148  for transform in self.transforms:
149  transform(inputCat, outputCat, wcs, photoCalib)
150  return outputCat
151 
152 
153 class RunTransformConfig(pexConfig.Config):
154  """!Configuration for RunTransformTaskBase derivatives."""
155  transform = pexConfig.ConfigurableField(
156  doc="Subtask which performs transformations",
157  target=TransformTask
158  )
159  inputConfigType = pexConfig.Field(
160  dtype=str,
161  doc="Dataset type of measurement operation configuration",
162  )
163 
164 
165 class RunTransformTaskBase(pipeBase.CmdLineTask):
166  r"""!
167  \anchor RunTransformTaskBase_
168 
169  \brief Command line interface for TransformTask.
170 
171  \section pipe_tasks_transform_Contents Contents
172 
173  - \ref pipe_tasks_runtransform_purpose
174  - \ref pipe_tasks_runtransform_invoke
175 
176  \section pipe_tasks_runtransform_purpose Description
177 
178  Provides a command-line task which can be used to run TransformTask.
179 
180  - Loads a plugin registry based on configuration;
181  - Loads configuration for the measurement task which was applied from a repository;
182  - Loads the SourceCatalog input schema from a repository;
183  - For each input dataRef, reads the SourceCatalog, WCS and calibration from the
184  repository and executes TransformTask.
185 
186  This is not a fully-fledged command line task: it requires specialization to a particular
187  source type by defining the variables indicated below.
188 
189  \section pipe_tasks_runtransform_invoke Task invocation
190 
191  \copydoc run
192  """
193  RunnerClass = pipeBase.ButlerInitializedTaskRunner
194  ConfigClass = RunTransformConfig
195 
196  # Subclasses should provide definitions for the attributes named below.
197  # Properties can be used if appropriate.
198  #
199  # Standard CmdLineTask attributes:
200  _DefaultName = None
201 
202  # Butler dataset type of the source type to be transformed ("src", "forced_src", etc):
203  sourceType = None
204 
205  # Butler dataset type of the calibration exposure to use when transforming ("calexp", etc):
206  calexpType = None
207 
208  @property
209  def inputSchemaType(self):
210  """!
211  The Butler dataset type for the schema of the input source catalog.
212 
213  By default, we append `_schema` to the input source type. Subclasses may customize
214  if required.
215  """
216  return self.sourceType + "_schema"
217 
218  @property
219  def outputDataset(self):
220  """!
221  The Butler dataset type for the schema of the output catalog.
222 
223  By default, we prepend `transformed_` to the input source type. Subclasses may
224  customize if required.
225  """
226  return 'transformed_' + self.sourceType
227 
228  @property
229  def measurementConfig(self):
230  """!
231  The configuration of the measurement operation used to generate the input catalog.
232 
233  By default we look for `measurement` under the root configuration of the
234  generating task. Subclasses may customize this (e.g. to `calibrate.measurement`)
235  if required.
236  """
237  return self.butler.get(self.config.inputConfigType).measurement.value
238 
239  def __init__(self, *args, **kwargs):
240  pipeBase.CmdLineTask.__init__(self, *args, config=kwargs['config'], log=kwargs['log'])
241  self.butler = kwargs['butler']
242  self.makeSubtask('transform', measConfig=self.measurementConfig,
243  inputSchema=self.butler.get(self.inputSchemaType).schema,
244  outputDataset=self.outputDataset)
245 
246  @pipeBase.timeMethod
247  def runDataRef(self, dataRef):
248  """!Transform the source catalog referred to by dataRef.
249 
250  The result is both returned and written as dataset type "transformed_" + the input
251  source dataset type to the provided dataRef.
252 
253  @param[in] dataRef Data reference for source catalog & calibrated exposure.
254 
255  @returns A BaseCatalog containing the transformed measurements.
256  """
257  inputCat = dataRef.get(self.sourceType)
258  wcs = dataRef.get(self.calexpType).getWcs()
259  photoCalib = dataRef.get(self.calexpType).getPhotoCalib()
260  outputCat = self.transform.run(inputCat, wcs, photoCalib)
261  dataRef.put(outputCat, self.outputDataset)
262  return outputCat
263 
264 
265 
271 
273  """!
274  \anchor SrcTransformTask_
275 
276  \brief Transform ``src`` measuremenents to calibrated form.
277 
278  This is a specialization of \ref RunTransformTaskBase_ "RunTransformTaskBase" which
279  operates on ``src`` measurements. Refer to the parent documentation for details.
280  """
281  _DefaultName = "transformSrcMeasurement"
282  sourceType = 'src'
283  calexpType = 'calexp'
284 
285  @property
286  def measurementConfig(self):
287  return self.butler.get(self.config.inputConfigType).calibrate.measurement.value
288 
289 
290 
296 
298  """!
299  \anchor ForcedSrcTransformTask_
300 
301  \brief Transform ``forced_src`` measuremenents to calibrated form.
302 
303  This is a specialization of \ref RunTransformTaskBase_ "RunTransformTaskBase" which
304  operates on ``forced_src`` measurements. Refer to the parent documentation for details.
305  """
306  _DefaultName = "transformForcedSrcMeasurement"
307  sourceType = 'forced_src'
308  calexpType = 'calexp'
309 
310 
311 
317 
319  """!
320  \anchor CoaddSrcTransformTask_
321 
322  \brief Transform measuremenents made on coadds to calibrated form.
323 
324  This is a specialization of \ref RunTransformTaskBase_ "RunTransformTaskBase" which
325  operates on measurements made on coadds. Refer to the parent documentation for details.
326  """
327  _DefaultName = "transformCoaddSrcMeasurement"
328 
329  @property
330  def coaddName(self):
331  return self.butler.get(self.config.inputConfigType).coaddName
332 
333  @property
334  def sourceType(self):
335  return self.coaddName + "Coadd_meas"
336 
337  @property
338  def calexpType(self):
339  return self.coaddName + "Coadd_calexp"
340 
341  def _getConfigName(self):
342  return "%s_transformCoaddSrcMeasurement_config" % (self.coaddName,)
343 
344  def _getMetaDataName(self):
345  return "%s_transformCoaddSrcMeasurement_metadata" % (self.coaddName,)
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase.runDataRef
def runDataRef(self, dataRef)
Transform the source catalog referred to by dataRef.
Definition: transformMeasurement.py:247
lsst.pipe.tasks.transformMeasurement.ForcedSrcTransformTask
Transform forced_src measuremenents to calibrated form.
Definition: transformMeasurement.py:297
transform
lsst.pipe.tasks.transformMeasurement.makeContiguous
def makeContiguous(catalog)
Return a version of the input catalog which is contiguous in memory.
Definition: transformMeasurement.py:30
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase
Command line interface for TransformTask.
Definition: transformMeasurement.py:165
lsst.pipe.tasks.transformMeasurement.TransformTask.transforms
transforms
Definition: transformMeasurement.py:120
ast::append
std::shared_ptr< FrameSet > append(FrameSet const &first, FrameSet const &second)
Construct a FrameSet that performs two transformations in series.
Definition: functional.cc:33
lsst.pipe.tasks.transformMeasurement.TransformTask.__init__
def __init__(self, measConfig, inputSchema, outputDataset, *args, **kwargs)
Initialize TransformTask.
Definition: transformMeasurement.py:97
lsst.pipe.tasks.transformMeasurement.TransformTask.mapper
mapper
Definition: transformMeasurement.py:114
lsst.pipe.tasks.transformMeasurement.TransformTask.getSchemaCatalogs
def getSchemaCatalogs(self)
Return a dict containing an empty catalog representative of this task's output.
Definition: transformMeasurement.py:126
lsst.pipe.tasks.assembleCoadd.run
def run(self, skyInfo, tempExpRefList, imageScalerList, weightList, altMaskList=None, mask=None, supplementaryData=None)
Definition: assembleCoadd.py:712
lsst.pipe.tasks.transformMeasurement.CoaddSrcTransformTask
Transform measuremenents made on coadds to calibrated form.
Definition: transformMeasurement.py:318
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase.__init__
def __init__(self, *args, **kwargs)
Definition: transformMeasurement.py:239
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase.outputDataset
def outputDataset(self)
The Butler dataset type for the schema of the output catalog.
Definition: transformMeasurement.py:219
lsst::afw::table::SchemaMapper
A mapping between the keys of two Schemas, used to copy data between them.
Definition: SchemaMapper.h:21
lsst::afw::table
Definition: table.dox:3
lsst.pipe.tasks.transformMeasurement.SrcTransformTask
Transform src measuremenents to calibrated form.
Definition: transformMeasurement.py:272
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase.sourceType
sourceType
Definition: transformMeasurement.py:203
lsst.pipe.tasks.transformMeasurement.TransformTask.outputDataset
outputDataset
Definition: transformMeasurement.py:111
lsst.pipe.tasks.transformMeasurement.TransformTask
Transform a SourceCatalog containing raw measurements to calibrated form.
Definition: transformMeasurement.py:54
lsst.pipe.tasks.transformMeasurement.TransformTask.run
def run(self, inputCat, wcs, photoCalib)
Transform raw source measurements to calibrated quantities.
Definition: transformMeasurement.py:131
lsst.pipe.tasks.transformMeasurement.RunTransformConfig
Configuration for RunTransformTaskBase derivatives.
Definition: transformMeasurement.py:153
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase.measurementConfig
def measurementConfig(self)
The configuration of the measurement operation used to generate the input catalog.
Definition: transformMeasurement.py:229
lsst.pipe.tasks.transformMeasurement.SrcTransformTask.measurementConfig
def measurementConfig(self)
The configuration of the measurement operation used to generate the input catalog.
Definition: transformMeasurement.py:286
lsst.pipe.tasks.transformMeasurement.CoaddSrcTransformTask.coaddName
def coaddName(self)
Definition: transformMeasurement.py:330
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase.calexpType
calexpType
Definition: transformMeasurement.py:206
lsst.pipe.tasks.transformMeasurement.TransformConfig
Configuration for TransformTask.
Definition: transformMeasurement.py:38
lsst.pipe.base
Definition: __init__.py:1
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase.butler
butler
Definition: transformMeasurement.py:241
lsst::afw::table::CatalogT< BaseRecord >
lsst.pipe.tasks.transformMeasurement.RunTransformTaskBase.inputSchemaType
def inputSchemaType(self)
The Butler dataset type for the schema of the input source catalog.
Definition: transformMeasurement.py:209