LSST Applications  21.0.0+04719a4bac,21.0.0-1-ga51b5d4+f5e6047307,21.0.0-11-g2b59f77+a9c1acf22d,21.0.0-11-ga42c5b2+86977b0b17,21.0.0-12-gf4ce030+76814010d2,21.0.0-13-g1721dae+760e7a6536,21.0.0-13-g3a573fe+768d78a30a,21.0.0-15-g5a7caf0+f21cbc5713,21.0.0-16-g0fb55c1+b60e2d390c,21.0.0-19-g4cded4ca+71a93a33c0,21.0.0-2-g103fe59+bb20972958,21.0.0-2-g45278ab+04719a4bac,21.0.0-2-g5242d73+3ad5d60fb1,21.0.0-2-g7f82c8f+8babb168e8,21.0.0-2-g8f08a60+06509c8b61,21.0.0-2-g8faa9b5+616205b9df,21.0.0-2-ga326454+8babb168e8,21.0.0-2-gde069b7+5e4aea9c2f,21.0.0-2-gecfae73+1d3a86e577,21.0.0-2-gfc62afb+3ad5d60fb1,21.0.0-25-g1d57be3cd+e73869a214,21.0.0-3-g357aad2+ed88757d29,21.0.0-3-g4a4ce7f+3ad5d60fb1,21.0.0-3-g4be5c26+3ad5d60fb1,21.0.0-3-g65f322c+e0b24896a3,21.0.0-3-g7d9da8d+616205b9df,21.0.0-3-ge02ed75+a9c1acf22d,21.0.0-4-g591bb35+a9c1acf22d,21.0.0-4-g65b4814+b60e2d390c,21.0.0-4-gccdca77+0de219a2bc,21.0.0-4-ge8a399c+6c55c39e83,21.0.0-5-gd00fb1e+05fce91b99,21.0.0-6-gc675373+3ad5d60fb1,21.0.0-64-g1122c245+4fb2b8f86e,21.0.0-7-g04766d7+cd19d05db2,21.0.0-7-gdf92d54+04719a4bac,21.0.0-8-g5674e7b+d1bd76f71f,master-gac4afde19b+a9c1acf22d,w.2021.13
LSST Data Management Base Package
processCcd.py
Go to the documentation of this file.
1 #
2 # LSST Data Management System
3 # Copyright 2008-2016 AURA/LSST.
4 #
5 # This product includes software developed by the
6 # LSST Project (http://www.lsst.org/).
7 #
8 # This program is free software: you can redistribute it and/or modify
9 # it under the terms of the GNU General Public License as published by
10 # the Free Software Foundation, either version 3 of the License, or
11 # (at your option) any later version.
12 #
13 # This program is distributed in the hope that it will be useful,
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 # GNU General Public License for more details.
17 #
18 # You should have received a copy of the LSST License Statement and
19 # the GNU General Public License along with this program. If not,
20 # see <https://www.lsstcorp.org/LegalNotices/>.
21 #
22 from lsst.ip.isr import IsrTask
23 import lsst.pex.config as pexConfig
24 import lsst.pipe.base as pipeBase
25 from .calibrate import CalibrateTask
26 from .characterizeImage import CharacterizeImageTask
27 
28 __all__ = ["ProcessCcdConfig", "ProcessCcdTask"]
29 
30 
31 class ProcessCcdConfig(pexConfig.Config):
32  """Config for ProcessCcd"""
33  isr = pexConfig.ConfigurableField(
34  target=IsrTask,
35  doc="""Task to perform instrumental signature removal or load a post-ISR image; ISR consists of:
36  - assemble raw amplifier images into an exposure with image, variance and mask planes
37  - perform bias subtraction, flat fielding, etc.
38  - mask known bad pixels
39  - provide a preliminary WCS
40  """,
41  )
42  charImage = pexConfig.ConfigurableField(
43  target=CharacterizeImageTask,
44  doc="""Task to characterize a science exposure:
45  - detect sources, usually at high S/N
46  - estimate the background, which is subtracted from the image and returned as field "background"
47  - estimate a PSF model, which is added to the exposure
48  - interpolate over defects and cosmic rays, updating the image, variance and mask planes
49  """,
50  )
51  doCalibrate = pexConfig.Field(
52  dtype=bool,
53  default=True,
54  doc="Perform calibration?",
55  )
56  calibrate = pexConfig.ConfigurableField(
57  target=CalibrateTask,
58  doc="""Task to perform astrometric and photometric calibration:
59  - refine the WCS in the exposure
60  - refine the PhotoCalib object in the exposure
61  - detect sources, usually at low S/N
62  """,
63  )
64 
65  def setDefaults(self):
66  self.isrisr.doWrite = False
67  self.charImagecharImage.doWriteExposure = False
68 
69 
75 
76 
77 class ProcessCcdTask(pipeBase.CmdLineTask):
78  r"""!Assemble raw data, fit the PSF, detect and measure, and fit WCS and zero-point
79 
80  @anchor ProcessCcdTask_
81 
82  @section pipe_tasks_processCcd_Contents Contents
83 
84  - @ref pipe_tasks_processCcd_Purpose
85  - @ref pipe_tasks_processCcd_Initialize
86  - @ref pipe_tasks_processCcd_IO
87  - @ref pipe_tasks_processCcd_Config
88  - @ref pipe_tasks_processCcd_Debug
89  - @ref pipe_tasks_processCcd_Example
90 
91  @section pipe_tasks_processCcd_Purpose Description
92 
93  Perform the following operations:
94  - Call isr to unpersist raw data and assemble it into a post-ISR exposure
95  - Call charImage subtract background, fit a PSF model, repair cosmic rays,
96  detect and measure bright sources, and measure aperture correction
97  - Call calibrate to perform deep detection, deblending and single-frame measurement,
98  refine the WCS and fit the photometric zero-point
99 
100  @section pipe_tasks_processCcd_Initialize Task initialisation
101 
102  @copydoc \_\_init\_\_
103 
104  @section pipe_tasks_processCcd_IO Invoking the Task
105 
106  This task is primarily designed to be run from the command line.
107 
108  The main method is `runDataRef`, which takes a single butler data reference for the raw input data.
109 
110  @section pipe_tasks_processCcd_Config Configuration parameters
111 
112  See @ref ProcessCcdConfig
113 
114  @section pipe_tasks_processCcd_Debug Debug variables
115 
116  ProcessCcdTask has no debug output, but its subtasks do.
117 
118  @section pipe_tasks_processCcd_Example A complete example of using ProcessCcdTask
119 
120  The following commands will process all raw data in obs_test's data repository.
121  Note: be sure to specify an `--output` that does not already exist:
122 
123  setup obs_test
124  setup pipe_tasks
125  processCcd.py $OBS_TEST_DIR/data/input --output processCcdOut --id
126 
127  The data is read from the small repository in the `obs_test` package and written `./processCcdOut`
128  (or whatever output you specified). Specifying `--id` with no values processes all data.
129  Add the option `--help` to see more options.
130  """
131  ConfigClass = ProcessCcdConfig
132  RunnerClass = pipeBase.ButlerInitializedTaskRunner
133  _DefaultName = "processCcd"
134 
135  def __init__(self, butler=None, psfRefObjLoader=None, astromRefObjLoader=None, photoRefObjLoader=None,
136  **kwargs):
137  """!
138  @param[in] butler The butler is passed to the refObjLoader constructor in case it is
139  needed. Ignored if the refObjLoader argument provides a loader directly.
140  @param[in] psfRefObjLoader An instance of LoadReferenceObjectsTasks that supplies an
141  external reference catalog for image characterization. May be None if the desired
142  loader can be constructed from the butler argument or all steps requiring a catalog
143  are disabled.
144  @param[in] astromRefObjLoader An instance of LoadReferenceObjectsTasks that supplies an
145  external reference catalog for astrometric calibration. May be None if the desired
146  loader can be constructed from the butler argument or all steps requiring a reference
147  catalog are disabled.
148  @param[in] photoRefObjLoader An instance of LoadReferenceObjectsTasks that supplies an
149  external reference catalog for photometric calibration. May be None if the desired
150  loader can be constructed from the butler argument or all steps requiring a reference
151  catalog are disabled.
152  @param[in,out] kwargs other keyword arguments for lsst.pipe.base.CmdLineTask
153  """
154  pipeBase.CmdLineTask.__init__(self, **kwargs)
155  self.makeSubtask("isr")
156  self.makeSubtask("charImage", butler=butler, refObjLoader=psfRefObjLoader)
157  self.makeSubtask("calibrate", butler=butler, icSourceSchema=self.charImage.schema,
158  astromRefObjLoader=astromRefObjLoader, photoRefObjLoader=photoRefObjLoader)
159 
160  @pipeBase.timeMethod
161  def runDataRef(self, sensorRef):
162  """Process one CCD
163 
164  The sequence of operations is:
165  - remove instrument signature
166  - characterize image to estimate PSF and background
167  - calibrate astrometry and photometry
168 
169  @param sensorRef: butler data reference for raw data
170 
171  @return pipe_base Struct containing these fields:
172  - charRes: object returned by image characterization task; an lsst.pipe.base.Struct
173  that will include "background" and "sourceCat" fields
174  - calibRes: object returned by calibration task: an lsst.pipe.base.Struct
175  that will include "background" and "sourceCat" fields
176  - exposure: final exposure (an lsst.afw.image.ExposureF)
177  - background: final background model (an lsst.afw.math.BackgroundList)
178  """
179  self.log.info("Processing %s" % (sensorRef.dataId))
180 
181  exposure = self.isr.runDataRef(sensorRef).exposure
182 
183  charRes = self.charImage.runDataRef(
184  dataRef=sensorRef,
185  exposure=exposure,
186  doUnpersist=False,
187  )
188  exposure = charRes.exposure
189 
190  if self.config.doCalibrate:
191  calibRes = self.calibrate.runDataRef(
192  dataRef=sensorRef,
193  exposure=charRes.exposure,
194  background=charRes.background,
195  doUnpersist=False,
196  icSourceCat=charRes.sourceCat,
197  )
198 
199  return pipeBase.Struct(
200  charRes=charRes,
201  calibRes=calibRes if self.config.doCalibrate else None,
202  exposure=exposure,
203  background=calibRes.background if self.config.doCalibrate else charRes.background,
204  )
205 
206  @classmethod
207  def _makeArgumentParser(cls):
208  """!Create and return an argument parser
209 
210  @param[in] cls the class object
211  @return the argument parser for this task.
212 
213  This override is used to delay making the data ref list until the dataset type is known;
214  this is done in @ref parseAndRun.
215  """
216  parser = pipeBase.ArgumentParser(name=cls._DefaultName_DefaultName)
217  parser.add_id_argument(name="--id",
218  datasetType=pipeBase.ConfigDatasetType(name="isr.datasetType"),
219  help="data IDs, e.g. --id visit=12345 ccd=1,2^0,3")
220  return parser
Assemble raw data, fit the PSF, detect and measure, and fit WCS and zero-point.
Definition: processCcd.py:77
def __init__(self, butler=None, psfRefObjLoader=None, astromRefObjLoader=None, photoRefObjLoader=None, **kwargs)
Definition: processCcd.py:136