LSSTApplications  21.0.0+1b62c9342b,21.0.0+45a059f35e,21.0.0-1-ga51b5d4+ceb9cf20a3,21.0.0-19-g7c7630f+a88ebbf2d9,21.0.0-2-g103fe59+3522cf3bc7,21.0.0-2-g1367e85+571a348718,21.0.0-2-g2909d54+45a059f35e,21.0.0-2-g45278ab+1b62c9342b,21.0.0-2-g4bc9b9f+35a70d5868,21.0.0-2-g5242d73+571a348718,21.0.0-2-g54e2caa+aa129c4686,21.0.0-2-g66bcc37+3caef57c29,21.0.0-2-g7f82c8f+6f9059e2fe,21.0.0-2-g8dde007+5d1b9cb3f5,21.0.0-2-g8f08a60+73884b2cf5,21.0.0-2-g973f35b+1d054a08b9,21.0.0-2-ga326454+6f9059e2fe,21.0.0-2-ga63a54e+3d2c655db6,21.0.0-2-gc738bc1+a567cb0f17,21.0.0-2-gde069b7+5a8f2956b8,21.0.0-2-ge17e5af+571a348718,21.0.0-2-ge712728+834f2a3ece,21.0.0-2-gecfae73+dfe6e80958,21.0.0-2-gfc62afb+571a348718,21.0.0-21-g006371a9+88174a2081,21.0.0-3-g4c5b185+7fd31a6834,21.0.0-3-g6d51c4a+3caef57c29,21.0.0-3-gaa929c8+55f5a6a5c9,21.0.0-3-gd222c45+afc8332dbe,21.0.0-3-gd5de2f2+3caef57c29,21.0.0-4-g3300ddd+1b62c9342b,21.0.0-4-g5873dc9+9a92674037,21.0.0-4-g8a80011+5955f0fd15,21.0.0-5-gb7080ec+8658c79ec4,21.0.0-5-gcff38f6+89f2a0074d,21.0.0-6-gd3283ba+55f5a6a5c9,21.0.0-8-g19111d86+2c4b0a9f47,21.0.0-9-g7bed000b9+c7d3cce47e,w.2021.03
LSSTDataManagementBasePackage
Classes | Functions
lsst.cp.pipe.astierCovPtcUtils Namespace Reference

Classes

class  CovFft
 
class  LoadParams
 

Functions

def fftSize (s)
 
def computeCovDirect (diffImage, weightImage, maxRange)
 
def covDirectValue (diffImage, weightImage, dx, dy)
 
def loadData (tupleName, params, expIdMask)
 
def fitData (tupleName, expIdMask, r=8)
 
def getFitDataFromCovariances (i, j, mu, fullCov, fullCovModel, fullCovSqrtWeights, gain=1.0, divideByMu=False, returnMasked=False)
 

Function Documentation

◆ computeCovDirect()

def lsst.cp.pipe.astierCovPtcUtils.computeCovDirect (   diffImage,
  weightImage,
  maxRange 
) 
Compute covariances of diffImage in real space.

For lags larger than ~25, it is slower than the FFT way.
Taken from https://github.com/PierreAstier/bfptc/

Parameters
----------
diffImage : `numpy.array`
    Image to compute the covariance of.

weightImage : `numpy.array`
    Weight image of diffImage (1's and 0's for good and bad pixels, respectively).

maxRange : `int`
    Last index of the covariance to be computed.

Returns
-------
outList : `list`
    List with tuples of the form (dx, dy, var, cov, npix), where:
    dx : `int`
        Lag in x
    dy : `int`
        Lag in y
    var : `float`
        Variance at (dx, dy).
    cov : `float`
        Covariance at (dx, dy).
    nPix : `int`
        Number of pixel pairs used to evaluate var and cov.

Definition at line 128 of file astierCovPtcUtils.py.

128 def computeCovDirect(diffImage, weightImage, maxRange):
129  """Compute covariances of diffImage in real space.
130 
131  For lags larger than ~25, it is slower than the FFT way.
132  Taken from https://github.com/PierreAstier/bfptc/
133 
134  Parameters
135  ----------
136  diffImage : `numpy.array`
137  Image to compute the covariance of.
138 
139  weightImage : `numpy.array`
140  Weight image of diffImage (1's and 0's for good and bad pixels, respectively).
141 
142  maxRange : `int`
143  Last index of the covariance to be computed.
144 
145  Returns
146  -------
147  outList : `list`
148  List with tuples of the form (dx, dy, var, cov, npix), where:
149  dx : `int`
150  Lag in x
151  dy : `int`
152  Lag in y
153  var : `float`
154  Variance at (dx, dy).
155  cov : `float`
156  Covariance at (dx, dy).
157  nPix : `int`
158  Number of pixel pairs used to evaluate var and cov.
159  """
160  outList = []
161  var = 0
162  # (dy,dx) = (0,0) has to be first
163  for dy in range(maxRange + 1):
164  for dx in range(0, maxRange + 1):
165  if (dx*dy > 0):
166  cov1, nPix1 = covDirectValue(diffImage, weightImage, dx, dy)
167  cov2, nPix2 = covDirectValue(diffImage, weightImage, dx, -dy)
168  cov = 0.5*(cov1 + cov2)
169  nPix = nPix1 + nPix2
170  else:
171  cov, nPix = covDirectValue(diffImage, weightImage, dx, dy)
172  if (dx == 0 and dy == 0):
173  var = cov
174  outList.append((dx, dy, var, cov, nPix))
175 
176  return outList
177 
178 

◆ covDirectValue()

def lsst.cp.pipe.astierCovPtcUtils.covDirectValue (   diffImage,
  weightImage,
  dx,
  dy 
) 
Compute covariances of diffImage in real space at lag (dx, dy).

Taken from https://github.com/PierreAstier/bfptc/ (c.f., appendix of Astier+19).

Parameters
----------
diffImage : `numpy.array`
    Image to compute the covariance of.

weightImage : `numpy.array`
    Weight image of diffImage (1's and 0's for good and bad pixels, respectively).

dx : `int`
    Lag in x.

dy : `int`
    Lag in y.

Returns
-------
cov : `float`
    Covariance at (dx, dy)

nPix : `int`
    Number of pixel pairs used to evaluate var and cov.

Definition at line 179 of file astierCovPtcUtils.py.

179 def covDirectValue(diffImage, weightImage, dx, dy):
180  """Compute covariances of diffImage in real space at lag (dx, dy).
181 
182  Taken from https://github.com/PierreAstier/bfptc/ (c.f., appendix of Astier+19).
183 
184  Parameters
185  ----------
186  diffImage : `numpy.array`
187  Image to compute the covariance of.
188 
189  weightImage : `numpy.array`
190  Weight image of diffImage (1's and 0's for good and bad pixels, respectively).
191 
192  dx : `int`
193  Lag in x.
194 
195  dy : `int`
196  Lag in y.
197 
198  Returns
199  -------
200  cov : `float`
201  Covariance at (dx, dy)
202 
203  nPix : `int`
204  Number of pixel pairs used to evaluate var and cov.
205  """
206  (nCols, nRows) = diffImage.shape
207  # switching both signs does not change anything:
208  # it just swaps im1 and im2 below
209  if (dx < 0):
210  (dx, dy) = (-dx, -dy)
211  # now, we have dx >0. We have to distinguish two cases
212  # depending on the sign of dy
213  if dy >= 0:
214  im1 = diffImage[dy:, dx:]
215  w1 = weightImage[dy:, dx:]
216  im2 = diffImage[:nCols - dy, :nRows - dx]
217  w2 = weightImage[:nCols - dy, :nRows - dx]
218  else:
219  im1 = diffImage[:nCols + dy, dx:]
220  w1 = weightImage[:nCols + dy, dx:]
221  im2 = diffImage[-dy:, :nRows - dx]
222  w2 = weightImage[-dy:, :nRows - dx]
223  # use the same mask for all 3 calculations
224  wAll = w1*w2
225  # do not use mean() because weightImage=0 pixels would then count
226  nPix = wAll.sum()
227  im1TimesW = im1*wAll
228  s1 = im1TimesW.sum()/nPix
229  s2 = (im2*wAll).sum()/nPix
230  p = (im1TimesW*im2).sum()/nPix
231  cov = p - s1*s2
232 
233  return cov, nPix
234 
235 

◆ fftSize()

def lsst.cp.pipe.astierCovPtcUtils.fftSize (   s) 
Calculate the size fof one dimension for the FFT

Definition at line 122 of file astierCovPtcUtils.py.

122 def fftSize(s):
123  """Calculate the size fof one dimension for the FFT"""
124  x = int(np.log(s)/np.log(2.))
125  return int(2**(x+1))
126 
127 

◆ fitData()

def lsst.cp.pipe.astierCovPtcUtils.fitData (   tupleName,
  expIdMask,
  r = 8 
) 
Fit data to models in Astier+19.

Parameters
----------
tupleName: `numpy.recarray`
    Recarray with rows with at least ( mu1, mu2, cov ,var, i, j, npix), where:
        mu1: mean value of flat1
        mu2: mean value of flat2
        cov: covariance value at lag (i, j)
        var: variance (covariance value at lag (0, 0))
        i: lag dimension
        j: lag dimension
        npix: number of pixels used for covariance calculation.

expIdMask : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the masked exposure pairs.

r: `int`, optional
    Maximum lag considered (e.g., to eliminate data beyond a separation "r": ignored in the fit).

Returns
-------
covFitList: `dict`
    Dictionary of CovFit objects, with amp names as keys.

covFitNoBList: `dict`
   Dictionary of CovFit objects, with amp names as keys (b=0 in Eq. 20 of Astier+19).

Notes
-----
The parameters of the full model for C_ij(mu) ("C_ij" and "mu" in ADU^2 and ADU, respectively)
in Astier+19 (Eq. 20) are:

    "a" coefficients (r by r matrix), units: 1/e
    "b" coefficients (r by r matrix), units: 1/e
    noise matrix (r by r matrix), units: e^2
    gain, units: e/ADU

"b" appears in Eq. 20 only through the "ab" combination, which is defined in this code as "c=ab".

Definition at line 310 of file astierCovPtcUtils.py.

310 def fitData(tupleName, expIdMask, r=8):
311  """Fit data to models in Astier+19.
312 
313  Parameters
314  ----------
315  tupleName: `numpy.recarray`
316  Recarray with rows with at least ( mu1, mu2, cov ,var, i, j, npix), where:
317  mu1: mean value of flat1
318  mu2: mean value of flat2
319  cov: covariance value at lag (i, j)
320  var: variance (covariance value at lag (0, 0))
321  i: lag dimension
322  j: lag dimension
323  npix: number of pixels used for covariance calculation.
324 
325  expIdMask : `dict`, [`str`, `list`]
326  Dictionary keyed by amp names containing the masked exposure pairs.
327 
328  r: `int`, optional
329  Maximum lag considered (e.g., to eliminate data beyond a separation "r": ignored in the fit).
330 
331  Returns
332  -------
333  covFitList: `dict`
334  Dictionary of CovFit objects, with amp names as keys.
335 
336  covFitNoBList: `dict`
337  Dictionary of CovFit objects, with amp names as keys (b=0 in Eq. 20 of Astier+19).
338 
339  Notes
340  -----
341  The parameters of the full model for C_ij(mu) ("C_ij" and "mu" in ADU^2 and ADU, respectively)
342  in Astier+19 (Eq. 20) are:
343 
344  "a" coefficients (r by r matrix), units: 1/e
345  "b" coefficients (r by r matrix), units: 1/e
346  noise matrix (r by r matrix), units: e^2
347  gain, units: e/ADU
348 
349  "b" appears in Eq. 20 only through the "ab" combination, which is defined in this code as "c=ab".
350  """
351 
352  lparams = LoadParams()
353  lparams.subtractDistantValue = False
354  lparams.r = r
355  covFitList = loadData(tupleName, lparams, expIdMask=expIdMask)
356  covFitNoBList = {} # [None]*(exts[-1]+1)
357  for ext, c in covFitList.items():
358  c.fitFullModel()
359  covFitNoBList[ext] = c.copy()
360  c.params['c'].release()
361  c.fitFullModel()
362  return covFitList, covFitNoBList
363 
364 

◆ getFitDataFromCovariances()

def lsst.cp.pipe.astierCovPtcUtils.getFitDataFromCovariances (   i,
  j,
  mu,
  fullCov,
  fullCovModel,
  fullCovSqrtWeights,
  gain = 1.0,
  divideByMu = False,
  returnMasked = False 
) 
Get measured signal and covariance, cov model, weigths, and mask at covariance lag (i, j).

Parameters
----------
i :  `int`
    Lag for covariance matrix.

j: `int`
    Lag for covariance matrix.

mu : `list`
    Mean signal values.

fullCov: `list` of `numpy.array`
    Measured covariance matrices at each mean signal level in mu.

fullCovSqrtWeights: `list` of `numpy.array`
    List of square root of measured covariances at each mean signal level in mu.

fullCovModel : `list` of `numpy.array`
    List of modeled covariances at each mean signal level in mu.

gain : `float`, optional
    Gain, in e-/ADU. If other than 1.0 (default), the returned quantities will be in
    electrons or powers of electrons.

divideByMu: `bool`, optional
    Divide returned covariance, model, and weights by the mean signal mu?

returnMasked : `bool`, optional
    Use mask (based on weights) in returned arrays (mu, covariance, and model)?

Returns
-------
mu : `numpy.array`
    list of signal values at (i, j).

covariance : `numpy.array`
    Covariance at (i, j) at each mean signal mu value (fullCov[:, i, j]).

covarianceModel : `numpy.array`
    Covariance model at (i, j).

weights : `numpy.array`
    Weights at (i, j).

maskFromWeights : `numpy.array`, optional
    Boolean mask of the covariance at (i,j), where the weights differ from 0.

Notes
-----
This function is a method of the `CovFit` class.

Definition at line 365 of file astierCovPtcUtils.py.

365 def getFitDataFromCovariances(i, j, mu, fullCov, fullCovModel, fullCovSqrtWeights, gain=1.0,
366  divideByMu=False, returnMasked=False):
367  """Get measured signal and covariance, cov model, weigths, and mask at covariance lag (i, j).
368 
369  Parameters
370  ----------
371  i : `int`
372  Lag for covariance matrix.
373 
374  j: `int`
375  Lag for covariance matrix.
376 
377  mu : `list`
378  Mean signal values.
379 
380  fullCov: `list` of `numpy.array`
381  Measured covariance matrices at each mean signal level in mu.
382 
383  fullCovSqrtWeights: `list` of `numpy.array`
384  List of square root of measured covariances at each mean signal level in mu.
385 
386  fullCovModel : `list` of `numpy.array`
387  List of modeled covariances at each mean signal level in mu.
388 
389  gain : `float`, optional
390  Gain, in e-/ADU. If other than 1.0 (default), the returned quantities will be in
391  electrons or powers of electrons.
392 
393  divideByMu: `bool`, optional
394  Divide returned covariance, model, and weights by the mean signal mu?
395 
396  returnMasked : `bool`, optional
397  Use mask (based on weights) in returned arrays (mu, covariance, and model)?
398 
399  Returns
400  -------
401  mu : `numpy.array`
402  list of signal values at (i, j).
403 
404  covariance : `numpy.array`
405  Covariance at (i, j) at each mean signal mu value (fullCov[:, i, j]).
406 
407  covarianceModel : `numpy.array`
408  Covariance model at (i, j).
409 
410  weights : `numpy.array`
411  Weights at (i, j).
412 
413  maskFromWeights : `numpy.array`, optional
414  Boolean mask of the covariance at (i,j), where the weights differ from 0.
415 
416  Notes
417  -----
418  This function is a method of the `CovFit` class.
419  """
420  mu = np.array(mu)
421  fullCov = np.array(fullCov)
422  fullCovModel = np.array(fullCovModel)
423  fullCovSqrtWeights = np.array(fullCovSqrtWeights)
424  covariance = fullCov[:, i, j]*(gain**2)
425  covarianceModel = fullCovModel[:, i, j]*(gain**2)
426  weights = fullCovSqrtWeights[:, i, j]/(gain**2)
427 
428  maskFromWeights = weights != 0
429  if returnMasked:
430  weights = weights[maskFromWeights]
431  covarianceModel = covarianceModel[maskFromWeights]
432  mu = mu[maskFromWeights]
433  covariance = covariance[maskFromWeights]
434 
435  if divideByMu:
436  covariance /= mu
437  covarianceModel /= mu
438  weights *= mu
439  return mu, covariance, covarianceModel, weights, maskFromWeights

◆ loadData()

def lsst.cp.pipe.astierCovPtcUtils.loadData (   tupleName,
  params,
  expIdMask 
) 
 Returns a list of CovFit objects, indexed by amp number.

Params
------
tupleName: `numpy.recarray`
    Recarray with rows with at least ( mu1, mu2, cov ,var, i, j, npix), where:
        mu1: mean value of flat1
        mu2: mean value of flat2
        cov: covariance value at lag (i, j)
        var: variance (covariance value at lag (0, 0))
        i: lag dimension
        j: lag dimension
        npix: number of pixels used for covariance calculation.

params: `covAstierptcUtil.LoadParams`
    Object with values to drive the bahaviour of fits.

expIdMask : `dict`, [`str`, `list`]
    Dictionary keyed by amp names containing the masked exposure pairs.

Returns
-------
covFitList: `dict`
    Dictionary with amps as keys, and CovFit objects as values.

Definition at line 265 of file astierCovPtcUtils.py.

265 def loadData(tupleName, params, expIdMask):
266  """ Returns a list of CovFit objects, indexed by amp number.
267 
268  Params
269  ------
270  tupleName: `numpy.recarray`
271  Recarray with rows with at least ( mu1, mu2, cov ,var, i, j, npix), where:
272  mu1: mean value of flat1
273  mu2: mean value of flat2
274  cov: covariance value at lag (i, j)
275  var: variance (covariance value at lag (0, 0))
276  i: lag dimension
277  j: lag dimension
278  npix: number of pixels used for covariance calculation.
279 
280  params: `covAstierptcUtil.LoadParams`
281  Object with values to drive the bahaviour of fits.
282 
283  expIdMask : `dict`, [`str`, `list`]
284  Dictionary keyed by amp names containing the masked exposure pairs.
285 
286  Returns
287  -------
288  covFitList: `dict`
289  Dictionary with amps as keys, and CovFit objects as values.
290  """
291 
292  exts = np.array(np.unique(tupleName['ampName']), dtype=str)
293  covFitList = {}
294  for ext in exts:
295  ntext = tupleName[tupleName['ampName'] == ext]
296  maskExt = expIdMask[ext]
297  if params.subtractDistantValue:
298  c = CovFit(ntext, params.r, maskExt)
299  c.subtractDistantOffset(params.r, params.start, params.offsetDegree)
300  else:
301  c = CovFit(ntext, params.r, maskExt)
302 
303  cc = c.copy()
304  cc.initFit() # allows to get a crude gain.
305  covFitList[ext] = cc
306 
307  return covFitList
308 
309 
lsst.cp.pipe.astierCovPtcUtils.fftSize
def fftSize(s)
Definition: astierCovPtcUtils.py:122
lsst.cp.pipe.astierCovPtcUtils.covDirectValue
def covDirectValue(diffImage, weightImage, dx, dy)
Definition: astierCovPtcUtils.py:179
lsst.cp.pipe.astierCovPtcUtils.computeCovDirect
def computeCovDirect(diffImage, weightImage, maxRange)
Definition: astierCovPtcUtils.py:128
lsst.cp.pipe.astierCovPtcUtils.fitData
def fitData(tupleName, expIdMask, r=8)
Definition: astierCovPtcUtils.py:310
lsst.cp.pipe.astierCovPtcUtils.getFitDataFromCovariances
def getFitDataFromCovariances(i, j, mu, fullCov, fullCovModel, fullCovSqrtWeights, gain=1.0, divideByMu=False, returnMasked=False)
Definition: astierCovPtcUtils.py:365
lsst.cp.pipe.astierCovPtcUtils.loadData
def loadData(tupleName, params, expIdMask)
Definition: astierCovPtcUtils.py:265
Generated on Thu Jan 21 2021 09:19:40 for LSSTApplications by doxygen 1.8.20