LSSTApplications  10.0-2-g4f67435,11.0.rc2+1,11.0.rc2+12,11.0.rc2+3,11.0.rc2+4,11.0.rc2+5,11.0.rc2+6,11.0.rc2+7,11.0.rc2+8
LSSTDataManagementBasePackage
Classes | Functions | Variables
lsst.utils.tests Namespace Reference

Classes

class  MemoryTestCase
 Check for memory leaks since memId0 was allocated. More...
 
class  TestCase
 Subclass of unittest.TestCase that adds some custom assertions for convenience. More...
 

Functions

def init
 
def run
 Exit with the status code resulting from running the provided test suite. More...
 
def findFileFromRoot
 Find file which is specified as a path relative to the toplevel directory; we start in $cwd and walk up until we find the file (or throw IOError if it doesn't exist) More...
 
def getTempFilePath
 Return a path suitable for a temporary file and try to delete the file on success. More...
 
def inTestCase
 A decorator to add a free function to our custom TestCase class, while also making it available as a free function. More...
 
def assertRaisesLsstCpp
 
def debugger
 Decorator to enter the debugger when there's an uncaught exception. More...
 
def plotImageDiff
 Plot the comparison of two 2-d NumPy arrays. More...
 
def assertClose
 Highly-configurable floating point comparisons for scalars and arrays. More...
 
def assertNotClose
 Fail a test if the given floating point values are completely equal to within the given tolerances. More...
 

Variables

 dafBase = None
 
int memId0 = 0
 
int nleakPrintMax = 20
 

Function Documentation

def lsst.utils.tests.assertClose (   testCase,
  lhs,
  rhs,
  rtol = sys.float_info.epsilon,
  atol = sys.float_info.epsilon,
  relTo = None,
  printFailures = True,
  plotOnFailure = False,
  plotFileName = None,
  invert = False 
)

Highly-configurable floating point comparisons for scalars and arrays.

The test assertion will fail if all elements lhs and rhs are not equal to within the tolerances specified by rtol and atol. More precisely, the comparison is:

abs(lhs - rhs) <= relTo*rtol OR abs(lhs - rhs) <= atol

If rtol or atol is None, that term in the comparison is not performed at all.

When not specified, relTo is the elementwise maximum of the absolute values of lhs and rhs. If set manually, it should usually be set to either lhs or rhs, or a scalar value typical of what is expected.

Parameters
[in]testCaseunittest.TestCase instance the test is part of
[in]lhsLHS value(s) to compare; may be a scalar or a numpy array of any dimension
[in]rhsRHS value(s) to compare; may be a scalar or a numpy array of any dimension
[in]rtolRelative tolerance for comparison; defaults to double-precision epsilon.
[in]atolAbsolute tolerance for comparison; defaults to double-precision epsilon.
[in]relToValue to which comparison with rtol is relative.
[in]printFailuresUpon failure, print all inequal elements as part of the message.
[in]plotOnFailureUpon failure, plot the originals and their residual with matplotlib. Only 2-d arrays are supported.
[in]plotFileNameFilename to save the plot to. If None, the plot will be displayed in a a window.
[in]invertIf True, invert the comparison and fail only if any elements are equal. Used to implement assertNotClose, which should generally be used instead for clarity.

Definition at line 285 of file tests.py.

286  printFailures=True, plotOnFailure=False, plotFileName=None, invert=False):
287  """!Highly-configurable floating point comparisons for scalars and arrays.
288 
289  The test assertion will fail if all elements lhs and rhs are not equal to within the tolerances
290  specified by rtol and atol. More precisely, the comparison is:
291 
292  abs(lhs - rhs) <= relTo*rtol OR abs(lhs - rhs) <= atol
293 
294  If rtol or atol is None, that term in the comparison is not performed at all.
295 
296  When not specified, relTo is the elementwise maximum of the absolute values of lhs and rhs. If
297  set manually, it should usually be set to either lhs or rhs, or a scalar value typical of what
298  is expected.
299 
300  @param[in] testCase unittest.TestCase instance the test is part of
301  @param[in] lhs LHS value(s) to compare; may be a scalar or a numpy array of any dimension
302  @param[in] rhs RHS value(s) to compare; may be a scalar or a numpy array of any dimension
303  @param[in] rtol Relative tolerance for comparison; defaults to double-precision epsilon.
304  @param[in] atol Absolute tolerance for comparison; defaults to double-precision epsilon.
305  @param[in] relTo Value to which comparison with rtol is relative.
306  @param[in] printFailures Upon failure, print all inequal elements as part of the message.
307  @param[in] plotOnFailure Upon failure, plot the originals and their residual with matplotlib.
308  Only 2-d arrays are supported.
309  @param[in] plotFileName Filename to save the plot to. If None, the plot will be displayed in a
310  a window.
311  @param[in] invert If True, invert the comparison and fail only if any elements *are* equal.
312  Used to implement assertNotClose, which should generally be used instead
313  for clarity.
314  """
315  if not numpy.isfinite(lhs).all():
316  testCase.fail("Non-finite values in lhs")
317  if not numpy.isfinite(rhs).all():
318  testCase.fail("Non-finite values in rhs")
319  diff = lhs - rhs
320  absDiff = numpy.abs(lhs - rhs)
321  if rtol is not None:
322  if relTo is None:
323  relTo = numpy.maximum(numpy.abs(lhs), numpy.abs(rhs))
324  else:
325  relTo = numpy.abs(relTo)
326  bad = absDiff > rtol*relTo
327  if atol is not None:
328  bad = numpy.logical_and(bad, absDiff > atol)
329  else:
330  if atol is None:
331  raise ValueError("rtol and atol cannot both be None")
332  bad = absDiff > atol
333  failed = numpy.any(bad)
334  if invert:
335  failed = not failed
336  bad = numpy.logical_not(bad)
337  cmpStr = "=="
338  failStr = "are the same"
339  else:
340  cmpStr = "!="
341  failStr = "differ"
342  msg = []
343  if failed:
344  if numpy.isscalar(bad):
345  msg = ["%s %s %s; diff=%s/%s=%s with rtol=%s, atol=%s"
346  % (lhs, cmpStr, rhs, absDiff, relTo, absDiff/relTo, rtol, atol)]
347  else:
348  msg = ["%d/%d elements %s with rtol=%s, atol=%s"
349  % (bad.sum(), bad.size, failStr, rtol, atol)]
350  if plotOnFailure:
351  if len(lhs.shape) != 2 or len(rhs.shape) != 2:
352  raise ValueError("plotOnFailure is only valid for 2-d arrays")
353  try:
354  plotImageDiff(lhs, rhs, bad, diff=diff, plotFileName=plotFileName)
355  except ImportError:
356  msg.append("Failure plot requested but matplotlib could not be imported.")
357  if printFailures:
358  # Make sure everything is an array if any of them are, so we can treat
359  # them the same (diff and absDiff are arrays if either rhs or lhs is),
360  # and we don't get here if neither is.
361  if numpy.isscalar(relTo):
362  relTo = numpy.ones(bad.shape, dtype=float) * relTo
363  if numpy.isscalar(lhs):
364  lhs = numpy.ones(bad.shape, dtype=float) * lhs
365  if numpy.isscalar(rhs):
366  rhs = numpy.ones(bad.shape, dtype=float) * rhs
367  for a, b, diff, rel in zip(lhs[bad], rhs[bad], absDiff[bad], relTo[bad]):
368  msg.append("%s %s %s (diff=%s/%s=%s)" % (a, cmpStr, b, diff, rel, diff/rel))
369  testCase.assertFalse(failed, msg="\n".join(msg))
370 
@inTestCase
bool all(CoordinateExpr< N > const &expr)
Return true if all elements are true.
def plotImageDiff
Plot the comparison of two 2-d NumPy arrays.
Definition: tests.py:229
def lsst.utils.tests.assertNotClose (   testCase,
  lhs,
  rhs,
  kwds 
)

Fail a test if the given floating point values are completely equal to within the given tolerances.

See assertClose for more information.

Definition at line 371 of file tests.py.

372 def assertNotClose(testCase, lhs, rhs, **kwds):
373  """!Fail a test if the given floating point values are completely equal to within the given tolerances.
374 
375  See assertClose for more information.
376  """
377  return assertClose(testCase, lhs, rhs, invert=True, **kwds)
def assertClose
Highly-configurable floating point comparisons for scalars and arrays.
Definition: tests.py:285
def assertNotClose
Fail a test if the given floating point values are completely equal to within the given tolerances...
Definition: tests.py:371
def lsst.utils.tests.assertRaisesLsstCpp (   testcase,
  excClass,
  callableObj,
  args,
  kwargs 
)

Definition at line 194 of file tests.py.

195 def assertRaisesLsstCpp(testcase, excClass, callableObj, *args, **kwargs):
196  warnings.warn("assertRaisesLsstCpp is deprecated; please just use TestCase.assertRaises",
197  DeprecationWarning)
198  return testcase.assertRaises(excClass, callableObj, *args, **kwargs)
199 
200 #-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
def assertRaisesLsstCpp
Definition: tests.py:194
def lsst.utils.tests.debugger (   exceptions)

Decorator to enter the debugger when there's an uncaught exception.

To use, just slap a "@debugger()" on your function.

You may provide specific exception classes to catch as arguments to the decorator function, e.g., "@debugger(RuntimeError, NotImplementedError)". This defaults to just 'AssertionError', for use on unittest.TestCase methods.

Code provided by "Rosh Oxymoron" on StackOverflow: http://stackoverflow.com/questions/4398967/python-unit-testing-automatically-running-the-debugger-when-a-test-fails

Definition at line 202 of file tests.py.

203 def debugger(*exceptions):
204  """!Decorator to enter the debugger when there's an uncaught exception
205 
206  To use, just slap a "@debugger()" on your function.
207 
208  You may provide specific exception classes to catch as arguments to
209  the decorator function, e.g., "@debugger(RuntimeError, NotImplementedError)".
210  This defaults to just 'AssertionError', for use on unittest.TestCase methods.
211 
212  Code provided by "Rosh Oxymoron" on StackOverflow:
213  http://stackoverflow.com/questions/4398967/python-unit-testing-automatically-running-the-debugger-when-a-test-fails
214  """
215  if not exceptions:
216  exceptions = (AssertionError, )
217  def decorator(f):
218  @functools.wraps(f)
219  def wrapper(*args, **kwargs):
220  try:
221  return f(*args, **kwargs)
222  except exceptions:
223  import sys, pdb
224  pdb.post_mortem(sys.exc_info()[2])
225  return wrapper
226  return decorator
227 
228 #-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
def debugger
Decorator to enter the debugger when there&#39;s an uncaught exception.
Definition: tests.py:202
def lsst.utils.tests.findFileFromRoot (   ifile)

Find file which is specified as a path relative to the toplevel directory; we start in $cwd and walk up until we find the file (or throw IOError if it doesn't exist)

This is useful for running tests that may be run from dir/tests or dir

Definition at line 92 of file tests.py.

92 
93 def findFileFromRoot(ifile):
94  """!Find file which is specified as a path relative to the toplevel directory;
95  we start in $cwd and walk up until we find the file (or throw IOError if it doesn't exist)
96 
97  This is useful for running tests that may be run from _dir_/tests or _dir_"""
98 
99  if os.path.isfile(ifile):
100  return ifile
101 
102  ofile = None
103  file = ifile
104  while file != "":
105  dirname, basename = os.path.split(file)
106  if ofile:
107  ofile = os.path.join(basename, ofile)
108  else:
109  ofile = basename
110 
111  if os.path.isfile(ofile):
112  return ofile
113 
114  file = dirname
115 
116  raise IOError, "Can't find %s" % ifile
117 
118 #-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
119 
@contextmanager
def findFileFromRoot
Find file which is specified as a path relative to the toplevel directory; we start in $cwd and walk ...
Definition: tests.py:92
def lsst.utils.tests.getTempFilePath (   ext)

Return a path suitable for a temporary file and try to delete the file on success.

If the with block completes successfully then the file is deleted, if possible; failure results in a printed warning. If the block exits with an exception the file if left on disk so it can be examined.

Parameters
[in]extfile name extension, e.g. ".fits"
Returns
path for a temporary file. The path is a combination of the caller's file path and the name of the top-level function, as per this simple example:
1 # file tests/testFoo.py
2 import unittest
3 import lsst.utils.tests
4 class FooTestCase(unittest.TestCase):
5  def testBasics(self):
6  self.runTest()
7 
8  def runTest(self):
9  with lsst.utils.tests.getTempFilePath(".fits") as tmpFile:
10  # if tests/.tests exists then tmpFile = "tests/.tests/testFoo_testBasics.fits"
11  # otherwise tmpFile = "testFoo_testBasics.fits"
12  ...
13  # at the end of this "with" block the path tmpFile will be deleted, but only if
14  # the file exists and the "with" block terminated normally (rather than with an exception)
15 ...

Definition at line 120 of file tests.py.

121 def getTempFilePath(ext):
122  """!Return a path suitable for a temporary file and try to delete the file on success
123 
124  If the with block completes successfully then the file is deleted, if possible;
125  failure results in a printed warning.
126  If the block exits with an exception the file if left on disk so it can be examined.
127 
128  @param[in] ext file name extension, e.g. ".fits"
129  @return path for a temporary file. The path is a combination of the caller's file path
130  and the name of the top-level function, as per this simple example:
131  @code
132  # file tests/testFoo.py
133  import unittest
134  import lsst.utils.tests
135  class FooTestCase(unittest.TestCase):
136  def testBasics(self):
137  self.runTest()
138 
139  def runTest(self):
140  with lsst.utils.tests.getTempFilePath(".fits") as tmpFile:
141  # if tests/.tests exists then tmpFile = "tests/.tests/testFoo_testBasics.fits"
142  # otherwise tmpFile = "testFoo_testBasics.fits"
143  ...
144  # at the end of this "with" block the path tmpFile will be deleted, but only if
145  # the file exists and the "with" block terminated normally (rather than with an exception)
146  ...
147  @endcode
148  """
149  stack = inspect.stack()
150  # get name of first function in the file
151  for i in range(2, len(stack)):
152  frameInfo = inspect.getframeinfo(stack[i][0])
153  if i == 2:
154  callerFilePath = frameInfo.filename
155  callerFuncName = frameInfo.function
156  elif callerFilePath == frameInfo.filename:
157  # this function called the previous function
158  callerFuncName = frameInfo.function
159  else:
160  break
161 
162  callerDir, callerFileNameWithExt = os.path.split(callerFilePath)
163  callerFileName = os.path.splitext(callerFileNameWithExt)[0]
164  outDir = os.path.join(callerDir, ".tests")
165  if not os.path.isdir(outDir):
166  outDir = ""
167  outName = "%s_%s%s" % (callerFileName, callerFuncName, ext)
168  outPath = os.path.join(outDir, outName)
169  yield outPath
170  if os.path.isfile(outPath):
171  try:
172  os.remove(outPath)
173  except OSError as e:
174  print "Warning: could not remove file %r: %s" % (outPath, e)
175  else:
176  print "Warning: could not find file %r" % (outPath,)
177 
178 #-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
def getTempFilePath
Return a path suitable for a temporary file and try to delete the file on success.
Definition: tests.py:120
def lsst.utils.tests.init ( )

Definition at line 46 of file tests.py.

46 
47 def init():
48  global memId0
49  if dafBase:
50  memId0 = dafBase.Citizen_getNextMemId() # used by MemoryTestCase
def lsst.utils.tests.inTestCase (   func)

A decorator to add a free function to our custom TestCase class, while also making it available as a free function.

Definition at line 184 of file tests.py.

185 def inTestCase(func):
186  """!A decorator to add a free function to our custom TestCase class, while also
187  making it available as a free function.
188  """
189  setattr(TestCase, func.__name__, func)
190  return func
191 
192 #-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
193 
@inTestCase
def inTestCase
A decorator to add a free function to our custom TestCase class, while also making it available as a ...
Definition: tests.py:184
def lsst.utils.tests.plotImageDiff (   lhs,
  rhs,
  bad = None,
  diff = None,
  plotFileName = None 
)

Plot the comparison of two 2-d NumPy arrays.

NOTE: this method uses matplotlib and imports it internally; it should be wrapped in a try/except block within packages that do not depend on matplotlib (including utils).

Parameters
[in]lhsLHS values to compare; a 2-d NumPy array
[in]rhsRHS values to compare; a 2-d NumPy array
[in]badA 2-d boolean NumPy array of values to emphasize in the plots
[in]diffdifference array; a 2-d NumPy array, or None to show lhs-rhs
[in]plotFileNameFilename to save the plot to. If None, the plot will be displayed in a a window.

Definition at line 229 of file tests.py.

230 def plotImageDiff(lhs, rhs, bad=None, diff=None, plotFileName=None):
231  """!Plot the comparison of two 2-d NumPy arrays.
232 
233  NOTE: this method uses matplotlib and imports it internally; it should be
234  wrapped in a try/except block within packages that do not depend on
235  matplotlib (including utils).
236 
237  @param[in] lhs LHS values to compare; a 2-d NumPy array
238  @param[in] rhs RHS values to compare; a 2-d NumPy array
239  @param[in] bad A 2-d boolean NumPy array of values to emphasize in the plots
240  @param[in] diff difference array; a 2-d NumPy array, or None to show lhs-rhs
241  @param[in] plotFileName Filename to save the plot to. If None, the plot will be displayed in a
242  a window.
243  """
244  from matplotlib import pyplot
245  if diff is None:
246  diff = lhs - rhs
247  pyplot.figure()
248  if bad is not None:
249  # make an rgba image that's red and transparent where not bad
250  badImage = numpy.zeros(bad.shape + (4,), dtype=numpy.uint8)
251  badImage[:,:,0] = 255
252  badImage[:,:,1] = 0
253  badImage[:,:,2] = 0
254  badImage[:,:,3] = 255*bad
255  vmin1 = numpy.minimum(numpy.min(lhs), numpy.min(rhs))
256  vmax1 = numpy.maximum(numpy.max(lhs), numpy.max(rhs))
257  vmin2 = numpy.min(diff)
258  vmax2 = numpy.max(diff)
259  for n, (image, title) in enumerate([(lhs, "lhs"), (rhs, "rhs"), (diff, "diff")]):
260  pyplot.subplot(2,3,n+1)
261  im1 = pyplot.imshow(image, cmap=pyplot.cm.gray, interpolation='nearest', origin='lower',
262  vmin=vmin1, vmax=vmax1)
263  if bad is not None:
264  pyplot.imshow(badImage, alpha=0.2, interpolation='nearest', origin='lower')
265  pyplot.axis("off")
266  pyplot.title(title)
267  pyplot.subplot(2,3,n+4)
268  im2 = pyplot.imshow(image, cmap=pyplot.cm.gray, interpolation='nearest', origin='lower',
269  vmin=vmin2, vmax=vmax2)
270  if bad is not None:
271  pyplot.imshow(badImage, alpha=0.2, interpolation='nearest', origin='lower')
272  pyplot.axis("off")
273  pyplot.title(title)
274  pyplot.subplots_adjust(left=0.05, bottom=0.05, top=0.92, right=0.75, wspace=0.05, hspace=0.05)
275  cax1 = pyplot.axes([0.8, 0.55, 0.05, 0.4])
276  pyplot.colorbar(im1, cax=cax1)
277  cax2 = pyplot.axes([0.8, 0.05, 0.05, 0.4])
278  pyplot.colorbar(im2, cax=cax2)
279  if plotFileName:
280  pyplot.savefig(plotFileName)
281  else:
282  pyplot.show()
283 
@inTestCase
def plotImageDiff
Plot the comparison of two 2-d NumPy arrays.
Definition: tests.py:229
def lsst.utils.tests.run (   suite,
  exit = True 
)

Exit with the status code resulting from running the provided test suite.

Definition at line 51 of file tests.py.

51 
52 def run(suite, exit=True):
53  """!Exit with the status code resulting from running the provided test suite"""
54 
55  if unittest.TextTestRunner().run(suite).wasSuccessful():
56  status = 0
57  else:
58  status = 1
59 
60  if exit:
61  sys.exit(status)
62  else:
63  return status
64 
65 #-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
def run
Exit with the status code resulting from running the provided test suite.
Definition: tests.py:51

Variable Documentation

lsst.utils.tests.dafBase = None

Definition at line 38 of file tests.py.

int lsst.utils.tests.memId0 = 0

Definition at line 43 of file tests.py.

int lsst.utils.tests.nleakPrintMax = 20

Definition at line 44 of file tests.py.