LSSTApplications  10.0+286,10.0+36,10.0+46,10.0-2-g4f67435,10.1+152,10.1+37,11.0,11.0+1,11.0-1-g47edd16,11.0-1-g60db491,11.0-1-g7418c06,11.0-2-g04d2804,11.0-2-g68503cd,11.0-2-g818369d,11.0-2-gb8b8ce7
LSSTDataManagementBasePackage
Classes | Functions | Variables
lsst.pipe.base.argumentParser Namespace Reference

Classes

class  DataIdContainer
 A container for data IDs and associated data references. More...
 
class  DataIdArgument
 Glorified struct for data about id arguments, used by ArgumentParser.add_id_argument. More...
 
class  DatasetArgument
 Specify that the dataset type should be a command-line option. More...
 
class  ArgumentParser
 An argument parser for pipeline tasks that is based on argparse.ArgumentParser. More...
 
class  ConfigValueAction
 argparse action callback to override config parameters using name=value pairs from the command line More...
 
class  ConfigFileAction
 argparse action to load config overrides from one or more files More...
 
class  IdValueAction
 argparse action callback to process a data ID into a dict More...
 
class  LogLevelAction
 argparse action to set log level More...
 

Functions

def _fixPath
 Apply environment variable as default root, if present, and abspath. More...
 
def getTaskDict
 Get a dictionary of task info for all subtasks in a config. More...
 
def obeyShowArgument
 Process arguments specified with –show (but ignores "data") More...
 
def showTaskHierarchy
 Print task hierarchy to stdout. More...
 
def setDottedAttr
 Like setattr, but accepts hierarchical names, e.g. More...
 
def getDottedAttr
 Like getattr, but accepts hierarchical names, e.g. More...
 
def dataExists
 Return True if data exists at the current level or any data exists at a deeper level, False otherwise. More...
 

Variables

list __all__ = ["ArgumentParser", "ConfigFileAction", "ConfigValueAction", "DataIdContainer", "DatasetArgument"]
 
string DEFAULT_INPUT_NAME = "PIPE_INPUT_ROOT"
 
string DEFAULT_CALIB_NAME = "PIPE_CALIB_ROOT"
 
string DEFAULT_OUTPUT_NAME = "PIPE_OUTPUT_ROOT"
 
 _pattern
 

Function Documentation

def lsst.pipe.base.argumentParser._fixPath (   defName,
  path 
)
private

Apply environment variable as default root, if present, and abspath.

Parameters
[in]defNamename of environment variable containing default root path; if the environment variable does not exist then the path is relative to the current working directory
[in]pathpath relative to default root path
Returns
abspath: path that has been expanded, or None if the environment variable does not exist and path is None

Definition at line 44 of file argumentParser.py.

44 
45 def _fixPath(defName, path):
46  """!Apply environment variable as default root, if present, and abspath
47 
48  @param[in] defName name of environment variable containing default root path;
49  if the environment variable does not exist then the path is relative
50  to the current working directory
51  @param[in] path path relative to default root path
52  @return abspath: path that has been expanded, or None if the environment variable does not exist
53  and path is None
54  """
55  defRoot = os.environ.get(defName)
56  if defRoot is None:
57  if path is None:
58  return None
59  return os.path.abspath(path)
60  return os.path.abspath(os.path.join(defRoot, path or ""))
61 
def _fixPath
Apply environment variable as default root, if present, and abspath.
def lsst.pipe.base.argumentParser.dataExists (   butler,
  datasetType,
  dataRef 
)

Return True if data exists at the current level or any data exists at a deeper level, False otherwise.

Parameters
[in]butlerdata butler (a lsst.daf.persistence.Butler)
[in]datasetTypedataset type (a str)
[in]dataRefbutler data reference (a lsst.daf.persistence.ButlerDataRef)

Definition at line 779 of file argumentParser.py.

780 def dataExists(butler, datasetType, dataRef):
781  """!Return True if data exists at the current level or any data exists at a deeper level, False otherwise
782 
783  @param[in] butler data butler (a \ref lsst.daf.persistence.butler.Butler
784  "lsst.daf.persistence.Butler")
785  @param[in] datasetType dataset type (a str)
786  @param[in] dataRef butler data reference (a \ref lsst.daf.persistence.butlerSubset.ButlerDataRef
787  "lsst.daf.persistence.ButlerDataRef")
788  """
789  subDRList = dataRef.subItems()
790  if subDRList:
791  for subDR in subDRList:
792  if dataExists(butler, datasetType, subDR):
793  return True
794  return False
795  else:
796  return butler.datasetExists(datasetType = datasetType, dataId = dataRef.dataId)
def dataExists
Return True if data exists at the current level or any data exists at a deeper level, False otherwise.
def lsst.pipe.base.argumentParser.getDottedAttr (   item,
  name 
)

Like getattr, but accepts hierarchical names, e.g.

foo.bar.baz

Parameters
[in]itemobject whose attribute is to be returned
[in]namename of item to get

For example if name is foo.bar.baz then returns item.foo.bar.baz

Definition at line 766 of file argumentParser.py.

767 def getDottedAttr(item, name):
768  """!Like getattr, but accepts hierarchical names, e.g. foo.bar.baz
769 
770  @param[in] item object whose attribute is to be returned
771  @param[in] name name of item to get
772 
773  For example if name is foo.bar.baz then returns item.foo.bar.baz
774  """
775  subitem = item
776  for subname in name.split("."):
777  subitem = getattr(subitem, subname)
778  return subitem
def getDottedAttr
Like getattr, but accepts hierarchical names, e.g.
def lsst.pipe.base.argumentParser.getTaskDict (   config,
  taskDict = None,
  baseName = "" 
)

Get a dictionary of task info for all subtasks in a config.

Designed to be called recursively; the user should call with only a config (leaving taskDict and baseName at their default values).

Parameters
[in]configconfiguration to process, an instance of lsst.pex.config.Config
[in,out]taskDictusers should not specify this argument; (supports recursion; if provided, taskDict is updated in place, else a new dict is started)
[in]baseNameusers should not specify this argument. (supports recursion: if a non-empty string then a period is appended and the result is used as a prefix for additional entries in taskDict; otherwise no prefix is used)
Returns
taskDict: a dict of config field name: task name

Definition at line 505 of file argumentParser.py.

506 def getTaskDict(config, taskDict=None, baseName=""):
507  """!Get a dictionary of task info for all subtasks in a config
508 
509  Designed to be called recursively; the user should call with only a config
510  (leaving taskDict and baseName at their default values).
511 
512  @param[in] config configuration to process, an instance of lsst.pex.config.Config
513  @param[in,out] taskDict users should not specify this argument;
514  (supports recursion; if provided, taskDict is updated in place, else a new dict is started)
515  @param[in] baseName users should not specify this argument.
516  (supports recursion: if a non-empty string then a period is appended and the result is used
517  as a prefix for additional entries in taskDict; otherwise no prefix is used)
518  @return taskDict: a dict of config field name: task name
519  """
520  if taskDict is None:
521  taskDict = dict()
522  for fieldName, field in config.iteritems():
523  if hasattr(field, "value") and hasattr(field, "target"):
524  subConfig = field.value
525  if isinstance(subConfig, pexConfig.Config):
526  subBaseName = "%s.%s" % (baseName, fieldName) if baseName else fieldName
527  try:
528  taskName = "%s.%s" % (field.target.__module__, field.target.__name__)
529  except Exception:
530  taskName = repr(field.target)
531  taskDict[subBaseName] = taskName
532  getTaskDict(config=subConfig, taskDict=taskDict, baseName=subBaseName)
533  return taskDict
def getTaskDict
Get a dictionary of task info for all subtasks in a config.
def lsst.pipe.base.argumentParser.obeyShowArgument (   showOpts,
  config = None,
  exit = False 
)

Process arguments specified with –show (but ignores "data")

Parameters
showOptsList of options passed to –show
configThe provided config
exitExit if "run" isn't included in showOpts

Supports the following options in showOpts:

Calls sys.exit(1) if any other option found.

Definition at line 534 of file argumentParser.py.

535 def obeyShowArgument(showOpts, config=None, exit=False):
536  """!Process arguments specified with --show (but ignores "data")
537 
538  @param showOpts List of options passed to --show
539  @param config The provided config
540  @param exit Exit if "run" isn't included in showOpts
541 
542  Supports the following options in showOpts:
543  - config[=PAT] Dump all the config entries, or just the ones that match the glob pattern
544  - tasks Show task hierarchy
545  - data Ignored; to be processed by caller
546  - run Keep going (the default behaviour is to exit if --show is specified)
547 
548  Calls sys.exit(1) if any other option found.
549  """
550  if not showOpts:
551  return
552 
553  for what in showOpts:
554  mat = re.search(r"^config(?:=(.+))?", what)
555  if mat:
556  pattern = mat.group(1)
557  if pattern:
558  class FilteredStream(object):
559  """A file object that only prints lines that match the glob "pattern"
560 
561  N.b. Newlines are silently discarded and reinserted; crude but effective.
562  """
563  def __init__(self, pattern):
564  self._pattern = pattern
565 
566  def write(self, str):
567  str = str.rstrip()
568  if str and fnmatch.fnmatch(str, self._pattern):
569  print str
570 
571  fd = FilteredStream(pattern)
572  else:
573  fd = sys.stdout
574 
575  config.saveToStream(fd, "config")
576  elif what == "data":
577  pass
578  elif what == "run":
579  pass
580  elif what == "tasks":
581  showTaskHierarchy(config)
582  else:
583  print >> sys.stderr, "Unknown value for show: %s (choose from '%s')" % \
584  (what, "', '".join("config[=XXX] data tasks run".split()))
585  sys.exit(1)
586 
587  if exit and "run" not in showOpts:
588  sys.exit(0)
def showTaskHierarchy
Print task hierarchy to stdout.
def obeyShowArgument
Process arguments specified with –show (but ignores "data")
def lsst.pipe.base.argumentParser.setDottedAttr (   item,
  name,
  value 
)

Like setattr, but accepts hierarchical names, e.g.

foo.bar.baz

Parameters
[in,out]itemobject whose attribute is to be set
[in]namename of item to set
[in]valuenew value for the item

For example if name is foo.bar.baz then item.foo.bar.baz is set to the specified value.

Definition at line 751 of file argumentParser.py.

752 def setDottedAttr(item, name, value):
753  """!Like setattr, but accepts hierarchical names, e.g. foo.bar.baz
754 
755  @param[in,out] item object whose attribute is to be set
756  @param[in] name name of item to set
757  @param[in] value new value for the item
758 
759  For example if name is foo.bar.baz then item.foo.bar.baz is set to the specified value.
760  """
761  subitem = item
762  subnameList = name.split(".")
763  for subname in subnameList[:-1]:
764  subitem = getattr(subitem, subname)
765  setattr(subitem, subnameList[-1], value)
def setDottedAttr
Like setattr, but accepts hierarchical names, e.g.
def lsst.pipe.base.argumentParser.showTaskHierarchy (   config)

Print task hierarchy to stdout.

Parameters
[in]config:configuration to process (an lsst.pex.config.Config)

Definition at line 589 of file argumentParser.py.

590 def showTaskHierarchy(config):
591  """!Print task hierarchy to stdout
592 
593  @param[in] config: configuration to process (an lsst.pex.config.Config)
594  """
595  print "Subtasks:"
596  taskDict = getTaskDict(config=config)
597 
598  fieldNameList = sorted(taskDict.keys())
599  for fieldName in fieldNameList:
600  taskName = taskDict[fieldName]
601  print "%s: %s" % (fieldName, taskName)
def showTaskHierarchy
Print task hierarchy to stdout.
def getTaskDict
Get a dictionary of task info for all subtasks in a config.

Variable Documentation

list lsst.pipe.base.argumentParser.__all__ = ["ArgumentParser", "ConfigFileAction", "ConfigValueAction", "DataIdContainer", "DatasetArgument"]

Definition at line 38 of file argumentParser.py.

lsst.pipe.base.argumentParser._pattern

Definition at line 563 of file argumentParser.py.

string lsst.pipe.base.argumentParser.DEFAULT_CALIB_NAME = "PIPE_CALIB_ROOT"

Definition at line 41 of file argumentParser.py.

string lsst.pipe.base.argumentParser.DEFAULT_INPUT_NAME = "PIPE_INPUT_ROOT"

Definition at line 40 of file argumentParser.py.

string lsst.pipe.base.argumentParser.DEFAULT_OUTPUT_NAME = "PIPE_OUTPUT_ROOT"

Definition at line 42 of file argumentParser.py.