LSSTApplications  1.1.2+25,10.0+13,10.0+132,10.0+133,10.0+224,10.0+41,10.0+8,10.0-1-g0f53050+14,10.0-1-g4b7b172+19,10.0-1-g61a5bae+98,10.0-1-g7408a83+3,10.0-1-gc1e0f5a+19,10.0-1-gdb4482e+14,10.0-11-g3947115+2,10.0-12-g8719d8b+2,10.0-15-ga3f480f+1,10.0-2-g4f67435,10.0-2-gcb4bc6c+26,10.0-28-gf7f57a9+1,10.0-3-g1bbe32c+14,10.0-3-g5b46d21,10.0-4-g027f45f+5,10.0-4-g86f66b5+2,10.0-4-gc4fccf3+24,10.0-40-g4349866+2,10.0-5-g766159b,10.0-5-gca2295e+25,10.0-6-g462a451+1
LSSTDataManagementBasePackage
Classes | Functions | Variables
lsst.pipe.base.argumentParser Namespace Reference

Classes

class  DataIdContainer
 A container for data IDs and associated data references. More...
 
class  DataIdArgument
 Glorified struct for data about id arguments, used by ArgumentParser.add_id_argument. More...
 
class  DatasetArgument
 Specify that the dataset type should be a command-line option. More...
 
class  ArgumentParser
 An argument parser for pipeline tasks that is based on argparse.ArgumentParser. More...
 
class  ConfigValueAction
 argparse action callback to override config parameters using name=value pairs from the command line More...
 
class  ConfigFileAction
 argparse action to load config overrides from one or more files More...
 
class  IdValueAction
 argparse action callback to process a data ID into a dict More...
 
class  TraceLevelAction
 argparse action to set trace level More...
 

Functions

def _fixPath
 Apply environment variable as default root, if present, and abspath. More...
 
def getTaskDict
 Get a dictionary of task info for all subtasks in a config. More...
 
def obeyShowArgument
 Process arguments specified with –show (but ignores "data") More...
 
def showTaskHierarchy
 Print task hierarchy to stdout. More...
 
def setDottedAttr
 Like setattr, but accepts hierarchical names, e.g. More...
 
def getDottedAttr
 Like getattr, but accepts hierarchical names, e.g. More...
 
def dataExists
 Return True if data exists at the current level or any data exists at a deeper level, False otherwise. More...
 

Variables

list __all__ = ["ArgumentParser", "ConfigFileAction", "ConfigValueAction", "DataIdContainer", "DatasetArgument"]
 
string DEFAULT_INPUT_NAME = "PIPE_INPUT_ROOT"
 
string DEFAULT_CALIB_NAME = "PIPE_CALIB_ROOT"
 
string DEFAULT_OUTPUT_NAME = "PIPE_OUTPUT_ROOT"
 
 _pattern
 

Function Documentation

def lsst.pipe.base.argumentParser._fixPath (   defName,
  path 
)
private

Apply environment variable as default root, if present, and abspath.

Parameters
[in]defNamename of environment variable containing default root path; if the environment variable does not exist then the path is relative to the current working directory
[in]pathpath relative to default root path
Returns
abspath: path that has been expanded, or None if the environment variable does not exist and path is None

Definition at line 44 of file argumentParser.py.

44 
45 def _fixPath(defName, path):
46  """!Apply environment variable as default root, if present, and abspath
47 
48  @param[in] defName name of environment variable containing default root path;
49  if the environment variable does not exist then the path is relative
50  to the current working directory
51  @param[in] path path relative to default root path
52  @return abspath: path that has been expanded, or None if the environment variable does not exist
53  and path is None
54  """
55  defRoot = os.environ.get(defName)
56  if defRoot is None:
57  if path is None:
58  return None
59  return os.path.abspath(path)
60  return os.path.abspath(os.path.join(defRoot, path or ""))
61 
def _fixPath
Apply environment variable as default root, if present, and abspath.
def lsst.pipe.base.argumentParser.dataExists (   butler,
  datasetType,
  dataRef 
)

Return True if data exists at the current level or any data exists at a deeper level, False otherwise.

Parameters
[in]butlerdata butler (a lsst.daf.persistence.Butler)
[in]datasetTypedataset type (a str)
[in]dataRefbutler data reference (a lsst.daf.persistence.ButlerDataRef)

Definition at line 780 of file argumentParser.py.

781 def dataExists(butler, datasetType, dataRef):
782  """!Return True if data exists at the current level or any data exists at a deeper level, False otherwise
783 
784  @param[in] butler data butler (a \ref lsst.daf.persistence.butler.Butler
785  "lsst.daf.persistence.Butler")
786  @param[in] datasetType dataset type (a str)
787  @param[in] dataRef butler data reference (a \ref lsst.daf.persistence.butlerSubset.ButlerDataRef
788  "lsst.daf.persistence.ButlerDataRef")
789  """
790  subDRList = dataRef.subItems()
791  if subDRList:
792  for subDR in subDRList:
793  if dataExists(butler, datasetType, subDR):
794  return True
795  return False
796  else:
797  return butler.datasetExists(datasetType = datasetType, dataId = dataRef.dataId)
def dataExists
Return True if data exists at the current level or any data exists at a deeper level, False otherwise.
def lsst.pipe.base.argumentParser.getDottedAttr (   item,
  name 
)

Like getattr, but accepts hierarchical names, e.g.

foo.bar.baz

Parameters
[in]itemobject whose attribute is to be returned
[in]namename of item to get

For example if name is foo.bar.baz then returns item.foo.bar.baz

Definition at line 767 of file argumentParser.py.

768 def getDottedAttr(item, name):
769  """!Like getattr, but accepts hierarchical names, e.g. foo.bar.baz
770 
771  @param[in] item object whose attribute is to be returned
772  @param[in] name name of item to get
773 
774  For example if name is foo.bar.baz then returns item.foo.bar.baz
775  """
776  subitem = item
777  for subname in name.split("."):
778  subitem = getattr(subitem, subname)
779  return subitem
def getDottedAttr
Like getattr, but accepts hierarchical names, e.g.
def lsst.pipe.base.argumentParser.getTaskDict (   config,
  taskDict = None,
  baseName = "" 
)

Get a dictionary of task info for all subtasks in a config.

Designed to be called recursively; the user should call with only a config (leaving taskDict and baseName at their default values).

Parameters
[in]configconfiguration to process, an instance of lsst.pex.config.Config
[in,out]taskDictusers should not specify this argument; (supports recursion; if provided, taskDict is updated in place, else a new dict is started)
[in]baseNameusers should not specify this argument. (supports recursion: if a non-empty string then a period is appended and the result is used as a prefix for additional entries in taskDict; otherwise no prefix is used)
Returns
taskDict: a dict of config field name: task name

Definition at line 517 of file argumentParser.py.

518 def getTaskDict(config, taskDict=None, baseName=""):
519  """!Get a dictionary of task info for all subtasks in a config
520 
521  Designed to be called recursively; the user should call with only a config
522  (leaving taskDict and baseName at their default values).
523 
524  @param[in] config configuration to process, an instance of lsst.pex.config.Config
525  @param[in,out] taskDict users should not specify this argument;
526  (supports recursion; if provided, taskDict is updated in place, else a new dict is started)
527  @param[in] baseName users should not specify this argument.
528  (supports recursion: if a non-empty string then a period is appended and the result is used
529  as a prefix for additional entries in taskDict; otherwise no prefix is used)
530  @return taskDict: a dict of config field name: task name
531  """
532  if taskDict is None:
533  taskDict = dict()
534  for fieldName, field in config.iteritems():
535  if hasattr(field, "value") and hasattr(field, "target"):
536  subConfig = field.value
537  if isinstance(subConfig, pexConfig.Config):
538  subBaseName = "%s.%s" % (baseName, fieldName) if baseName else fieldName
539  try:
540  taskName = "%s.%s" % (field.target.__module__, field.target.__name__)
541  except Exception:
542  taskName = repr(field.target)
543  taskDict[subBaseName] = taskName
544  getTaskDict(config=subConfig, taskDict=taskDict, baseName=subBaseName)
545  return taskDict
def getTaskDict
Get a dictionary of task info for all subtasks in a config.
def lsst.pipe.base.argumentParser.obeyShowArgument (   showOpts,
  config = None,
  exit = False 
)

Process arguments specified with –show (but ignores "data")

Parameters
showOptsList of options passed to –show
configThe provided config
exitExit if "run" isn't included in showOpts

Supports the following options in showOpts:

Calls sys.exit(1) if any other option found.

Definition at line 546 of file argumentParser.py.

547 def obeyShowArgument(showOpts, config=None, exit=False):
548  """!Process arguments specified with --show (but ignores "data")
549 
550  @param showOpts List of options passed to --show
551  @param config The provided config
552  @param exit Exit if "run" isn't included in showOpts
553 
554  Supports the following options in showOpts:
555  - config[=PAT] Dump all the config entries, or just the ones that match the glob pattern
556  - tasks Show task hierarchy
557  - data Ignored; to be processed by caller
558  - run Keep going (the default behaviour is to exit if --show is specified)
559 
560  Calls sys.exit(1) if any other option found.
561  """
562  if not showOpts:
563  return
564 
565  for what in showOpts:
566  mat = re.search(r"^config(?:=(.+))?", what)
567  if mat:
568  pattern = mat.group(1)
569  if pattern:
570  class FilteredStream(object):
571  """A file object that only prints lines that match the glob "pattern"
572 
573  N.b. Newlines are silently discarded and reinserted; crude but effective.
574  """
575  def __init__(self, pattern):
576  self._pattern = pattern
577 
578  def write(self, str):
579  str = str.rstrip()
580  if str and fnmatch.fnmatch(str, self._pattern):
581  print str
582 
583  fd = FilteredStream(pattern)
584  else:
585  fd = sys.stdout
586 
587  config.saveToStream(fd, "config")
588  elif what == "data":
589  pass
590  elif what == "run":
591  pass
592  elif what == "tasks":
593  showTaskHierarchy(config)
594  else:
595  print >> sys.stderr, "Unknown value for show: %s (choose from '%s')" % \
596  (what, "', '".join("config[=XXX] data tasks run".split()))
597  sys.exit(1)
598 
599  if exit and "run" not in showOpts:
600  sys.exit(0)
def showTaskHierarchy
Print task hierarchy to stdout.
def obeyShowArgument
Process arguments specified with –show (but ignores "data")
def lsst.pipe.base.argumentParser.setDottedAttr (   item,
  name,
  value 
)

Like setattr, but accepts hierarchical names, e.g.

foo.bar.baz

Parameters
[in,out]itemobject whose attribute is to be set
[in]namename of item to set
[in]valuenew value for the item

For example if name is foo.bar.baz then item.foo.bar.baz is set to the specified value.

Definition at line 752 of file argumentParser.py.

753 def setDottedAttr(item, name, value):
754  """!Like setattr, but accepts hierarchical names, e.g. foo.bar.baz
755 
756  @param[in,out] item object whose attribute is to be set
757  @param[in] name name of item to set
758  @param[in] value new value for the item
759 
760  For example if name is foo.bar.baz then item.foo.bar.baz is set to the specified value.
761  """
762  subitem = item
763  subnameList = name.split(".")
764  for subname in subnameList[:-1]:
765  subitem = getattr(subitem, subname)
766  setattr(subitem, subnameList[-1], value)
def setDottedAttr
Like setattr, but accepts hierarchical names, e.g.
def lsst.pipe.base.argumentParser.showTaskHierarchy (   config)

Print task hierarchy to stdout.

Parameters
[in]config:configuration to process (an lsst.pex.config.Config)

Definition at line 601 of file argumentParser.py.

602 def showTaskHierarchy(config):
603  """!Print task hierarchy to stdout
604 
605  @param[in] config: configuration to process (an lsst.pex.config.Config)
606  """
607  print "Subtasks:"
608  taskDict = getTaskDict(config=config)
609 
610  fieldNameList = sorted(taskDict.keys())
611  for fieldName in fieldNameList:
612  taskName = taskDict[fieldName]
613  print "%s: %s" % (fieldName, taskName)
def showTaskHierarchy
Print task hierarchy to stdout.
def getTaskDict
Get a dictionary of task info for all subtasks in a config.

Variable Documentation

list lsst.pipe.base.argumentParser.__all__ = ["ArgumentParser", "ConfigFileAction", "ConfigValueAction", "DataIdContainer", "DatasetArgument"]

Definition at line 38 of file argumentParser.py.

lsst.pipe.base.argumentParser._pattern

Definition at line 575 of file argumentParser.py.

string lsst.pipe.base.argumentParser.DEFAULT_CALIB_NAME = "PIPE_CALIB_ROOT"

Definition at line 41 of file argumentParser.py.

string lsst.pipe.base.argumentParser.DEFAULT_INPUT_NAME = "PIPE_INPUT_ROOT"

Definition at line 40 of file argumentParser.py.

string lsst.pipe.base.argumentParser.DEFAULT_OUTPUT_NAME = "PIPE_OUTPUT_ROOT"

Definition at line 42 of file argumentParser.py.