lsst.daf.persistence.fsScanner.FsScanner Class Reference

Public Member Functions
def	__init__ (self, pathTemplate)
def	getFields (self)
def	isNumeric (self, name)
def	isInt (self, name)
def	isFloat (self, name)
def	processPath (self, location)

Public Attributes
	globString
	fields
	reString

Detailed Description

Class to scan a filesystem location for paths matching a template.

Decomposes the resulting paths into fields and passes them to a callback
function.

Definition at line 32 of file fsScanner.py.

Constructor & Destructor Documentation

__init__()

def lsst.daf.persistence.fsScanner.FsScanner.__init__	(		self,
			pathTemplate
	)

Constructor.  Takes the path template, which should be in the form
of a Python string with named format substitution specifications.
Such a template would be suitable for generating a path given a set of
fields in a dictionary.  Does not handle hex (%x or %X).

Example:
    %(field)s/%(visit)d/%(exposure)d/raw-%(visit)d-e%(exposure)03d-c%(ccd)03d-a%(amp)03d.fits

Note that fields may appear multiple times; the second and subsequent
appearances of such fields will have "_{number}" appended to them to
disambiguate, although it is typically assumed that they will all be
identical.

Trailing brackets (and their contents) can be used to indicate which HDU from a file should
be used. They will not be included in the filename search.

Definition at line 39 of file fsScanner.py.

    def __init__(self, pathTemplate):
        """Constructor.  Takes the path template, which should be in the form
        of a Python string with named format substitution specifications.
        Such a template would be suitable for generating a path given a set of
        fields in a dictionary.  Does not handle hex (%x or %X).
 
        Example:
            %(field)s/%(visit)d/%(exposure)d/raw-%(visit)d-e%(exposure)03d-c%(ccd)03d-a%(amp)03d.fits
 
        Note that fields may appear multiple times; the second and subsequent
        appearances of such fields will have "_{number}" appended to them to
        disambiguate, although it is typically assumed that they will all be
        identical.
 
        Trailing brackets (and their contents) can be used to indicate which HDU from a file should
        be used. They will not be included in the filename search.
        """
 
        # Trim any trailing braces off the end of the path template.
        if pathTemplate.endswith(']'):
            pathTemplate = pathTemplate[0:pathTemplate.rfind('[')]
 
        # Change template into a globbable path specification.
        fmt = re.compile(r'%\‍((\w+)\‍).*?([dioueEfFgGcrs])')
 
        self.globString = fmt.sub('*', pathTemplate)
 
        # Change template into a regular expression.
        last = 0
        self.fields = {}
        self.reString = ""
        n = 0
        pos = 0
        for m in fmt.finditer(pathTemplate):
            fieldName = m.group(1)
            if fieldName in self.fields:
                fieldName += "_%d" % (n,)
                n += 1
 
            prefix = pathTemplate[last:m.start(0)]
            last = m.end(0)
            self.reString += prefix
 
            if m.group(2) in 'crs':
                fieldType = str
                self.reString += r'(?P<' + fieldName + '>.+)'
            elif m.group(2) in 'eEfFgG':
                fieldType = float
                self.reString += r'(?P<' + fieldName + r'>[\d.eE+-]+)'
            else:
                fieldType = int
                self.reString += r'(?P<' + fieldName + r'>[\d+-]+)'
 
            self.fields[fieldName] = dict(pos=pos, fieldType=fieldType)
            pos += 1
 
        self.reString += pathTemplate[last:]
 

Member Function Documentation

getFields()

def lsst.daf.persistence.fsScanner.FsScanner.getFields

(

self

)

Return the list of fields that will be returned from matched
paths, in order.

Definition at line 97 of file fsScanner.py.

    def getFields(self):
        """Return the list of fields that will be returned from matched
        paths, in order."""
 
        fieldList = ["" for i in range(len(self.fields))]
        for f in list(self.fields.keys()):
            fieldList[self.fields[f]['pos']] = f
        return fieldList
 

isFloat()

def lsst.daf.persistence.fsScanner.FsScanner.isFloat	(		self,
			name
	)

Return true if the given field contains an float.

Definition at line 116 of file fsScanner.py.

    def isFloat(self, name):
        """Return true if the given field contains an float."""
 
        return self.fields[name]['fieldType'] == float
 

isInt()

def lsst.daf.persistence.fsScanner.FsScanner.isInt	(		self,
			name
	)

Return true if the given field contains an integer.

Definition at line 111 of file fsScanner.py.

    def isInt(self, name):
        """Return true if the given field contains an integer."""
 
        return self.fields[name]['fieldType'] == int
 

isNumeric()

def lsst.daf.persistence.fsScanner.FsScanner.isNumeric	(		self,
			name
	)

Return true if the given field contains a number.

Definition at line 106 of file fsScanner.py.

    def isNumeric(self, name):
        """Return true if the given field contains a number."""
 
        return self.fields[name]['fieldType'] in (float, int)
 

processPath()

def lsst.daf.persistence.fsScanner.FsScanner.processPath	(		self,
			location
	)

Scan a given path location. Return info about paths that conform to the path template:
:param location:
:return: Path info: {path: {key:value ...}, ...} e.g.:
    {'0239622/instcal0239622.fits.fz': {'visit_0': 239622, 'visit': 239622}}

Definition at line 121 of file fsScanner.py.

    def processPath(self, location):
        """
        Scan a given path location. Return info about paths that conform to the path template:
        :param location:
        :return: Path info: {path: {key:value ...}, ...} e.g.:
            {'0239622/instcal0239622.fits.fz': {'visit_0': 239622, 'visit': 239622}}
        """
        ret = {}
        curdir = os.getcwd()
        os.chdir(location)
        pathList = glob.glob(self.globString)
        for path in pathList:
            m = re.search(self.reString, path)
            if m:
                dataId = m.groupdict()
                for f in self.fields:
                    if self.isInt(f):
                        dataId[f] = int(dataId[f])
                    elif self.isFloat(f):
                        dataId[f] = float(dataId[f])
                ret[path] = dataId
            else:
                print("Warning: unmatched path: %s" % (path,), file=sys.stderr)
        os.chdir(curdir)
        return ret

Member Data Documentation

fields

lsst.daf.persistence.fsScanner.FsScanner.fields

Definition at line 68 of file fsScanner.py.

globString

lsst.daf.persistence.fsScanner.FsScanner.globString

Definition at line 64 of file fsScanner.py.

reString

lsst.daf.persistence.fsScanner.FsScanner.reString

Definition at line 69 of file fsScanner.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-0.7.0/Linux64/daf_persistence/22.0.1-4-g243d05b+871c1b8305/python/lsst/daf/persistence/fsScanner.py