lsst.base.packages Namespace Reference

Classes
class	Packages

Functions
def	getVersionFromPythonModule
def	getPythonPackages
def	getEnvironmentPackages

Variables
list	__all__ = ["getVersionFromPythonModule", "getPythonPackages", "getEnvironmentPackages", "Packages"]
tuple	BUILDTIME = set(["boost", "eigen", "tmv"])
tuple	PYTHON = set(["galsim"])
tuple	ENVIRONMENT = set(["astrometry_net", "astrometry_net_data", "minuit2", "xpa"])
	_eups = None

Detailed Description

Determine which packages are being used in the system and their versions

There are a few different types of packages, and their versions are collected in different ways:
1. Run-time libraries (e.g., cfitsio, fftw): we get their version from interrogating the dynamic library
2. Python modules (e.g., afw, numpy; galsim is also in this group even though we only use it through the
   library, because no version information is currently provided through the library): we get their version
   from the __version__ module variable. Note that this means that we're only aware of modules that have
   already been imported.
3. Other packages provide no run-time accessible version information (e.g., astrometry_net): we get their
   version from interrogating the environment. Currently, that means EUPS; if EUPS is replaced or dropped then
   we'll need to consider an alternative means of getting this version information.
4. Local versions of packages (a non-installed EUPS package, selected with "setup -r /path/to/package"): we
   identify these through the environment (EUPS again) and use as a version the path supplemented with the
   git SHA and, if the git repo isn't clean, an MD5 of the diff.

These package versions are collected and stored in a Packages object, which provides useful comparison and
persistence features.

Example usage:

from lsst.base import Packages
pkgs = Packages.fromSystem()
print "Current packages:", pkgs
old = Packages.read("/path/to/packages.pickle")
print "Old packages:", old
print "Missing packages compared to before:", pkgs.missing(old)
print "Extra packages compared to before:", pkgs.extra(old)
print "Different packages: ", pkgs.difference(old)
old.update(pkgs)  # Include any new packages in the old
old.write("/path/to/packages.pickle")

Function Documentation

def lsst.base.packages.getEnvironmentPackages

(

)

Provide a dict of products and their versions from the environment

We use EUPS to determine the version of certain products (those that don't provide
a means to determine the version any other way) and to check if uninstalled packages
are being used. We only report the product/version for these packages.

Definition at line 132 of file packages.py.

def getEnvironmentPackages():
    """Provide a dict of products and their versions from the environment

    We use EUPS to determine the version of certain products (those that don't provide
    a means to determine the version any other way) and to check if uninstalled packages
    are being used. We only report the product/version for these packages.
    """
    try:
        from eups import Eups
        from eups.Product import Product
    except:
        from lsst.pex.logging import getDefaultLog
        getDefaultLog().warn("Unable to import eups, so cannot determine package versions from environment")
        return {}

    # Cache eups object since creating it can take a while
    global _eups
    if not _eups:
        _eups = Eups()
    products = _eups.findProducts(tags=["setup"])

    # Get versions for things we can't determine via runtime mechanisms
    # XXX Should we just grab everything we can, rather than just a predetermined set?
    packages = {prod.name: prod.version for prod in products if prod in ENVIRONMENT}

    # The string 'LOCAL:' (the value of Product.LocalVersionPrefix) in the version name indicates uninstalled
    # code, so the version could be different than what's being reported by the runtime environment (because
    # we don't tend to run "scons" every time we update some python file, and even if we did sconsUtils
    # probably doesn't check to see if the repo is clean).
    for prod in products:
        if not prod.version.startswith(Product.LocalVersionPrefix):
            continue
        ver = prod.version

        gitDir = os.path.join(prod.dir, ".git")
        if os.path.exists(gitDir):
            # get the git revision and an indication if the working copy is clean
            revCmd = ["git", "--git-dir=" + gitDir, "--work-tree=" + prod.dir, "rev-parse", "HEAD"]
            diffCmd = ["git", "--no-pager", "--git-dir=" + gitDir, "--work-tree=" + prod.dir, "diff",
                       "--patch"]
            try:
                rev = subprocess.check_output(revCmd).decode().strip()
                diff = subprocess.check_output(diffCmd)
            except:
                ver += "@GIT_ERROR"
            else:
                ver += "@" + rev
                if diff:
                    ver += "+" + hashlib.md5(diff).hexdigest()
        else:
            ver += "@NO_GIT"

        packages[prod.name] = ver
    return packages