Inheritance diagram for lsst.dax.apdb.apdbSchema.ApdbSchema:

Public Member Functions
def	__init__ (self, engine, dia_object_index, dia_object_nightly, schema_file, extra_schema_file=None, column_map=None, afw_schemas=None, prefix="")

def	makeSchema (self, drop=False, mysql_engine='InnoDB', oracle_tablespace=None, oracle_iot=False)

def	getAfwSchema (self, table_name, columns=None)

def	getAfwColumns (self, table_name)

def	getColumnMap (self, table_name)

Public Attributes
	objects

	objects_nightly

	objects_last

	sources

	forcedSources

	visits

Detailed Description

Class for management of APDB schema.

Attributes
----------
objects : `sqlalchemy.Table`
    DiaObject table instance
objects_nightly : `sqlalchemy.Table`
    DiaObjectNightly table instance, may be None
objects_last : `sqlalchemy.Table`
    DiaObjectLast table instance, may be None
sources : `sqlalchemy.Table`
    DiaSource table instance
forcedSources : `sqlalchemy.Table`
    DiaForcedSource table instance
visits : `sqlalchemy.Table`
    ApdbProtoVisits table instance

Parameters
----------
engine : `sqlalchemy.engine.Engine`
    SQLAlchemy engine instance
dia_object_index : `str`
    Indexing mode for DiaObject table, see `ApdbConfig.dia_object_index`
    for details.
dia_object_nightly : `bool`
    If `True` then create per-night DiaObject table as well.
schema_file : `str`
    Name of the YAML schema file.
extra_schema_file : `str`, optional
    Name of the YAML schema file with extra column definitions.
column_map : `str`, optional
    Name of the YAML file with column mappings.
afw_schemas : `dict`, optional
    Dictionary with table name for a key and `afw.table.Schema`
    for a value. Columns in schema will be added to standard APDB
    schema (only if standard schema does not have matching column).
prefix : `str`, optional
    Prefix to add to all scheam elements.

Definition at line 162 of file apdbSchema.py.

Constructor & Destructor Documentation

◆ init()

def lsst.dax.apdb.apdbSchema.ApdbSchema.__init__	(	self,
		engine,
		dia_object_index,
		dia_object_nightly,
		schema_file,
		extra_schema_file = `None`,
		column_map = `None`,
		afw_schemas = `None`,
		prefix = `""`
	)

Definition at line 219 of file apdbSchema.py.

                  afw_schemas=None, prefix=""):
  
         self._engine = engine
         self._dia_object_index = dia_object_index
         self._dia_object_nightly = dia_object_nightly
         self._prefix = prefix
  
         self._metadata = MetaData(self._engine)
  
         self.objects = None
         self.objects_nightly = None
         self.objects_last = None
         self.sources = None
         self.forcedSources = None
         self.visits = None
  
         if column_map:
             column_map = os.path.expandvars(column_map)
             _LOG.debug("Reading column map file %s", column_map)
             with open(column_map) as yaml_stream:
                 # maps cat column name to afw column name
                 self._column_map = yaml.load(yaml_stream, Loader=yaml.SafeLoader)
                 _LOG.debug("column map: %s", self._column_map)
         else:
             _LOG.debug("No column map file is given, initialize to empty")
             self._column_map = {}
         self._column_map_reverse = {}
         for table, cmap in self._column_map.items():
             # maps afw column name to cat column name
             self._column_map_reverse[table] = {v: k for k, v in cmap.items()}
         _LOG.debug("reverse column map: %s", self._column_map_reverse)
  
         # build complete table schema
         self._schemas = self._buildSchemas(schema_file, extra_schema_file,
                                            afw_schemas)
  
         # map cat column types to alchemy
         self._type_map = dict(DOUBLE=self._getDoubleType(),
                               FLOAT=sqlalchemy.types.Float,
                               DATETIME=sqlalchemy.types.TIMESTAMP,
                               BIGINT=sqlalchemy.types.BigInteger,
                               INTEGER=sqlalchemy.types.Integer,
                               INT=sqlalchemy.types.Integer,
                               TINYINT=sqlalchemy.types.Integer,
                               BLOB=sqlalchemy.types.LargeBinary,
                               CHAR=sqlalchemy.types.CHAR,
                               BOOL=sqlalchemy.types.Boolean)
  
         # generate schema for all tables, must be called last
         self._makeTables()
  

Member Function Documentation

◆ getAfwColumns()

def lsst.dax.apdb.apdbSchema.ApdbSchema.getAfwColumns	(	self,
		table_name
	)

Returns mapping of afw column names to Column definitions.

Parameters
----------
table_name : `str`
    One of known APDB table names.

Returns
-------
column_map : `dict`
    Mapping of afw column names to `ColumnDef` instances.

Definition at line 437 of file apdbSchema.py.

     def getAfwColumns(self, table_name):
         """Returns mapping of afw column names to Column definitions.
  
         Parameters
         ----------
         table_name : `str`
             One of known APDB table names.
  
         Returns
         -------
         column_map : `dict`
             Mapping of afw column names to `ColumnDef` instances.
         """
         table = self._schemas[table_name]
         col_map = self._column_map.get(table_name, {})
  
         cmap = {}
         for column in table.columns:
             afw_name = col_map.get(column.name, column.name)
             cmap[afw_name] = column
         return cmap
  

◆ getAfwSchema()

def lsst.dax.apdb.apdbSchema.ApdbSchema.getAfwSchema	(	self,
		table_name,
		columns = `None`
	)

Return afw schema for given table.

Parameters
----------
table_name : `str`
    One of known APDB table names.
columns : `list` of `str`, optional
    Include only given table columns in schema, by default all columns
    are included.

Returns
-------
schema : `lsst.afw.table.Schema`
column_map : `dict`
    Mapping of the table/result column names into schema key.

Definition at line 369 of file apdbSchema.py.

     def getAfwSchema(self, table_name, columns=None):
         """Return afw schema for given table.
  
         Parameters
         ----------
         table_name : `str`
             One of known APDB table names.
         columns : `list` of `str`, optional
             Include only given table columns in schema, by default all columns
             are included.
  
         Returns
         -------
         schema : `lsst.afw.table.Schema`
         column_map : `dict`
             Mapping of the table/result column names into schema key.
         """
  
         table = self._schemas[table_name]
         col_map = self._column_map.get(table_name, {})
  
         # make a schema
         col2afw = {}
         schema = afwTable.SourceTable.makeMinimalSchema()
         for column in table.columns:
             if columns and column.name not in columns:
                 continue
             afw_col = col_map.get(column.name, column.name)
             if afw_col in schema.getNames():
                 # Continue if the column is already in the minimal schema.
                 key = schema.find(afw_col).getKey()
             elif column.type in ("DOUBLE", "FLOAT") and column.unit == "deg":
                 #
                 # NOTE: degree to radian conversion is not supported (yet)
                 #
                 # angles in afw are radians and have special "Angle" type
                 key = schema.addField(afw_col,
                                       type="Angle",
                                       doc=column.description or "",
                                       units="rad")
             elif column.type == "BLOB":
                 # No BLOB support for now
                 key = None
             else:
                 units = column.unit or ""
                 # some units in schema are not recognized by afw but we do not care
                 if self._afw_type_map_reverse[column.type] == 'String':
                     key = schema.addField(afw_col,
                                           type=self._afw_type_map_reverse[column.type],
                                           doc=column.description or "",
                                           units=units,
                                           parse_strict="silent",
                                           size=10)
                 elif units == "deg":
                     key = schema.addField(afw_col,
                                           type='Angle',
                                           doc=column.description or "",
                                           parse_strict="silent")
                 else:
                     key = schema.addField(afw_col,
                                           type=self._afw_type_map_reverse[column.type],
                                           doc=column.description or "",
                                           units=units,
                                           parse_strict="silent")
             col2afw[column.name] = key
  
         return schema, col2afw
  

◆ getColumnMap()

def lsst.dax.apdb.apdbSchema.ApdbSchema.getColumnMap	(	self,
		table_name
	)

Returns mapping of column names to Column definitions.

Parameters
----------
table_name : `str`
    One of known APDB table names.

Returns
-------
column_map : `dict`
    Mapping of column names to `ColumnDef` instances.

Definition at line 459 of file apdbSchema.py.

     def getColumnMap(self, table_name):
         """Returns mapping of column names to Column definitions.
  
         Parameters
         ----------
         table_name : `str`
             One of known APDB table names.
  
         Returns
         -------
         column_map : `dict`
             Mapping of column names to `ColumnDef` instances.
         """
         table = self._schemas[table_name]
         cmap = {column.name: column for column in table.columns}
         return cmap
  

◆ makeSchema()

def lsst.dax.apdb.apdbSchema.ApdbSchema.makeSchema	(	self,
		drop = `False`,
		mysql_engine = `'InnoDB'`,
		oracle_tablespace = `None`,
		oracle_iot = `False`
	)

Create or re-create all tables.

Parameters
----------
drop : `bool`, optional
    If True then drop tables before creating new ones.
mysql_engine : `str`, optional
    MySQL engine type to use for new tables.
oracle_tablespace : `str`, optional
    Name of Oracle tablespace, only useful with oracle
oracle_iot : `bool`, optional
    Make Index-organized DiaObjectLast table.

Definition at line 339 of file apdbSchema.py.

     def makeSchema(self, drop=False, mysql_engine='InnoDB', oracle_tablespace=None, oracle_iot=False):
         """Create or re-create all tables.
  
         Parameters
         ----------
         drop : `bool`, optional
             If True then drop tables before creating new ones.
         mysql_engine : `str`, optional
             MySQL engine type to use for new tables.
         oracle_tablespace : `str`, optional
             Name of Oracle tablespace, only useful with oracle
         oracle_iot : `bool`, optional
             Make Index-organized DiaObjectLast table.
         """
  
         # re-make table schema for all needed tables with possibly different options
         _LOG.debug("clear metadata")
         self._metadata.clear()
         _LOG.debug("re-do schema mysql_engine=%r oracle_tablespace=%r",
                    mysql_engine, oracle_tablespace)
         self._makeTables(mysql_engine=mysql_engine, oracle_tablespace=oracle_tablespace,
                          oracle_iot=oracle_iot)
  
         # create all tables (optionally drop first)
         if drop:
             _LOG.info('dropping all tables')
             self._metadata.drop_all()
         _LOG.info('creating all tables')
         self._metadata.create_all()
  

Member Data Documentation

◆ forcedSources

lsst.dax.apdb.apdbSchema.ApdbSchema.forcedSources

Definition at line 234 of file apdbSchema.py.

◆ objects

lsst.dax.apdb.apdbSchema.ApdbSchema.objects

Definition at line 230 of file apdbSchema.py.

◆ objects_last

lsst.dax.apdb.apdbSchema.ApdbSchema.objects_last

Definition at line 232 of file apdbSchema.py.

◆ objects_nightly

lsst.dax.apdb.apdbSchema.ApdbSchema.objects_nightly

Definition at line 231 of file apdbSchema.py.

◆ sources

lsst.dax.apdb.apdbSchema.ApdbSchema.sources

Definition at line 233 of file apdbSchema.py.

◆ visits

lsst.dax.apdb.apdbSchema.ApdbSchema.visits

Definition at line 235 of file apdbSchema.py.

The documentation for this class was generated from the following file:

/j/snowflake/release/lsstsw/stack/lsst-scipipe-0.7.0/Linux64/dax_apdb/22.0.1-2-gb66926d+6312710a6c/python/lsst/dax/apdb/apdbSchema.py

Public Member Functions

Public Attributes