lsst.pipe.tasks.diff_matched_tract_catalog.DiffMatchedTractCatalogTask Class Reference

Inheritance diagram for lsst.pipe.tasks.diff_matched_tract_catalog.DiffMatchedTractCatalogTask:

Public Member Functions
	runQuantum (self, butlerQC, inputRefs, outputRefs)
pipeBase.Struct	run (self, astropy.table.Table catalog_ref, astropy.table.Table catalog_target, astropy.table.Table catalog_match_ref, astropy.table.Table catalog_match_target, afwGeom.SkyWcs wcs=None)

Static Public Attributes
	ConfigClass = DiffMatchedTractCatalogConfig

Static Protected Attributes
str	_DefaultName = "DiffMatchedTractCatalog"

Detailed Description

Load subsets of matched catalogs and output a merged catalog of matched sources.

Definition at line 312 of file diff_matched_tract_catalog.py.

Member Function Documentation

run()

pipeBase.Struct lsst.pipe.tasks.diff_matched_tract_catalog.DiffMatchedTractCatalogTask.run	(		self,
		astropy.table.Table	catalog_ref,
		astropy.table.Table	catalog_target,
		astropy.table.Table	catalog_match_ref,
		astropy.table.Table	catalog_match_target,
		afwGeom.SkyWcs	wcs = None )

Load matched reference and target (measured) catalogs, measure summary statistics, and output
a combined matched catalog with columns from both inputs.

Parameters
----------
catalog_ref : `astropy.table.Table`
    A reference catalog to diff objects/sources from.
catalog_target : `astropy.table.Table`
    A target catalog to diff reference objects/sources to.
catalog_match_ref : `astropy.table.Table`
    A catalog with match indices of target sources and selection flags
    for each reference source.
catalog_match_target : `astropy.table.Table`
    A catalog with selection flags for each target source.
wcs : `lsst.afw.image.SkyWcs`
    A coordinate system to convert catalog positions to sky coordinates,
    if necessary.

Returns
-------
retStruct : `lsst.pipe.base.Struct`
    A struct with output_ref and output_target attribute containing the
    output matched catalogs.

Definition at line 341 of file diff_matched_tract_catalog.py.

    ) -> pipeBase.Struct:
        """Load matched reference and target (measured) catalogs, measure summary statistics, and output
        a combined matched catalog with columns from both inputs.
 
        Parameters
        ----------
        catalog_ref : `astropy.table.Table`
            A reference catalog to diff objects/sources from.
        catalog_target : `astropy.table.Table`
            A target catalog to diff reference objects/sources to.
        catalog_match_ref : `astropy.table.Table`
            A catalog with match indices of target sources and selection flags
            for each reference source.
        catalog_match_target : `astropy.table.Table`
            A catalog with selection flags for each target source.
        wcs : `lsst.afw.image.SkyWcs`
            A coordinate system to convert catalog positions to sky coordinates,
            if necessary.
 
        Returns
        -------
        retStruct : `lsst.pipe.base.Struct`
            A struct with output_ref and output_target attribute containing the
            output matched catalogs.
        """
        # Would be nice if this could refer directly to ConfigClass
        config: DiffMatchedTractCatalogConfig = self.config
 
        # Strip any provenance from tables before merging to prevent
        # warnings from conflicts being issued by astropy.utils.merge during
        # vstack or hstack calls.
        DatasetProvenance.strip_provenance_from_flat_dict(catalog_ref.meta)
        DatasetProvenance.strip_provenance_from_flat_dict(catalog_target.meta)
        DatasetProvenance.strip_provenance_from_flat_dict(catalog_match_ref.meta)
        DatasetProvenance.strip_provenance_from_flat_dict(catalog_match_target.meta)
 
        # It would be nice to make this a Selector but those are
        # only available in analysis_tools for now
        select_ref, select_target = (
            (catalog[column] if column else np.ones(len(catalog), dtype=bool))
            for catalog, column in (
                (catalog_match_ref, self.config.column_match_candidate_ref),
                (catalog_match_target, self.config.column_match_candidate_target),
            )
        )
        # Add additional selection criteria for target sources beyond those for matching
        # (not recommended, but can be done anyway)
        for column in config.columns_target_select_true:
            select_target &= catalog_target[column]
        for column in config.columns_target_select_false:
            select_target &= ~catalog_target[column]
 
        ref, target = config.coord_format.format_catalogs(
            catalog_ref=catalog_ref, catalog_target=catalog_target,
            select_ref=None, select_target=select_target, wcs=wcs, radec_to_xy_func=radec_to_xy,
        )
        cat_ref = ref.catalog
        cat_target = target.catalog
        n_target = len(cat_target)
 
        if not config.filter_on_match_candidate:
            for cat_add, cat_match, column in (
                (cat_ref, catalog_match_ref, config.column_match_candidate_ref),
                (cat_target, catalog_match_target, config.column_match_candidate_target),
            ):
                if column is not None:
                    cat_add[column] = cat_match[column]
 
        match_row = catalog_match_ref['match_row']
        matched_ref = match_row >= 0
        matched_row = match_row[matched_ref]
        matched_target = np.zeros(n_target, dtype=bool)
        matched_target[matched_row] = True
 
        # Add/compute distance columns
        coord1_target_err, coord2_target_err = config.columns_target_coord_err
        column_dist, column_dist_err = 'match_distance', 'match_distanceErr'
        dist = np.full(n_target, np.nan)
 
        target_match_c1, target_match_c2 = (coord[matched_row] for coord in (target.coord1, target.coord2))
        target_ref_c1, target_ref_c2 = (coord[matched_ref] for coord in (ref.coord1, ref.coord2))
 
        dist_err = np.full(n_target, np.nan)
        dist[matched_row] = sphdist(
            target_match_c1, target_match_c2, target_ref_c1, target_ref_c2
        ) if config.coord_format.coords_spherical else np.hypot(
            target_match_c1 - target_ref_c1, target_match_c2 - target_ref_c2,
        )
        cat_target_matched = cat_target[matched_row]
        # This will convert a masked array to an array filled with nans
        # wherever there are bad values (otherwise sphdist can raise)
        c1_err, c2_err = (
            np.ma.getdata(cat_target_matched[c_err]) for c_err in (coord1_target_err, coord2_target_err)
        )
        # Should probably explicitly add cosine terms if ref has errors too
        dist_err[matched_row] = sphdist(
            target_match_c1, target_match_c2, target_match_c1 + c1_err, target_match_c2 + c2_err
        ) if config.coord_format.coords_spherical else np.hypot(c1_err, c2_err)
        cat_target[column_dist], cat_target[column_dist_err] = dist, dist_err
 
        # Create a matched table, preserving the target catalog's named index (if it has one)
        cat_left = cat_target[matched_row]
        cat_right = cat_ref[matched_ref]
        if config.column_matched_prefix_target:
            cat_left.rename_columns(
                list(cat_left.columns),
                new_names=[f'{config.column_matched_prefix_target}{col}' for col in cat_left.columns],
            )
        if config.column_matched_prefix_ref:
            cat_right.rename_columns(
                list(cat_right.columns),
                new_names=[f'{config.column_matched_prefix_ref}{col}' for col in cat_right.columns],
            )
        cat_matched = astropy.table.hstack((cat_left, cat_right))
 
        if config.include_unmatched:
            # Create an unmatched table with the same schema as the matched one
            # ... but only for objects with no matches (for completeness/purity)
            # and that were selected for matching (or inclusion via config)
            cat_right = astropy.table.Table(
                cat_ref[~matched_ref & select_ref]
            )
            cat_right.rename_columns(
                cat_right.colnames,
                [f"{config.column_matched_prefix_ref}{col}" for col in cat_right.colnames],
            )
            match_row_target = catalog_match_target['match_row']
            cat_left = cat_target[~(match_row_target >= 0) & select_target]
            cat_left.rename_columns(
                cat_left.colnames,
                [f"{config.column_matched_prefix_target}{col}" for col in cat_left.colnames],
            )
            # This may be slower than pandas but will, for example, create
            # masked columns for booleans, which pandas does not support.
            # See https://github.com/pandas-dev/pandas/issues/46662
            cat_unmatched = astropy.table.vstack([cat_left, cat_right])
 
        for columns_convert_base, prefix in (
            (config.columns_ref_mag_to_nJy, config.column_matched_prefix_ref),
            (config.columns_target_mag_to_nJy, ""),
        ):
            if columns_convert_base:
                columns_convert = {
                    f"{prefix}{k}": f"{prefix}{v}" for k, v in columns_convert_base.items()
                } if prefix else columns_convert_base
                to_convert = [cat_matched]
                if config.include_unmatched:
                    to_convert.append(cat_unmatched)
                for cat_convert in to_convert:
                    cat_convert.rename_columns(
                        tuple(columns_convert.keys()),
                        tuple(columns_convert.values()),
                    )
                    for column_flux in columns_convert.values():
                        cat_convert[column_flux] = u.ABmag.to(u.nJy, cat_convert[column_flux])
 
        if config.include_unmatched:
            # This is probably less efficient than just doing an outer join originally; worth checking
            cat_matched = astropy.table.vstack([cat_matched, cat_unmatched])
            if (prefix_coord := config.prefix_best_coord) is not None:
                columns_coord_best = (
                    f"{prefix_coord}{col_coord}" for col_coord in (
                        ("ra", "dec") if config.coord_format.coords_spherical else ("coord1", "coord2")
                    )
                )
                for column_coord_best, column_coord_ref, column_coord_target in zip(
                    columns_coord_best,
                    (config.coord_format.column_ref_coord1, config.coord_format.column_ref_coord2),
                    (config.coord_format.column_target_coord1, config.coord_format.column_target_coord2),
                ):
                    column_full_ref = f'{config.column_matched_prefix_ref}{column_coord_ref}'
                    column_full_target = f'{config.column_matched_prefix_target}{column_coord_target}'
                    values = cat_matched[column_full_ref]
                    unit = values.unit
                    values_bad = np.ma.masked_invalid(values).mask
                    # Cast to an unmasked array - there will be no bad values
                    values = np.array(values)
                    values[values_bad] = cat_matched[column_full_target][values_bad]
                    cat_matched[column_coord_best] = values
                    cat_matched[column_coord_best].unit = unit
                    cat_matched[column_coord_best].description = (
                        f"Best {columns_coord_best} value from {column_full_ref} if available"
                        f" else {column_full_target}"
                    )
 
        retStruct = pipeBase.Struct(cat_matched=cat_matched)
        return retStruct

runQuantum()

lsst.pipe.tasks.diff_matched_tract_catalog.DiffMatchedTractCatalogTask.runQuantum	(		self,
			butlerQC,
			inputRefs,
			outputRefs )

Definition at line 318 of file diff_matched_tract_catalog.py.

    def runQuantum(self, butlerQC, inputRefs, outputRefs):
        inputs = butlerQC.get(inputRefs)
        skymap = inputs.pop("skymap")
 
        columns_match_ref = ['match_row']
        if (column := self.config.column_match_candidate_ref) is not None:
            columns_match_ref.append(column)
 
        columns_match_target = ['match_row']
        if (column := self.config.column_match_candidate_target) is not None and (
            column in inputs['columns_match_target']
        ):
            columns_match_target.append(column)
 
        outputs = self.run(
            catalog_ref=inputs['cat_ref'].get(parameters={'columns': self.config.columns_in_ref}),
            catalog_target=inputs['cat_target'].get(parameters={'columns': self.config.columns_in_target}),
            catalog_match_ref=inputs['cat_match_ref'].get(parameters={'columns': columns_match_ref}),
            catalog_match_target=inputs['cat_match_target'].get(parameters={'columns': columns_match_target}),
            wcs=skymap[butlerQC.quantum.dataId["tract"]].wcs,
        )
        butlerQC.put(outputs, outputRefs)
 

Member Data Documentation

_DefaultName

str lsst.pipe.tasks.diff_matched_tract_catalog.DiffMatchedTractCatalogTask._DefaultName = "DiffMatchedTractCatalog"

staticprotected

Definition at line 316 of file diff_matched_tract_catalog.py.

ConfigClass

lsst.pipe.tasks.diff_matched_tract_catalog.DiffMatchedTractCatalogTask.ConfigClass = DiffMatchedTractCatalogConfig

static

Definition at line 315 of file diff_matched_tract_catalog.py.

---

The documentation for this class was generated from the following file:

• /j/snowflake/release/lsstsw/stack/lsst-scipipe-12.0.0/Linux/pipe_tasks/gcedae5159b+afaec0eb3d/python/lsst/pipe/tasks/diff_matched_tract_catalog.py