Inheritance diagram for lsst.ip.diffim.imageMapReduce.ImageMapReduceTask:
Public Member Functions | |
| __init__ (self, *args, **kwargs) | |
| run (self, exposure, **kwargs) | |
| plotBoxes (self, fullBBox, skip=3) | |
Public Attributes | |
| boxes0 | |
| boxes1 | |
Static Public Attributes | |
| ConfigClass = ImageMapReduceConfig | |
Protected Member Functions | |
| _runMapper (self, exposure, doClone=False, **kwargs) | |
| _reduceImage (self, mapperResults, exposure, **kwargs) | |
| _generateGrid (self, exposure, forceEvenSized=False, **kwargs) | |
| _plotBoxGrid (self, boxes, bbox, **kwargs) | |
Static Protected Attributes | |
| str | _DefaultName = "ip_diffim_imageMapReduce" |
Detailed Description
Split an Exposure into subExposures (optionally on a grid) and perform the same operation on each. Perform 'simple' operations on a gridded set of subExposures of a larger Exposure, and then (by default) have those subExposures stitched back together into a new, full-sized image. Contrary to the expectation given by its name, this task does not perform these operations in parallel, although it could be updatd to provide such functionality. The actual operations are performed by two subTasks passed to the config. The exposure passed to this task's `run` method will be divided, and those subExposures will be passed to the subTasks, along with the original exposure. The reducing operation is performed by the second subtask.
Definition at line 488 of file imageMapReduce.py.
Constructor & Destructor Documentation
◆ __init__()
| lsst.ip.diffim.imageMapReduce.ImageMapReduceTask.__init__ | ( | self, | |
| * | args, | ||
| ** | kwargs ) |
Create the image map-reduce task
Parameters
----------
args :
arguments to be passed to
`lsst.pipe.base.task.Task.__init__`
kwargs :
additional keyword arguments to be passed to
`lsst.pipe.base.task.Task.__init__`
Definition at line 509 of file imageMapReduce.py.
509 def __init__(self, *args, **kwargs):
510 """Create the image map-reduce task
511
512 Parameters
513 ----------
514 args :
515 arguments to be passed to
516 `lsst.pipe.base.task.Task.__init__`
517 kwargs :
518 additional keyword arguments to be passed to
519 `lsst.pipe.base.task.Task.__init__`
520 """
521 pipeBase.Task.__init__(self, *args, **kwargs)
522
523 self.boxes0 = self.boxes1 = None
524 self.makeSubtask("mapper")
525 self.makeSubtask("reducer")
526
Member Function Documentation
◆ _generateGrid()
|
protected |
Generate two lists of bounding boxes that evenly grid `exposure`
Unless the config was provided with `cellCentroidsX` and
`cellCentroidsY`, grid (subimage) centers are spaced evenly
by gridStepX/Y. Then the grid is adjusted as little as
possible to evenly cover the input exposure (if
adjustGridOption is not 'none'). Then the second set of
bounding boxes is expanded by borderSizeX/Y. The expanded
bounding boxes are adjusted to ensure that they intersect the
exposure's bounding box. The resulting lists of bounding boxes
and corresponding expanded bounding boxes are set to
`self.boxes0`, `self.boxes1`.
Parameters
----------
exposure : `lsst.afw.image.Exposure`
input exposure whose full bounding box is to be evenly gridded.
forceEvenSized : `bool`
force grid elements to have even-valued x- and y- dimensions?
(Potentially useful if doing Fourier transform of subExposures.)
Definition at line 623 of file imageMapReduce.py.
623 def _generateGrid(self, exposure, forceEvenSized=False, **kwargs):
624 """Generate two lists of bounding boxes that evenly grid `exposure`
625
626 Unless the config was provided with `cellCentroidsX` and
627 `cellCentroidsY`, grid (subimage) centers are spaced evenly
628 by gridStepX/Y. Then the grid is adjusted as little as
629 possible to evenly cover the input exposure (if
630 adjustGridOption is not 'none'). Then the second set of
631 bounding boxes is expanded by borderSizeX/Y. The expanded
632 bounding boxes are adjusted to ensure that they intersect the
633 exposure's bounding box. The resulting lists of bounding boxes
634 and corresponding expanded bounding boxes are set to
635 `self.boxes0`, `self.boxes1`.
636
637 Parameters
638 ----------
639 exposure : `lsst.afw.image.Exposure`
640 input exposure whose full bounding box is to be evenly gridded.
641 forceEvenSized : `bool`
642 force grid elements to have even-valued x- and y- dimensions?
643 (Potentially useful if doing Fourier transform of subExposures.)
644 """
645 # kwargs are ignored, but necessary to enable optional passing of
646 # `forceEvenSized` from `_runMapper`.
647 bbox = exposure.getBBox()
648
649 # Extract the config parameters for conciseness.
650 cellCentroidsX = self.config.cellCentroidsX
651 cellCentroidsY = self.config.cellCentroidsY
652 cellSizeX = self.config.cellSizeX
653 cellSizeY = self.config.cellSizeY
654 gridStepX = self.config.gridStepX
655 gridStepY = self.config.gridStepY
656 borderSizeX = self.config.borderSizeX
657 borderSizeY = self.config.borderSizeY
658 adjustGridOption = self.config.adjustGridOption
659 scaleByFwhm = self.config.scaleByFwhm
660
661 if cellCentroidsX is None or len(cellCentroidsX) <= 0:
662 # Not given centroids; construct them from cellSize/gridStep
663 psf = exposure.getPsf()
664 psfFwhm = (psf.computeShape(psf.getAveragePosition()).getDeterminantRadius()
665 * 2.*np.sqrt(2.*np.log(2.)))
666 if scaleByFwhm:
667 self.log.info("Scaling grid parameters by %f", psfFwhm)
668
669 def rescaleValue(val):
670 if scaleByFwhm:
671 return np.rint(val*psfFwhm).astype(int)
672 else:
673 return np.rint(val).astype(int)
674
675 cellSizeX = rescaleValue(cellSizeX)
676 cellSizeY = rescaleValue(cellSizeY)
677 gridStepX = rescaleValue(gridStepX)
678 gridStepY = rescaleValue(gridStepY)
679 borderSizeX = rescaleValue(borderSizeX)
680 borderSizeY = rescaleValue(borderSizeY)
681
682 nGridX = bbox.getWidth()//gridStepX
683 nGridY = bbox.getHeight()//gridStepY
684
685 if adjustGridOption == 'spacing':
686 # Readjust spacings so that they fit perfectly in the image.
687 nGridX = bbox.getWidth()//cellSizeX + 1
688 nGridY = bbox.getHeight()//cellSizeY + 1
689 xLinSpace = np.linspace(cellSizeX//2, bbox.getWidth() - cellSizeX//2, nGridX)
690 yLinSpace = np.linspace(cellSizeY//2, bbox.getHeight() - cellSizeY//2, nGridY)
691
692 elif adjustGridOption == 'size':
693 cellSizeX = gridStepX
694 cellSizeY = gridStepY
695 xLinSpace = np.arange(cellSizeX//2, bbox.getWidth() + cellSizeX//2, cellSizeX)
696 yLinSpace = np.arange(cellSizeY//2, bbox.getHeight() + cellSizeY//2, cellSizeY)
697 cellSizeX += 1 # add 1 to make sure there are no gaps
698 cellSizeY += 1
699
700 else:
701 xLinSpace = np.arange(cellSizeX//2, bbox.getWidth() + cellSizeX//2, gridStepX)
702 yLinSpace = np.arange(cellSizeY//2, bbox.getHeight() + cellSizeY//2, gridStepY)
703
704 cellCentroids = [(x, y) for x in xLinSpace for y in yLinSpace]
705
706 else:
707 # in py3 zip returns an iterator, but want to test length below, so use this instead:
708 cellCentroids = [(cellCentroidsX[i], cellCentroidsY[i]) for i in range(len(cellCentroidsX))]
709
710 # first "main" box at 0,0
712 # first expanded box
713 bbox1 = geom.Box2I(bbox0)
714 bbox1.grow(geom.Extent2I(borderSizeX, borderSizeY))
715
716 self.boxes0 = [] # "main" boxes; store in task so can be extracted if needed
717 self.boxes1 = [] # "expanded" boxes
718
719 def _makeBoxEvenSized(bb):
720 """Force a bounding-box to have dimensions that are modulo 2."""
721
722 if bb.getWidth() % 2 == 1: # grow to the right
724 bb.clip(bbox)
725 if bb.getWidth() % 2 == 1: # clipped at right -- so grow to the left
726 bb.include(geom.Point2I(bb.getMinX()-1, bb.getMaxY()))
727 bb.clip(bbox)
728 if bb.getHeight() % 2 == 1: # grow upwards
730 bb.clip(bbox)
731 if bb.getHeight() % 2 == 1: # clipped upwards -- so grow down
732 bb.include(geom.Point2I(bb.getMaxX(), bb.getMinY()-1))
733 bb.clip(bbox)
734 if bb.getWidth() % 2 == 1 or bb.getHeight() % 2 == 1: # Box is probably too big
735 raise RuntimeError('Cannot make bounding box even-sized. Probably too big.')
736
737 return bb
738
739 # Use given or grid-parameterized centroids as centers for bounding boxes
740 if cellCentroids is not None and len(cellCentroids) > 0:
741 for x, y in cellCentroids:
742 centroid = geom.Point2D(x, y)
743 bb0 = geom.Box2I(bbox0)
744 xoff = int(np.floor(centroid.getX())) - bb0.getWidth()//2
745 yoff = int(np.floor(centroid.getY())) - bb0.getHeight()//2
746 bb0.shift(geom.Extent2I(xoff, yoff))
747 bb0.clip(bbox)
748 if forceEvenSized:
749 bb0 = _makeBoxEvenSized(bb0)
750 bb1 = geom.Box2I(bbox1)
751 bb1.shift(geom.Extent2I(xoff, yoff))
752 bb1.clip(bbox)
753 if forceEvenSized:
754 bb1 = _makeBoxEvenSized(bb1)
755
756 if bb0.getArea() > 1 and bb1.getArea() > 1:
757 self.boxes0.append(bb0)
758 self.boxes1.append(bb1)
759
760 return self.boxes0, self.boxes1
761
◆ _plotBoxGrid()
|
protected |
Plot a grid of boxes using matplotlib.
Parameters
----------
boxes : `list` of `lsst.geom.Box2I`
a list of bounding boxes.
bbox : `lsst.geom.Box2I`
an overall bounding box
**kwargs
additional keyword arguments for matplotlib
Definition at line 785 of file imageMapReduce.py.
785 def _plotBoxGrid(self, boxes, bbox, **kwargs):
786 """Plot a grid of boxes using matplotlib.
787
788 Parameters
789 ----------
790 boxes : `list` of `lsst.geom.Box2I`
791 a list of bounding boxes.
792 bbox : `lsst.geom.Box2I`
793 an overall bounding box
794 **kwargs
795 additional keyword arguments for matplotlib
796 """
797 import matplotlib.pyplot as plt
798
799 def plotBox(box):
800 corners = np.array([np.array([pt.getX(), pt.getY()]) for pt in box.getCorners()])
801 corners = np.vstack([corners, corners[0, :]])
802 plt.plot(corners[:, 0], corners[:, 1], **kwargs)
803
804 for b in boxes:
805 plotBox(b)
806 plt.xlim(bbox.getBeginX(), bbox.getEndX())
807 plt.ylim(bbox.getBeginY(), bbox.getEndY())
◆ _reduceImage()
|
protected |
Reduce/merge a set of sub-exposures into a final result
Return an exposure of the same dimensions as `exposure`.
`mapperResults` is expected to have been produced by `runMapper`.
Parameters
----------
mapperResults : `list`
`list` of `lsst.pipe.base.Struct`, each of which was produced by
`config.mapper`
exposure : `lsst.afw.image.Exposure`
the original exposure
**kwargs
additional keyword arguments
Returns
-------
Output of `reducer.run` which is a `pipeBase.Struct`.
Definition at line 600 of file imageMapReduce.py.
600 def _reduceImage(self, mapperResults, exposure, **kwargs):
601 """Reduce/merge a set of sub-exposures into a final result
602
603 Return an exposure of the same dimensions as `exposure`.
604 `mapperResults` is expected to have been produced by `runMapper`.
605
606 Parameters
607 ----------
608 mapperResults : `list`
609 `list` of `lsst.pipe.base.Struct`, each of which was produced by
610 `config.mapper`
611 exposure : `lsst.afw.image.Exposure`
612 the original exposure
613 **kwargs
614 additional keyword arguments
615
616 Returns
617 -------
618 Output of `reducer.run` which is a `pipeBase.Struct`.
619 """
620 result = self.reducer.run(mapperResults, exposure, **kwargs)
621 return result
622
◆ _runMapper()
|
protected |
Perform `mapper.run` on each sub-exposure
Perform `mapper.run` on each sub-exposure across a
grid on `exposure` generated by `_generateGrid`. Also pass to
`mapper.run` an 'expanded sub-exposure' containing the
same region as the sub-exposure but with an expanded bounding box.
Parameters
----------
exposure : `lsst.afw.image.Exposure`
the original exposure which is used as the template
doClone : `bool`
if True, clone the subimages before passing to subtask;
in that case, the sub-exps do not have to be considered as read-only
kwargs :
additional keyword arguments to be passed to
`mapper.run` and `self._generateGrid`, including `forceEvenSized`.
Returns
-------
a list of `pipeBase.Struct`s as returned by `mapper.run`.
Definition at line 555 of file imageMapReduce.py.
555 def _runMapper(self, exposure, doClone=False, **kwargs):
556 """Perform `mapper.run` on each sub-exposure
557
558 Perform `mapper.run` on each sub-exposure across a
559 grid on `exposure` generated by `_generateGrid`. Also pass to
560 `mapper.run` an 'expanded sub-exposure' containing the
561 same region as the sub-exposure but with an expanded bounding box.
562
563 Parameters
564 ----------
565 exposure : `lsst.afw.image.Exposure`
566 the original exposure which is used as the template
567 doClone : `bool`
568 if True, clone the subimages before passing to subtask;
569 in that case, the sub-exps do not have to be considered as read-only
570 kwargs :
571 additional keyword arguments to be passed to
572 `mapper.run` and `self._generateGrid`, including `forceEvenSized`.
573
574 Returns
575 -------
576 a list of `pipeBase.Struct`s as returned by `mapper.run`.
577 """
578 if self.boxes0 is None:
579 self._generateGrid(exposure, **kwargs) # possibly pass `forceEvenSized`
580 if len(self.boxes0) != len(self.boxes1):
581 raise ValueError('Bounding boxes list and expanded bounding boxes list are of different lengths')
582
583 self.log.info("Processing %d sub-exposures", len(self.boxes0))
584 mapperResults = []
585 for box0, box1 in zip(self.boxes0, self.boxes1):
586 subExp = exposure.Factory(exposure, box0)
587 expandedSubExp = exposure.Factory(exposure, box1)
588 if doClone:
589 subExp = subExp.clone()
590 expandedSubExp = expandedSubExp.clone()
591 result = self.mapper.run(subExp, expandedSubExp, exposure.getBBox(), **kwargs)
592 if self.config.returnSubImages:
593 toAdd = pipeBase.Struct(inputSubExposure=subExp,
594 inputExpandedSubExposure=expandedSubExp)
595 result.mergeItems(toAdd, 'inputSubExposure', 'inputExpandedSubExposure')
596 mapperResults.append(result)
597
598 return mapperResults
599
◆ plotBoxes()
| lsst.ip.diffim.imageMapReduce.ImageMapReduceTask.plotBoxes | ( | self, | |
| fullBBox, | |||
| skip = 3 ) |
Plot both grids of boxes using matplotlib.
Will compute the grid via `_generateGrid` if
`self.boxes0` and `self.boxes1` have not already been set.
Parameters
----------
exposure : `lsst.afw.image.Exposure`
Exposure whose bounding box is gridded by this task.
skip : `int`
Plot every skip-ped box (help make plots less confusing)
Definition at line 762 of file imageMapReduce.py.
762 def plotBoxes(self, fullBBox, skip=3):
763 """Plot both grids of boxes using matplotlib.
764
765 Will compute the grid via `_generateGrid` if
766 `self.boxes0` and `self.boxes1` have not already been set.
767
768 Parameters
769 ----------
770 exposure : `lsst.afw.image.Exposure`
771 Exposure whose bounding box is gridded by this task.
772 skip : `int`
773 Plot every skip-ped box (help make plots less confusing)
774 """
775 import matplotlib.pyplot as plt
776
777 if self.boxes0 is None:
778 raise RuntimeError('Cannot plot boxes. Run _generateGrid first.')
779 self._plotBoxGrid(self.boxes0[::skip], fullBBox, ls='--')
780 # reset the color cycle -- see
781 # http://stackoverflow.com/questions/24193174/reset-color-cycle-in-matplotlib
782 plt.gca().set_prop_cycle(None)
783 self._plotBoxGrid(self.boxes1[::skip], fullBBox, ls=':')
784
◆ run()
| lsst.ip.diffim.imageMapReduce.ImageMapReduceTask.run | ( | self, | |
| exposure, | |||
| ** | kwargs ) |
Perform a map-reduce operation on the given exposure.
Split the exposure into sub-expposures on a grid (parameters
given by `ImageMapReduceConfig`) and perform
`config.mapper.run()` on each. Reduce the resulting
sub-exposures by running `config.reducer.run()`.
Parameters
----------
exposure : `lsst.afw.image.Exposure`
the full exposure to process
kwargs :
additional keyword arguments to be passed to
subtask `run` methods
Returns
-------
output of `reducer.run()`
Definition at line 528 of file imageMapReduce.py.
528 def run(self, exposure, **kwargs):
529 """Perform a map-reduce operation on the given exposure.
530
531 Split the exposure into sub-expposures on a grid (parameters
532 given by `ImageMapReduceConfig`) and perform
533 `config.mapper.run()` on each. Reduce the resulting
534 sub-exposures by running `config.reducer.run()`.
535
536 Parameters
537 ----------
538 exposure : `lsst.afw.image.Exposure`
539 the full exposure to process
540 kwargs :
541 additional keyword arguments to be passed to
542 subtask `run` methods
543
544 Returns
545 -------
546 output of `reducer.run()`
547
548 """
549 self.log.info("Mapper sub-task: %s", self.mapper._DefaultName)
550 mapperResults = self._runMapper(exposure, **kwargs)
551 self.log.info("Reducer sub-task: %s", self.reducer._DefaultName)
552 result = self._reduceImage(mapperResults, exposure, **kwargs)
553 return result
554
Member Data Documentation
◆ _DefaultName
|
staticprotected |
Definition at line 507 of file imageMapReduce.py.
◆ boxes0
| lsst.ip.diffim.imageMapReduce.ImageMapReduceTask.boxes0 |
Definition at line 523 of file imageMapReduce.py.
◆ boxes1
| lsst.ip.diffim.imageMapReduce.ImageMapReduceTask.boxes1 |
Definition at line 585 of file imageMapReduce.py.
◆ ConfigClass
|
static |
Definition at line 506 of file imageMapReduce.py.
The documentation for this class was generated from the following file:
- /j/snowflake/release/lsstsw/stack/lsst-scipipe-8.0.0/Linux64/ip_diffim/g180d380827+a53a32eff8/python/lsst/ip/diffim/imageMapReduce.py
