Public Member Functions |
Public Attributes |
Static Public Attributes |
Protected Attributes |
List of all members
lsst.dax.apdb.cassandra.partitioner.Partitioner Class Reference
Public Member Functions | |
| __init__ (self, ApdbCassandraConfig config) | |
| int | pixel (self, sphgeom.UnitVector3d direction) |
| int | time_partition (self, float|astropy.time.Time time) |
| tuple[astropy.time.Time, astropy.time.Time] | partition_period (self, int time_partition) |
| tuple[list[tuple[str, tuple]], int] | spatial_where (self, sphgeom.Region|None region, *, bool use_ranges=False, bool for_prepare=False) |
| tuple[list[str], list[tuple[str, tuple]]] | temporal_where (self, ApdbTables table, float|astropy.time.Time start_time, float|astropy.time.Time end_time, *, bool|None query_per_time_part=None, bool for_prepare=False, ApdbCassandraTimePartitionRange|None partitons_range=None) |
Public Attributes | |
| pixelization | |
Static Public Attributes | |
| partition_zero_epoch = astropy.time.Time(0, format="unix_tai") | |
Protected Attributes | |
| _config = config | |
| _epoch = float(self.partition_zero_epoch.mjd) | |
Detailed Description
Logic for temporal and spacial partitiong of APDB tables.
Parameters
----------
config : `ApdbCassandraConfig`
Configuration object.
Definition at line 36 of file partitioner.py.
Constructor & Destructor Documentation
◆ __init__()
| lsst.dax.apdb.cassandra.partitioner.Partitioner.__init__ | ( | self, | |
| ApdbCassandraConfig | config ) |
Definition at line 48 of file partitioner.py.
48 def __init__(self, config: ApdbCassandraConfig):
49 self._config = config
50 self.pixelization = Pixelization(
51 config.partitioning.part_pixelization,
52 config.partitioning.part_pix_level,
53 config.partitioning.part_pix_max_ranges,
54 )
55 self._epoch = float(self.partition_zero_epoch.mjd)
56
Member Function Documentation
◆ partition_period()
| tuple[astropy.time.Time, astropy.time.Time] lsst.dax.apdb.cassandra.partitioner.Partitioner.partition_period | ( | self, | |
| int | time_partition ) |
Return time period for specified taime partition.
Parameters
----------
time_partition : `int`
Time partition.
Returns
-------
start : `astropy.time.Time`
Start of the period, inclusive boundary.
end : `astropy.time.Time`
Start of the period, exclusive boundary.
Definition at line 94 of file partitioner.py.
94 def partition_period(self, time_partition: int) -> tuple[astropy.time.Time, astropy.time.Time]:
95 """Return time period for specified taime partition.
96
97 Parameters
98 ----------
99 time_partition : `int`
100 Time partition.
101
102 Returns
103 -------
104 start : `astropy.time.Time`
105 Start of the period, inclusive boundary.
106 end : `astropy.time.Time`
107 Start of the period, exclusive boundary.
108 """
109 partition_days = self._config.partitioning.time_partition_days
110 start_mjd = self._epoch + partition_days * time_partition
111 end_mjd = self._epoch + partition_days * (time_partition + 1)
112 start = astropy.time.Time(start_mjd, format="mjd", scale="tai")
113 end = astropy.time.Time(end_mjd, format="mjd", scale="tai")
114 return (start, end)
115
◆ pixel()
| int lsst.dax.apdb.cassandra.partitioner.Partitioner.pixel | ( | self, | |
| sphgeom.UnitVector3d | direction ) |
Compute the index of the pixel for given direction.
Parameters
----------
direction : `lsst.sphgeom.UnitVector3d`
Spatial position.
Returns
-------
pixel : `int`
Pixel index.
Definition at line 57 of file partitioner.py.
57 def pixel(self, direction: sphgeom.UnitVector3d) -> int:
58 """Compute the index of the pixel for given direction.
59
60 Parameters
61 ----------
62 direction : `lsst.sphgeom.UnitVector3d`
63 Spatial position.
64
65 Returns
66 -------
67 pixel : `int`
68 Pixel index.
69 """
70 return self.pixelization.pixel(direction)
71
◆ spatial_where()
| tuple[list[tuple[str, tuple]], int] lsst.dax.apdb.cassandra.partitioner.Partitioner.spatial_where | ( | self, | |
| sphgeom.Region | None | region, | ||
| * | , | ||
| bool | use_ranges = False, | ||
| bool | for_prepare = False ) |
Generate expressions for spatial part of WHERE clause.
Parameters
----------
region : `sphgeom.Region`
Spatial region for query results.
use_ranges : `bool`, optional
If True then use pixel ranges ("apdb_part >= p1 AND apdb_part <=
p2") instead of exact list of pixels. Should be set to True for
large regions covering very many pixels.
for_prepare : `bool`, optional
If True then use placeholders for prepared statement (?), otherwise
produce regulr statement placeholders (%s).
Returns
-------
expressions : `list` [ `tuple` ]
Empty list is returned if ``region`` is `None`, otherwise a list
of one or more ``(expression: str, parameters: tuple)`` tuples.
partition_count : `int`
Number of spatial partitions in the result.
Definition at line 116 of file partitioner.py.
118 ) -> tuple[list[tuple[str, tuple]], int]:
119 """Generate expressions for spatial part of WHERE clause.
120
121 Parameters
122 ----------
123 region : `sphgeom.Region`
124 Spatial region for query results.
125 use_ranges : `bool`, optional
126 If True then use pixel ranges ("apdb_part >= p1 AND apdb_part <=
127 p2") instead of exact list of pixels. Should be set to True for
128 large regions covering very many pixels.
129 for_prepare : `bool`, optional
130 If True then use placeholders for prepared statement (?), otherwise
131 produce regulr statement placeholders (%s).
132
133 Returns
134 -------
135 expressions : `list` [ `tuple` ]
136 Empty list is returned if ``region`` is `None`, otherwise a list
137 of one or more ``(expression: str, parameters: tuple)`` tuples.
138 partition_count : `int`
139 Number of spatial partitions in the result.
140 """
141 if region is None:
142 return [], 0
143
144 token = "?" if for_prepare else "%s"
145
146 count = 0
147 expressions: list[tuple[str, tuple]] = []
148 if use_ranges:
149 pixel_ranges = self.pixelization.envelope(region)
150 for lower, upper in pixel_ranges:
151 upper -= 1
152 if lower == upper:
153 expressions.append((f'"apdb_part" = {token}', (lower,)))
154 count += 1
155 elif lower + 1 == upper:
156 expressions.append((f'"apdb_part" = {token}', (lower,)))
157 expressions.append((f'"apdb_part" = {token}', (upper,)))
158 count += 2
159 else:
160 count += upper - lower + 1
161 expressions.append((f'"apdb_part" >= {token} AND "apdb_part" <= {token}', (lower, upper)))
162 else:
163 pixels = self.pixelization.pixels(region)
164 count = len(pixels)
165 if self._config.partitioning.query_per_spatial_part:
166 expressions.extend((f'"apdb_part" = {token}', (pixel,)) for pixel in pixels)
167 else:
168 pixels_str = ",".join([str(pix) for pix in pixels])
169 expressions.append((f'"apdb_part" IN ({pixels_str})', ()))
170
171 return expressions, count
172
◆ temporal_where()
| tuple[list[str], list[tuple[str, tuple]]] lsst.dax.apdb.cassandra.partitioner.Partitioner.temporal_where | ( | self, | |
| ApdbTables | table, | ||
| float | astropy.time.Time | start_time, | ||
| float | astropy.time.Time | end_time, | ||
| * | , | ||
| bool | None | query_per_time_part = None, | ||
| bool | for_prepare = False, | ||
| ApdbCassandraTimePartitionRange | None | partitons_range = None ) |
Generate table names and expressions for temporal part of WHERE
clauses.
Parameters
----------
table : `ApdbTables`
Table to select from.
start_time : `astropy.time.Time` or `float`
Starting Datetime of MJD value of the time range.
end_time : `astropy.time.Time` or `float`
Starting Datetime of MJD value of the time range.
query_per_time_part : `bool`, optional
If None then use ``query_per_time_part`` from configuration.
for_prepare : `bool`, optional
If True then use placeholders for prepared statement (?), otherwise
produce regulr statement placeholders (%s).
partitons_range : `ApdbCassandraTimePartitionRange` or `None`
Partitions range to further restrict time range.
Returns
-------
tables : `list` [ `str` ]
List of the table names to query. Empty list is returned when time
range does not overlap ``partitons_range``.
expressions : `list` [ `tuple` ]
A list of zero or more ``(expression: str, parameters: tuple)``
tuples.
Definition at line 173 of file partitioner.py.
182 ) -> tuple[list[str], list[tuple[str, tuple]]]:
183 """Generate table names and expressions for temporal part of WHERE
184 clauses.
185
186 Parameters
187 ----------
188 table : `ApdbTables`
189 Table to select from.
190 start_time : `astropy.time.Time` or `float`
191 Starting Datetime of MJD value of the time range.
192 end_time : `astropy.time.Time` or `float`
193 Starting Datetime of MJD value of the time range.
194 query_per_time_part : `bool`, optional
195 If None then use ``query_per_time_part`` from configuration.
196 for_prepare : `bool`, optional
197 If True then use placeholders for prepared statement (?), otherwise
198 produce regulr statement placeholders (%s).
199 partitons_range : `ApdbCassandraTimePartitionRange` or `None`
200 Partitions range to further restrict time range.
201
202 Returns
203 -------
204 tables : `list` [ `str` ]
205 List of the table names to query. Empty list is returned when time
206 range does not overlap ``partitons_range``.
207 expressions : `list` [ `tuple` ]
208 A list of zero or more ``(expression: str, parameters: tuple)``
209 tuples.
210 """
211 tables: list[str]
212 temporal_where: list[tuple[str, tuple]] = []
213 # First and last partition.
214 time_part_start = self.time_partition(start_time)
215 time_part_end = self.time_partition(end_time)
216 if partitons_range:
217 # Check for non-overlapping ranges.
218 if time_part_start > partitons_range.end or time_part_end < partitons_range.start:
219 return [], []
220 if time_part_start < partitons_range.start:
221 time_part_start = partitons_range.start
222 if time_part_end > partitons_range.end:
223 time_part_end = partitons_range.end
224 # Inclusive range.
225 time_parts = list(range(time_part_start, time_part_end + 1))
226 if self._config.partitioning.time_partition_tables:
227 tables = [table.table_name(self._config.prefix, part) for part in time_parts]
228 else:
229 token = "?" if for_prepare else "%s"
230 tables = [table.table_name(self._config.prefix)]
231 if query_per_time_part is None:
232 query_per_time_part = self._config.partitioning.query_per_time_part
233 if query_per_time_part:
234 temporal_where = [(f'"apdb_time_part" = {token}', (time_part,)) for time_part in time_parts]
235 else:
236 time_part_list = ",".join([str(part) for part in time_parts])
237 temporal_where = [(f'"apdb_time_part" IN ({time_part_list})', ())]
238
239 return tables, temporal_where
◆ time_partition()
| int lsst.dax.apdb.cassandra.partitioner.Partitioner.time_partition | ( | self, | |
| float | astropy.time.Time | time ) |
Calculate time partition number for a given time.
Parameters
----------
time : `float` or `astropy.time.Time`
Time for which to calculate partition number. Can be float to mean
MJD or `astropy.time.Time`
Returns
-------
partition : `int`
Partition number for a given time.
Definition at line 72 of file partitioner.py.
72 def time_partition(self, time: float | astropy.time.Time) -> int:
73 """Calculate time partition number for a given time.
74
75 Parameters
76 ----------
77 time : `float` or `astropy.time.Time`
78 Time for which to calculate partition number. Can be float to mean
79 MJD or `astropy.time.Time`
80
81 Returns
82 -------
83 partition : `int`
84 Partition number for a given time.
85 """
86 if isinstance(time, astropy.time.Time):
87 mjd = float(time.mjd)
88 else:
89 mjd = time
90 days_since_epoch = mjd - self._epoch
91 partition = int(days_since_epoch) // self._config.partitioning.time_partition_days
92 return partition
93
Member Data Documentation
◆ _config
|
protected |
Definition at line 49 of file partitioner.py.
◆ _epoch
|
protected |
Definition at line 55 of file partitioner.py.
◆ partition_zero_epoch
|
static |
Definition at line 45 of file partitioner.py.
◆ pixelization
| lsst.dax.apdb.cassandra.partitioner.Partitioner.pixelization |
Initial value:
= Pixelization(
config.partitioning.part_pixelization,
config.partitioning.part_pix_level,
config.partitioning.part_pix_max_ranges,
)
Definition at line 50 of file partitioner.py.
The documentation for this class was generated from the following file:
- /j/snowflake/release/lsstsw/stack/lsst-scipipe-12.1.0/Linux/dax_apdb/gb88ae4c679+7dec8f19df/python/lsst/dax/apdb/cassandra/partitioner.py
Generated on Thu Nov 27 2025 09:15:10 for LSST Applications by
1.13.2