Skip to content

feat: Implement date_part scalar function #27005

Draft
devanbenz wants to merge 59 commits intomaster-1.xfrom
db/76/date_part
Draft

feat: Implement date_part scalar function #27005
devanbenz wants to merge 59 commits intomaster-1.xfrom
db/76/date_part

Conversation

@devanbenz
Copy link

@devanbenz devanbenz commented Dec 3, 2025

Implement date_part(part, expression)

Arguments

- part: Part of the date to return. The following date parts are supported:
   - year
    - quarter (emits value in inclusive range [1, 4] based on which quartile of the year the date is in)
    - month
    - week (week of the year)
    - day (day of the month)
    - hour
    - minute
    - second
    - millisecond
    - microsecond
    - nanosecond
    - dow (day of the week where Sunday is 0)
    - doy (day of the year)
    - epoch (seconds since Unix epoch)
    - isodow (day of the week where Monday is 0)
- expression: Time expression to operate on. Must be time VarRef

Example usage:

SELECT * FROM some_measurement
WHERE time >= now() - 10d AND time <= now() AND (date_part(time, dow) != 0 AND date_part(time, dow) != 6)
SELECT value, date_part('hour', time) FROM some_measurement

Please see #27001 (comment) for additional information on date_part limitations in 1.x.

@devanbenz devanbenz self-assigned this Dec 4, 2025
@devanbenz devanbenz linked an issue Dec 8, 2025 that may be closed by this pull request
@devanbenz devanbenz marked this pull request as ready for review December 9, 2025 21:50

const (
DatePartString = "date_part"
DatePartTimeString = "date_part_time"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is date_part_time? I don't see any tests for it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used to create a reference to time since time is an auxiliary field https://github.com/influxdata/influxdb/pull/27005/files#diff-609a7e16be956ed6386e1a4a4efadf600b7d4de7dcfea27330dc692d1e901dc8R930-R944 I'm going to create some ValueMapper tests for this.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gwossum I can add tests for this but it would likely require exporting

type valueMapper struct {
and testing it. We don't currently have any valueMapper specific tests. It's basically just a struct filled with maps so we would likely just be testing go's map functionality, which may not be worth the effort?

Copy link
Contributor

@davidby-influx davidby-influx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some changes from the first pass. Will review again after changes.

// Multiple selectors WITH date_part should also error
{s: `SELECT value, first(value), last(value), date_part('dow', time) FROM cpu`, err: `mixing multiple selector functions with tags or fields is not supported`},
// date_part subquery validation - cannot be sole field
{s: `SELECT date_part('dow', value) FROM (SELECT value FROM cpu)`, err: `date_part: second argument must be time VarRef`},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the spec, it says:

expression: Time expression to operate on. Can be a constant, column, or function.

Here it looks like you are prohibiting anything not the time column. Which is correct?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to modify our version of the spec. I think it should only operate on the time column. For v3 they can decide if they would like to add more features and allow it to work on constants.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just changed the spec in the PR description to match: #27001 I recall we had a conversation yesterday during standup about the usefulness of constant timestamps. I verified that for other functions in influxdb 1.x (math functions predominately) we don't allow constants, so implementing a constant here would be non-trivial.

@devanbenz
Copy link
Author

Testing with influx cli changes:

> SELECT count(*) FROM cpu GROUP BY date_part('year', time),date_part('month', time),host
name: cpu
tags: host=server01
group: month
time count_value year month
---- ----------- ---- -----
0    4                1
0    2                4
0    1                7

name: cpu
tags: host=server01
group: year
time count_value year month
---- ----------- ---- -----
0    7           2023

name: cpu
tags: host=server02
group: month
time count_value year month
---- ----------- ---- -----
0    2                1
0    2                2
0    2                5
0    2                8
0    1                10
0    1                11
0    1                12

name: cpu
tags: host=server02
group: year
time count_value year month
---- ----------- ---- -----
0    2           2023
0    9           2024

name: cpu
tags: host=server03
group: month
time count_value year month
---- ----------- ---- -----
0    1                1
0    1                6
0    5                9
0    1                12

name: cpu
tags: host=server03
group: year
time count_value year month
---- ----------- ---- -----
0    1           2024
0    7           2025

@devanbenz devanbenz marked this pull request as ready for review March 20, 2026 15:15
@davidby-influx davidby-influx requested a review from Copilot March 20, 2026 16:37
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds InfluxQL support for a date_part(part, time) scalar function (including validation, evaluation, and query planning hooks) and extends query execution to support GROUP BY date_part(...) by threading date-part-derived grouping metadata through iterators and result emission.

Changes:

  • Implement date_part parsing/validation, runtime evaluation (DatePartValuer), and date-part extraction helpers.
  • Add iterator/planner plumbing to (a) evaluate date_part() in WHERE conditions using the point timestamp and (b) support GROUP BY date_part(...) via an extensible DimensionGrouper.
  • Extend result/CLI structures to carry and display non-tag grouping metadata (grouping_keys) and add broad integration tests.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tsdb/shard.go Switch illegal "time" tag/field checks to use models.TimeBytes.
tsdb/field_validator.go Switch illegal "time" field checks to use models.TimeBytes.
tsdb/engine/tsm1/iterator.gen.go.tmpl Add timestamp mapping for condition eval and compute date-part aux values for grouping.
tsdb/engine/tsm1/iterator.gen.go Generated iterator changes mirroring template updates (time mapping + date-part aux).
query/date_part.go New: date_part core implementation (validation, extraction, valuer, grouper + encoding).
query/dimension_grouper.go New: abstraction for non-tag/time grouping key resolution.
query/iterator.go Extend IteratorOptions with date-part dimension support and cache NeedTimeRef.
query/select.go Add date-part GROUP BY dimensions as output columns and adjust mapper handling for date_part.
query/iterator.gen.go.tmpl Teach scanners/reducers about date-part grouping entries and time constant usage.
query/iterator.gen.go Generated query iterator changes for grouping-key resolution and scan handling.
query/cursor.go Add grouping_keys propagation via scanner cursor and enable date_part evaluation via timestamp injection.
query/emitter.go Include grouping keys in row emission and header suppression behavior.
query/functions.go Register date_part return type as Integer.
query/compile.go Allow date_part() in dimensions and conditions; validate date_part calls.
query/compile_test.go Add compile success/failure coverage for date_part.
query/date_part_test.go New: unit tests for parsing/validation/valuer and date-part grouper round-trips.
models/time.go Add models.TimeBytes/models.TimeString shared constants.
models/rows.go Add GroupingKeys to API row model and incorporate it into series equality logic.
coordinator/statement_executor.go Use models.TimeString for result column naming.
cmd/influx/cli/cli.go Suppress headers only when grouping keys match; display grouping keys in formatted output.
tests/server_test.go Large integration suite for date_part in WHERE/SELECT/GROUP BY/subqueries.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@devanbenz devanbenz marked this pull request as draft March 20, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.x area/influxql Issues related to InfluxQL query language kind/enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[1.x] Add date_part scalar function to influxdb

6 participants