Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Invalid handling of time filter in PPL #9104

Open
Smjert opened this issue Dec 20, 2024 · 2 comments · May be fixed by #9379
Open

[BUG] Invalid handling of time filter in PPL #9104

Smjert opened this issue Dec 20, 2024 · 2 comments · May be fixed by #9379
Assignees
@abbyhu2000
Labels
bug Something isn't working discover for discover reinvent

Comments

@Smjert
Copy link

Smjert commented Dec 20, 2024

Describe the bug

Invalid handling of time filter in PPL

To Reproduce

Steps to reproduce the behavior:

  1. Login anonymous in the future playground.
  2. Select the OTEL demo environment.
  3. Select Discover from the menu on the left hand side
  4. In the data source selection dropdown, type ecomm, then select the first item opensearch*ecomm*
  5. Using PPL query mode, set the time filter to absolute mode starting at Oct 20, 2024 @ 00:00:00.00 and ending at Oct 20, 2024 @23:00:00.00 then run source = opensearch*ecomm*,
    • Results having timestamps on both Oct 19 and Oct 20 are shown. As no timezone information is presented the assumption is that it is at least the same timezone and thus incorrect.

Expected behavior

  • Step 5: Results are shown in a list of entries with Time and _source, all within the selected timeframe.

OpenSearch Version

2.18

Dashboards Version

2.18

Plugins

Alerting, anomaly-detection, asynchronous-search, cross-cluster-replication, custom-codecs, flow-framework, geospatial, index-management, job-scheduler, knn, ml, neural-search, notifications, notifications-core, observability, performance-analyzer, reports-scheduler, security, security-analytics, skills, sql, system-templates, query-insights

Screenshots

N/A

Host/Environment:

  • OS: macOS
  • Browser and version Google Chrome, version 131.0.6778.205

Additional context

N/A
@Smjert Smjert added bug Something isn't working untriaged labels Dec 20, 2024
@krisfreedain
Copy link
Member

Catch All Triage - 1, 2, 3

@abbyhu2000 abbyhu2000 self-assigned this Feb 11, 2025
@abbyhu2000
Copy link
Member

abbyhu2000 commented Feb 12, 2025 •
edited
Loading

PPL are actually returning the correct results using the selected time filter, but when it is rendering the time column in the discover table, PPL renders time column using UTC timezone(however the time filters will use the default browser time zone or a user selected time zone by advanced setting) so it looks like it is not handling the time filter correctly.

One way to verify is that sending the requests with the same time filter using DQL and PPL, they are actually getting the same hits back, but PPL returns those hits with a timestamp field as 2025-02-10 08:28:19 while DQL will return as 2025-02-10T08:28:19+00:00. And when discover table renders the timestamp column, PPL will just render Feb 10, 2025 @ 08:28:19.000 while DQL will convert it and render Feb 10, 2025 @ 00:28:19.000

Working on the fix.

@abbyhu2000 abbyhu2000 added the discover for discover reinvent label Feb 13, 2025
@abbyhu2000 abbyhu2000 linked a pull request Feb 13, 2025 that will close this issue
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abbyhu2000 abbyhu2000

Labels
bug Something isn't working discover for discover reinvent
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Render PPL time column using the correct time zone abbyhu2000/OpenSearch-Dashboards
3 participants
@Smjert @krisfreedain @abbyhu2000