Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Scale to Zero with Reader/Writer Separation #16726

Closed

Conversation

prudhvigodithi
Copy link
Member

@prudhvigodithi prudhvigodithi commented Nov 26, 2024

Description

At high level

  • Add new setting to handle scale to zero.
    • Setting validation logic.
  • Routing table updates.
    • Handle Scale up.
    • Handle Scale down.
  • Health check changes.
    • For Overall Cluster.
    • For Specific Index
    • At shard level

[Will update with More PR description shortly]

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@prudhvigodithi prudhvigodithi requested a review from mch2 November 26, 2024 22:06
@prudhvigodithi
Copy link
Member Author

prudhvigodithi commented Nov 26, 2024

Adding @mch2 to please go over the initial version, I will continue refining the code for cleanup, quality improvement and add some tests, but I’ve raised the PR to gather feedback on the approach.
Thank you
@getsaurabh02 @vinaykpud

@@ -45,6 +45,17 @@ testClusters {
plugin('plugins:'.concat(p))
}
}
setting 'opensearch.experimental.feature.read.write.split.enabled', 'true'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove these settings once I have the PR ready for review, these changes are to ensure I can test scale to zero implementation with remote store and read write split enabled.

Copy link
Contributor

❌ Gradle check result for cdbef91: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 7e64d56: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Dec 2, 2024

❌ Gradle check result for aa96f64: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Prudhvi Godithi <[email protected]>
Copy link
Contributor

github-actions bot commented Dec 4, 2024

❌ Gradle check result for ca070bc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@opensearch-trigger-bot
Copy link
Contributor

This PR is stalled because it has been open for 30 days with no activity.

@opensearch-trigger-bot opensearch-trigger-bot bot added the stalled Issues that have stalled label Jan 4, 2025
@prudhvigodithi
Copy link
Member Author

Coming from the discussion part of the #16720, we can go with new API for this scale to zero concept. This PR is based on an index setting but I will close this PR and create a new PR that has the logic to use a new API for the scale to zero concept. The rest of the logic for routing table updates and health check changes remains the same.
Thank you
@mch2 @getsaurabh02

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stalled Issues that have stalled
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant