Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Refactor integration tests #9615

Open
TaoChenOSU opened this issue Nov 7, 2024 · 0 comments
Open

Python: Refactor integration tests #9615

TaoChenOSU opened this issue Nov 7, 2024 · 0 comments
Labels
needs discussion Issues that require discussion by the internal Semantic Kernel team before proceeding python Pull requests for the Python Semantic Kernel

Comments

@TaoChenOSU
Copy link
Contributor

TaoChenOSU commented Nov 7, 2024

We should consider further refactoring our integration test pipeline to improve the speed and reliability. Below is a list of items we work on:

  • Set a timeout on the tests so that they don't hang and block the merge queue.
  • Split the test into multiple jobs so that they can run in parallel. Currently, we run the tests in groups sequentially. For instance, tests on completion and tests on memory can run in parallel.
  • Run the jobs in larger runners so that test that require local resource become less flaky.
  • Instead of running all the tests in every single PR merge, we can group a set of less important test to be run less frequently. For example, we can run the tests on the samples once per day at midnight.
@markwallace-microsoft markwallace-microsoft added python Pull requests for the Python Semantic Kernel triage labels Nov 7, 2024
@TaoChenOSU TaoChenOSU added needs discussion Issues that require discussion by the internal Semantic Kernel team before proceeding and removed triage labels Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs discussion Issues that require discussion by the internal Semantic Kernel team before proceeding python Pull requests for the Python Semantic Kernel
Projects
Status: No status
Development

No branches or pull requests

2 participants