Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep track of the current attempt count and make it accessible in the TestContext when running tests with the new Retry Attribute #4692

Open
radmorecameron opened this issue Jan 16, 2025 · 2 comments

Comments

@radmorecameron
Copy link

radmorecameron commented Jan 16, 2025

Summary

I think it could be useful to have the current retry attempt count be accessibile from TestContext.

Background and Motivation

One of the test projects I have creates files as part of each test so that test runs can be easily analyzed without rerunning tests in debug mode (this is useful for finding errors in CI). The names of the files involve the test's display name as well as what the test is currently doing.

Proposed Feature

It'd be useful if the TestContext had an attribute that was able to keep track of the retry count.

[TestClass]
public class MyTestClass {
   public TestContext TestContext {get; set; } = null!;
   
   protected string GetFileName() {
        var testName = (TestContext.TestName != TestContext.TestDisplayName) ? $"{TestContext.TestName}.{TestContext.TestDisplayName}" : $"{TestContext.TestDisplayName}";

       // use new TestAttemptCount property so the name is unique and we don't have to manually track it for each test method:
       return $"{testName}-{TestContext.TestAttemptCount}"
   }

   [TestMethod]
   [Retry(3)]
   public async TaskFlakeyTest() {
      // send http request to external server/interact with an external service
      // validate response
   }

  [TestCleanup]
  public async Task Cleanup() {
    if (TestContext.CurrentTestOutcome == UnitTestOutcome.Failed) {
      string fileName = GetFileName();
      // save response to a file (in my case failing Playwright traces are saved to a file)
    }
  }
}

Alternative Designs

Leave it as is, if people need to keep track of the Retry count then they can handle it themselves with a static variable.
As done here:
https://github.com/microsoft/testfx/blob/main/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/RetryTests.cs#L94

@Youssef1313
Copy link
Member

@Evangelink Per the current design, I think the only way is to add TestContext to RetryContext and leave it up to individual retry attribute implementations to properly update a property on TestContext. But we can't guarantee that attribute authors will respect this properly. What do you think?

@Evangelink
Copy link
Member

Yes for adding it to the TestContext and yes for the suggestion @Youssef1313. Adding to 3.8 milestone, if you want to jump on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants