[Testing] Fix for flaky UITests in CI that occasionally fail - 3 #27905

NafeelaNazhir · 2025-02-19T16:21:24Z

Description of Change

This pull request includes various changes to the test cases in the src/Controls/tests/TestCases.Shared.Tests/Tests directory, primarily focusing on improving test stability and addressing platform-specific issues. The most important changes include modifying test methods to remove unnecessary async and await keywords, adding platform-specific handling to address CI flakiness, and updating conditional compilation directives.

src/Controls/tests/TestCases.Shared.Tests/Tests/CarouselViewUITests.cs: Removed unnecessary async and await keywords from multiple test methods to simplify the code and potentially improve test stability. [1] [2] [3]
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue24574.cs: Added a retry mechanism for double-tap actions on Android to address CI flakiness.
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25943.cs: Added a wait for the "OK" button on Android to ensure the date picker dialog is fully loaded before interacting with it.
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/TimePickerUITest.cs: Updated the conditional compilation directive to include iOS due to inconsistent picker popup layout.
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/XFIssue/Issue1614.cs: Updated the conditional compilation directive to include Android due to an issue with the cancel button not displaying after orientation changes.
src/Controls/tests/TestCases.HostApp/Issues/Issue19152.xaml: Updated the Entry control to use a custom UITestEntry control with additional properties.
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue19152.cs: Added a sleep delay to ensure the keyboard appears before dismissing it on Android.

TestCases

Issue19152
CarouselViewUITests
Issue24574
Issue25943
TimePickerUITest
Issue1614

dotnet-policy-service · 2025-02-19T16:21:50Z

Hey there @NafeelaNazhir! Thank you so much for your PR! Someone from the team will get assigned to your PR shortly and we'll get it reviewed.

PureWeen · 2025-02-19T17:04:53Z

/azp run

azure-pipelines · 2025-02-19T17:05:15Z

Azure Pipelines successfully started running 3 pipeline(s).

jsuarezruiz · 2025-02-20T06:44:38Z

/azp run

azure-pipelines · 2025-02-20T06:44:59Z

Azure Pipelines successfully started running 3 pipeline(s).

Copilot

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (1)

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue19152.cs:22

[nitpick] Consider replacing the fixed Thread.Sleep delay with a more robust wait method (e.g. waiting for the keyboard element) to reduce potential flakiness on slower devices.

Thread.Sleep(500); // Wait for the keyboard to appear

Copilot · 2025-02-20T11:04:32Z

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue24574.cs

@@ -25,6 +25,19 @@ public void TapThenDoubleTap()
 			App.WaitForElement("Single");

 			App.DoubleTap("TapLabel");
+
+#if ANDROID


Consider adding a log or comment inside the retry block to record when the first double tap attempt fails. This can help diagnose intermittent CI issues on Android.

Copilot · 2025-02-20T11:04:33Z

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/XFIssue/Issue1614.cs

@@ -1,4 +1,5 @@
-#if TEST_FAILS_ON_CATALYST && TEST_FAILS_ON_WINDOWS //The test fails on Windows and MacCatalyst because the SetOrientation method, which is intended to change the device orientation, is only supported on mobile platforms Android and iOS.
+#if TEST_FAILS_ON_CATALYST && TEST_FAILS_ON_WINDOWS && TEST_FAILS_ON_ANDROID //The test fails on Windows and MacCatalyst because the SetOrientation method, which is intended to change the device orientation, is only supported on mobile platforms Android and iOS.


Ensure that the inclusion of the TEST_FAILS_ON_ANDROID flag is documented either in the test case or related docs to clarify its purpose for addressing the cancel button display issue.

Copilot · 2025-02-20T11:04:33Z

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25943.cs

@@ -20,6 +20,7 @@ public void DatePickerShouldDisplayProperSelectedDate()
 			App.WaitForElement("DatePicker");
 			App.Tap("DatePicker");
 #if ANDROID
+            App.WaitForElement("OK");


Confirm that waiting for the 'OK' element is sufficiently robust across various Android devices and consider specifying a timeout if unexpected delays could occur.

Copilot · 2025-02-20T11:04:33Z

src/Controls/tests/TestCases.Shared.Tests/Tests/CarouselViewUITests.cs

@@ -10,8 +10,7 @@ public class CarouselViewUITests : UITest
 	{
 		const string CarouselViewGallery = "CarouselView Gallery";

-		public CarouselViewUITests(TestDevice device)
-			: base(device)
+		public CarouselViewUITests(TestDevice device): base(device)


Ensure consistent formatting in constructor declarations to adhere to the project's coding style guidelines.

Copilot · 2025-02-20T11:04:34Z

src/Controls/tests/TestCases.Shared.Tests/Tests/CarouselViewUITests.cs

@@ -25,19 +24,18 @@ public override void TestSetup()

 		[Test]
 		[Category(UITestCategories.CarouselView)]
-		public async Task CarouselViewSetPosition()
+		public void CarouselViewSetPosition()


Verify that removing async/await and fixed delays does not result in race conditions; consider whether additional waiting for elements is needed to ensure test stability across environments.

Copilot · 2025-02-20T11:04:34Z

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/TimePickerUITest.cs

@@ -1,4 +1,5 @@
-#if TEST_FAILS_ON_CATALYST // TimePicker not opens the dialog, issue: https://github.com/dotnet/maui/issues/10827 
+#if TEST_FAILS_ON_CATALYST && TEST_FAILS_ON_IOS // TimePicker not opens the dialog, issue: https://github.com/dotnet/maui/issues/10827 


Ensure that the updated conditional directive for iOS accurately targets devices with inconsistent picker popup layouts and that associated documentation is updated to reflect this change.

jsuarezruiz · 2025-02-21T11:35:29Z

/rebase

jsuarezruiz · 2025-02-21T12:06:00Z

/azp run

azure-pipelines · 2025-02-21T12:06:21Z

Azure Pipelines successfully started running 3 pipeline(s).

dotnet-policy-service bot added community ✨ Community Contribution partner/syncfusion Issues / PR's with Syncfusion collaboration labels Feb 19, 2025

NafeelaNazhir added area-testing Unit tests, device tests and removed community ✨ Community Contribution labels Feb 19, 2025

NafeelaNazhir changed the title ~~Enabled flaky testcases 3~~ [Testing] Fix for flaky UITests in CI that occasionally fail - 3 Feb 19, 2025

anandhan-rajagopal added the community ✨ Community Contribution label Feb 19, 2025

NafeelaNazhir marked this pull request as ready for review February 20, 2025 11:03

Copilot bot review requested due to automatic review settings February 20, 2025 11:03

NafeelaNazhir requested a review from a team as a code owner February 20, 2025 11:03

NafeelaNazhir requested review from jfversluis and mattleibow February 20, 2025 11:03

Copilot AI reviewed Feb 20, 2025

View reviewed changes

NafeelaNazhir added 5 commits February 21, 2025 11:36

enabled flaky UITest cases

8ee9d1c

updated changes

0ec0c0a

updated changes

3da3952

Updated Image

abf7c42

updated Android image

cda5bb0

github-actions bot force-pushed the enabled-flaky-testcases-3 branch from 16a66c2 to cda5bb0 Compare February 21, 2025 11:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Testing] Fix for flaky UITests in CI that occasionally fail - 3 #27905

[Testing] Fix for flaky UITests in CI that occasionally fail - 3 #27905

NafeelaNazhir commented Feb 19, 2025

dotnet-policy-service bot commented Feb 19, 2025

PureWeen commented Feb 19, 2025

azure-pipelines bot commented Feb 19, 2025

jsuarezruiz commented Feb 20, 2025

azure-pipelines bot commented Feb 20, 2025

Copilot AI left a comment

Copilot AI Feb 20, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Copilot AI Feb 20, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Copilot AI Feb 20, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Copilot AI Feb 20, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Copilot AI Feb 20, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Copilot AI Feb 20, 2025

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

jsuarezruiz commented Feb 21, 2025

jsuarezruiz commented Feb 21, 2025

azure-pipelines bot commented Feb 21, 2025

		@@ -1,4 +1,5 @@
		#if TEST_FAILS_ON_CATALYST && TEST_FAILS_ON_WINDOWS //The test fails on Windows and MacCatalyst because the SetOrientation method, which is intended to change the device orientation, is only supported on mobile platforms Android and iOS.
		#if TEST_FAILS_ON_CATALYST && TEST_FAILS_ON_WINDOWS && TEST_FAILS_ON_ANDROID //The test fails on Windows and MacCatalyst because the SetOrientation method, which is intended to change the device orientation, is only supported on mobile platforms Android and iOS.

		@@ -1,4 +1,5 @@
		#if TEST_FAILS_ON_CATALYST // TimePicker not opens the dialog, issue: https://github.com/dotnet/maui/issues/10827
		#if TEST_FAILS_ON_CATALYST && TEST_FAILS_ON_IOS // TimePicker not opens the dialog, issue: https://github.com/dotnet/maui/issues/10827

[Testing] Fix for flaky UITests in CI that occasionally fail - 3 #27905

Are you sure you want to change the base?

[Testing] Fix for flaky UITests in CI that occasionally fail - 3 #27905

Conversation

NafeelaNazhir commented Feb 19, 2025

Description of Change

TestCases

dotnet-policy-service bot commented Feb 19, 2025

PureWeen commented Feb 19, 2025

azure-pipelines bot commented Feb 19, 2025

jsuarezruiz commented Feb 20, 2025

azure-pipelines bot commented Feb 20, 2025

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot AI Feb 20, 2025

Choose a reason for hiding this comment

Copilot AI Feb 20, 2025

Choose a reason for hiding this comment

Copilot AI Feb 20, 2025

Choose a reason for hiding this comment

Copilot AI Feb 20, 2025

Choose a reason for hiding this comment

Copilot AI Feb 20, 2025

Choose a reason for hiding this comment

Copilot AI Feb 20, 2025

Choose a reason for hiding this comment

jsuarezruiz commented Feb 21, 2025

jsuarezruiz commented Feb 21, 2025

azure-pipelines bot commented Feb 21, 2025