[Mobile] Add BrowserStack Android MAUI Test #23383
Conversation
| /// </summary> | ||
| /// <param name="text">Log text to send.</param> | ||
| /// <param name="logLevel">Log level -- choose between info, debug, warning, and error</param> | ||
| public void browserStackLog(String text, String logLevel = "info") |
There was a problem hiding this comment.
It's not used anywhere at the moment, but I thought that I would leave the function in in case anybody wanted to write logs to send to BrowserStack for debugging purposes
There was a problem hiding this comment.
we can add it back later if we want to use it. in general, I'd prefer to not have unused code checked in. it may potentially be useful but also adds complexity.
csharp/test/Microsoft.ML.OnnxRuntime.Tests.Android/RunAllTest.cs
Outdated
Show resolved
Hide resolved
...rp/test/Microsoft.ML.OnnxRuntime.Tests.Android/Microsoft.ML.OnnxRuntime.Tests.Android.csproj
Outdated
Show resolved
Hide resolved
csharp/test/Microsoft.ML.OnnxRuntime.Tests.MAUI/Microsoft.ML.OnnxRuntime.Tests.MAUI.csproj
Outdated
Show resolved
Hide resolved
csharp/test/Microsoft.ML.OnnxRuntime.Tests.Android/BrowserStackTest.cs
Outdated
Show resolved
Hide resolved
csharp/test/Microsoft.ML.OnnxRuntime.Tests.Android/RunAllTest.cs
Outdated
Show resolved
Hide resolved
|
|
||
| if (element.Text.Equals("✔")) | ||
| { | ||
| i++; |
There was a problem hiding this comment.
do we need to increment i in these two if conditions or can we just let the loop run?
There was a problem hiding this comment.
Ah, the number of tests that passed is stored as a separate text element from the "✔", so I increment to make sure that the element directly after the indicator is parsed as the number passed.
…rStack.Android, applied suggestions
| Assert.Pass(); | ||
| return; |
There was a problem hiding this comment.
Assert.Pass() throws so we can just return.
| Assert.Pass(); | |
| return; | |
| return; |
|
|
||
| // click into test results if tests have failed | ||
| FindAppiumElementThenClick("//android.widget.TextView", "⛔"); | ||
| await Task.Delay(500); |
There was a problem hiding this comment.
how reliable is this delay? would the test be less prone to flakiness if we explicitly wait for an expected UI element to become available?
| public async Task ClickRunAllTest() | ||
| { | ||
| // XAML for the main page: | ||
| // https://github.com/mattleibow/DeviceRunners/blob/main/src/DeviceRunners.VisualRunners.Maui/App/Pages/HomePage.xaml |
There was a problem hiding this comment.
also use a permalink here
| // https://github.com/mattleibow/DeviceRunners/blob/main/src/DeviceRunners.VisualRunners.Maui/App/Pages/HomePage.xaml | |
| // https://github.com/mattleibow/DeviceRunners/blob/cba7644e07b305ba64dc930b01c3eee55ef2b93d/src/DeviceRunners.VisualRunners.Maui/App/Pages/HomePage.xaml |
| /// </summary> | ||
| /// <param name="text">Log text to send.</param> | ||
| /// <param name="logLevel">Log level -- choose between info, debug, warning, and error</param> | ||
| public void browserStackLog(String text, String logLevel = "info") |
There was a problem hiding this comment.
we can add it back later if we want to use it. in general, I'd prefer to not have unused code checked in. it may potentially be useful but also adds complexity.
|
will this test be run from a pipeline? will that be done in another PR? |
Yep, the test will be run in the NuGet packaging pipeline which will be done in a follow-up PR |
Description
Add test project that will perform an automated UI test that runs the unit tests on Android.
Motivation
Context
Microsoft.ML.OnnxRuntime.Tests.MAUI uses DeviceRunners.VisualRunners to allow running the unit tests (found in Microsoft.ML.OnnxRuntime.Tests.Common) across multiple devices. DeviceRunners.VisualRunners provides a simple UI with a button that will run the unit tests and a panel with the unit test results.
In order to automate the process of running the unit tests across mobile devices, Appium is used for UI testing orchestration (it provides a way to interact with the UI), and BrowserStack automatically runs these Appium tests across different mobile devices.
This project does not include the capability to start an Appium server locally or attach to a local emulator or device.
Build & run instructions
Requirements
dotnet workload install mauidotnet workload install maui-androidRun instructions
dotnet publish -c Release -f net8.0-androidin the Microsoft.ML.OnnxRuntime.Tests.MAUI directory.bin\Release\net8.0-android\publish.browserstack.ymlfile to target the path to the signed APK.dotnet test