Xcode 16 Degraded Performance Since 9/25

[Nathan-Molby]

Describe the bug
Xcode 16 seems to have had degraded performance on MacOS 14 since 9/24. We updated our pipelines to Xcode 16 on 9/23, and saw consistent successful test runs (unit and UI tests) on both 9/23 and 9/24.

Starting on 9/25, however, we have identified three separate degradations with Xcode 16.

1. UI tests fail consistently. The same UI tests that were working on 9/24 now fail every time since 9/25.
2. Unit and UI tests take ~50% longer to run. Before 9/25, our tests suite took about 20 minutes to run. Now it takes 30+ minutes.
3. Fastlane scan fails due to inability to find a simulator. On 9/24, this command:

xcodebuild -showBuildSettings -scheme ******** -project ********* -configuration Debug -destination platform\=iOS\ Simulator,name\=iPhone\ 15,OS\=18.0

succeeded. On 9/25, the same command resulted in an inability to find the iPhone 15 18.0 simulator.

Due to the correlation of these three issues on the same date, I am fairly confident that a runner upgrade resulted in degraded performance for Xcode 16.

To Reproduce
Steps to reproduce the behavior:

1. Attempt to run a unit or UI test suite on MacOS 14 Xcode 16
2. Notice that it fails

Expected behavior

1. UI tests succeed as they did on 9/24
2. Unit and UI tests take the same amount of time they did on 9/24
3. Fastlane scan succeeds

Runner Version and Platform

Version of the successful runner: macos-14-arm64 20240918.8
Version of the failing runner: macos-14-arm64 20240922.1

Both runners indicate that they are version 2.319.1.

What's not working?

UI test error:
The tests fail due to timing out on the first step.

Fastlane error:

No device found with name 'iPhone 15'

[Nathan-Molby]

Since this repo appears inactive, I created a corresponding issue here