apt update fails on Ubuntu 24.04 if arm64 is enabled

[Ravbug]

Description

apt update fails with 404 errors on the Ubuntu 24.04 runner when adding ARM64 as an allowed architecture. It works correctly on the Ubuntu 22.04 runner and my local 24.04 VM. I cannot use the older Ubuntu runner because I need the gcc-13 cross compiler which is only available on the 24.04 runner. The error is 100% repeatable.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

Image: ubuntu-24.04
Version: 20241016.1.0
Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20241016.1/images/ubuntu/Ubuntu2404-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20241016.1

Is it regression?

Image: ubuntu-22.04 Version: 20240922.1.0 Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240922.1/images/ubuntu/Ubuntu2204-Readme.md Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240922.1

Expected behavior

apt update should succeed, like it does when arm64 packages are not enabled.

Actual behavior

apt update fails with 404 errors:

...
Ign:50 http://azure.archive.ubuntu.com/ubuntu noble-security/restricted arm64 Packages
Ign:55 http://azure.archive.ubuntu.com/ubuntu noble-security/multiverse arm64 Packages
Err:40 http://azure.archive.ubuntu.com/ubuntu noble-security/main arm64 Packages
  404  Not Found [IP: 52.252.75.106 80]
Ign:44 http://azure.archive.ubuntu.com/ubuntu noble-security/universe arm64 Packages
Ign:50 http://azure.archive.ubuntu.com/ubuntu noble-security/restricted arm64 Packages
Ign:55 http://azure.archive.ubuntu.com/ubuntu noble-security/multiverse arm64 Packages
Hit:60 https://ppa.launchpadcontent.net/ubuntu-toolchain-r/test/ubuntu noble InRelease
Fetched 378 kB in 1s (441 kB/s)
Reading package lists...
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/dists/noble/main/binary-arm64/Packages  404  Not Found [IP: 52.252.75.106 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/dists/noble-updates/main/binary-arm64/Packages  404  Not Found [IP: 52.252.75.106 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/dists/noble-backports/universe/binary-arm64/Packages  404  Not Found [IP: 52.252.75.106 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/dists/noble-security/main/binary-arm64/Packages  404  Not Found [IP: 52.252.75.106 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.

Example of it failing: https://github.com/RavEngine/Samples/actions/runs/11644598134/job/32426671409

Repro steps

1. Set the contents of /etc/apt/sources.list to the following:

deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ noble main restricted universe
deb [arch=amd64] http://security.ubuntu.com/ubuntu/ noble-security main restricted universe
deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ noble-updates main restricted universe

deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ noble main restricted multiverse universe
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ noble-updates main restricted multiverse universe

2. sudo dpkg --add-architecture arm64
3. sudo add-apt-repository ppa:ubuntu-toolchain-r/test
4. sudo apt update
5. Observe 404 errors.

[kishorekumar-anchala]

Hi @Ravbug ,

We're working on it , we will share the latest informational ASAP. thank you !

[subir0071]

Hi @Ravbug , We were able to reproduce the error at our end. Apparently the http://azure.archive.ubuntu.com/ is missing the binary-arm64 path.
Hence the 404 error being received.

Have raised a bug report with ubuntu for the missing package.

Here is a screen shot from the azure archive url showing the missing package path -

[subir0071]

Hi @Ravbug - Could you please try using the workaround provided in the comment received on the Ubuntu bug report?

[Ravbug]

@subir0071 Unfortunately changing my URLs did not work.
/etc/apt/sources.lst:

deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ noble main restricted universe
deb [arch=amd64] http://security.ubuntu.com/ubuntu/ noble-security main restricted universe
deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ noble-updates main restricted universe

deb [arch=arm64] http://azure.ports.ubuntu.com/ubuntu-ports/ noble main restricted multiverse universe
deb [arch=arm64] http://azure.ports.ubuntu.com/ubuntu-ports/ noble-updates main restricted multiverse universe

Err:36 http://azure.archive.ubuntu.com/ubuntu noble-security/main arm64 Packages
  404  Not Found [IP: 52.252.75.106 80]
Ign:40 http://azure.archive.ubuntu.com/ubuntu noble-security/universe arm64 Packages
Ign:43 http://azure.archive.ubuntu.com/ubuntu noble-security/restricted arm64 Packages
Ign:45 http://azure.archive.ubuntu.com/ubuntu noble-security/multiverse arm64 Packages
Hit:47 https://ppa.launchpadcontent.net/ubuntu-toolchain-r/test/ubuntu noble InRelease
Fetched 378 kB in 1s (416 kB/s)
Reading package lists...
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/dists/noble/main/binary-arm64/Packages  404  Not Found [IP: 52.252.75.106 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/dists/noble-updates/main/binary-arm64/Packages  404  Not Found [IP: 52.252.75.106 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/dists/noble-backports/universe/binary-arm64/Packages  404  Not Found [IP: 52.252.75.106 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/dists/noble-security/main/binary-arm64/Packages  404  Not Found [IP: 52.252.75.106 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.

Looks like it's still trying to go to http://azure.archive.ubuntu.com/ubuntu for arm64 packages. Failing workflow: https://github.com/RavEngine/Samples/actions/runs/11705750275/job/32601202947