OpenWrt image pull failure: failed to register layer: lsetxattr security.capability /usr/bin/caddy: operation not supported

[zhengliw]

Steps to reproduce

1. Use https://github.com/nextcloud/all-in-one/blob/main/compose.yaml to define docker-compose file...
2. docker compose up
3. Initial image pull fails with error

Expected behavior

Image downloads successfully and container starts up

Actual behavior

failed to register layer: lsetxattr security.capability /usr/bin/caddy: operation not supported

Host OS

OpenWrt 23.05.3, Docker 26.1.0
using Docker Compose

Nextcloud AIO version

not related

Current channel

nextcloud/all-in-one:latest

Other valuable info

Based on my research, my guess would be that something is trying to access xattr related functionality although OpenWrt doesn't include these in the default kernel. Maybe we could identify the component that's using it...

This is a snippet around the end....

   ✔ 4abcf2066143 Already exists                                           0.0s 
   ✔ 64d2971301bc Pull complete                                            1.0s 
   ✔ cc6eca4f1862 Pull complete                                            1.0s 
   ✔ b5d35b189a56 Pull complete                                            1.0s 
   ✔ e4edc397ae5a Pull complete                                            1.3s 
   ✔ 61392208410a Pull complete                                            1.3s 
   ✔ bee5804676fa Pull complete                                            3.2s 
   ✔ 51b55d7a64eb Pull complete                                            3.2s 
   ✔ 1de2bd55d517 Pull complete                                            6.6s 
   ✔ a326e5f4c37a Pull complete                                            6.6s 
   ⠿ d630d07ded49 Extracting       14.64MB/14.64...                        9.8s 
   ✔ a4242c1aebc5 Download complete                                        7.4s 
   ✔ 79c7df51942c Download complete                                        7.0s 
   ✔ 3302431a828c Download complete                                        7.9s 
   ✔ f52aa223cc7a Download complete                                        7.8s 
   ✔ 32420f8e2209 Download complete                                        8.2s 
   ✔ 930390356495 Download complete                                        8.4s 
   ✔ 267204048c7c Download complete                                        8.6s 
failed to register layer: lsetxattr security.capability /usr/bin/caddy: operation not supported

[szaimen]

failed to register layer: lsetxattr security.capability

Hi, I found this: https://www.google.com/search?q=failed%20to%20register%20layer%3A%20lsetxattr%20security.capability

[zhengliw]

First, thanks for responding so quickly.
I've looked into all relevant sources but all of them suggested either using another kernel (building OpenWrt myself) or some workarounds that do not work. The top search result is for Adguard Home and contains an Adguard Home specific workaround, as you may have noticed.

Let's try to collect the sparse information that can be found on the web, related or possibly related to the topic:

This one, a thread with a few links:

https://discourse.pi-hole.net/t/lsetxattr-secuirty-capability-bin-ping-operation-not-supported/69514/4

I looked into this thread:
https://forum.openwrt.org/t/docker-error-failed-to-register-layer-lsetxattr-security-capability-usr-libexec-gstreamer-1-0-gst-ptp-helper-operation-not-supported/198293/4

It suggested using seccomp=unconfined, which only applies when an actual container is run. The error is appearent in the image pulling stage already.

Others suggested using btrfs, I will have to try this out tomorrow. See:
https://forum.openwrt.org/t/docker-pull-fails-failed-to-register-layer-operation-not-supported/138253/52

And here's how to adapt Docker, although might be optional since just switching to btrfs might as well be enough:
https://docs.docker.com/storage/storagedriver/btrfs-driver/

I do have the ability to google, and I only open issues when I really can't fix something with the help of online sources. This is why I'm here in the first place...

[vtcosta]

If you were having this problem in a EXT4 filesystem, it has been fixed in the latest snapshot.

For more information, see PR below:
openwrt/openwrt#16181

[zhengliw]

that's great to hear, thanks!
also, running openwrt on an x86 machine for self-hosting wasn't the greatest idea...

[namchuai]

I'm still having this issue. Just install openwrt yesterday on my rpi4.

NAME="OpenWrt"
VERSION="23.05.5"
ID="openwrt"
ID_LIKE="lede openwrt"
PRETTY_NAME="OpenWrt 23.05.5"
VERSION_ID="23.05.5"
HOME_URL="https://openwrt.org/"
BUG_URL="https://bugs.openwrt.org/"
SUPPORT_URL="https://forum.openwrt.org/"
BUILD_ID="r24106-10cc5fcd00"
OPENWRT_BOARD="bcm27xx/bcm2711"
OPENWRT_ARCH="aarch64_cortex-a72"
OPENWRT_TAINTS=""
OPENWRT_DEVICE_MANUFACTURER="OpenWrt"
OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/"
OPENWRT_DEVICE_PRODUCT="Generic"
OPENWRT_DEVICE_REVISION="v0"
OPENWRT_RELEASE="OpenWrt 23.05.5 r24106-10cc5fcd00"

[cronner]

same issue in 23.05.5

[vtcosta]

It seems to be fixed only for the SNAPSHOT versions of OpenWrt. Please, see the latest responses from the following thread:
https://forum.openwrt.org/t/docker-pull-fails-failed-to-register-layer-operation-not-supported/138253/75

If you need to use the release 23.05.5, try to configure docker on a BTRFS filesystem:
https://docs.docker.com/engine/storage/drivers/btrfs-driver/