Merge `bootc` branch to master #2180

evan-goode · 2024-12-13T19:09:16Z

This PR, along with rpm-software-management/libdnf#1683, merges support for the --transient flag and the persistence=auto|persist|transient configuration option for https://issues.redhat.com/browse/RHEL-70917. See also #2155.

Also included are some build files for Copr that may be useful for future feature branches similar to bootc.

Merging this PR doesn't mean that work on the --transient feature is 100% done, just that we're confident enough we're not going to totally scrap it and we want to continue its development on the master branch.

Support automatic builds on git commits. Used to generate up to date RPMs of dnf for testing and development purposes.

Text alignment and formatting in the spec file.

Adds support for the --transient option on all transactions. Passing --transient on a bootc system will call `bootc usr-overlay` to create a transient writeable /usr and continue the transaction. Specifying --transient on a non-bootc system will throw an error; we don't want to mislead users to thinking this feature works on non-bootc systems. If --transient is not specified and the bootc system is in a locked state, the operation will be aborted and a message will be printed suggesting to try again with --transient.

evan-goode · 2024-12-17T19:09:58Z

#2186 should be merged first.

The --transient flag only works on a bootc system, but we should test that it throws the correct error and doesn't do anything unexpected on non-bootc systems. Requires rpm-software-management/dnf#2180 and rpm-software-management/libdnf#1683.

Documents the new `--transient` command-line argument and `persistence` configuration option. I tried to use a table for listing the valid options for `persistence`, but RST does not automatically wrap table cells containing long lines, so a list was much easier.

Using libostree gives us more detail about the current state of the deployment than only checking whether /usr is writable.

To keep /usr read-only after DNF is finished with a transient transaction, we call `ostree admin unlock --transient` to mount the /usr overlay as read-only by default. Then, we create a private mount namespace for DNF and its child processes and remount the /usr overlayfs as read/write in the private mountns. os.unshare is unfortunately only available in Python >= 3.12, so we have to call libc.unshare via Python ctypes here and hardcode the CLONE_NEWNS flag that we need to pass.

evan-goode · 2025-01-30T23:31:43Z

For testing this, you'll need to do the following broad steps:

Write a Containerfile. For example:

# Containerfile
FROM quay.io/centos-bootc/centos-bootc:stream9

RUN echo 'root:password' | chpasswd

# Use the dnf-bootc Copr built from the bootc branches, or alternatively you could COPY your own RPMs into the container
RUN dnf -y install dnf-plugins-core && \
	dnf -y copr enable dcantrell/dnf-bootc && \
	dnf -y distro-sync --repo copr:copr.fedorainfracloud.org:dcantrell:dnf-bootc

EDIT: DO NOT USE dnf -y update IN YOUR CONTAINERFILE. See https://docs.fedoraproject.org/en-US/bootc/building-containers/#_dnf_y_update.

Build a container from the Containerfile using podman
Build a qcow2 image from the container using podman-bootc or bootc-image-builder
Boot the qcow2 image in a VM using standalone QEMU, libvirt, or podman-bootc

podman-bootc is the recommended way to do all this, but from my experience it can be a little unreliable. I personally use these scripts based on bootc-image-builder and virt-install.

dnf-bootc's only job is to Require python3-gobject-base and ostree-libs, which are needed to check the unlock status on bootc systems. We don't want to add these dependencies on `python3-dnf` because we don't want them on non-bootc systems, so we use a subpackage.

.copr/Makefile

dnf.spec

dnf/cli/cli.py

dnf.spec

ppisar · 2025-02-04T15:59:05Z

I was unable to test it in RHEL 9.6 bootc image as dbus fails to start for me there. I will try testing it within Fedora 40 bootc image.

evan-goode · 2025-02-04T20:30:19Z

I was unable to test it in RHEL 9.6 bootc image as dbus fails to start for me there. I will try testing it within Fedora 40 bootc image.

Today I noticed that my VMs based on quay.io/centos-bootc/centos-bootc:stream9 started breaking (I experienced kernel panics due to missing rootfs and also DBus failing to start) since my Containerfile had the line dnf -y upgrade. Which is precisely not recommended because it breaks images unpredictably. Maybe that is the problem in your case?

ppisar · 2025-02-05T09:33:46Z

I do update all packages. I found the cause why dbus fails to start: /etc/dbus-1/system.d has a wrong selinux label inside the image. It is etc_t, while it should be dbusd_etc_t. As far as I know ostree does not directly support extended attributes, hence the bootc tooling manages them in a separate database. Obviously the database went mad.

ppisar · 2025-02-05T12:48:04Z

Since today I also get a failure in mounting the root file system. Not updating all packages indeed works it around.

ppisar · 2025-02-05T12:49:22Z

I tried applying the commits to F40 package and have a problem getting the code working. Installing or uninstalling any package in nonbootc system raises this exception:

AttributeError: 'ConfigMain' object has no attribute 'persistence'. Did you mean: 'persistdir'?

The same happens when I run DNF from the bootc branch directly. Maybe I did something wrong.

I'd like to know whether you are going to squash some commits and especially to rebase this bootc branch to the current master? Because the commits are, from an archeological point of view, one big mess.

ppisar · 2025-02-05T13:35:06Z

I tried applying the commits to F40 package and have a problem getting the code working. Installing or uninstalling any package in nonbootc system raises this exception:
AttributeError: 'ConfigMain' object has no attribute 'persistence'. Did you mean: 'persistdir'?

That's because this pull request requires libdnf change rpm-software-management/libdnf#1683.

ppisar · 2025-02-05T15:25:05Z

I tried applying the commits to F40 package and have a problem getting the code working. Installing or uninstalling any package in nonbootc system raises this exception:
AttributeError: 'ConfigMain' object has no attribute 'persistence'. Did you mean: 'persistdir'?
That's because this pull request requires libdnf change rpm-software-management/libdnf#1683.

To prevent this we should require a minimal version of libdnf from python3-dnf package. It's not so important for upstream spec files, but handy for dnf.spec in the distributions.

evan-goode · 2025-02-05T15:39:25Z

I'd like to know whether you are going to squash some commits and especially to rebase this bootc branch to the current master? Because the commits are, from an archeological point of view, one big mess.

Sure, I rebased onto master and reordered/squashed some commits just now on my evan-goode/bootc branch. I think I'll have to create a new PR unless I want to rewrite history on the bootc branch, GitHub does not give me the option to change the source branch here.

To prevent this we should require a minimal version of libdnf from python3-dnf package. It's not so important for upstream spec files, but handy for dnf.spec in the distributions.

Done on https://github.com/evan-goode/dnf/tree/evan-goode/bootc.

Bumping libdnf version to 0.74.0 in rpm-software-management/libdnf#1692.

evan-goode · 2025-02-05T15:42:16Z

Closing in favor of #2203, I'm more comfortable rewriting history on my own branch than the bootc branch here on upstream.

The --transient flag only works on a bootc system, but we should test that it throws the correct error and doesn't do anything unexpected on non-bootc systems. Requires rpm-software-management/dnf#2180 and rpm-software-management/libdnf#1683.

dcantrell and others added 3 commits November 14, 2024 13:34

copr: Add Copr build files

877a8f2

Support automatic builds on git commits. Used to generate up to date RPMs of dnf for testing and development purposes.

copr: Small Copr build file improvements

6bbe920

Text alignment and formatting in the spec file.

evan-goode mentioned this pull request Dec 13, 2024

Merge bootc branch to master rpm-software-management/libdnf#1683

Merged

evan-goode mentioned this pull request Jan 15, 2025

Add test for --transient and persistence option rpm-software-management/ci-dnf-stack#1618

Merged

evan-goode marked this pull request as ready for review January 15, 2025 18:50

evan-goode mentioned this pull request Jan 15, 2025

Merge bootc branch to dnf-4-stack rpm-software-management/ci-dnf-stack#1619

Merged

evan-goode added 3 commits January 20, 2025 10:40

doc: Note transient transactions will appear in history

742f1cd

bootc: Use ostree GObject API to get deployment status

337795f

Using libostree gives us more detail about the current state of the deployment than only checking whether /usr is writable.

mcurlej assigned mcurlej, ppisar and jrohel and unassigned mcurlej Feb 3, 2025

ppisar reviewed Feb 3, 2025

View reviewed changes

.copr/Makefile Outdated Show resolved Hide resolved