Support fine-grained database schema migrations

[edolstra]

Motivation

Backward-compatible schema changes (e.g. those that add tables or nullable columns) now no longer need a change to the global schema file (/nix/var/nix/db/schema). Thus, old Nix versions can continue to access the database.

This is especially useful for schema changes required by experimental features. In particular, it replaces the ad-hoc handling of the schema changes for CA derivations (i.e. the file /nix/var/nix/db/ca-schema).

Schema versions 8 and 10 could have been handled by this mechanism in a backward-compatible way as well.

Context

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[Mic92]

Does this also help with the deadlock that are seeing when enabling ca derivations? Currently it requires a reboot of the machine because existing running nix processes will not allow the schema migration from happening as they hold the global lock file.

[edolstra]

@Mic92 It should help with the deadlock since it doesn't hold the global lock anymore while doing SQLite schema migrations. (Maybe it should, but AFAIK SQLite doesn't need it.)

[Mic92]

Great that's my main road block to deploy this by default on a larger amount of machines

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-10-16-nix-team-meeting-minutes-187/54835/1

[bryanhonof]

To me, this looks good. Haven't tested it out though, also not sure how one might test this.

[Mic92]

To me, this looks good. Haven't tested it out though, also not sure how one might test this.

On a system without ca-derivations, enable ca-derivations and see if they work.