Implement timoni mod build

[jmgilman]

Currently, it appears the only way to publish a Timoni module in OCI format is to use timoni mod push. This is unfortunate, as it limits what an end-user can do with the OCI artifact prior to pushing it to a registry. This is especially true in existing CI systems, where the desire may be to integrate Timoni as another step (i.e. to produce container images, but not necessarily publish them until later).

In this sense, I believe something like timoni mod build would be a helpful intermediate step that performs the same operations as timoni mod publish but without pushing the image to a registry.

[jmgilman]

@stefanprodan FYI I will see about attempting this if the use case sounds good to you. If you have any pointers feel free to share them. It will probably take me a bit to read through the code base.

[stefanprodan]

@jmgilman you could add a new function to the OCI package, same code as in push, but after this line dump the image to disk instead of pushing it:

timoni/internal/oci/push_module.go

Line 108 in da003ac

if err := crane.Push(img, ref.String(), opts...); err != nil {

The command could look like timoni mod build --output my-module.tgz.

[rp-thomas]

@stefanprodan have another question on this. Would it make sense to be able to reference the my-module.tgz in a bundle file?

Usecase: I'm currently working on a bunch of different modules for a bigger application and want to bundle these in a bundle.cue.
If I want to test the bundle creation e.g. timoni bundle build or apply it in a local cluster, I have to publish each module every time to the registry before I can test the whole bundle.

[stefanprodan]

For local testing, @b4nst was suggesting to allow local paths to modules. The issue is that a bundle can be made of many files at diffrent paths on disk, if the path to some module is relative to the bundle file path it can become a nightmare really fast. I guess we could use the path of the first bundle file passed to the CLI as the base for the relative paths to modules...

[b4nst]

Yeah, I still did not find time to work on that proposal. My initial idea was to make the path relative to the bundle where the module is referenced. There's a way with CUE to get the path where the concrete value has been set IIRC. I think from an UX perspective that would be the most natural. Inside each bundle file the path is relative to that bundle file.
It's still on my todo!

[stefanprodan]

Inside each bundle file the path is relative to that bundle file.

But a bundle can be composed of many files at different paths e.g.:

timoni bundle apply -f ./some/path/to/bundle.cue -f /some/other/path/bundle.extas.cue

After we unify the bundle definition which would be the root for the relative paths?

[b4nst]

Nah what I was saying is that if we setup the CUE build properly, I think we can still find reference to the original file even after unification.

So everything in ./some/path/to/bundle.cue should be relative to ./some/path/to/bundle.cue and everything in /some/other/path/bundle.extras.cue would be relative to /some/other/path/bundle.extras.cue

[b4nst]

Okay I just confirmed ☝️. I made a quick bundle debug command, which is basically a copy pasta of bundle build but as soon as it get to v, err := bm.Build() I shortened it to:

	pos := v.LookupPath(cue.ParsePath("bundle.instances.redis.module.url")).Pos()
	fmt.Printf("pos: %+v\n", pos)

	return nil

Run it with those 2 files

// bundle.cue
bundle: {
	apiVersion: "v1alpha1"
	name:       "podinfo"
	instances: redis: module: url: string
}

// redis.cue
bundle: {
	instances: redis: {
		module: {
			url:     "oci://ghcr.io/stefanprodan/modules/redis"
			version: "7.2.3"
		}
		namespace: "default"
	}
}

and you get:

pos: /var/folders/vl/j712bqm55gd754rr5tdm8bxm0000gn/T/timoni1061324987/1.redis.cue.cue:4:13

The Pos reflects the original place of the redis file, which is indeed the one with the concrete value for module url.

So the only thing I need to do is either keeping a map of Bundle Workspace -> Original path during bundle.InitWorkspace, or even better sending the correct path to CUE.

I'll start a PR.