Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bake): allow concurrent projects with x-depot bake extension #288

Merged
merged 5 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 40 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,12 @@ For all other platforms, you can download the binary directly from [the latest r

Run a Docker build from a HCL, JSON, or Compose file using Depot's remote builder infrastructure. This command accepts all the command line flags as Docker's `docker buildx bake` command, you can run `depot bake --help` for the full list.

The `bake` command needs to know which [project](https://depot.dev/docs/core-concepts#projects) id to route the build to. For passing the project id you have three options available to you:
The `bake` command needs to know which [project](https://depot.dev/docs/core-concepts#projects) id to route the build to. For passing the project id you have four options available to you:

1. Run `depot init` at the root of your repository and commit the resulting `depot.json` file
2. Use the `--project` flag in your `depot bake` command
3. Set the `DEPOT_PROJECT_ID` environment variable which will be automatically detected
3. Set the `DEPOT_PROJECT_ID` environment variable which will be automatically detected.
4. Use [`x-depot` ](http://depot.dev/docs/cli/reference#compose-support) extension field in your `docker-compose.yml` file.

By default, `depot bake` will leave the built image in the remote builder cache. If you would like to download the image to your local Docker daemon (for instance, to `docker run` the result), you can use the `--load` flag.

Expand Down Expand Up @@ -109,6 +110,43 @@ If you want to build a specific target in the bake file, you can specify it in t
depot bake -f docker-bake.hcl original
```

#### compose support

Depot supports using bake to build [Docker Compose](https://depot.dev/blog/depot-with-docker-compose) files.

To use `depot bake` with a Docker Compose file, you can specify the file with the `-f` flag:

```shell
depot bake -f docker-compose.yml
```

Compose files have special extensions prefixed with `x-` to give additional information to the build process.

In this example, the `x-bake` extension is used to specify the tags for each service
and the `x-depot` extension is used to specify different project IDs for each.

```yaml
services:
mydb:
build:
dockerfile: ./Dockerfile.db
x-bake:
tags:
- ghcr.io/myorg/mydb:latest
- ghcr.io/myorg/mydb:v1.0.0
x-depot:
project-id: 1234567890
myapp:
build:
dockerfile: ./Dockerfile.app
x-bake:
tags:
- ghcr.io/myorg/myapp:latest
- ghcr.io/myorg/myapp:v1.0.0
x-depot:
project-id: 9876543210
```

#### Flags for `bake`

| Name | Description |
Expand Down
51 changes: 45 additions & 6 deletions pkg/buildx/bake/bake.go
Original file line number Diff line number Diff line change
Expand Up @@ -628,6 +628,8 @@ type Target struct {

// linked is a private field to mark a target used as a linked one
linked bool

ProjectID string `json:"project_id,omitempty" hcl:"project_id,optional" cty:"project_id"`
}

var _ hclparser.WithEvalContexts = &Target{}
Expand Down Expand Up @@ -730,6 +732,9 @@ func (t *Target) Merge(t2 *Target) {
if t2.NoCacheFilter != nil { // merge
t.NoCacheFilter = append(t.NoCacheFilter, t2.NoCacheFilter...)
}
if t2.ProjectID != "" {
t.ProjectID = t2.ProjectID
}
t.Inherits = append(t.Inherits, t2.Inherits...)
}

Expand Down Expand Up @@ -927,16 +932,50 @@ func (t *Target) GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(
return value.AsString(), nil
}

func TargetsToBuildOpt(m map[string]*Target, inp *Input) (map[string]build.Options, error) {
m2 := make(map[string]build.Options, len(m))
for k, v := range m {
bo, err := toBuildOpt(v, inp)
type DepotBakeOptions struct {
ProjectTargetOptions map[string]map[string]build.Options
}

// input is only used for remote bake.
func NewDepotBakeOptions(defaultProjectID string, targets map[string]*Target, input *Input) (*DepotBakeOptions, error) {
opts := &DepotBakeOptions{
ProjectTargetOptions: map[string]map[string]build.Options{},
}

for targetName, target := range targets {
projectID := target.ProjectID
if projectID == "" {
projectID = defaultProjectID
}
if projectID == "" {
return nil, errors.Errorf("Project ID is missing for target %s, please specify with --project, DEPOT_PROJECT_ID, or run `depot init`", targetName)
}
buildOpt, err := toBuildOpt(target, input)
if err != nil {
return nil, err
}
m2[k] = *bo

if _, ok := opts.ProjectTargetOptions[projectID]; !ok {
opts.ProjectTargetOptions[projectID] = map[string]build.Options{}
}
opts.ProjectTargetOptions[projectID][targetName] = *buildOpt
}

return opts, nil
}

// ProjectOpts returns the targeted build options for a specific project ID.
func (o *DepotBakeOptions) ProjectOpts(id string) map[string]build.Options {
return o.ProjectTargetOptions[id]
}

// ProjectIDs returns the x-depot project IDs.
func (o *DepotBakeOptions) ProjectIDs() []string {
projectIDs := make([]string, 0, len(o.ProjectTargetOptions))
for projectID := range o.ProjectTargetOptions {
projectIDs = append(projectIDs, projectID)
}
return m2, nil
return projectIDs
}

func updateContext(t *build.Inputs, inp *Input) {
Expand Down
16 changes: 16 additions & 0 deletions pkg/buildx/bake/compose.go
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,10 @@ type xbake struct {
// docs/manuals/bake/compose-file.md#extension-field-with-x-bake
}

type XDepot struct {
ProjectID string `yaml:"project-id,omitempty"`
}

type stringMap map[string]string
type stringArray []string

Expand All @@ -253,6 +257,18 @@ func (sa *stringArray) UnmarshalYAML(unmarshal func(interface{}) error) error {
// composeExtTarget converts Compose build extension x-bake to bake Target
// https://github.com/compose-spec/compose-spec/blob/master/spec.md#extension
func (t *Target) composeExtTarget(exts map[string]interface{}) error {
var xdepot XDepot
buf, ok := exts["x-depot"]
if ok && buf != nil {
yb, _ := yaml.Marshal(buf)
if err := yaml.Unmarshal(yb, &xdepot); err != nil {
return err
}
if xdepot.ProjectID != "" {
t.ProjectID = xdepot.ProjectID
}
}

var xb xbake

ext, ok := exts["x-bake"]
Expand Down
Loading
Loading