Set devcontainer.json

[eitsupi]

How about setting up devcontainer(s) to easily configure the development environment?
https://containers.dev/

Setting this up could make it easier to contribute, as the same environment can be created in GitHub Codespaces.
(I think the current setup with Taskfile is well done, but I personally have some resistance to it because it includes the kind of things I don't want to install locally.)

I think devcontainer is popular enough, but it is a bad point that it currently only has real value in VS Code Dev Containers or Codespaces, which are proprietary software, so it has no value for those who do not use these IDEs It is not good for those who do not use these IDEs.
(It is possible to create a container using the OSS Dev Container CLI and install an editor inside the container to utilize it from other editors, but installing the Dev Container CLI is a pain)

I have been developing on Dev Containers for years and would be happy to help if you need setup.

[max-sixty]

I think we discussed this a while ago. I think this would be great to have if folks would use it.

I've tried hard to make it easy to contribute — some parts of it are good — but it still requires installing things like task up front, and installing lots of stuff. And it's only really supported on Mac. A devcontainer would give a faster, easier way of getting started, even if the cap is a bit lower.

We could possibly start from the existing Dockerfile?

(btw @eitsupi what would you prefer the Taskfile not install? Things like elixir & hugo?)

[eitsupi]

Thank you for your reply.

I've tried hard to make it easy to contribute — some parts of it are good — but it still requires installing things like task up front, and installing lots of stuff. And it's only really supported on Mac. A devcontainer would give a faster, easier way of getting started, even if the cap is a bit lower.

Yes, the Taskfile in this repository is great!
However, an installation that assumes Homebrew will of course not work on Windows (I love the fact that Task also works on Windows due to the built-in shell.), and may not be ideal on Linux either.

Therefore, my suggestion was to install runtimes, etc. that need to be installed by Homebrew, etc. on Dev Container, and continue to install by Cargo, etc. using Taskfile, so that it can work in a wider range of applications.

We could possibly start from the existing Dockerfile?

As far as I can see, a Dockerfile is not required.
For now I am using the following json file without committing.
(of course this doesn't cover everything in the repository, just Rust, Node and Python for now)

{
  "image": "mcr.microsoft.com/devcontainers/rust:0-1-bullseye",
  "features": {
    "ghcr.io/eitsupi/devcontainer-features/go-task:1": {},
    "ghcr.io/devcontainers/features/python:1": {},
    "ghcr.io/devcontainers/features/node:1": {},
    "ghcr.io/rocker-org/devcontainer-features/apt-packages:1": {
      "packages": "clang"
    }
  },
  "mounts": [
    {
      "source": "devcontainer-cargo-cache",
      "target": "/usr/local/cargo",
      "type": "volume"
    }
  ]
}

By the way, prqlr can prepare a development environment with R and Rust installed in devcontainer.json, so if you have time, try running this in Codespaces and see how it works.
https://github.com/eitsupi/prqlr/blob/6818f68e0d867e1880086355e3ec511717048b99/.devcontainer/devcontainer.json

(btw @eitsupi what would you prefer the Taskfile not install? Things like elixir & hugo?)

I'm quite fastidious and avoid installing language runtimes and Python packages.
For the past few years, my development environment has always been on Docker.

Definitely I am just being extreme and I am sorry if I have offended you.

[max-sixty]

Definitely I am just being extreme and I am sorry if I have offended you.

Sorry I didn't mean this at all! I was genuinely curious to learn. I totally understand your perspective. Sorry if my question sounded abrupt, I didn't mean it like that at all — my sense is the same as yours — that some people like the Taskfile but others don't at all (@aljazerzen doesn't use it IIUC)

[max-sixty]

As far as I can see, a Dockerfile is not required. For now I am using the following json file without committing. (of course this doesn't cover everything in the repository, just Rust, Node and Python for now)

Ah interesting, they're simpler to define now.

I'm trying the one for prqlr now, it's good. We could add things like rust-analyzer extension — that's really useful for navigating rust files. We also put our suggested extensions in https://github.com/PRQL/prql/blob/main/.vscode/extensions.json, so VSCode will suggest them when someone opens the project (useful for folks not using a Dev Container)

Though one nice thing about using a bare Dockerfile rather than the vscode definition is that we can share code with folks who want to use a docker container outside of VSCode. But not a big deal.

(FWIW if you want to make one for prql, running task setup-dev doesn't quite work, since it requires brew being installed, so we'd have to adjust that a bit)

Overall I'm +1 on adding this if someone would find it helpful. I wouldn't use it for PRQL, but I would for prqlr, since my R environment is not well-developed.

[eitsupi]

Thank you for taking a look at this. I would like to work on this.

We could add things like rust-analyzer extension — that's really useful for navigating rust files.

These key extensions may be automatically installed in the container as part of the Dev Container Features.

They are actually defined in the following location, and this will be recorded as metadata on the Docker image label, and VS Code Dev Containers or Codespaces reading the metadata will automatically install the extension. So there is no need for us to specify it explicitly.
https://github.com/devcontainers/features/blob/4a9929f96485061e3778b35848e21d7c3c193480/src/rust/devcontainer-feature.json#L36

[richb-hanover]

@max-sixty @eitsupi I was gearing up to learn more about docker-compose to work on #1563 (caching Rust build products, likely using docker-compose) when I saw this item. The little I know about a PRQL Dev Container is that could address my concerns:

• Work on the Book, Playground, and Website
• Work on the Rust pieces (compiler, language bindings, etc.)
• Without installing Rust or other tools on my laptop

My questions:

1. Are those assumptions correct (that the Dev Container could do this)?
2. Where does the Dev Container stand today? It looks like @eitsupi has some of this working
3. Are there places where I could help? (Even though I don't "speak Rust", and likely wouldn't venture into that corner, I think this would be valuable for the project)

Thanks.

[eitsupi]

Are those assumptions correct (that the Dev Container could do this)?

Correct. I have Rust installed on my machine, probably as a dependency of some CLI tool, but I have never used it for development.

Only VS Code, VS Code Dev Containers and Docker need to be installed.
When using Codespaces, everything is done in the browser, so nothing is needed.

Where does the Dev Container stand today? It looks like @eitsupi has some of this working

Sorry but I have not yet done the work to create a PR for this. I will create an initial PR today.

Are there places where I could help? (Even though I don't "speak Rust", and likely wouldn't venture into that corner, I think this would be valuable for the project)

It would be helpful if you could test what I have created on Codespaces and other places.

[richb-hanover]

Great. I'll ask for a couple hints getting started with Dev Containers and Codespaces, but that could be fun. Thanks.

[eitsupi]

To use Codespaces, simply press the button in two steps as shown in the image below.

Since the devcontainer.json file is in devcontainer branch now (#1893), it may be necessary to copy the branch for use in your fork.

[richb-hanover]

I built the Dev Container with VS Code on my Macbook. It did take a while (just under 20 minutes) to build on my 2.3GHz i9. (I also built the Dev Container it on a M2 Mac mini - it took less than 12 minutes.)

As I watched the messages scroll by, it's obvious that it's doing all the things that happened when running docker build. (I guess there are no surprises here.)

Now I'm looking at the messages below, and I'm not sure what to do next.

1. These look like suggested commands I should run. How do I run them "in the container"?
2. How do I work on various projects, such as cd book; hugo server...?

Thanks!

...
🟢 Setup complete! ✅🚀
[1191536 ms] Start: Run in container: # Test for /home/vscode/.gitconfig and git
[1191540 ms] 
[1191540 ms] 
[1191541 ms] Start: Run in container: # Copy /Users/richb/.gitconfig to /home/vscode/.gitconfig
[1191548 ms] 
[1191548 ms] 
[1191550 ms] Start: Run in container: command -v git >/dev/null 2>&1 && git config --global --replace-all credential.helper '!f() { /home/vscode/.vscode-server/bin/441438abd1ac652551dbe4d408dfcec8a499b8bf/node /tmp/vscode-remote-containers-fc5f59b7a2dc05bd17e4948731fc8f191a2fd774.js git-credential-helper $*; }; f' || true
[1191555 ms] 
[1191555 ms] 
[1191556 ms] Start: Run in container: git config --global --add safe.directory '/workspaces/dc-prql'
[1191560 ms] 
[1191560 ms] 

[eitsupi]

Thanks for testing this!

These look like suggested commands I should run. How do I run them "in the container"?

All of these are executed within the container.

How do I work on various projects, such as cd book; hugo server...?

These are definitely areas that need to be documented.
These commonly used commands are defined in the Taskfile and task is already installed in the container. (By the way, zsh in the container should be able to auto-complete for task.)

You can display predefined tasks with task -l in project root.

For example, executing task run-website will launch the website.

prql/Taskfile.yml

Lines 288 to 292 in d8b7e63

	run-website:
	desc: Build & serve the static website.
	dir: website
	cmds:
	- hugo server

[richb-hanover]

Yes - I figured out how to run task run-website. (THIS IS SO COOL!)

There definitely needs to be a "Howto for Newcomers" that shows how to do the simple (but inscrutable, until someone shows you) steps to get the whole Dev Container built and running. I'm taking notes as I go and will write something up for comment by people who know more than I do. Thanks.