Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adjusting README and Dockerfile for x86_64 and arm64 usage #22

Merged
merged 8 commits into from
Apr 22, 2024
34 changes: 29 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,18 +1,42 @@
FROM python:3.11.8
################################################################################
# getml-base
FROM python:3.11.8 AS getml-base

RUN useradd getml
USER getml
WORKDIR /home/getml

COPY --chown=getml:getml --chmod=0777 ./requirements.txt /home/getml/requirements.txt

ENV PATH="$PATH:/home/getml/.local/bin"
ENV PATH="/home/getml/.local/bin:$PATH"

RUN python3.11 \
-mpip install \
-r /home/getml/requirements.txt

RUN mkdir /home/getml/.getML /home/getml/.getML/logs /home/getml/.getML/projects /home/getml/demo
RUN curl https://storage.googleapis.com/static.getml.com/download/1.4.0/getml-1.4.0-x64-linux.tar.gz | tar -C /home/getml/.getML -xvzf -
RUN mkdir /home/getml/.getML


################################################################################
# getml-demo
FROM getml-base AS getml-demo

ARG TARGETARCH
ARG TARGETOS
RUN \
if [ "${TARGETARCH}" = "amd64" ]; then \
export GETML_ARCH="x64" ;\
else \
export GETML_ARCH="${TARGETARCH}" ;\
fi; \
export GETML_VERSION=$(grep -o "^getml==.*$" requirements.txt | cut -b8-) ;\
export GETML_BUCKET="https://storage.googleapis.com/static.getml.com/download" ;\
export GETML_ENGINE_FILE="getml-${GETML_VERSION}-${GETML_ARCH}-${TARGETOS}.tar.gz" ;\
export GETML_ENGINE_URL="${GETML_BUCKET}/${GETML_VERSION}/${GETML_ENGINE_FILE}" ;\
echo "Downloading getML engine from ${GETML_ENGINE_URL}" ;\
curl ${GETML_ENGINE_URL} | tar -C /home/getml/.getML -xvzf -

COPY --chown=getml:getml . /home/getml/demo/

EXPOSE 1709 8888
CMD [ "/home/getml/.local/bin/jupyter", "lab", "--ip='*'", "--port=8888", "--notebook-dir='/home/getml/demo'" ]
CMD [ "/home/getml/.local/bin/jupyter", "lab", "--ip='*'", "--port=8888", "--notebook-dir='/home/getml/demo'" ]
27 changes: 16 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ This repository contains different [Jupyter Notebooks](https://jupyter.org/) to
* [Usage](#usage)
* [Reading Online](#reading-online)
* [Experimenting Locally](#experimenting-locally)
* [Using Docker or Podman](#using-docker-or-podman)
* [On the Machine (Linux/x64)](#on-the-machine-linuxx64)
* [Using Docker](#using-docker)
* [On the Machine (Linux/x64 & arm64)](#on-the-machine-linuxx64--arm64)
Urfoex marked this conversation as resolved.
Show resolved Hide resolved
* [Notebooks](#notebooks)
* [Overview](#overview)
* [Descriptions](#descriptions)
Expand All @@ -51,26 +51,23 @@ As github renders the notebooks, they can each be viewed by just opening and scr

To experiment with the notebooks, such as playing with different pipelines and predictors, it is best to run them on a local machine. Linux users with an x64 architecture can choose from one of the options provided below. Soon, we will offer a simple, container-based solution compatible with all major systems (Windows, Mac) and will also support ARM-based architectures.

### Using Docker or Podman
### Using Docker

There are a `docker-compose.yml` and a `Dockerfile` for easy usage provided.

Simply clone this repository and command to start the `notebooks` service. The image, it depends on, will be build if it is not already available.

```
$ git clone https://github.com/getml/getml-demo.git
$ docker-compose up notebooks
$ docker compose up notebooks
```

> [!NOTE]
> The files are set up to also work with [podman](https://podman.io/) and [podman-compose](https://github.com/containers/podman-compose)

To open Jupyter Lab in the browser, look for the following lines in the output and copy-paste it in your browser:

```
Or copy and paste one of these URLs:

http://localhost:8888/lab?token=<randomly_generated_token>
http://localhost:8888/lab?token=<generated_token>
```

After the first `getml.engine.launch(...)` is executed and the engine is started, its monitor can be opened in the browser under
Expand All @@ -79,7 +76,15 @@ After the first `getml.engine.launch(...)` is executed and the engine is started
http://localhost:1709/#/token/token
```

### On the Machine (Linux/x64)
> [!NOTE]
> Using alternatives to [Docker Desktop](https://www.docker.com/products/docker-desktop/) like
> * [Podman](https://podman.io/),
> * [Podman Desktop](https://podman-desktop.io/) or
> * [Rancher Desktop](https://rancherdesktop.io/) with a container engine like dockerd(moby) or containerd(nerdctl)
>
> allows bind-mounting the notebooks in a user-writeable way (this might need to include `userns_mode: keep-id`) instead of having to `COPY` them in. In combination with volume-binding `/home/getml/.getML/logs` and `/home/getml/.getML/projects`, runs and changes can be persisted across containers.

### On the Machine (Linux/x64 & arm64)

Alternatively, getML and the notebooks can be run natively on the local Linux machine by having certain software installed, like Python and some Python libraries, Jupyter-Lab and the getML engine. The [getML Python library](https://github.com/getml/getml-community/) provides an engine version without [enterprise features](https://www.getml.com/pricing). But as those features are shown in the demonstration notebooks, the [trail of the enterprise version](https://www.getml.com/download) can be used for those cases.

Expand All @@ -91,7 +96,7 @@ $ cd getml-demo
$ pipx install hatch
$ hatch env create
$ hatch shell
$ pip install -r requirements/requirements.3.11.txt
$ pip install -r requirements.txt
$ jupyter-lab
```

Expand All @@ -103,7 +108,7 @@ With the last command, Jupyter-Lab should automatically open in the browser. If
```
Or copy and paste one of these URLs:

http://localhost:8888/lab?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
http://localhost:8888/lab?token=<generated_token>
```

After the first `getml.engine.launch(...)` is executed and the engine is started, its monitor can be opened in the browser under
Expand Down
10 changes: 1 addition & 9 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,12 @@ services:
build:
context: ./
dockerfile: Dockerfile
target: getml-demo
networks:
- notebooks_network
ports:
- "1709:1709"
- "8888:8888"
userns_mode: keep-id
volumes:
- ./:/home/getml/demo
- notebooks_logs_volume:/home/getml/.getML/logs
- notebooks_projects_volume:/home/getml/.getML/projects

networks:
notebooks_network:

volumes:
notebooks_logs_volume:
notebooks_projects_volume:
65 changes: 36 additions & 29 deletions dodgers.ipynb

Large diffs are not rendered by default.

3 changes: 1 addition & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,4 @@ seaborn==0.13.2
ipywidgets==8.1.2
plotly==5.18.0
prophet==1.1.5
matplotlib==3.8.2
pystan==3.9.0
matplotlib==3.8.2