Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adjusting README and Dockerfile for x86_64 and arm64 usage #22

Merged
merged 8 commits into from
Apr 22, 2024
Merged
1 change: 1 addition & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
GETML_VERSION=1.4.0
34 changes: 30 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,18 +1,44 @@
FROM python:3.11.8
################################################################################
# getml-base
FROM python:3.11.8 AS getml-base

RUN useradd getml
USER getml
WORKDIR /home/getml

COPY --chown=getml:getml --chmod=0777 ./requirements.txt /home/getml/requirements.txt

ENV PATH="$PATH:/home/getml/.local/bin"
ENV PATH="/home/getml/.local/bin:$PATH"

RUN python3.11 \
-mpip install \
-r /home/getml/requirements.txt

RUN mkdir /home/getml/.getML /home/getml/.getML/logs /home/getml/.getML/projects /home/getml/demo
RUN curl https://storage.googleapis.com/static.getml.com/download/1.4.0/getml-1.4.0-x64-linux.tar.gz | tar -C /home/getml/.getML -xvzf -
RUN mkdir /home/getml/.getML

################################################################################
# getml-amd64
FROM getml-base AS getml-amd64
ARG GETML_ARCH=x64

################################################################################
# getml-arm64
FROM getml-base AS getml-arm64
ARG GETML_ARCH=arm64

################################################################################
# getml-demo
FROM getml-${TARGETARCH} AS getml-demo
Urfoex marked this conversation as resolved.
Show resolved Hide resolved

ARG TARGETOS
ARG GETML_VERSION
ARG GETML_BUCKET="https://storage.googleapis.com/static.getml.com/download"
ARG GETML_ENGINE_FILE="getml-${GETML_VERSION}-${GETML_ARCH}-${TARGETOS}.tar.gz"
Urfoex marked this conversation as resolved.
Show resolved Hide resolved
ARG GETML_ENGINE_URL="${GETML_BUCKET}/${GETML_VERSION}/${GETML_ENGINE_FILE}"

RUN curl "${GETML_ENGINE_URL}" | tar -C /home/getml/.getML -xvzf -

COPY --chown=getml:getml . /home/getml/demo/

EXPOSE 1709 8888
CMD [ "/home/getml/.local/bin/jupyter", "lab", "--ip='*'", "--port=8888", "--notebook-dir='/home/getml/demo'" ]
27 changes: 16 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ This repository contains different [Jupyter Notebooks](https://jupyter.org/) to
* [Usage](#usage)
* [Reading Online](#reading-online)
* [Experimenting Locally](#experimenting-locally)
* [Using Docker or Podman](#using-docker-or-podman)
* [On the Machine (Linux/x64)](#on-the-machine-linuxx64)
* [Using Docker](#using-docker)
* [On the Machine (Linux/x64 & arm64)](#on-the-machine-linuxx64--arm64)
Urfoex marked this conversation as resolved.
Show resolved Hide resolved
* [Notebooks](#notebooks)
* [Overview](#overview)
* [Descriptions](#descriptions)
Expand All @@ -51,26 +51,23 @@ As github renders the notebooks, they can each be viewed by just opening and scr

To experiment with the notebooks, such as playing with different pipelines and predictors, it is best to run them on a local machine. Linux users with an x64 architecture can choose from one of the options provided below. Soon, we will offer a simple, container-based solution compatible with all major systems (Windows, Mac) and will also support ARM-based architectures.

### Using Docker or Podman
### Using Docker

There are a `docker-compose.yml` and a `Dockerfile` for easy usage provided.

Simply clone this repository and command to start the `notebooks` service. The image, it depends on, will be build if it is not already available.

```
$ git clone https://github.com/getml/getml-demo.git
$ docker-compose up notebooks
$ docker compose up notebooks
```

> [!NOTE]
> The files are set up to also work with [podman](https://podman.io/) and [podman-compose](https://github.com/containers/podman-compose)

To open Jupyter Lab in the browser, look for the following lines in the output and copy-paste it in your browser:

```
Or copy and paste one of these URLs:

http://localhost:8888/lab?token=<randomly_generated_token>
http://localhost:8888/lab?token=<generated_token>
```

After the first `getml.engine.launch(...)` is executed and the engine is started, its monitor can be opened in the browser under
Expand All @@ -79,7 +76,15 @@ After the first `getml.engine.launch(...)` is executed and the engine is started
http://localhost:1709/#/token/token
```

### On the Machine (Linux/x64)
> [!NOTE]
> Using alternatives to [Docker Desktop](https://www.docker.com/products/docker-desktop/) like
> * [Podman](https://podman.io/),
> * [Podman Desktop](https://podman-desktop.io/) or
> * [Rancher Desktop](https://rancherdesktop.io/) with a container engine like dockerd(moby) or containerd(nerdctl)
>
> allows bind-mounting the notebooks in a user-writeable way (this might need to include `userns_mode: keep-id`) instead of having to `COPY` them in. In combination with volume-binding `/home/getml/.getML/logs` and `/home/getml/.getML/projects`, runs and changes can be persisted across containers.

### On the Machine (Linux/x64 & arm64)

Alternatively, getML and the notebooks can be run natively on the local Linux machine by having certain software installed, like Python and some Python libraries, Jupyter-Lab and the getML engine. The [getML Python library](https://github.com/getml/getml-community/) provides an engine version without [enterprise features](https://www.getml.com/pricing). But as those features are shown in the demonstration notebooks, the [trail of the enterprise version](https://www.getml.com/download) can be used for those cases.

Expand All @@ -91,7 +96,7 @@ $ cd getml-demo
$ pipx install hatch
$ hatch env create
$ hatch shell
$ pip install -r requirements/requirements.3.11.txt
$ pip install -r requirements.txt
$ jupyter-lab
```

Expand All @@ -103,7 +108,7 @@ With the last command, Jupyter-Lab should automatically open in the browser. If
```
Or copy and paste one of these URLs:

http://localhost:8888/lab?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
http://localhost:8888/lab?token=<generated_token>
```

After the first `getml.engine.launch(...)` is executed and the engine is started, its monitor can be opened in the browser under
Expand Down
14 changes: 4 additions & 10 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,14 @@ services:
build:
context: ./
dockerfile: Dockerfile
target: getml-demo
args:
- GETML_VERSION=$GETML_VERSION
networks:
- notebooks_network
ports:
- "1709:1709"
- "8888:8888"
userns_mode: keep-id
volumes:
- ./:/home/getml/demo
- notebooks_logs_volume:/home/getml/.getML/logs
- notebooks_projects_volume:/home/getml/.getML/projects

networks:
notebooks_network:

volumes:
notebooks_logs_volume:
notebooks_projects_volume:
notebooks_network:
65 changes: 36 additions & 29 deletions dodgers.ipynb

Large diffs are not rendered by default.

3 changes: 1 addition & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,4 @@ seaborn==0.13.2
ipywidgets==8.1.2
plotly==5.18.0
prophet==1.1.5
matplotlib==3.8.2
pystan==3.9.0
matplotlib==3.8.2