Skip to content

Commit

Permalink
Add a tutorial that uses cue-lang to generate larger configuration fi…
Browse files Browse the repository at this point in the history
…les (#149)
  • Loading branch information
svrnm authored Feb 6, 2025
1 parent f0cf899 commit 1b1d6d3
Show file tree
Hide file tree
Showing 7 changed files with 168 additions and 2 deletions.
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ learn using it with the step by step tutorial:
1. [Two services](./docs/tutorial/1-two-services.md)
2. [A database and more services](./docs/tutorial/2-a-database-and-more-services.md)
3. [Errors and randomness](./docs/tutorial/3-errors-and-randomness.md)
4. [Observability with OpenTelemetry](./docs/tutorial/4-observability-with-opentelemetry.md)
4. [Observability with cilium Hubble](./docs/tutorial/4-observability-with-cilium-hubble.md)
5. [Observability with OpenTelemetry](./docs/tutorial/5-observability-with-opentelemetry.md)
6. [Large deployments with cue](./docs/tutorial/5-large-deployments-with-cue/README.md)

## Configuration specification

Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial/1-two-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,4 +67,4 @@ simulation:
kubectl apply -f deployments/
```

In both cases you will be see three containers being started.
In both cases you will be see three containers being started.
12 changes: 12 additions & 0 deletions docs/tutorial/4-observability-with-cilium-hubble.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Tutorial 4: Observability with cilium hubble

> [!NOTE]
>
> This tutorial is work in progress.
If you run your application simulation on kubernetes you can use [cilium Hubble's ui](https://github.com/cilium/hubble).

Follow the instructions to install [cilium](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/) and [enable the Hubble UI](https://docs.cilium.io/en/stable/observability/hubble/hubble-ui/#hubble-ui).

If you already have deployed your application simulation in a previous tutorial (for example the [two services from tutorial 1](./1-two-services.md)), you can navigate to `http://localhost:12000/?namespace=app-sim-tutorial1`
in your browser, and you will see the interaction between the load generator, and the two services.
5 changes: 5 additions & 0 deletions docs/tutorial/5-observability-with-opentelemetry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Tutorial 4: Observability with OpenTelemetry

> [!NOTE]
>
> This tutorial is work in progress.
113 changes: 113 additions & 0 deletions docs/tutorial/6-large-deployments-with-cue/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Tutorial 5: Large deployments with cue

> [!NOTE]
>
> This tutorial is work in progress.
With application simulator you are able to describe a complex multi-service architecture without thinking about the business logic of each service that is part of your setup.
This allows you to create arbitrarily large deployments for your purposes, e.g. if you'd like to generate lots of telemetry data for your observability backend.

The only limitation that remains, is that YAML does not provide you with a level of programmability, that allows you to write a lot of similar services in one place. You have
to write them down one by one.

Instead of enriching the application simulation configuration file format with such a functionality, you can leverage configuration languages like [`pkl`](https://pkl-lang.org/), [`dhall`](https://dhall-lang.org/) or [`cue`](https://cuelang.org/).
In this tutorial we will use `cue` to demonstrate this use case.

## Requirements

Make sure you have `cue` installed locally, following [these instructions](https://cuelang.org/docs/introduction/installation/)

## Create cue file

To get started create the following file named [`large-application.cue`](../../examples/cue/large-application.cue) in an empty folder:

```cue
import "list"
import "math"
#depth: int @tag(depth,type=int)
#adjectives: ["swift", "happy", "brave", "clever", "silent", "bold", "lucky", "fierce", "gentle", "mighty", "shy", "curious", "wise", "playful", "proud", "loyal"]
#nouns: ["lion", "tiger", "eagle", "panda", "fox", "wolf", "hawk", "bear", "otter", "falcon", "rabbit", "panther", "deer", "owl", "cheetah", "dolphin"]
loaders: {
"user1": {
type: "curl",
wait: 5,
sleep: 2,
urls: [ "http://swift-lion/next" ]
}
}
services: {
for x in list.Range(0, #depth+1, 1) {
for y in list.Range(0, x+1, 1) {
"\(#adjectives[x])-\(#nouns[y])": {
type: "java"
endpoints: {
http: {
"/next": list.Concat([[
for i in [y, y+1] if x < #depth {
"http://\(#adjectives[x+1])-\(#nouns[i])/next"
}
], ["sleep,\(math.Exp2(x)+(y*10))"]])
}
}
}
}
}
}
```

## Export cue to YAML

Run the following command to generate a YAML file:

```shell
cue -t depth=3 export large-application.cue --out yaml > config.yaml
```

This will generate an application simulation configuration file with 10 services, since the `for` loop within the `services` block creates a tree of services, where each node talks to two children, until `depth` is reached:

```mermaid
flowchart TD
user1 --> swift-lion
swift-lion --> happy-lion
swift-lion --> happy-tiger
happy-lion --> brave-lion
happy-lion --> brave-tiger
happy-tiger --> brave-tiger
happy-tiger --> brave-eagle
brave-lion --> clever-lion
brave-lion --> clever-tiger
brave-tiger --> clever-tiger
brave-tiger --> clever-eagle
brave-eagle --> clever-eagle
brave-eagle --> cleaver-panda
```

By increasing the `depth` parameter, you can add additional layers, e.g. `depth=4` will create 15 services, `depth=5` will create 21 services and `depth=6` will create 28 services.

Additionally, each service will introduce some milliseconds of delay using `sleep` depending on their position in the tree.

We recommend that you continue the tutorial with `depth=3`. Afterwards you can increase the value step by step and check how many nodes you can run simultaneously.

## Create your k8s deployment files

To turn the application simulation configuration file from the last step into manifests for kubernetes, run the generator:

```
docker run --rm -t -i -v ${PWD}/deployments:/app/deployments -v ${PWD}:/mnt ghcr.io/cisco-open/app-simulator-generators-k8s:latest --config /mnt/config.yaml
```

This will create a folder `deployments` that will continue all the files you need to run your simulation on your cluster, using the following command:

```
kubectl create namespace app-sim-cue
kubectl apply -f deployments/ --namespace app-sim-cue
```

This will spin up your sample application. You can now use [OpenTelemetry](./5-observability-with-opentelemetry.md) or [cilium hubble](./4-observability-with-cilium-hubble.md to visualize your application.

For example, the following image shows an application with `depth=4` in cilium hubble ui:

![](./depth-4-in-hubble-ui.png)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
34 changes: 34 additions & 0 deletions examples/cue/large-application.cue
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import "list"
import "math"

#depth: int @tag(depth,type=int)
#adjectives: ["swift", "happy", "brave", "clever", "silent", "bold", "lucky", "fierce", "gentle", "mighty", "shy", "curious", "wise", "playful", "proud", "loyal"]
#nouns: ["lion", "tiger", "eagle", "panda", "fox", "wolf", "hawk", "bear", "otter", "falcon", "rabbit", "panther", "deer", "owl", "cheetah", "dolphin"]

loaders: {
"user1": {
type: "curl",
wait: 5,
sleep: 2,
urls: [ "http://swift-lion/next" ]
}
}

services: {
for x in list.Range(0, #depth+1, 1) {
for y in list.Range(0, x+1, 1) {
"\(#adjectives[x])-\(#nouns[y])": {
type: "java"
endpoints: {
http: {
"/next": list.Concat([[
for i in [y, y+1] if x < #depth {
"http://\(#adjectives[x+1])-\(#nouns[i])/next"
}
], ["sleep,\(math.Exp2(x)+(y*10))"]])
}
}
}
}
}
}

0 comments on commit 1b1d6d3

Please sign in to comment.