Skip to content

Commit

Permalink
update: rewrite clojure with AWS
Browse files Browse the repository at this point in the history
  • Loading branch information
@practicalli-johnny
practicalli-johnny committed Feb 4, 2024
1 parent 260213f commit 354b8d7
Showing 1 changed file with 88 additions and 64 deletions.
152 changes: 88 additions & 64 deletions docs/posts/access-aws-with-clojure-from-scratch.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,25 @@ tags:
- cognitect-labs
---

![alternative text](https://github.com/practicalli/graphic-design/blob/live/topic-images/aws-logo-name.png?raw=true){align=right loading=lazy style="width:240px"}
![Practicalli AWS topic logo](https://github.com/practicalli/graphic-design/blob/live/topic-images/aws-logo-name.png?raw=true){align=right loading=lazy style="width:240px"}

[Cognitect Labs AWS API](https://github.com/cognitect-labs/aws-api) is an excellent data driven Clojure library for interacting with AWS from code. Essentially a client for an AWS Service is created which operations are invoked. The project includes working code examples (_once a working AWS account is configured_).
[:fontawesome-brands-github: Cognitect Labs AWS API](https://github.com/cognitect-labs/aws-api) is an excellent data driven Clojure library for interacting with AWS from code. The project includes working code examples.

The hardest is configuring suitable AWS accounts, programmatic access and service permissions, especially when new to AWS. This article starts with nothing and steps through the simplest and quickest AWS account and permission setup, using the ["AWS Free Tier"](https://aws.amazon.com/free/). Note: every service has quota limits and some services are only free for a period of time (free trials & 12 months free).
The biggest challenge for those new to AWS is to configuring suitable AWS accounts, programmatic access and service permissions.

The approach outlined is not intended for commercial work, only the bare essentials to start exploring what is possible with the Cognitect Labs library. (Or you could try out [LocalStack if you have docker installed locally](https://localstack.cloud/)).
This article starts with nothing and steps through the simplest and quickest AWS account and permission setup, using the [:globe_with_meridians: AWS Free Tier](https://aws.amazon.com/free/).

!!! INFO "Service quota limits"
Each service has itos own quota limits and some services are only free for a period of time (free trials & 12 months free).

The approach outlined is not intended for commercial work, only the bare essentials to start exploring what is possible with the Cognitect Labs library. (Or you could try out [:globe_with_meridians: LocalStack if you have docker installed locally](https://localstack.cloud/)).

<!-- more -->

!!! WARNING "Avoid this approach for sesitive accounts"
Avoid approach for personal or company projects containing sensitive information**
??? WARNING "Avoid this approach for sensitive accounts"
Avoid approach for personal or company projects containing sensitive information

!!! WARNING "Credit card required"
??? WARNING "Credit card required"
A working credit card is required to setup a root account and AWS will charge the card $1 to verify identity and that the card is legitimate. This amount will be returned within 3-5 days.

Apart from the temporary charge, this approach should not incur any charges so long as the account stays within the free plan limits. Recommend using a digital card that can be set to frozen when not explicitly used as a safety precaution.
Expand All @@ -31,7 +36,7 @@ The approach outlined is not intended for commercial work, only the bare essenti

A root account is used for administration of an Amazon Web Service (AWS) account. This includes management of IAM accounts which are used for day to day work and programmatic (code) access.

Create a root account by [singing up for AWS](https://portal.aws.amazon.com/billing/signup#/start/email)
Create a root account by [:globe_with_meridians: signing up for an AWS account](https://portal.aws.amazon.com/billing/signup#/start/email){target=_blank}

Enter an email for the root account and a name for the AWS account

Expand All @@ -50,7 +55,7 @@ Generate a secure password for the root account, e.g. using a Password Manager s

Select **Personal** plan as this account is only used for personal projects & hacking (not cracking).

Complete contact information with your actual details (used when AWS confirms the credit card is legitimate) and confirm the [AWS Customer Agreement](https://aws.amazon.com/agreement/).
Complete contact information with your actual details (used when AWS confirms the credit card is legitimate) and confirm the [:globe_with_meridians: AWS Customer Agreement](https://aws.amazon.com/agreement/).

![AWS sign up website - contact details](https://github.com/practicalli/graphic-design/blob/live/cloud-services/aws/aws-sign-up-webpage-contact-information.png?raw=true)

Expand Down Expand Up @@ -131,41 +136,46 @@ Retrieve the access key details so they can be used when configuring the AWS CLI

## Install AWS CLI 2

[AWS Command Line Interface (AWS CLI)](https://aws.amazon.com/cli/) is a unified tool to control multiple AWS services from the command line and automate them through scripts.
[:globe_with_meridians: AWS Command Line Interface (AWS CLI)](https://aws.amazon.com/cli/) is a unified tool to control multiple AWS services from the command line and automate them through scripts.

The AWS CLI configuration can also be used by Clojure code using the Cognitect Labs AWS API library.

Download the install script archive file

```shell
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
```
!!! NOTE ""
```shell
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
```

Extract the install script which extract to a directory called `aws`

```shell
unzip awscliv2.zip
```
!!! NOTE ""
```shell
unzip awscliv2.zip
```

Run the install script, providing the user password when prompted

```shell
sudo ~/aws/install
```
!!! NOTE ""
```shell
sudo ~/aws/install
```

The install script can also be used to update an existing version of the AWS CLI version 2 (but not AWS CLI version 1)

```shell
sudo ~/aws/install --bin-dir /usr/local/bin --install-dir /usr/local/aws-cli --update
```
!!! NOTE ""
```shell
sudo ~/aws/install --bin-dir /usr/local/bin --install-dir /usr/local/aws-cli --update
```

## Configure AWS CLI

Use AWS CLI to create a configuration, using its wizard

```shell
aws configure
```
!!! NOTE ""
```shell
aws configure
```

Enter the access key and access key secret from the IAM account when prompted

Expand All @@ -180,86 +190,94 @@ The wizard creates two files
## Install Clojure

[Practicalli guide to install Clojure CLI](https://practical.li/clojure/install/), Java and Practicalli Clojure CLI Config for a wide range of community tools.
[:fontawesome-solid-book-open: Practicalli guide to install Clojure CLI](https://practical.li/clojure/install/), Java and Practicalli Clojure CLI Config for a wide range of community tools.

> Leiningen should also work, although this tool was not tested
## Cognitect Labs AWS API's

Now for the easy part :)
Now for the simpler part :)

[Create a new Clojure project](https://practical.li/clojure/clojure-cli/projects/), e.g. using the `:project/create` from Practicalli Clojure CLI
[Create a new Clojure project](https://practical.li/clojure/clojure-cli/projects/), e.g. using the `:project/create` alias from [Practicalli Clojure CLI Config](https://practical.li/clojure/clojure-cli/practicalli-config/).

```shell
clojure -T:project/create :template app :name practicalli/bucket-of-data
```
!!! NOTE "Create minimal Clojure CLI project"
```shell
clojure -T:project/create :template :practicalli/minimal :name practicalli/bucket-of-data
```

Edit the `deps.edn` file and add the Cognitect Labs AWS API, Endpoints and S3 libraries

```clojure
com.cognitect.aws/api {:mvn/version "0.8.666"}
com.cognitect.aws/endpoints {:mvn/version "1.1.12.456"}
com.cognitect.aws/s3 {:mvn/version "847.2.1365.0"}
```
!!! NOTE "Add Congnitect Labs AWS API libraries as project dependencies"
```clojure title="Project deps.edn"
com.cognitect.aws/api {:mvn/version "0.8.692"}
com.cognitect.aws/endpoints {:mvn/version "1.1.12.504"}
com.cognitect.aws/s3 {:mvn/version "848.2.1413.0"}
```

> [latest-release.edn](https://github.com/cognitect-labs/aws-api/blob/main/latest-releases.edn) listed all the libraries and their current version
> [:fontawesome-brands-github: latest-release.edn](https://github.com/cognitect-labs/aws-api/blob/main/latest-releases.edn) listed all the libraries and their current version
Start repl the repl, e.g. using the [Practicalli REPL Reloaded workflow](https://practical.li/clojure/clojure-cli/repl-reloaded/) or from an editor
Start repl the repl, e.g. using the [:fontawesome-solid-book-open: Practicalli REPL Reloaded workflow](https://practical.li/clojure/clojure-cli/repl-reloaded/) or from an editor

```shell
clojure -M:repl/reloaded
```
!!! NOTE ""
```shell
clojure -M:repl/reloaded
```

Open `src/practicalli/bucket_of_data.clj` file in a Clojure aware editor & connect to the REPL process (if the editor was not used to start the REPL process)

Create a rich comment and start to experiment with the Cognitect Labs AWS API library.

Example creating an S3 bucket, adding a file and reading the contents of a file.

Require the main library

```clojure
(require '[cognitect.aws.client.api :as aws])
```

Define a client for the AWS S3 service
!!! EXAMPLE "Require Cognitect Labs AWS API"
```clojure
(require '[cognitect.aws.client.api :as aws])
```

```clojure
(def data-bucket (aws/client {:api :s3}))
```
!!! EXAMPLE "Define a client for the AWS S3 service"
```clojure
(def data-bucket (aws/client {:api :s3}))
```

Validate requests to the S3 service to report incorrect arguments (especially useful when starting out with the library). Underlying this is a Clojure specification derived from the AWS specifications.

```clojure
(aws/validate-requests data-bucket true)
```
!!! EXAMPLE "Valicate S3 bucket requests"
```clojure
(aws/validate-requests data-bucket true)
```

Credentials are required for this request, so its a useful test to ensure the AWS CLI is configured correctly.

```clojure
(aws/invoke data-bucket {:op :ListBuckets})
```
!!! EXAMPLE "List S3 buckets to check credentials are working"
```clojure
(aws/invoke data-bucket {:op :ListBuckets})
```

> A REPL restart (stop/start) is required if the AWS environment variables are changed, otherwise the updated values in those environment variables will not be used.
As Cognitect Labs AWS API has a complete specification for each service, all the available operations for a specific service can be listed (even if the credentials are not working)

List the S3 Service operations. Note, this returns a big data structure, so I find it useful to look at it in a data inspector, e.g. Portal, Cider Inspector or Morse

```clojure
(aws/ops data-bucket)
```
!!! EXAMPLE "List S3 service operations"
```clojure
(aws/ops data-bucket)
```

Once the name of the operation is know, view the documentation to see how it is used.

```clojure
(aws/doc data-bucket :CreateBucket)
```
!!! EXAMPLE "Create S3 bucket"
```clojure
(aws/doc data-bucket :CreateBucket)
```

Portal provides an effective way to navigate

![Clojure Cognitect Labs AWS API - navigate AWS S3 Bucket operations in Portal - create bucket](https://github.com/practicalli/graphic-design/blob/live/cloud-services/aws/clojure-aws-api-s3-operations-create-bucket.png?raw=true)

> The Practicalli REPL Reloaded workflow launches Portal at startup and all evaluations are sent to the Portal UI, so its easy to navigate the large list of operations for an AWS service.

## SSO Account approach

Single sign-on access can be used for engineers and command line access.
Expand All @@ -277,8 +295,14 @@ This is only the start of using AWS from Clojure and almost all the work was to

There are lots of services to explore on AWS and with a working account and credentials and the excellent Cognitect Labs AWS API library it should be relatively straight forward (fingers crossed).

Please to keep a considered eye on how much of AWS you are using. Even though a free plan was used, it can be quite easy to leave things running and exceed quotas. Its not clear as yet if AWS will stop the service or simply send a bill. Taking a look at the quotas allowed for a service is a valuable step to ensure you can remain within the free plan.
!!! WARNING "Regularly review AWS usage"
Keep a considered eye on how much of AWS you are using.

Even when using a "free plan", it is possible to incur costs, e.g. forgetting to switch services off and exceeding quotas.

Its not clear as yet if AWS will stop the service or simply send a bill.

Being aware of the AWS quotas allowed for a service is a valuable step to ensure you can remain within the free plan.

---
Thank you.
Expand Down

0 comments on commit 354b8d7

Please sign in to comment.