camunda · jbergjan · Nov 11, 2024
@@ -0,0 +1,5 @@
+Copyright Camunda Services GmbH and/or licensed to Camunda Services GmbH under one or more contributor license agreements and licensed to you under a proprietary license.
+You may not use this file except in compliance with the proprietary license.
+The proprietary license can be either the Camunda Self-Managed Free Edition license (available on Camunda’s website) or the Camunda Self-Managed Enterprise Edition license (a copy you obtain when you contact Camunda).
+The Camunda Self-Managed Free Edition comes for free but only allows for usage of the software (file) in non-production environments.
+If you want to use the software (file) in production, you need to purchase the Camunda Self-Managed Enterprise Edition.
@@ -0,0 +1,152 @@
+# AWS S3 Connector
+
+Camunda Outbound Connector to interact with the content of an S3 bucket
+
+DISCLAIMER: You are responsible for your AWS configuration in your environment, keep in mind that you have to make
+sure that you keep credentials in a safe place and only give access to specific resources, and be as restrictive as 
+possible. This is not a security tutorial for AWS. You should definitively know what you are doing!
+
+## Compatibility
+
+- JDK 21+
+- Camunda Platform v8.7.x
+- Connector SDK v8.7.x
+- AWS SDK v2.x
+
+## Features
+
+- Upload a generated file to an AWS S3 bucket
+- Delete a file from an AWS S3 bucket
+- Download a file from an AWS S3 bucket
+- Files are saved in the local filesystem to allow interaction between activities
+
+
+## Setup
+
+### Connector configuration
+
+| name            | description                                | example                     |
+|-----------------|--------------------------------------------|-----------------------------|
+| `accessKey`     | the AWS access key for the authorized user | `secrets.AWS_ACCESS_KEY`    |
+| `secretKey`     | the AWS secret key for the authorized user | `secrets.AWS_SECRET_KEY`    |
+| `region`        | the AWS region of your S3 bucket           | eu-central-1                |
+| `bucketName`    | the name of the S3 bucket                  | camunda-s3-connector-bucket |
+| `objectKey`     | path + file name in the s3 bucket          | `="invoice/"+fileName`      |
+| `operationType` | what to do on s3                           | `PUT_OBJECT`                |
+| `filePath`      | absolute path to the file to upload        | `=filePath`                 |
+| `contentType`   | the content type of the content            | `=contentType`              |
+
+NOTE: please do not put secrets directly into your configuration. See the secrets section for more details.
+
+#### How it looks in the modeler
+<img src="./assets/connector-config-example.png" alt="how it looks like in the modeler" width="400" />
+
+### required AWS Resources
+- S3 bucket (non-public) with server-side encryption and versioning enabled
+- IAM User with putObject, deleteObject and getObject rights to the bucket
+- Access key and Secret key for the IAM user
+
+### create AWS Resources if not yet present
+#### 1. create AWS S3 Bucket:
+   First, you need to log into the AWS management console and navigate to S3.
+    There you can create a new bucket by setting your AWS region and a unique name.
+   <img src="./assets/awsScreenshots/screenshot_bucket_creation.png" width="600" />
+
+Moreover, making sure that the “Block all public access” option is enabled to keep the bucket private.
+<img src="./assets/awsScreenshots/screenshot_public_access.png" width="600" />
+
+#### 2. create IAM User and Access/Security Key
+Once the bucket has been set up, you must create an IAM (Identity and Access Management) user in the IAM area. 
+To do this, navigate to IAM > Users and add a new user.
+
+##### Get credentials from AWS
+
+In order to access AWS from your connector you need the above mentioned user with a IAM policy for S3. Additionally
+you need to generated credentials for the user. You can do this over the AWS console in IAM:
+
+- Log into your AWS console
+- Go to IAM > Users
+- Select your created user, e.g. `camunda-s3-connector-user`
+- Go to Security credentials > Access keys
+- Click `Create access key`
+- Select `Third-party service`
+- Check that you understand the risk of having a permanent access key
+- add a tag if wished
+- Click on create and save your credentials in a save place
+
+ATTENTION: There are many ways to achieve this in IAM, I only describe the easiest,
+but possibly not the safest way to do it
+
+
+#### 3. IAM policy
+
+Next, you create an IAM policy that grants the user authorization to upload, delete and retrieve objects in the bucket. 
+To do this, you must create an IAM policy in the IAM area and create a JSON policy with the corresponding authorizations.
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "AllowPutAndDeleteInS3",
+            "Effect": "Allow",
+            "Action": [
+                "s3:PutObject",
+                "s3:DeleteObject",
+                "s3:GetObject"
+            ],
+            "Resource": "arn:aws:s3:::<bucket-name>/*"
+        }
+    ]
+}
+```
+Afterwards, you need to add the created policy to your previously created IAM user.
+
+## Runtime
+
+### Running the connector
+
+- You can run it as a standalone connector described here: [run a standalone connector](https://github.com/NovatecConsulting/camunda-aws-s3-connector/tree/main/connector-aws-s3-standalone)
+- Or you can run it together with some JobWorkers described here: [run a connector](https://github.com/NovatecConsulting/camunda-aws-s3-connector/tree/main/connector-aws-s3-example)
+
+### Handling secrets
+Since your connector needs to run in a custom connector runtime, you cannot just add secrets in the cloud console since
+they will not be auto-magically transported into your connector runtime. You can provide them by:
+
+- adding them as environment variables (e.g. when you use the SpringBoot connector runtime)
+- adding them as an env file to your docker container
+
+NOTE: This behaviour will most likely be improved in the future
+
+## File Handling
+
+The connector has two file adapters:
+
+- *cloud file adapter* to S3
+- *local file adapter* to the local file system
+
+They are implementations of the [Connector File API](./fileapi)
+
+### Why is this necessary?
+If you are handling a lot of files (maybe even big files) it is a best practice approach to NOT store your file or the content
+in a process variable for several reasons:
+
+- Zeebe currently only support variables < 2MB
+- The files become part of the process state
+- You have no way to clean it up (yet)
+
+With the local file adapter the file can be written to a temp directory and the file path can be handed to other activities.
+If you want to move files to another process instance, just upload it back to S3 and start another process with the
+input variables needed to download the file again from S3.
+
+### Current Restrictions
+
+It is currently not possible to receive process instance specific variables like the process instance key in the connector 
+or with a feel expression. Both improvements exist as a feature request. The only way to get an instance key is by calling
+the value from an activated job in a JobWorker and setting it as a process variable for others to pick up.
+
+NOTE: the process instance key is used for generating file paths so local and remote files are not overwritten by other
+instances
+
+### This connector was initially developed by the BPM team at [Novatec Consulting GmbH](https://www.novatec-gmbh.de)
+<img src="./assets/novatec.png" width="600" />
@@ -0,0 +1,204 @@
+{
+  "$schema": "https://unpkg.com/@camunda/zeebe-element-templates-json-schema/resources/schema.json",
+  "name": "Connector AWS S3",
+  "id": "6f60159e-f5f5-49d0-805b-9320aab39ee5",
+  "description": "Manage your files on AWS S3",
+  "version": 2,
+  "icon": {
+    "contents": "data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPHN2ZyB3aWR0aD0iODBweCIgaGVpZ2h0PSI4MHB4IiB2aWV3Qm94PSIwIDAgNjAgNjAiIHZlcnNpb249IjEuMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayI+CiAgICA8dGl0bGU+Q29ubmVjdG9yL0xvZ28vUzM8L3RpdGxlPgogICAgPGcgaWQ9IkNvbm5lY3Rvci9Mb2dvL1MzIiBzdHJva2U9Im5vbmUiIHN0cm9rZS13aWR0aD0iMSIgZmlsbD0ibm9uZSIgZmlsbC1ydWxlPSJldmVub2RkIj4KICAgICAgICA8ZyBpZD0iQ29ubmVjdG9yL0xvZ28vUzMvU3F1YXJlIiBmaWxsPSIjRkZGRkZGIj4KICAgICAgICAgICAgPHJlY3QgaWQ9IlJlY3RhbmdsZSIgeD0iMCIgeT0iMCIgd2lkdGg9IjYwIiBoZWlnaHQ9IjYwIj48L3JlY3Q+CiAgICAgICAgPC9nPgogICAgICAgIDxnIGlkPSJDb25uZWN0b3IvTG9nby9TMy9CdWNrZXQiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDIuOTk5OTAwLCAxLjk5OTYwMCkiIGZpbGw9IiMwMDAwMDAiPgogICAgICAgICAgICA8cGF0aCBzdHJva2U9ImJsYWNrIiBzdHJva2Utd2lkdGg9IjEuNSIgZD0iTTQ3LjgzNiwzMC44OTMgTDQ4LjIyLDI4LjE4OSBDNTEuNzYxLDMwLjMxIDUxLjgwNywzMS4xODYgNTEuODA2MDEzMiwzMS4yMSBDNTEuOCwzMS4yMTUgNTEuMTk2LDMxLjcxOSA0Ny44MzYsMzAuODkzIEw0Ny44MzYsMzAuODkzIFogTTQ1Ljg5MywzMC4zNTMgQzM5Ljc3MywyOC41MDEgMzEuMjUsMjQuNTkxIDI3LjgwMSwyMi45NjEgQzI3LjgwMSwyMi45NDcgMjcuODA1LDIyLjkzNCAyNy44MDUsMjIuOTIgQzI3LjgwNSwyMS41OTUgMjYuNzI3LDIwLjUxNyAyNS40MDEsMjAuNTE3IEMyNC4wNzcsMjAuNTE3IDIyLjk5OSwyMS41OTUgMjIuOTk5LDIyLjkyIEMyMi45OTksMjQuMjQ1IDI0LjA3NywyNS4zMjMgMjUuNDAxLDI1LjMyMyBDMjUuOTgzLDI1LjMyMyAyNi41MTEsMjUuMTA2IDI2LjkyOCwyNC43NjEgQzMwLjk4NiwyNi42ODIgMzkuNDQzLDMwLjUzNSA0NS42MDgsMzIuMzU1IEw0My4xNyw0OS41NjEgQzQzLjE2Myw0OS42MDggNDMuMTYsNDkuNjU1IDQzLjE2LDQ5LjcwMiBDNDMuMTYsNTEuMjE3IDM2LjQ1Myw1NCAyNS40OTQsNTQgQzE0LjQxOSw1NCA3LjY0MSw1MS4yMTcgNy42NDEsNDkuNzAyIEM3LjY0MSw0OS42NTYgNy42MzgsNDkuNjExIDcuNjMyLDQ5LjU2NiBMMi41MzgsMTIuMzU5IEM2Ljk0NywxNS4zOTQgMTYuNDMsMTcgMjUuNSwxNyBDMzQuNTU2LDE3IDQ0LjAyMywxNS40IDQ4LjQ0MSwxMi4zNzQgTDQ1Ljg5MywzMC4zNTMgWiBNMiw4LjQ3OCBDMi4wNzIsNy4xNjIgOS42MzQsMiAyNS41LDIgQzQxLjM2NCwyIDQ4LjkyNyw3LjE2MSA0OSw4LjQ3OCBMNDksOC45MjcgQzQ4LjEzLDExLjg3OCAzOC4zMywxNSAyNS41LDE1IEMxMi42NDgsMTUgMi44NDMsMTEuODY4IDIsOC45MTMgTDIsOC40NzggWiBNNTEsOC41IEM1MSw1LjAzNSA0MS4wNjYsMCAyNS41LDAgQzkuOTM0LDAgMCw1LjAzNSAwLDguNSBMMC4wOTQsOS4yNTQgTDUuNjQyLDQ5Ljc3OCBDNS43NzUsNTQuMzEgMTcuODYxLDU2IDI1LjQ5NCw1NiBDMzQuOTY2LDU2IDQ1LjAyOSw1My44MjIgNDUuMTU5LDQ5Ljc4MSBMNDcuNTU1LDMyLjg4NCBDNDguODg4LDMzLjIwMyA0OS45ODUsMzMuMzY2IDUwLjg2NiwzMy4zNjYgQzUyLjA0OSwzMy4zNjYgNTIuODQ5LDMzLjA3NyA1My4zMzQsMzIuNDk5IEM1My43MzIsMzIuMDI1IDUzLjg4NCwzMS40NTEgNTMuNzcsMzAuODQgQzUzLjUxMSwyOS40NTYgNTEuODY4LDI3Ljk2NCA0OC41MjIsMjYuMDU1IEw1MC44OTgsOS4yOTMgTDUxLDguNSBaIiBpZD0iUzNfTG9nbyI+PC9wYXRoPgogICAgICAgIDwvZz4KICAgIDwvZz4KPC9zdmc+"
+  },
+  "category": {
+    "id": "connectors",
+    "name": "Connectors"
+  },
+  "appliesTo": [
+    "bpmn:Task"
+  ],
+  "elementType": {
+    "value": "bpmn:ServiceTask"
+  },
+  "groups": [
+    {
+      "id": "authentication",
+      "label": "Authentication"
+    },
+    {
+      "id": "operation",
+      "label": "Select Operation"
+    },
+    {
+      "id": "operationDetails",
+      "label": "Operation Details"
+    },
+    {
+      "id": "output",
+      "label": "Output Mapping"
+    },
+    {
+      "id": "errors",
+      "label": "Error Handling"
+    }
+  ],
+  "properties": [
+    {
+      "type": "Hidden",
+      "value": "io.camunda.connector.awss3:aws-s3:1",
+      "binding": {
+        "type": "zeebe:taskDefinition:type"
+      }
+    },
+    {
+      "label": "Access Key",
+      "description": "Provide an access key of a user with permissions to access the specified AWS S3 bucket",
+      "group": "authentication",
+      "type": "String",
+      "binding": {
+        "type": "zeebe:input",
+        "name": "authentication.accessKey"
+      },
+      "constraints": {
+        "notEmpty": true
+      }
+    },
+    {
+      "label": "Secret Key",
+      "description": "Provide a secret key of a user with permissions to access the specified AWS S3 bucket",
+      "group": "authentication",
+      "type": "String",
+      "binding": {
+        "type": "zeebe:input",
+        "name": "authentication.secretKey"
+      },
+      "constraints": {
+        "notEmpty": true
+      }
+    },
+    {
+      "id": "operationType",
+      "group": "operation",
+      "type": "Dropdown",
+      "value": "PUT_OBJECT",
+      "choices": [
+        {
+          "name": "Upload file",
+          "value": "PUT_OBJECT"
+        },
+        {
+          "name": "Delete file",
+          "value": "DELETE_OBJECT"
+        },
+        {
+          "name": "Download file",
+          "value": "GET_OBJECT"
+        }
+      ],
+      "binding": {
+        "type": "zeebe:input",
+        "name": "requestDetails.operationType",
+        "key": "requestDetails.operationType"
+      }
+    },
+    {
+      "label": "AWS Region",
+      "description": "Specify an AWS region",
+      "group": "operationDetails",
+      "type": "String",
+      "feel": "optional",
+      "binding": {
+        "type": "zeebe:input",
+        "name": "requestDetails.region"
+      },
+      "constraints": {
+        "notEmpty": true,
+        "maxLength": 255
+      }
+    },
+    {
+      "label": "Bucket name",
+      "description": "Enter the name of your s3 bucket",
+      "group": "operationDetails",
+      "type": "String",
+      "feel": "optional",
+      "binding": {
+        "type": "zeebe:input",
+        "name": "requestDetails.bucketName"
+      },
+      "constraints": {
+        "notEmpty": true,
+        "maxLength": 255
+      }
+    },
+    {
+      "label": "Object key",
+      "description": "Provide a key for your upload, relative from the bucket (e.g. my/files/message.xml)",
+      "group": "operationDetails",
+      "type": "String",
+      "feel": "optional",
+      "binding": {
+        "type": "zeebe:input",
+        "name": "requestDetails.objectKey"
+      },
+      "constraints": {
+        "notEmpty": true,
+        "maxLength": 255
+      }
+    },
+    {
+      "label": "File Path",
+      "description": "Provide the path to a local file, default is the same as objectKey",
+      "group": "operationDetails",
+      "type": "String",
+      "feel": "optional",
+      "binding": {
+        "type": "zeebe:input",
+        "name": "requestDetails.filePath"
+      }
+    },
+    {
+      "label": "Content Type",
+      "description": "Provide a content type (e.g. application/xml)",
+      "group": "operationDetails",
+      "type": "String",
+      "feel": "optional",
+      "binding": {
+        "type": "zeebe:input",
+        "name": "requestDetails.contentType"
+      },
+      "constraints": {
+        "notEmpty": false
+      }
+    },
+    {
+      "label": "Result Variable",
+      "description": "Name of variable to store the response in",
+      "group": "output",
+      "type": "String",
+      "binding": {
+        "type": "zeebe:taskHeader",
+        "key": "resultVariable"
+      }
+    },
+    {
+      "label": "Result Expression",
+      "description": "Expression to map the response into process variables",
+      "group": "output",
+      "type": "Text",
+      "feel": "required",
+      "binding": {
+        "type": "zeebe:taskHeader",
+        "key": "resultExpression"
+      }
+    },
+    {
+      "label": "Error Expression",
+      "description": "Expression to handle errors. Details in the <a href=\"https://docs.camunda.io/docs/components/connectors/use-connectors/\" target=\"_blank\">documentation</a>.",
+      "group": "errors",
+      "type": "Text",
+      "feel": "required",
+      "binding": {
+        "type": "zeebe:taskHeader",
+        "key": "errorExpression"
+      }
+    }
+  ]
+}
diff --git a/connectors/aws/aws-s3/connector-aws-s3/pom.xml b/connectors/aws/aws-s3/connector-aws-s3/pom.xml
@@ -0,0 +1,31 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>io.camunda.connector</groupId>
+        <artifactId>connector-aws-s3-parent</artifactId>
+        <version>8.7.0-SNAPSHOT</version>
+        <relativePath>../pom.xml</relativePath>
+    </parent>
+
+    <name>connector-aws-s3</name>
+    <groupId>io.camunda.connector.awss3</groupId>
+    <artifactId>connector-aws-s3</artifactId>
+
+    <properties>
+        <maven.compiler.source>21</maven.compiler.source>
+        <maven.compiler.target>21</maven.compiler.target>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+    </properties>
+
+    <dependencies>
+        <dependency>
+            <groupId>io.camunda.connector.fileapi</groupId>
+            <artifactId>connector-aws-fileapi</artifactId>
+            <version>8.7.0-SNAPSHOT</version>
+        </dependency>
+    </dependencies>
+
+</project>