Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC]Issues with multiple documents #9160

Open
1 of 4 tasks
soghoyanaws opened this issue Feb 6, 2025 · 0 comments
Open
1 of 4 tasks

[DOC]Issues with multiple documents #9160

soghoyanaws opened this issue Feb 6, 2025 · 0 comments

Comments

@soghoyanaws
Copy link

soghoyanaws commented Feb 6, 2025

What do you want to do?

  • Request a change to existing documentation
  • Add new documentation
  • Report a technical problem with the documentation
  • Other

Tell us about your request. Provide a summary of the request.
Issue 1: The documentation in the link suggests using a event_collect_timeout parameter is an option, but it is not. Is it called something else or does it not exist?
https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/processors/aws-lambda/

Issue 2: the documentation doesn't really say what args lambda will receive and doesn't specify what exactly the lambda should return.
There's a basic example that shows a lambda return an array of events, but in our testing this does not work. it seems like the pipeline expects the lambda to return an object with the events nested under that same batch key_name.
https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configure-client-lambda.html

Issue 3: As far as the lambda processor for the Opensearch pipeline, the docs here suggest that you can give a max_retries parameter, but it results in an error as seen in the second image. The first picture is the config, which looks like this
processor:
- aws_lambda:
function_name: "{{lambda_name}}"
response_events_match: true
tags_on_failure: ["lambda_failure"]
batch:
key_name: "events"
max_retries: 0
aws:
region: us-east-1
sts_role_arn: "{{pipeline_iam_role_arn}}"
- drop_events:
drop_when: '(/drop == true)'

Image
Image

Version: List the OpenSearch version to which this issue applies, e.g. 2.14, 2.12--2.14, or all.

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

@soghoyanaws soghoyanaws changed the title [DOC] [DOC]Issues with multiple documents Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant