Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out_blob: consider auto_create_container #9457

Merged
merged 2 commits into from
Oct 31, 2024

Conversation

uristernik
Copy link
Contributor

@uristernik uristernik commented Oct 2, 2024

Consider auto_create_container when ensuring container exist

  1. Today we are not considering this setting
  2. Changing the behaviour that setting auto_create_container to false should return FLB_TRUE. If auto_create_container is false return FLB_FALSE, fluent-bit will never flush data to storage account, it will retry and eventually fail.

Fixes #9459


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • [N/A] Example configuration file for the change
  • [N/A] Debug log output from testing the change
    No need for debug log output, attaching info log output

With auto_create_container on:

Fluent Bit v3.2.0
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _           _____  _____ 
|  ___| |                | |   | ___ (_) |         |____ |/ __  \
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __   / /`' / /'
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / /   \ \  / /  
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /.___/ /./ /___
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/ \____(_)_____/


[2024/10/04 08:22:02] [ info] [fluent bit] version=3.2.0, commit=88f44c6c80, pid=1
[2024/10/04 08:22:02] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/10/04 08:22:02] [ info] [cmetrics] version=0.9.6
[2024/10/04 08:22:02] [ info] [ctraces ] version=0.5.6
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] initializing
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] multiline core started
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] db: delete unmonitored stale inodes from the database: count=0
[2024/10/04 08:22:02] [ info] [filter:kubernetes:kubernetes.1] https=1 host=10.96.0.1 port=443
[2024/10/04 08:22:02] [ info] [filter:kubernetes:kubernetes.1]  token updated
[2024/10/04 08:22:02] [ info] [filter:kubernetes:kubernetes.1] local POD info OK
[2024/10/04 08:22:02] [ info] [filter:kubernetes:kubernetes.1] testing connectivity with API server...
[2024/10/04 08:22:02] [ info] [filter:kubernetes:kubernetes.1] connectivity OK
[2024/10/04 08:22:02] [ info] [output:azure_blob:azure_blob.0] account_name=xxx, container_name=logs, blob_type=blockblob, emulator_mode=no, endpoint=https://xxx/, auth_type=sas
[2024/10/04 08:22:02] [ info] [output:azure_blob:azure_blob.0] initializing worker
[2024/10/04 08:22:02] [ info] [output:azure_blob:azure_blob.0] worker #0 started
[2024/10/04 08:22:02] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2024/10/04 08:22:02] [ info] [sp] stream processor started
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361836 watch_fd=1 name=/var/log/containers/coredns-5dd5756b68-zxntv_kube-system_coredns-1151680f0cfdb669fb0aa0b67002b3e7cebde20951de9708cdefec126fb3622b.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1352059 watch_fd=2 name=/var/log/containers/coredns-5dd5756b68-zxntv_kube-system_coredns-61f36c9a9001fe1445db694e2b3bfbd5d29f29c311f117f7fca185769df06b2c.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361296 watch_fd=3 name=/var/log/containers/etcd-minikube_kube-system_etcd-94227cefaff3b3b2d3fba53eb691ae2c8e835b46c0dd91dbb6fdc764ff92b34a.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351393 watch_fd=4 name=/var/log/containers/etcd-minikube_kube-system_etcd-b820386d0619804d710ac9ca4d9511ab20f4cce008a7c6f0fbd5fe282453f6c0.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1180622 watch_fd=5 name=/var/log/containers/fluent-bit-d8kcc_default_fluent-bit-5efb3c8e4e9a655178c845674ee1e975593b44c1d35300bf7fd29b0d6f8684cd.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361715 watch_fd=6 name=/var/log/containers/kindnet-gdhj4_kube-system_kindnet-cni-96c5cef5c116d8723302b4c7b78251834315280ee2c5feb42e87fa0a54b70c70.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1352078 watch_fd=7 name=/var/log/containers/kindnet-gdhj4_kube-system_kindnet-cni-afcf957e18dad68983bcd576a11addc2616571923a3fa2fd2148e0efdcfe91e7.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351390 watch_fd=8 name=/var/log/containers/kube-apiserver-minikube_kube-system_kube-apiserver-73ef43bb9d99f99928e80273731975e6da9b10cc725d7484270e597f2d61ebcb.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361289 watch_fd=9 name=/var/log/containers/kube-apiserver-minikube_kube-system_kube-apiserver-fd0c9e5f665024632fe49cb12b5f17c9619b053cafdc6cc249091ed6dc2916d2.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351389 watch_fd=10 name=/var/log/containers/kube-controller-manager-minikube_kube-system_kube-controller-manager-ac5d19593b98a1d539cc5643c947fe0cd80279fac29237d1f79611e288480e8a.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361297 watch_fd=11 name=/var/log/containers/kube-controller-manager-minikube_kube-system_kube-controller-manager-e686474dfc80364029317aa43140e4dc5b47cad951aa472b1d1faff95a1e29e3.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361690 watch_fd=12 name=/var/log/containers/kube-proxy-7bmp5_kube-system_kube-proxy-4659c90c8afbc32e6494399f10ba19f156fa9cd5187a2eabf4ec8a9d877bfe57.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351834 watch_fd=13 name=/var/log/containers/kube-proxy-7bmp5_kube-system_kube-proxy-948535a87eef9ef20735cd86b9d025924f0a52455a4fa7d7e0da4a93ee811020.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361294 watch_fd=14 name=/var/log/containers/kube-scheduler-minikube_kube-system_kube-scheduler-357922c70e5c6ed04fc9bd7817a9aae39d076815838f3016c0897549bf11bf12.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351395 watch_fd=15 name=/var/log/containers/kube-scheduler-minikube_kube-system_kube-scheduler-b746b05b59757cac516f7a8e9fc6f3d42f867ad6884b96e0677cc445e36f381a.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361846 watch_fd=16 name=/var/log/containers/metrics-server-7c66d45ddc-w6lmf_kube-system_metrics-server-27ea1e3df003a11e68a56473622dfbde505b9e692eeeb0e6e6a0fc1ba8d0642d.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1357830 watch_fd=17 name=/var/log/containers/metrics-server-7c66d45ddc-w6lmf_kube-system_metrics-server-300fbb3cd38c48fba82effcd2ed5327f9ea14e18b0c992776d31694854cd64f6.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361693 watch_fd=18 name=/var/log/containers/storage-provisioner_kube-system_storage-provisioner-6f2fe8443b054cb954b588121eef231a8ab9177b073f52134f5b7cff161c0f12.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351900 watch_fd=19 name=/var/log/containers/storage-provisioner_kube-system_storage-provisioner-8a0e04f927948876fb14d3b4ed7e4fd3217335d808c80ae63dede52eb3d02f1c.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1180474 watch_fd=20 name=/var/log/containers/test_default_upload-1803814d839588840da38a65a5b6e974ffa65368146078d5ccef4ce16f835665.log
[2024/10/04 08:22:02] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1180641 watch_fd=21 name=/var/log/containers/fluent-bit-fb2bz_default_fluent-bit-2a04054b3448f7b1f83c2b71674ff262975f483b7eab473a2f009c4d4c8ea79b.log
[2024/10/04 08:22:03] [ info] [input:tail:tail.0] inotify_fs_remove(): inode=1180622 watch_fd=5
[2024/10/04 08:22:12] [error] [output:azure_blob:azure_blob.0] failed getting container 'logs', access denied
[2024/10/04 08:22:12] [error] [output:azure_blob:azure_blob.0] cannot ensure container 'logs' exists
[2024/10/04 08:22:12] [ warn] [engine] failed to flush chunk '1-1728030122.304633888.flb', retry in 10 seconds: task_id=1, input=tail.0 > output=azure_blob.0 (out_id=0)
[2024/10/04 08:22:12] [error] [output:azure_blob:azure_blob.0] failed getting container 'logs', access denied
[2024/10/04 08:22:12] [error] [output:azure_blob:azure_blob.0] cannot ensure container 'logs' exists
[2024/10/04 08:22:12] [ warn] [engine] failed to flush chunk '1-1728030125.262367626.flb', retry in 10 seconds: task_id=3, input=tail.0 > output=azure_blob.0 (out_id=0)
[2024/10/04 08:22:12] [error] [output:azure_blob:azure_blob.0] failed getting container 'logs', access denied
[2024/10/04 08:22:12] [error] [output:azure_blob:azure_blob.0] cannot ensure container 'logs' exists
[2024/10/04 08:22:12] [ warn] [engine] failed to flush chunk '1-1728030123.339554250.flb', retry in 9 seconds: task_id=2, input=tail.0 > output=azure_blob.0 (out_id=0)
[2024/10/04 08:22:12] [error] [output:azure_blob:azure_blob.0] failed getting container 'logs', access denied
[2024/10/04 08:22:12] [ warn] [engine] failed to flush chunk '1-1728030122.261692013.flb', retry in 6 seconds: task_id=0, input=tail.0 > output=azure_blob.0 (out_id=0)
[2024/10/04 08:22:12] [error] [output:azure_blob:azure_blob.0] cannot ensure container 'logs' exists

With auto_create_container off:

Fluent Bit v3.2.0
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _           _____  _____ 
|  ___| |                | |   | ___ (_) |         |____ |/ __  \
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __   / /`' / /'
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / /   \ \  / /  
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /.___/ /./ /___
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/ \____(_)_____/


[2024/10/04 08:28:48] [ info] [fluent bit] version=3.2.0, commit=88f44c6c80, pid=1
[2024/10/04 08:28:48] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/10/04 08:28:48] [ info] [cmetrics] version=0.9.6
[2024/10/04 08:28:48] [ info] [ctraces ] version=0.5.6
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] initializing
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] multiline core started
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] db: delete unmonitored stale inodes from the database: count=0
[2024/10/04 08:28:48] [ info] [filter:kubernetes:kubernetes.1] https=1 host=10.96.0.1 port=443
[2024/10/04 08:28:48] [ info] [filter:kubernetes:kubernetes.1]  token updated
[2024/10/04 08:28:48] [ info] [filter:kubernetes:kubernetes.1] local POD info OK
[2024/10/04 08:28:48] [ info] [filter:kubernetes:kubernetes.1] testing connectivity with API server...
[2024/10/04 08:28:48] [ info] [filter:kubernetes:kubernetes.1] connectivity OK
[2024/10/04 08:28:48] [ info] [output:azure_blob:azure_blob.0] account_name=xxx, container_name=logs, blob_type=blockblob, emulator_mode=no, endpoint=https://xxx/, auth_type=sas
[2024/10/04 08:28:48] [ info] [output:azure_blob:azure_blob.0] initializing worker
[2024/10/04 08:28:48] [ info] [output:azure_blob:azure_blob.0] worker #0 started
[2024/10/04 08:28:48] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2024/10/04 08:28:48] [ info] [sp] stream processor started
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1180747 watch_fd=1 name=/var/log/containers/coredns-5d756df69c-8l9w9_kube-system_coredns-a379d17d983271d154da6d29351b48ffcb394d979f848faeac4ed0192a4ff2b3.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1180648 watch_fd=2 name=/var/log/containers/coredns-5d756df69c-zstzm_kube-system_coredns-1bf55c770e6a7080452d77109f8a64c16bfbde112ee31da3f7c0db7bb6750c68.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361296 watch_fd=3 name=/var/log/containers/etcd-minikube_kube-system_etcd-94227cefaff3b3b2d3fba53eb691ae2c8e835b46c0dd91dbb6fdc764ff92b34a.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351393 watch_fd=4 name=/var/log/containers/etcd-minikube_kube-system_etcd-b820386d0619804d710ac9ca4d9511ab20f4cce008a7c6f0fbd5fe282453f6c0.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361715 watch_fd=5 name=/var/log/containers/kindnet-gdhj4_kube-system_kindnet-cni-96c5cef5c116d8723302b4c7b78251834315280ee2c5feb42e87fa0a54b70c70.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1352078 watch_fd=6 name=/var/log/containers/kindnet-gdhj4_kube-system_kindnet-cni-afcf957e18dad68983bcd576a11addc2616571923a3fa2fd2148e0efdcfe91e7.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351390 watch_fd=7 name=/var/log/containers/kube-apiserver-minikube_kube-system_kube-apiserver-73ef43bb9d99f99928e80273731975e6da9b10cc725d7484270e597f2d61ebcb.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361289 watch_fd=8 name=/var/log/containers/kube-apiserver-minikube_kube-system_kube-apiserver-fd0c9e5f665024632fe49cb12b5f17c9619b053cafdc6cc249091ed6dc2916d2.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351389 watch_fd=9 name=/var/log/containers/kube-controller-manager-minikube_kube-system_kube-controller-manager-ac5d19593b98a1d539cc5643c947fe0cd80279fac29237d1f79611e288480e8a.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361297 watch_fd=10 name=/var/log/containers/kube-controller-manager-minikube_kube-system_kube-controller-manager-e686474dfc80364029317aa43140e4dc5b47cad951aa472b1d1faff95a1e29e3.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361690 watch_fd=11 name=/var/log/containers/kube-proxy-7bmp5_kube-system_kube-proxy-4659c90c8afbc32e6494399f10ba19f156fa9cd5187a2eabf4ec8a9d877bfe57.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351834 watch_fd=12 name=/var/log/containers/kube-proxy-7bmp5_kube-system_kube-proxy-948535a87eef9ef20735cd86b9d025924f0a52455a4fa7d7e0da4a93ee811020.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361294 watch_fd=13 name=/var/log/containers/kube-scheduler-minikube_kube-system_kube-scheduler-357922c70e5c6ed04fc9bd7817a9aae39d076815838f3016c0897549bf11bf12.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351395 watch_fd=14 name=/var/log/containers/kube-scheduler-minikube_kube-system_kube-scheduler-b746b05b59757cac516f7a8e9fc6f3d42f867ad6884b96e0677cc445e36f381a.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361846 watch_fd=15 name=/var/log/containers/metrics-server-7c66d45ddc-w6lmf_kube-system_metrics-server-27ea1e3df003a11e68a56473622dfbde505b9e692eeeb0e6e6a0fc1ba8d0642d.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1357830 watch_fd=16 name=/var/log/containers/metrics-server-7c66d45ddc-w6lmf_kube-system_metrics-server-300fbb3cd38c48fba82effcd2ed5327f9ea14e18b0c992776d31694854cd64f6.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1361693 watch_fd=17 name=/var/log/containers/storage-provisioner_kube-system_storage-provisioner-6f2fe8443b054cb954b588121eef231a8ab9177b073f52134f5b7cff161c0f12.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1351900 watch_fd=18 name=/var/log/containers/storage-provisioner_kube-system_storage-provisioner-8a0e04f927948876fb14d3b4ed7e4fd3217335d808c80ae63dede52eb3d02f1c.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1180474 watch_fd=19 name=/var/log/containers/test_default_upload-1803814d839588840da38a65a5b6e974ffa65368146078d5ccef4ce16f835665.log
[2024/10/04 08:28:48] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1180580 watch_fd=20 name=/var/log/containers/fluent-bit-8krf8_default_fluent-bit-2eb1d8867ac496f86b811eee21c59d2a6b4d1fd43c8930323bd67a3cf60c0b68.log
[2024/10/04 08:28:58] [ info] [output:azure_blob:azure_blob.0] auto_create_container is disabled, assuming container 'logs' already exists
[2024/10/04 08:28:58] [ info] [output:azure_blob:azure_blob.0] auto_create_container is disabled, assuming container 'logs' already exists
[2024/10/04 08:28:58] [ info] [output:azure_blob:azure_blob.0] content uploaded successfully: file=kube.var.log.containers.kindnet-gdhj4_kube-system_kindnet-cni-96c5cef5c116d8723302b4c7b78251834315280ee2c5feb42e87fa0a54b70c70.log.1728030538236
[2024/10/04 08:28:58] [ info] [output:azure_blob:azure_blob.0] blob id ZmxiLTE3MjgwMzA1MzguMjM2OC5pZA== committed successfully
[2024/10/04 08:28:59] [ info] [output:azure_blob:azure_blob.0] content uploaded successfully: file=kube.var.log.containers.fluent-bit-8krf8_default_fluent-bit-2eb1d8867ac496f86b811eee21c59d2a6b4d1fd43c8930323bd67a3cf60c0b68.log.1728030538233

  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • [N/A] Documentation required for this feature

Backporting

  • Backport to latest stable release.
    Will do after this one is merged

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@uristernik uristernik force-pushed the out-azure-fix-auto-create-container branch from 3c540f3 to 2e6f8ef Compare October 2, 2024 10:35
@uristernik uristernik changed the title Consider auto_create_container when ensuring container exist out_blob: consider auto_create_container Oct 2, 2024
@uristernik
Copy link
Contributor Author

@uristernik
Copy link
Contributor Author

I'll add some more context here - we are using SAS token authentication and want to make sure all fluent-bit clients have permissions to write but not to read (so clients from different clusters could not read each other's data).

When creating a SAS token with write only permissions we are seeing the following errors:

[2024/10/01 15:40:33] [error] [engine] chunk '17-1727797215.111211921.flb' cannot be retried: task_id=1, input=tail.0 > output=azure_blob.0
[2024/10/01 15:40:35] [ warn] [engine] failed to flush chunk '17-1727797225.102833884.flb', retry in 7 seconds: task_id=0, input=tail.0 > output=azure_blob.0 (out_id=0)
[2024/10/01 15:40:42] [error] [engine] chunk '17-1727797225.102833884.flb' cannot be retried: task_id=0, input=tail.0 > output=azure_blob.0
[2024/10/01 15:40:45] [ warn] [engine] failed to flush chunk '17-1727797235.100613583.flb', retry in 9 seconds: task_id=0, input=tail.0 > output=azure_blob.0 (out_id=0)

This is because the SAS token doesn't have permissions call GetContainerProperties to ensure the container exists. There no error printed (even in debug). We can see we have a retry before failing but this happens across all chunks effectively dropping everything.

1. Today we are not considering this setting
2. Changing the behaviour that setting auto_create_container to false should return FLB_TRUE. If auto_create_container=false return FLB_FALSE, fluent-bit will never flush data to storage account, it will retry and eventually fail.

Signed-off-by: Uri Sternik <[email protected]>
@uristernik
Copy link
Contributor Author

@leonardo-albertovich any chance to get some 👀 here?
Thank you 🙏

Copy link
Collaborator

@leonardo-albertovich leonardo-albertovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once that single issue is addressed this can be approved.

Please correct me if I missed something and my comment is wrong.

plugins/out_azure_blob/azure_blob.c Outdated Show resolved Hide resolved
Signed-off-by: Uri Sternik <[email protected]>
@uristernik uristernik force-pushed the out-azure-fix-auto-create-container branch from f51340b to e2aaa87 Compare October 22, 2024 08:36
@uristernik
Copy link
Contributor Author

Hey @leonardo-albertovich 👋
I fixed the above comments

@edsiper edsiper merged commit 104aceb into fluent:master Oct 31, 2024
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

write-only SAS token fails sending data to blob
3 participants