Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[elasticsearchexporter] support compression level #37260

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .chloggen/add-compression-level-esexporter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: elasticsearchexporter

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Syncs support for compression level on elasticsearch exporter with confighttp. The default compression level is 1 if not set.

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [37260]
2 changes: 1 addition & 1 deletion exporter/elasticsearchexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ service:

### HTTP settings

The Elasticsearch exporter supports common [HTTP Configuration Settings][confighttp]. Gzip compression is enabled by default. To disable compression, set `compression` to `none`.
The Elasticsearch exporter supports common [HTTP Configuration Settings][confighttp]. Gzip compression is enabled by default. To disable compression, set `compression` to `none`. Default Compression Level is set to 1.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The Elasticsearch exporter supports common [HTTP Configuration Settings][confighttp]. Gzip compression is enabled by default. To disable compression, set `compression` to `none`. Default Compression Level is set to 1.
The Elasticsearch exporter supports common [HTTP Configuration Settings][confighttp]. Gzip compression is enabled by default. To disable compression, set `compression` to `none`. Default Compression Level is set to 1 (gzip.BestSpeed).

As a consequence of supporting [confighttp], the Elasticsearch exporter also supports common [TLS Configuration Settings][configtls].

The Elasticsearch exporter sets `timeout` (HTTP request timeout) to 90s by default.
Expand Down
7 changes: 6 additions & 1 deletion exporter/elasticsearchexporter/bulkindexer.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,12 @@
}
var compressionLevel int
if config.Compression == configcompression.TypeGzip {
compressionLevel = gzip.BestSpeed
if config.CompressionParams.Level {

Check failure on line 75 in exporter/elasticsearchexporter/bulkindexer.go

View workflow job for this annotation

GitHub Actions / govulncheck (exporter-1)

config.CompressionParams undefined (type *Config has no field or method CompressionParams)
compressionLevel = config.CompressionParams.Level

Check failure on line 76 in exporter/elasticsearchexporter/bulkindexer.go

View workflow job for this annotation

GitHub Actions / govulncheck (exporter-1)

config.CompressionParams undefined (type *Config has no field or method CompressionParams)
} else {
compressionLevel = gzip.BestSpeed

}
}
return docappender.BulkIndexerConfig{
Client: client,
Expand Down
9 changes: 9 additions & 0 deletions exporter/elasticsearchexporter/bulkindexer_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import (
"github.com/elastic/go-elasticsearch/v7"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/config/configcompression"
"go.opentelemetry.io/collector/config/confighttp"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
Expand Down Expand Up @@ -263,6 +264,14 @@ func TestAsyncBulkIndexer_logRoundTrip(t *testing.T) {
Flush: FlushSettings{Interval: time.Hour, Bytes: 1e+8},
},
},
{
name: "compression gzip - level 5",
config: Config{
NumWorkers: 1,
ClientConfig: confighttp.ClientConfig{Compression: "gzip", CompressionParams: configcompression.CompresionParams{Level: 5}},
Flush: FlushSettings{Interval: time.Hour, Bytes: 1e+8},
},
},
}

for _, tt := range tests {
Expand Down
Loading