Skip to content

Commit

Permalink
Added end-to-end PPL tests for Spark (#1028)
Browse files Browse the repository at this point in the history 
* Added end-to-end PPL tests for Spark

Signed-off-by: Norman Jordan <[email protected]>

* Added instructions for adding end-to-end tests

Signed-off-by: Norman Jordan <[email protected]>

---------

Signed-off-by: Norman Jordan <[email protected]>
  • Loading branch information
@normanj-bitquill
normanj-bitquill authored Feb 6, 2025
1 parent eff717a commit 4d6ba7d
Show file tree
Hide file tree
Showing 455 changed files with 2,791 additions and 0 deletions.
80 changes: 80 additions & 0 deletions docs/docker/integ-test/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,86 @@ Sample response:
}
```

## Adding Tests

### Spark Tests

| | Query File Extension | Directory |
|-----|----------------------|-----------------------------------------------|
| PPL | .ppl | e2e-test/src/test/resources/spark/queries/ppl |
| SQL | .sql | e2e-test/src/test/resources/spark/queries/sql |

A test consists of a query and the expected results. Choose a base filename. The query file uses the base filename
with the extension above. The results file uses the base filename with a `.results` extension. The results file is
in CSV format with a field headers line.

Example: Adding a PPL Test

1. Create a file named `e2e-test/src/test/resources/spark/queries/ppl/sample-test.ppl` with the following contents:
```
source=my-table | fields x, y
```
2. Generate the expected results file named `e2e-test/src/test/resources/spark/queries/ppl/sample-test.results`
It is a CSV file with the field name header. For example:
```
x,y
1,1
2,4
3,9
```

### Async API Tests

| | Query File Extension | Directory |
|-----|----------------------|----------------------------------------------------|
| PPL | .ppl | e2e-test/src/test/resources/opensearch/queries/ppl |
| SQL | .sql | e2e-test/src/test/resources/opensearch/queries/sql |

A test consists of a query and the expected results. Choose a base filename. The query file uses the base filename
with the extension above. The results file uses the base filename with a `.results` extension. The results file is
in JSON format is the response when retrieving the results using the Async Query API.

Example: Adding a PPL Test

1. Create a file named `e2e-test/src/test/resources/opensearch/queries/ppl/sample-test.ppl` with the following
contents:
```
source=my-table | fields x, y
```
2. Generate the expected results file named `e2e-test/src/test/resources/opensearch/queries/ppl/sample-test.results`
It is a JSON file. For example:
```
{
"status": "SUCCESS",
"schema": [
{
"name": "x",
"type": "integer"
},
{
"name": "y",
"type": "integer"
}
],
"datarows": [
[
1,
1
],
[
2,
4
],
[
3,
9
]
],
"total": 3,
"size": 3
}
```

## Configuration of the Cluster

There are several settings that can be adjusted for the cluster.
Expand Down
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_001.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | dedup 1 status | fields @timestamp, clientip, status, size | head 10
3 changes: 3 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_001.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
@timestamp,clientip,status,size
2023-10-01T10:00:00.000Z,40.135.0.0,200,24736
2023-10-01T10:20:00.000Z,247.37.0.0,304,0
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_002.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | dedup status, size | head 10
4 changes: 4 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_002.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_003.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | dedup 1 status keepempty=true | head 10
3 changes: 3 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_003.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_004.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | dedup status, size keepempty=true | head 10
4 changes: 4 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_004.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_005.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | dedup 2 status | head 10
4 changes: 4 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_005.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_006.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | dedup 2 status, size | head 10
5 changes: 5 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_006.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_007.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | dedup 2 status, size keepempty=true | head 10
5 changes: 5 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_007.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_008.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | sort status | fields @timestamp, clientip, status | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_008.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp,clientip,status
2023-10-01T10:00:00.000Z,40.135.0.0,200
2023-10-01T10:05:00.000Z,232.0.0.0,200
2023-10-01T10:10:00.000Z,26.1.0.0,200
2023-10-01T10:15:00.000Z,247.37.0.0,200
2023-10-01T10:25:00.000Z,252.0.0.0,200
2023-10-01T10:20:00.000Z,247.37.0.0,304
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_009.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | fields - @timestamp, clientip, status | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_009.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
request,size,year,day,month
GET /images/hm_bg.jpg HTTP/1.0,24736,2023,1,10
GET /images/hm_bg.jpg HTTP/1.0,24736,2023,1,10
GET /images/hm_bg.jpg HTTP/1.0,24736,2023,1,10
GET /french/splash_inet.html HTTP/1.0,3781,2023,1,10
GET /images/hm_nbg.jpg HTTP/1.0,0,2023,1,10
GET /images/hm_bg.jpg HTTP/1.0,24736,2023,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_010.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval new_time = @timestamp, new_clientip = clientip | fields - new_time, new_clientip, status | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_010.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp,request,size,year,clientip,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_011.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval new_clientip = lower(clientip) | fields - new_clientip | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_011.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp,request,size,year,clientip,status,day,month,new_clientip
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10,40.135.0.0
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10,232.0.0.0
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10,26.1.0.0
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10,247.37.0.0
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10,247.37.0.0
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10,252.0.0.0
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_012.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | fields + @timestamp, clientip, status | fields - clientip, status | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_012.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp
2023-10-01T10:00:00.000Z
2023-10-01T10:05:00.000Z
2023-10-01T10:10:00.000Z
2023-10-01T10:15:00.000Z
2023-10-01T10:20:00.000Z
2023-10-01T10:25:00.000Z
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_013.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where status = 200 | head 10
6 changes: 6 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_013.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_014.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where status != 200 | head 10
2 changes: 2 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_014.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_015.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where size > 0 | head 10
6 changes: 6 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_015.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_016.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where size <= 0 | head 10
2 changes: 2 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_016.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_017.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where clientip = '236.14.2.0' | head 10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_017.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@timestamp,request,size,year,clientip,status,day,month
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_018.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where size > 0 AND status = 200 OR clientip = '236.14.2.0' | head 100
6 changes: 6 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_018.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_019.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where size <= 0 AND like(request, 'GET%') | head 10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_019.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@timestamp,request,size,year,clientip,status,day,month
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_020.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs status = 200 | head 10
6 changes: 6 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_020.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_021.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs size > 0 AND status = 200 OR clientip = '236.14.2.0' | head 100
6 changes: 6 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_021.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_022.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs size <= 0 AND like(request, 'GET%') | head 10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_022.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@timestamp,request,size,year,clientip,status,day,month
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_023.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where isempty(size)
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_023.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@timestamp,request,size,year,clientip,status,day,month
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_024.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where ispresent(size)
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_024.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_025.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where isnull(size) | head 10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_025.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@timestamp,request,size,year,clientip,status,day,month
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_026.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where isnotnull(size) | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_026.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_027.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where like(request, 'GET%') | head 10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_027.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@timestamp,request,size,year,clientip,status,day,month
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_028.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where like(request, '%bordeaux%') | head 10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_028.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@timestamp,request,size,year,clientip,status,day,month
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_029.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | where case(status = 200, 'success' else 'failed') = 'success' | head 10
6 changes: 6 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_029.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
@timestamp,request,size,year,clientip,status,day,month
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_030.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval h = "Hello", w = "World" | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_030.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp,request,size,year,clientip,status,day,month,h,w
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10,Hello,World
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10,Hello,World
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10,Hello,World
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10,Hello,World
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10,Hello,World
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10,Hello,World
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_031.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval @h = "Hello" | eval @w = "World" | fields @timestamp, @h, @w
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_031.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp,@h,@w
2023-10-01T10:00:00.000Z,Hello,World
2023-10-01T10:05:00.000Z,Hello,World
2023-10-01T10:10:00.000Z,Hello,World
2023-10-01T10:15:00.000Z,Hello,World
2023-10-01T10:20:00.000Z,Hello,World
2023-10-01T10:25:00.000Z,Hello,World
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_032.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval newF = clientip | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_032.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
@timestamp,request,size,year,clientip,status,day,month,newF
2023-10-01T10:00:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,40.135.0.0,200,1,10,40.135.0.0
2023-10-01T10:05:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,232.0.0.0,200,1,10,232.0.0.0
2023-10-01T10:10:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,26.1.0.0,200,1,10,26.1.0.0
2023-10-01T10:15:00.000Z,GET /french/splash_inet.html HTTP/1.0,3781,2023,247.37.0.0,200,1,10,247.37.0.0
2023-10-01T10:20:00.000Z,GET /images/hm_nbg.jpg HTTP/1.0,0,2023,247.37.0.0,304,1,10,247.37.0.0
2023-10-01T10:25:00.000Z,GET /images/hm_bg.jpg HTTP/1.0,24736,2023,252.0.0.0,200,1,10,252.0.0.0
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_033.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval newF = clientip | fields clientip, newF | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_033.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
clientip,newF
40.135.0.0,40.135.0.0
232.0.0.0,232.0.0.0
26.1.0.0,26.1.0.0
247.37.0.0,247.37.0.0
247.37.0.0,247.37.0.0
252.0.0.0,252.0.0.0
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_034.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval f = size | where f > 1 | sort f | fields size, clientip, status | head 10
6 changes: 6 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_034.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
size,clientip,status
3781,247.37.0.0,200
24736,40.135.0.0,200
24736,232.0.0.0,200
24736,26.1.0.0,200
24736,252.0.0.0,200
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_035.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval f = status * 2 | eval h = f * 2 | fields status, f, h | head 10
7 changes: 7 additions & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_035.results
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
status,f,h
200,400,800
200,400,800
200,400,800
200,400,800
304,608,1216
200,400,800
1 change: 1 addition & 0 deletions e2e-test/src/test/resources/spark/queries/ppl/query_036.ppl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source = dev.default.http_logs | eval f = size * 2, h = status | stats sum(f) by h
Loading

0 comments on commit 4d6ba7d

Please sign in to comment.