Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PPL: Add json_object command #3242

Open
wants to merge 29 commits into
base: main
Choose a base branch
from

Conversation

acarbonetto
Copy link
Collaborator

@acarbonetto acarbonetto commented Jan 13, 2025

This PR depends on the json_valid command: #3230

Description

For OpenSearch-PPL, adds the json_object() functions that takes a even number of varargs (key/value pairs). Returns an expression type that is valid json object (STRUCT).

Description
>>>>>>>>>>>

Usage: `json_object(<key>, <value>[, <key>, <value>]...)` returns a JSON object from key-value pairs.

Argument type:
- A \<key\> must be STRING.
- A \<value\> can be a scalar, another json object, or json array type.  Note: scalar fields will be treated as single-value.  Use `json_array` to construct an array value from a multi-value.

Return type: STRUCT

Example:

    os> source=people | eval result = json_object('key', 123.45) | fields result
    fetched rows / total rows = 1/1
    +------------------+
    | result           |
    +------------------+
    | {"key":123.45}   |
    +------------------+

    os> source=people | eval result = json_object('outer', json_object('inner', 123.45)) | fields result
    fetched rows / total rows = 1/1
    +------------------------------+
    | result                       |
    +------------------------------+
    | {"outer":{"inner":123.45}}   |
    +------------------------------+

    os> source=people | eval result = json_object('array_doc', json_array(123.45, "string", true, null)) | fields result
    fetched rows / total rows = 1/1
    +------------------------------------------------+
    | result                                         |
    +------------------------------------------------+
    | {"array_doc":[123.45, "string", true, null]}   |
    +------------------------------------------------+

Related Issues

Resolves #3208

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@acarbonetto acarbonetto force-pushed the feature/acarbo_json_object_ppl branch from 27299f2 to 7eb5979 Compare January 20, 2025 18:02
}

@Override
public Pair<FunctionSignature, FunctionBuilder> resolve(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@acarbonetto a naive question - do we have more use cases of kv pairs such as this one ?
if so would it make sense to move this code into a more generic place?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting question - no. json_object is very unique. Although we do have cases of path-value pairings, we're going to treat them differently.


List<String> validJsonStrings =
List.of(
// test json objects are valid
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you plz add more complex nested use cases here ?
for example inner obj with and array side by an array ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the json_array function to do that. I've added a TODO in the tests to add this test in the future.

docs/user/ppl/functions/json.rst Show resolved Hide resolved
docs/user/ppl/functions/json.rst Outdated Show resolved Hide resolved
14yapkc1 and others added 5 commits February 11, 2025 22:19
kenrickyap and others added 21 commits February 11, 2025 22:20
Signed-off-by: Kenrick Yap <[email protected]>
Signed-off-by: Kenrick Yap <[email protected]>
Signed-off-by: Kenrick Yap <[email protected]>
….java

Co-authored-by: Andrew Carbonetto <[email protected]>
Signed-off-by: kenrickyap <[email protected]>
Signed-off-by: Kenrick Yap <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
@acarbonetto acarbonetto force-pushed the feature/acarbo_json_object_ppl branch from 50449b0 to 4759311 Compare February 12, 2025 06:37
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Copy link
Collaborator

@Yury-Fridlyand Yury-Fridlyand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of that API? It looks kinda useless.
Are there future plans for other features where it could be used?

@acarbonetto
Copy link
Collaborator Author

What is the purpose of that API? It looks kinda useless. Are there future plans for other features where it could be used?

As mentioned in the RFC, this API mirrors the Splunk SP2 api and would be helpful for migrating to sharing data between apis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Add json_object constructor function to PPL
6 participants