Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve incorrect type handling on columnar #2462

Open
PSeitz opened this issue Jul 29, 2024 · 0 comments
Open

Improve incorrect type handling on columnar #2462

PSeitz opened this issue Jul 29, 2024 · 0 comments

Comments

@PSeitz
Copy link
Contributor

PSeitz commented Jul 29, 2024

The way incorrect types on typed columns are handled in columnar is a footgun. We should ignore it or return an error.

#[test]
fn test_date_range_query() {
    let mut schema_builder = Schema::builder();
    let date_field = schema_builder.add_date_field("date", FAST);
    let schema = schema_builder.build();

    let index = Index::create_in_ram(schema.clone());
    
      // This is adds a string and creates a string column
      index_writer
          .add_document(doc!(date_field => "2022-12-01T00:00:01Z"))
          .unwrap();
          
      let mut index_writer = index.writer_with_num_threads(1, 50_000_000).unwrap();
      // Parses the string to a Date. Now we have 2 columns under "date"
      let doc = TantivyDocument::parse_json(&schema, r#"{ "date": "2022-12-01T00:00:01Z" }"#)
          .unwrap();
      index_writer.add_document(doc).unwrap();
      
      // Adds a Date
      index_writer
          .add_document(doc!(date_field => DateTime::from_utc(
              OffsetDateTime::parse("2023-12-01T00:00:01Z", &Rfc3339).unwrap(),
          )))
          .unwrap();

      index_writer.commit().unwrap();
    
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant