You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
10 on segment 1 is getting coerced to a f64 and is not the same as 10 i64 on segment. So these would be considered two values in the cardinality aggregation.
One way to fix this would be to normalize the data back to i64 after retrieving the value from the column.
This normalization may be quite expensive though, so we need to check the performance to see if want to make that tradeoff for the improved accuracy.
Only cases where the columns are coerced to different types on different segments are affected. In cases where all segment are the same type, the normalization would be wasted CPU cycles.
The text was updated successfully, but these errors were encountered:
We coerce numerical values into a common numerical column. That may affect the precision of the cardinality aggregation. E.g.
10
on segment 1 is getting coerced to a f64 and is not the same as10
i64 on segment. So these would be considered two values in the cardinality aggregation.One way to fix this would be to normalize the data back to i64 after retrieving the value from the column.
This normalization may be quite expensive though, so we need to check the performance to see if want to make that tradeoff for the improved accuracy.
Only cases where the columns are coerced to different types on different segments are affected. In cases where all segment are the same type, the normalization would be wasted CPU cycles.
The text was updated successfully, but these errors were encountered: