You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Throughout the repo we use u64 types as well as i64 types, I suppose the latter is due to postgres using i64 to represent their BIGINT but this of course can lead to loss of precision, particularly when converting from floating point types as well. So we need a consolidated approach to handling these types.
My recommendation would be to standardize signed integer types across the entire repo.
The text was updated successfully, but these errors were encountered:
@jorgeantonio21 you are correct, forgive my poor wording, I should have said "converting to floating point types"
But we actually do cast from i64 to f64 such as here for instance, and casting i64 to f64 causes a loss of precision (i64 is 64 bits wide, but f64's mantissa is only 52 bits wide)
Throughout the repo we use
u64
types as well asi64
types, I suppose the latter is due to postgres usingi64
to represent their BIGINT but this of course can lead to loss of precision, particularly when converting from floating point types as well. So we need a consolidated approach to handling these types.My recommendation would be to standardize signed integer types across the entire repo.
The text was updated successfully, but these errors were encountered: