You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm using astro_float to validate function approximations in my code. I noticed that one of my tests fails with a massive discrepancy in computed results. Here's a minimal example which reproduces the issue:
use astro_float::ctx::Context;use astro_float::{expr,Consts,RoundingMode};fnmain(){letmut ctx = Context::new(1024,RoundingMode::Down,Consts::new().expect("constants cache initialized"),
-10000,10000,);let z = &astro_float::BigFloat::from(-2.0);let d = &astro_float::BigFloat::from(1.0e-64);println!("{}", expr!(z - d, ctx));}
Note the exponent: e-32. The exponent should be e+0. Interestingly, this gives the correct result when using a precision of 64, 128, 256 and 512, but it fails with 1024.
The text was updated successfully, but these errors were encountered:
I'm using
astro_float
to validate function approximations in my code. I noticed that one of my tests fails with a massive discrepancy in computed results. Here's a minimal example which reproduces the issue:Output:
Note the exponent:
e-32
. The exponent should bee+0
. Interestingly, this gives the correct result when using a precision of 64, 128, 256 and 512, but it fails with 1024.The text was updated successfully, but these errors were encountered: