You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #682 I have found that currently numbat uses 64bit floats and for high precision calculations that could be quite an important detail (e.g. numbers could get rounded, etc.) - it makes sense to document this.
If there are some plans(tickets?) to support more representations (BigDecimal?) then that could be also useful.
The text was updated successfully, but these errors were encountered:
In #682 I have found that currently
numbat
uses 64bit floats and for high precision calculations that could be quite an important detail (e.g. numbers could get rounded, etc.) - it makes sense to document this.If there are some plans(tickets?) to support more representations (BigDecimal?) then that could be also useful.
The text was updated successfully, but these errors were encountered: