[Maxima] fpprintprec
Stavros Macrakis
macrakis at alum.mit.edu
Wed Oct 8 08:42:54 CDT 2008
On Wed, Oct 8, 2008 at 9:25 AM, Raymond Toy <raymond.toy at ericsson.com>wrote:
> > ... Also, should we round the printed result?
> >
> > fpprec:16;
> > fpprintprec:4;
> > 1.56789b0 -> 1.567b0
> >
> > Yes, the correct result here would be 1.568b0.
>
> Hmm. That's not so easy with the current algorithm because the number
> has already been converted to a list of individual digits. Doing the
> example above is easy, but not so easy for something like 1.999999b0.
>
It is important to round correctly. I am surprised that the current
algorithm first generates the digits and then rounds/truncates. That is not
a good way to do it....
-s
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.math.utexas.edu/pipermail/maxima/attachments/20081008/299e3d0f/attachment.htm
More information about the Maxima
mailing list