[Maxima] fpprintprec

Stavros Macrakis macrakis at alum.mit.edu
Wed Oct 8 08:42:54 CDT 2008


On Wed, Oct 8, 2008 at 9:25 AM, Raymond Toy <raymond.toy at ericsson.com>wrote:

> >    ... Also, should we round the printed result?
> >
> >     fpprec:16;
> >     fpprintprec:4;
> >     1.56789b0 -> 1.567b0
> >
> > Yes, the correct result here would be 1.568b0.
>
> Hmm.  That's not so easy with the current algorithm because the number
> has already been converted to a list of individual digits.  Doing the
> example above is easy, but not so easy for something like 1.999999b0.
>

It is important to round correctly.  I am surprised that the current
algorithm first generates the digits and then rounds/truncates.  That is not
a good way to do it....

       -s
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.math.utexas.edu/pipermail/maxima/attachments/20081008/299e3d0f/attachment.htm 


More information about the Maxima mailing list