Residue on big numbers _maybe_ computes...

If we don't know about 'em, we can't fix 'em.

Residue on big numbers _maybe_ computes...

PostPosted by turnip » Thu May 04, 2017 3:12 pm

Okay, this is kind of a weird one, and probably has to do with default variable types. First consider this:
(23*2)|5*⍳30
5 25 125 96 480 284 362 223 57 285 367 248 182 381 318 3 15 75 375 288 382 323 0 0 0 0 0 0 0 0

Now, those zeroes after 5*22 don't necessarily bother me. After all, it's not like I used variable precision. But now, watch this:

(23*2)|5*22
323
(23*2)|5*23
28
(23*2)|5*24
140
(23*2)|5*25
171
(23*2)|5*26
326
(23*2)|5*27
43
(23*2)|5*28
0

Why, now it seems to be actually able to handle 5*23 through 5*27 with no difficulty. Why the difference? Again, I assume this has something to do with the way the precision is handled differently in vectors vs scalars?

--Mike Turniansky
turnip
 
Posts: 21
Joined: Wed Jan 11, 2012 12:52 pm

Re: Residue on big numbers _maybe_ computes...

PostPosted by forummaster » Sat May 06, 2017 3:24 pm

turnip wrote:Okay, this is kind of a weird one, and probably has to do with default variable types. First consider this:
(23*2)|5*⍳30
5 25 125 96 480 284 362 223 57 285 367 248 182 381 318 3 15 75 375 288 382 323 0 0 0 0 0 0 0 0

Now, those zeroes after 5*22 don't necessarily bother me. After all, it's not like I used variable precision. But now, watch this:

(23*2)|5*22
323
(23*2)|5*23
28
(23*2)|5*24
140
(23*2)|5*25
171
(23*2)|5*26
326
(23*2)|5*27
43
(23*2)|5*28
0

Why, now it seems to be actually able to handle 5*23 through 5*27 with no difficulty. Why the difference? Again, I assume this has something to do with the way the precision is handled differently in vectors vs scalars?

--Mike Turniansky

The reason for the difference is that 5*⍳30 is Floating Point because 5*28 and higher powers are. Compare these two expressions

      (23*2)|5*⍳27
5 25 125 96 480 284 362 223 57 285 367 248 182 381 318 3 15 75 375 288 382 323 28 140 171 326 43
(23*2)|5*⍳30
5 25 125 96 480 284 362 223 57 285 367 248 182 381 318 3 15 75 375 288 382 323 0 0 0 0 0 0 0 0

In other words the scalar lines you types were all Integer Residue up to and including 5*27. Beyond that, it was Floating Point Residue, but the vector lines were all calculated as Floating Point Residue.
forummaster
 
Posts: 567
Joined: Wed Jan 23, 2013 1:00 pm


Return to Bug Reports

Who is online

Users browsing this forum: No registered users and 2 guests

cron