Large Matrices in R: long vectors not supported yet -

Large Matrices in R: long vectors not supported yet -

i running 64 bit r 3.1 in 64bit ubuntu environment 400gb of ram, , encountering unusual limitation when dealing big matrices.

i have numeric matrix called a, 4000 rows 950,000 columns. when seek access element in it, receive next error:

error: long vectors not supported yet: subset.c:733

although matrix read in via scan, can replicate next code

test <- matrix(1,4000,900000) #no error test[1,1] #error

my googling reveals mutual error message prior r 3.0, vector of size 2^31-1 limit. however, not case, given environment.

should not using native matrix type kind of matrix?

a matrix atomic vector dimension attribute allows r access matrix. matrix vector of length 4000*9000000 3.6e+10 elements (the largest integer value approx 2.147e+9). subsetting long vector is supported atomic vectors (i.e. accessing elements beyond 2.147e+9 limit). treat matrix long vector.

if remember default r fills matrices column-wise if wanted retrieve value @ test[ 2701 , 850000 ] access via:

i <- ( 2701 - 1 ) * 850000 + 2701 test[i] #[1] 1

note long vector subsetting because:

2701l * 850000l #[1] na #warning message: #in 2701l * 850000l : nas produced integer overflow

r vector matrix