Saturday, 15 January 2011

java - Read files into string array quickly -



java - Read files into string array quickly -

i need read file java has 500,000+ lines , wondering whether there's anyway speed process compared code:

scanner s1 = new scanner(new fileinputstream(args[0])); while(s1.hasnextline()) { temp += s1.nextline() + "\n"; } info = temp.split("\\s+");

it's fine @ start after 200000 lines

temp += s1.nextline() + "\n"

does end taking while. final format need string array of every word.

the reason temp += s1.nextline() + "\n" taking long time generating lot of strings. in fact, n characters read, generating o(n) big strings, , copying o(n^2) characters.

the solution (just) append stringbuilder instead of using string concatenation. however, that's not real solution here, because temp string not ultimate goal. ultimate goal create array of words.

what really need split each line words, , accumulate words. accumulating them straight array won't work ... because arrays cannot extended. recommend following:

create arraylist<string> hold of words read , split each line array of words append words in array list of words when finished, utilize list.toarray produce final array of words ... or maybe leave words in list, if more appropriate.

the final format need string array of every word.

i read above meaning want list of of words in file. if word appears multiple times in file, should appear multiple times in list.

on other hand, if want list of distinct words in file, should utilize set rather list accumulate words. depending on want words next, hashset, treeset or linkedhashset appropriate.

java

No comments:

Post a Comment