algorithm - How to sort a big file (not fitting in RAM) -

algorithm - How to sort a big file (not fitting in RAM) -

say there algorithm x requires 2 steps final output file.

collect data sort data

let collected info big held in ram , written file before step 2 takes action.

for example, take file 500gb contains numbers, output step 1. 1 number in each line. step 2 must sort lines in ascending order.

how step 2 go efficiently sorting numbers without reading input file whole?

most efficient increment swap space 500 gb , single sort, letting os memory manager handle cache.

an alternative split info pieces fit, 250 2gb files. sort each one, merge sort on result.

algorithm sorting order bigdata