hadoop - How Block size varies from Cluster1 to Cluster2, if we use DistCp command? -
i processing "distcp" command move few critical files form cluster1 cluster2. these critical files residing blocksize 64mb, before. , moved cluster2 [it got 128mb blocksize).
after distcp move, how does critical files performance increment new blocksize in cluster2..performance increment or decreases..???
it depends on files. hadoop files supposed read sequentially , if files big(let's gbs or tbs) increment performance if increment blocksize, because decrease number of tasks performed. copying distcp not maintain block properties of file since block configurations varies cluster cluster.
hadoop distcp
No comments:
Post a Comment