entropy - Entropie (information theory) calculation -

entropy - Entropie (information theory) calculation -

i have basic question calculating entropy of split.

assumed have set 2 classes, yes , no. in set have 3 samples yes , 2 samples no.

if calculate entropy of set obtain:

-(2/5)*(log(2/5)/log(2))-(3/5)*(log(3/5)/log(2))=0.9710

now, gets me confused. if entropy 0 have samples of 1 class. if entropy 0.5 (for 2 classes) have 50% yes , 50% no samples. value close 1 tells me now?

a pointer please, sense not seeing obvious here, don't understand when entropy can reach 1?

in binary illustration such yours, entropy of scheme approach 1 if distributed each of possible outcomes (10 samples, 5 yes, 5 no). farther distribution closer 0. can see binary entropy plot on wikipedia.

more perfect distribution of entropy sum log2(numclasses). 2 == log2(2) == 1.

entropy