[Information Theory] Entropy, KL Divergence, and Mutual Information

Contents

• Entropy, Cross Entropy, Conditional Entropy
• KL Divergence
• Mutual Information
• KL Divergence of Two Different Normal Distribution & Bernoulli Distribution

Notes

• Entropy, KL Divergence, and Mutual Information