LaTeX
I(X;Y)=\sum_{x,y}p(x,y)\log\frac{p(x,y)}{p(x)p(y)}Variables
- p(x,y): joint distribution
- p(x), p(y): marginals
How to use this formula
Measures how much knowing one random variable reduces uncertainty about another.
Important notes
- The value is nonnegative and zero for independent variables.
Quick example
Used in feature selection and communication theory.
Applicability, worked calculation, and verification
Assumptions and domain checks
- The value is nonnegative and zero for independent variables.
- For the Mutual Information, every denominator must be nonzero, and the numerator and denominator must remain correctly grouped.
- Logarithm arguments must be positive in the real domain, and the base must be stated when it is not implied.
- For the Mutual Information, the index variable, lower bound, upper bound, and any empty-sum or empty-product convention must be clear.
Worked example
Used in feature selection and communication theory.
Common mistakes
- When copying Mutual Information, keep the complete numerator and denominator grouped; a missing brace or parenthesis changes the result.
- Do not compare entropy or information values computed with different logarithm bases without converting the units.
Continue the workflow
Use Mutual Information in your own work
- Check the domainMatch the variables and assumptions to the problem before substituting values.
- Copy the exact notationPreserve grouping, signs, and exponents in
I(X;Y)=\sum_{x,y}p(x,y)\log\frac{p(x,y)}{p(x)p(y)}. - Edit or convertOpen the expression in the LaTeX editor, then export it for your document or web page.
Review and verification
Last reviewed: 2026-07-23
Automated quality check: Kept noindex until the missing evidence is supplied.
Formula references
- Digital Library of Mathematical FunctionsNational Institute of Standards and Technology — Definitions, notation, identities, and reference material for mathematical functions.
Frequently asked questions
What is the Mutual Information used for?
Measures how much knowing one random variable reduces uncertainty about another.
Can I copy this formula as LaTeX?
Yes. Copy I(X;Y)=\sum_{x,y}p(x,y)\log\frac{p(x,y)}{p(x)p(y)} or open it in the LaTeX editor.
What should I check before using it?
Confirm that each variable, unit, domain restriction, and assumption matches the problem.