LaTeX
D_{\mathrm{KL}}(p\|q)=\sum_x p(x)\log\frac{p(x)}{q(x)}Variables
- p, q: probability distributions
How to use this formula
Measures directed discrepancy from a reference distribution q to a target distribution p.
Important notes
- KL divergence is not symmetric and is not a metric.
Quick example
D_KL(p||q)=0 only when p and q agree almost everywhere.
Applicability, worked calculation, and verification
Assumptions and domain checks
- KL divergence is not symmetric and is not a metric.
- For the Kullback–Leibler Divergence, every denominator must be nonzero, and the numerator and denominator must remain correctly grouped.
- Logarithm arguments must be positive in the real domain, and the base must be stated when it is not implied.
- For the Kullback–Leibler Divergence, the index variable, lower bound, upper bound, and any empty-sum or empty-product convention must be clear.
Worked example
D_KL(p||q)=0 only when p and q agree almost everywhere.
Common mistakes
- When copying Kullback–Leibler Divergence, keep the complete numerator and denominator grouped; a missing brace or parenthesis changes the result.
- Do not compare entropy or information values computed with different logarithm bases without converting the units.
Continue the workflow
Use Kullback–Leibler Divergence in your own work
- Check the domainMatch the variables and assumptions to the problem before substituting values.
- Copy the exact notationPreserve grouping, signs, and exponents in
D_{\mathrm{KL}}(p\|q)=\sum_x p(x)\log\frac{p(x)}{q(x)}. - Edit or convertOpen the expression in the LaTeX editor, then export it for your document or web page.
Review and verification
Last reviewed: 2026-07-23
Automated quality check: Kept noindex until the missing evidence is supplied.
Formula references
- Digital Library of Mathematical FunctionsNational Institute of Standards and Technology — Definitions, notation, identities, and reference material for mathematical functions.
Frequently asked questions
What is the Kullback–Leibler Divergence used for?
Measures directed discrepancy from a reference distribution q to a target distribution p.
Can I copy this formula as LaTeX?
Yes. Copy D_{\mathrm{KL}}(p\|q)=\sum_x p(x)\log\frac{p(x)}{q(x)} or open it in the LaTeX editor.
What should I check before using it?
Confirm that each variable, unit, domain restriction, and assumption matches the problem.