LaTeX
H(p,q)=-\sum_x p(x)\log q(x)Variables
- p: target distribution
- q: model distribution
How to use this formula
Measures the expected coding cost when q is used for data distributed as p.
Important notes
- q(x) must be positive wherever p(x)>0.
Quick example
Classification loss often averages cross-entropy across examples.
Applicability, worked calculation, and verification
Assumptions and domain checks
- q(x) must be positive wherever p(x)>0.
- Logarithm arguments must be positive in the real domain, and the base must be stated when it is not implied.
- For the Cross-Entropy, the index variable, lower bound, upper bound, and any empty-sum or empty-product convention must be clear.
- State the logarithm base, probability model, and whether the reported unit is bits, nats, or another convention.
Worked example
Classification loss often averages cross-entropy across examples.
Common mistakes
- Do not omit the index or bounds in Cross-Entropy; changing either one changes which terms are included.
- Do not compare entropy or information values computed with different logarithm bases without converting the units.
Continue the workflow
Use Cross-Entropy in your own work
- Check the domainMatch the variables and assumptions to the problem before substituting values.
- Copy the exact notationPreserve grouping, signs, and exponents in
H(p,q)=-\sum_x p(x)\log q(x). - Edit or convertOpen the expression in the LaTeX editor, then export it for your document or web page.
Review and verification
Last reviewed: 2026-07-23
Automated quality check: Kept noindex until the missing evidence is supplied.
Formula references
- Digital Library of Mathematical FunctionsNational Institute of Standards and Technology — Definitions, notation, identities, and reference material for mathematical functions.
Frequently asked questions
What is the Cross-Entropy used for?
Measures the expected coding cost when q is used for data distributed as p.
Can I copy this formula as LaTeX?
Yes. Copy H(p,q)=-\sum_x p(x)\log q(x) or open it in the LaTeX editor.
What should I check before using it?
Confirm that each variable, unit, domain restriction, and assumption matches the problem.