Naive Bayes classifier

A probabilistic classifier that applies Bayes’ theorem while assuming features are conditionally independent given the class. It estimates class probabilities from feature values and selects a likely class.

Bayes’ theorem: A rule relating conditional probabilities that updates the probability of a hypothesis using evidence. It converts class likelihoods and prior probabilities into posterior class probabilities.
Conditional probability: The probability of an event given that another event has occurred. Class-conditional feature probabilities are the core quantities estimated by the classifier.
Multinomial Naive Bayes: A Naive Bayes variant that models feature counts with a multinomial distribution. Word-count vectors make this variant a standard choice for text categorization.
Logistic regression: A statistical model that estimates class probabilities using a linear function of input features. Unlike Naive Bayes, it models class probabilities directly without multiplying feature likelihoods.
Laplace smoothing: A smoothing method that adds a small positive pseudocount to possible outcomes when estimating probabilities. It prevents unseen feature values from assigning a class zero likelihood.
Conditional independence: Two variables are conditionally independent when knowing one gives no further information about the other after conditioning on a third variable. Naive Bayes assumes every feature pair is independent once the class is known.
Probability distribution: A mathematical description assigning probabilities to possible outcomes of a random variable. Each feature model specifies a distribution of values within each class.
Bernoulli Naive Bayes: A Naive Bayes variant that models binary feature presence or absence with Bernoulli distributions. It suits documents represented by whether each word appears, rather than how often.
Decision tree: A predictive model that partitions inputs through a sequence of feature-based decision rules. Trees can represent feature interactions that the independence assumption omits.
Calibration (statistics): The agreement between predicted probabilities and observed outcome frequencies. Naive Bayes can rank classes well yet produce overconfident probability estimates.

Linked from 8 pages