Independent and identically distributed random variables

A collection of random variables is independent and identically distributed (i.i.d.) when the variables are mutually independent and share one probability distribution.

Random variable: A measurable function that assigns a numerical value to each outcome of a random experiment. Each member of an i.i.d. collection is a random variable.
Product measure: A measure on a product space whose values encode the measures of its component spaces. Independent variables with a common law have a joint law formed from repeated copies of that law.
Independent and identically distributed sampling: A sampling model in which observations are independent draws from one fixed population distribution. It is the standard setting for many methods that treat data points as repeated draws.
Markov chain: A stochastic process in which the next state depends on the present state, conditional on the past. Successive states are generally dependent, unlike members of an i.i.d. sequence.
Statistical independence: A property in which knowing the values of some variables does not change the joint probabilities of the others. Mutual independence rules out probabilistic dependence among every finite subset.
Probability generating function: A power series encoding the probabilities of a nonnegative integer-valued random variable. For independent integer-valued variables, the sum's generating function is the product of theirs.
Law of large numbers: A theorem stating conditions under which sample averages converge to an expected value. I.i.d. samples provide a central setting where averages stabilize around their common mean.
Autoregressive model: A time-series model that expresses each value as a function of earlier values and an error term. Its observations typically depend on preceding observations rather than being independent.
Probability distribution: A rule assigning probabilities to possible values or sets of values of a random variable. Identical distribution means every variable has this same probability law.
Moment-generating function: A function of a real parameter whose derivatives, when they exist, encode a distribution's moments. Independence turns the moment-generating function of a sum into a product of identical functions.

Linked from 20 pages