To solve many practical problems it is necessary to know the complex of conditions due to which the result of the cumulative impact large quantities random factors are almost independent of chance. These conditions are described in several theorems called common name law large numbers, where the random variable k is equal to 1 or 0 depending on whether the result of the kth trial is success or failure. Thus, Sn is the sum of n mutually independent random variables, each of which takes the values 1 and 0 with probabilities p and q.

The simplest form of the law of large numbers is Bernoulli's theorem, which states that if the probability of an event is the same in all trials, then as the number of trials increases, the frequency of the event tends to the probability of the event and ceases to be random.

Poisson's theorem states that the frequency of an event in a series independent tests tends to the arithmetic mean of its probabilities and ceases to be random.

Limit theorems of probability theory, the Moivre-Laplace theorem explain the nature of the stability of the frequency of occurrence of an event. This nature lies in the fact that the limiting distribution of the number of occurrences of an event with an unlimited increase in the number of trials (if the probability of the event is the same in all trials) is a normal distribution.

Central limit theorem explains the widespread distribution of the normal distribution law. The theorem states that whenever a random variable is formed as a result of addition large number independent random variables with finite variances, the distribution law of this random variable turns out to be an almost normal law.

Lyapunov's theorem explains the widespread distribution of the normal distribution law and explains the mechanism of its formation. The theorem allows us to state that whenever a random variable is formed as a result of the addition of a large number of independent random variables, the variances of which are small compared to the dispersion of the sum, the distribution law of this random variable turns out to be an almost normal law. And since random variables are always generated infinite number reasons and most often none of them has a dispersion comparable to the dispersion of the random variable itself, then most random variables encountered in practice are subject to the normal distribution law.

The qualitative and quantitative statements of the law of large numbers are based on Chebyshev inequality. It determines the upper limit of the probability that the deviation of the value of a random variable from its mathematical expectation is greater than a certain given number. It is remarkable that Chebyshev’s inequality gives an estimate of the probability of an event for a random variable whose distribution is unknown, only its mathematical expectation and dispersion.

Chebyshev's inequality. If a random variable x has variance, then for any x > 0 the inequality is true, where M x and D x - mathematical expectation and variance of the random variable x.

Bernoulli's theorem. Let x n be the number of successes in n Bernoulli trials and p the probability of success in an individual trial. Then for any s > 0 it is true.

Lyapunov's theorem. Let s 1, s 2, …, s n, …- unlimited sequence independent random variables with mathematical expectations m 1, m 2, …, m n, … and variances s 1 2, s 2 2, …, s n 2 …. Let's denote.

Then = Ф(b) - Ф(a) for any real numbers a and b, where Ф(x) is the normal distribution function.

Let a discrete random variable be given. Let us consider the dependence of the number of successes Sn on the number of trials n. On each trial, Sn increases by 1 or 0. This statement can be written as:

Sn = 1 +…+ n. (1.1)

Law of Large Numbers. Let (k) be a sequence of mutually independent random variables with identical distributions. If the mathematical expectation = M(k) exists, then for any > 0 for n

In other words, the probability that the average S n /n differs from the mathematical expectation by less than an arbitrarily given value tends to one.

Central limit theorem. Let (k) be a sequence of mutually independent random variables with identical distributions. Let's assume that they exist. Let Sn = 1 +…+ n , Then for any fixed

F () -- F () (1.3)

Here F (x) -- normal function I distribute. This theorem was formulated and proved by Linlberg. Lyapunov and other authors proved it earlier, under more restrictive conditions. It is necessary to imagine that the theorem formulated above is only a very special case of a much more general theorem, which in turn is closely related to many other limit theorems. Note that (1.3) is much stronger than (1.2), since (1.3) gives an estimate for the probability that the difference is greater than. On the other hand, the law of large numbers (1.2) is true even if the random variables k do not have finite variance, so it applies to more general case than the central limit theorem (1.3). Let us illustrate the last two theorems with examples.

Examples. a) Consider a sequence of independent throws of a symmetrical die. Let k be the number of points obtained during the kth throw. Then

M(k)=(1+2+3+4+5+6)/6=3.5,

a D(k)=(1 2 +2 2 +3 2 +4 2 +5 2 +6 2)/6-(3.5) 2 =35/12 and S n /n

is the average number of points resulting from n throws.

The law of large numbers states that it is plausible that for large n this average will be close to 3.5. The Central Limit Theorem states the probability that |Sn -- 3.5n |< (35n/12) 1/2 близка к Ф() -- Ф(-). При n = 1000 и а=1 мы находим, что вероятность неравенства 3450 < Sn < 3550 равна примерно 0,68. Выбрав для а значение а 0 = 0,6744, удовлетворяющее соотношению Ф(0)-- Ф(-- 0)=1/2, мы получим, что для Sn шансы находиться внутри или вне интервала 3500 36 примерно одинаковы.

b) Sampling. Let's assume that in population,

consisting of N families, Nk families have exactly k children each

(k = 0, 1 ...; Nk = N). If a family is selected at random, then the number of children in it is a random variable that takes a value with probability p = N/N. In back-to-back selection, one can view a sample of size n as a collection of n independent random variables or "observations" 1, ..., n that all have the same distribution; S n /n is the sample mean. The law of large numbers states that for a sufficiently large random sample its mean will probably be close to, that is, the population mean. The central limit theorem allows one to estimate the likely magnitude of the discrepancy between these means and determine the sample size required for a reliable estimate. In practice, and and are usually unknown; however, in most cases it is easy to obtain a preliminary estimate for and can always be enclosed within reliable boundaries. If we want a probability of 0.99 or greater that the sample mean S n /n differs from the unknown population mean by less than 1/10, then the sample size must be taken such that

The x root of the equation Ф(x) - Ф(-- x) = 0.99 is equal to x = 2.57 ..., and therefore n must be such that 2.57 or n > 660. A careful preliminary estimate makes it possible to find the required sample size.

c) Poisson distribution.

Suppose that the random variables k have a Poisson distribution (p(k;)). Then Sn has a Poisson distribution with mean and variance equal to n.

By writing instead of n, we conclude that for n

The summation is performed over all k from 0 to. Ph-la (1.5) also holds when in an arbitrary way.

Coursework

on the topic: “Laws of large numbers”

Identically distributed random variables

To solve many practical problems, it is necessary to know a set of conditions due to which the result of the combined influence of a large number of random factors is almost independent of chance. These conditions are described in several theorems, collectively called the law of large numbers, where the random variable k is equal to 1 or 0 depending on whether the result of the kth trial is success or failure. Thus, Sn is the sum of n mutually independent random variables, each of which takes the values 1 and 0 with probabilities p and q.

Poisson's theorem states that the frequency of an event in a series of independent trials tends to the arithmetic mean of its probabilities and ceases to be random.

The central limit theorem explains the widespread distribution of the normal distribution law. The theorem states that whenever a random variable is formed as a result of the addition of a large number of independent random variables with finite variances, the distribution law of this random variable turns out to be an almost normal law.

Lyapunov's theorem explains the widespread distribution of the normal distribution law and explains the mechanism of its formation. The theorem allows us to state that whenever a random variable is formed as a result of the addition of a large number of independent random variables, the variances of which are small compared to the dispersion of the sum, the distribution law of this random variable turns out to be an almost normal law. And since random variables are always generated by an infinite number of causes and most often none of them has a dispersion comparable to the dispersion of the random variable itself, most random variables encountered in practice are subject to the normal distribution law.

The qualitative and quantitative statements of the law of large numbers are based on Chebyshev inequality. It determines the upper bound on the probability that the deviation of the value of a random variable from its mathematical expectation is greater than a certain specified number. It is remarkable that Chebyshev’s inequality gives an estimate of the probability of an event for a random variable whose distribution is unknown, only its mathematical expectation and variance are known.

Chebyshev's inequality. If a random variable x has variance, then for any x > 0 the inequality is true, where M x and D x - mathematical expectation and variance of the random variable x.

Bernoulli's theorem. Let x n be the number of successes in n Bernoulli trials and p the probability of success in an individual trial. Then for any s > 0, .

Lyapunov's theorem. Let s 1 , s 2 , …, s n , … be an unlimited sequence of independent random variables with mathematical expectations m 1 , m 2 , …, m n , … and variances s 1 2 , s 2 2 , …, s n 2 … . Let's denote , , , .

Then = Ф(b) - Ф(a) for any real numbers a and b, where Ф(x) is the normal distribution function.

Sn = 1 +…+ n. (1.1)

Law of large numbers. Let (k) be a sequence of mutually independent random variables with identical distributions. If the mathematical expectation = M(k) exists, then for any > 0 for n

In other words, the probability that the average S n /n differs from the mathematical expectation by less than an arbitrarily given value tends to one.

Central limit theorem. Let (k) be a sequence of mutually independent random variables with identical distributions. Let's assume that they exist. Let Sn = 1 +…+ n , Then for any fixed

F () - F () (1.3)

Here Ф(х) is the normal distribution function. This theorem was formulated and proved by Linlberg. Lyapunov and other authors proved it earlier, under more restrictive conditions. It is necessary to imagine that the theorem formulated above is only a very special case of a much more general theorem, which in turn is closely related to many other limit theorems. Note that (1.3) is much stronger than (1.2), since (1.3) gives an estimate for the probability that the difference is greater than . On the other hand, the law of large numbers (1.2) is true even if the random variables k do not have finite variance, so it applies to a more general case than the central limit theorem (1.3). Let us illustrate the last two theorems with examples.

Examples. a) Consider a sequence of independent throws of a symmetrical die. Let k be the number of points obtained during the kth throw. Then

M( k)=(1+2+3+4+5+6)/6=3.5,

a D( k)=(1 2 +2 2 +3 2 +4 2 +5 2 +6 2)/6-(3.5) 2 =35/12 and S n /n

is the average number of points resulting from n throws.

The law of large numbers states that it is plausible that for large n this average will be close to 3.5. The central limit theorem states the probability that |Sn - 3.5n |< (35n/12) 1/2 близка к Ф() - Ф(- ). При n = 1000 и а=1 мы находим, что вероятность неравенства 3450 < Sn < 3550 равна примерно 0,68. Выбрав для а значение а 0 = 0,6744, удовлетворяющее соотношению Ф( 0)- Ф(- 0)=1/2, мы получим, что для Sn шансы находиться внутри или вне интервала 3500 36 примерно одинаковы.

b) Sampling. Let us assume that in the population,

consisting of N families, Nk families have exactly k children each

(k = 0, 1 ...; Nk = N). If a family is selected at random, then the number of children in it is a random variable that takes a value with probability p = N /N. In back-to-back selection, one can view a sample of size n as a collection of n independent random variables or "observations" 1, ..., n that all have the same distribution; S n /n is the sample mean. The Law of Large Numbers states that for a large enough random sample, the mean is likely to be close to , that is, the population mean. The central limit theorem allows one to estimate the likely magnitude of the discrepancy between these means and determine the sample size required for a reliable estimate. In practice, and and are usually unknown; however, in most cases it is easy to obtain a preliminary estimate for and can always be enclosed within reliable boundaries. If we want a probability of 0.99 or greater that the sample mean S n /n differs from the unknown population mean by less than 1/10, then the sample size must be taken such that

The x root of the equation F(x) - F(- x) = 0.99 is x = 2.57..., and therefore n must be such that 2.57 or n > 660. A careful preliminary estimate makes it possible to find the required sample size.

c) Poisson distribution.

Suppose that the random variables k have a Poisson distribution (p(k; )). Then Sn has a Poisson distribution with mean and variance equal to n.

Writing instead of n, we conclude that for n

The summation is performed over all k from 0 to . Ph-la (1.5) also holds when in an arbitrary way.

Above we considered the question of finding the PDF for the sum of statistically independent random variables. In this section we will look again at the amount statistically independent quantities, but our approach will be different and does not depend on the partial PDFs of random variables in the sum. In particular, assume that the sum terms are statistically independent and identically distributed random variables, each of which has bounded means and bounded variance.

Let be defined as the normalized sum, called the sample mean

First, we will determine upper bounds on the tail probability, and then we will prove a very important theorem that determines the PDF in the limit when it tends to infinity.

The random variable defined by (2.1.187) is often encountered when estimating the average of a random variable over a number of observations, . In other words, can be considered as independent sample realizations from a distribution, and is an estimate of the mean.

The mathematical expectation is

The variance is

If we consider it as an estimate of the mean, we see that its mathematical expectation is equal to , and its dispersion decreases with increasing sample size. If it increases without limit, the variance tends to zero. Parameter estimate (in in this case), which satisfies the conditions that its mathematical expectation tends to true meaning parameter, and the variance is strictly zero, is called a consistent estimate.

The tail probability of a random variable can be estimated from above using the bounds given in Section. 2.1.5. Chebyshev’s inequality in relation to has the form

. (2.1.188)

In the limit when , from (2.1.188) it follows

. (2.1.189)

Consequently, the probability that the estimate of the mean differs from the true value by more than , tends to zero if it grows without limit. This statement is a form of the law of large numbers. Since the upper bound converges to zero relatively slowly, i.e. inversely proportional. expression (2.1.188) is called weak law of large numbers.

If we apply the Chernoff bound to the random variable, which contains an exponential dependence on , then we obtain a tight upper bound for the single-tail probability. Following the procedure outlined in Sect. 2.1.5, we find that the tail probability for is determined by the expression

where and . But , are statistically independent and identically distributed. Hence,

where is one of the quantities. The parameter , which gives the most accurate upper bound, is obtained by differentiating (2.1.191) and equating the derivative to zero. This leads to the equation

(2.1.192)

Let us denote the solution (2.1.192) by . Then the bound for the upper tail probability is

, . (2.1.193)

Similarly, we will find that the lower tail probability has the bound

, . (2.1.194)

Example 2.1.7. Let , be a series of statistically independent random variables defined as follows:

We want to define a tight upper bound on the probability that the sum of is greater than zero. Since , the amount will have negative value for the mathematical expectation (average), therefore, we will look for the upper tail probability. For in (2.1.193) we have

, (2.1.195)

where is the solution to the equation

Hence,

. (2.1.197)

Consequently, for the boundary in (2.1.195) we obtain

We see that upper limit decreases exponentially with , as expected. In contrast, according to the Chebyshev bound, the tail probability decreases inversely with .

Central limit theorem. In this section, we consider an extremely useful theorem concerning the IDF of a sum of random variables in the limit when the number of terms of the sum increases without limit. There are several versions of this theorem. Let us prove the theorem for the case when the random summable variables , , are statistically independent and identically distributed, each of them has a limited mean and limited variance.

For convenience, we define a normalized random variable

Thus, it has zero mean and unit variance.

Now let

Since each summand of the sum has zero mean and unit variance, the value normalized (by the factor ) has zero mean and unit variance. We want to define the FMI for the limit when .

The characteristic function is equal to

, (2.1.200).

or, equivalently,

. (2.1.206)

But that's just it characteristic function Gaussian random variable with zero mean and unit variance. Thus we have important result; The PDF of the sum of statistically independent and identically distributed random variables with limited mean and variance approaches Gaussian at . This result is known as central limit theorem.

Although we have assumed that the random variables in the sum are equally distributed, this assumption can be relaxed provided that certain additional restrictions are still imposed on the properties of the random variables being summed. There is one variation of the theorem, for example, when the assumption of an identical distribution of random variables is abandoned in favor of a condition imposed on the third absolute moment of the random variables of the sum. For a discussion of this and other versions of the central limit theorem, the reader is referred to Cramer (1946).

The central limit theorem is a group of theorems devoted to establishing the conditions under which normal law distributions, and the violation of which leads to a distribution different from normal. Various shapes The central limit theorem differs from each other in the conditions imposed on the distributions of the random terms forming the sum. Let's prove one of the most simple shapes this theorem, namely, the central limit theorem for independent identically distributed terms.

Consider a sequence of independent identically distributed random variables that have a mathematical expectation. Let's also assume that variance exists. Let us introduce the notation. The law of large numbers for this sequence can be represented in the following form:

where convergence can be understood both in the sense of convergence in probability (weak law of large numbers) and in the sense of convergence with probability, equal to one(strengthened law of large numbers).

Theorem (central limit theorem for independent identically distributed random variables). Let be a sequence of independent identically distributed random variables, . Then there is uniform relative to () convergence

where is the function of the standard normal distribution(with parameters):

If the condition for such convergence is satisfied, the sequence is called asymptotically normal.

Theorems of Lyapunov and Lindeberg

Let's consider the case when random variables have different distributions - they are independent with different distributions.

Theorem (Lindeberg). Let be a sequence of independent random variables with finite variances. If the Lindeberg condition is satisfied for this sequence:

where, then the central limit theorem holds for it.

Since directly checking the Lindeberg condition is difficult, we consider some other condition under which the central limit theorem holds, namely the condition of the Lyapunov theorem.

Theorem (Lyapunov). If the Lyapunov condition is satisfied for a sequence of random variables:

then the sequence is asymptotically normal, i.e. the central limit theorem holds.

The fulfillment of the Lyapunov condition implies the fulfillment of the Lindeberg condition, and from it follows the central limit theorem.

It is already known that according to the distribution law one can find numerical characteristics random variable. It follows that if several random variables have identical distributions, then their numerical characteristics are the same.

Let's consider n mutually independent random variables X 1 , X 2 , ...., X p, which have the same distributions, and therefore the same characteristics (mathematical expectation, dispersion, etc.). Of greatest interest is the study of the numerical characteristics of the arithmetic mean of these quantities, which is what we will do in this section.

Let us denote the arithmetic mean of the random variables under consideration by :

= (X 1 +X 2 +…+X n)/n.

The following three provisions establish a connection between the numerical characteristics of the arithmetic mean X and the corresponding characteristics of each individual quantity.

1. The mathematical expectation of the arithmetic mean of identically distributed mutually independent random variables is equal to the mathematical expectation of each of the values:

M()=a

Proof. Using the properties of mathematical expectation ( constant factor can be taken out as a sign of mathematical expectation; the mathematical expectation of the sum is equal to the sum of the mathematical expectations of the terms), we have

M( )= M

Taking into account that the mathematical expectation of each of the quantities according to the condition is equal to A, we get

M()=na/n=a.

2. The dispersion of the arithmetic mean of n identically distributed mutually independent random variables is n times less than the dispersion D of each of the values:

D()=D/n.(* )

Proof. Using the properties of dispersion (the constant factor can be taken out of the dispersion sign by squaring it; the dispersion of the sum of independent quantities is equal to the sum of the dispersions of the terms), we have

D( )=D

Taking into account that the dispersion of each of the quantities according to the condition is equal to D, we get

D( )= nD/n 2 =D/n.

3. Average standard deviation the arithmetic mean of n identically distributed mutually independent random variables is several times less than the mean quadratic deviation s each of the quantities:

Proof. Because D()= D/n, then the standard deviation is equal to

s ( )= .

General conclusion from formulas (*) and (**): remembering that dispersion and standard deviation serve as measures of the dispersion of a random variable, we conclude that the arithmetic mean of a sufficiently large number of mutually independent random variables has significantly less dispersion than each individual value.

Let us explain with an example the significance of this conclusion for practice.

Example. Usually to measure some physical quantity make several measurements, and then find the arithmetic mean of the obtained numbers, which is taken as an approximate value of the measured value. Assuming that the measurements are made under the same conditions, prove:

a) the arithmetic mean gives a more reliable result than individual measurements;

b) with an increase in the number of measurements, the reliability of this result increases.

Solution. a) It is known that individual measurements give different values of the measured quantity. The result of each measurement depends on many random reasons (temperature changes, instrument fluctuations, etc.), which cannot be fully taken into account in advance.

Therefore, we have the right to consider possible results n individual measurements as random variables X 1 , X 2 , ..., X p(the index indicates the measurement number). These quantities have equal distribution probabilities (measurements are made using the same method and with the same instruments), and therefore the same numerical characteristics; in addition, they are mutually independent (the result of each individual measurement does not depend on other measurements).

We already know that the arithmetic mean of such quantities has less dispersion than each individual quantity. In other words, the arithmetic mean turns out to be closer to the true value of the measured value than the result of a separate measurement. This means that the arithmetic mean of several measurements gives a more reliable result than a single measurement.

b) We already know that as the number of individual random variables increases, the dispersion of the arithmetic mean decreases. This means that as the number of measurements increases, the arithmetic mean of several measurements differs less and less from the true value of the measured value. Thus, by increasing the number of measurements, a more reliable result is obtained.

For example, if the standard deviation of an individual measurement is s = 6 m, and a total of n= 36 measurements, then the standard deviation of the arithmetic mean of these measurements is only 1 m. Indeed,

s ( )=

We see that the arithmetic mean of several measurements, as one would expect, turned out to be closer to the true value of the measured value than the result of a separate measurement.

Identically distributed random variables. Open Library - open library of educational information

Theorems of Lyapunov and Lindeberg