Angular minors of the Hessian matrix. Hessian functions

Size: px

Start showing from the page:

Transcript

1 Definition. Point 0 is called the local maximum point of the function neighborhood of point 0, which for all of this neighborhood f f 0. Definition. Point 0 is called the local minimum point of the function, a neighborhood of point 0, that for all of this neighborhood f f 0. f f, if such exists, if such exists The value of the function at the maximum point is called the local maximum, the value of the function at the minimum point is the local minimum of this function. The maximum and minimum of a function are called its local extrema. The term “local extremum” is due to the fact that the introduced concept of extremum is associated with the neighborhood of a given point in the domain of definition of the function, and not with this entire domain. A function can have several extrema, and the minimum at one point may be greater than the maximum at another. Usually in the literature the terms “extremum”, “maximum”, “minimum” are used to denote a strict local extremum, a strict local maximum, a strict local minimum. Definition. Point 0 is called a point of strict local maximum of a function such a neighborhood of point 0 such that for everyone in this neighborhood f f 0. f, if Definition exists. Point 0 is called a point of strict local minimum of a function such a neighborhood of point 0 such that for everyone in this neighborhood f f 0. Or, point 0 is called a point of strict local minimum of function 0: 0 f f. 0 0 f if f if exists Definition. The largest (smallest) value of a function on an interval is called the global extremum. The global extremum can be reached either at the points of the local extremum or at the ends of the segment. A matrix composed of the second derivatives of a function is called the Hessian matrix: f f n d f T d d f f... n 1 n (we will agree to call the determinant of the Hessian matrix the Hessian; similarly: a matrix composed of the first derivatives of a function is called the Jacobian matrix, and its determinant is called the Jacobian. Literature: 1) Malugin V.A. "Mathematical analysis, course of lectures (mathematics for economists)", 005, p. 105 (the concept of extremum of a function);) Malugin V.A. “Mathematical analysis, problems and exercises (mathematics for economists)”, 006, p. 13 (global extremum; necessary condition for extremum, 1st and 1st sufficient conditions for extremum); 3) Written D.T. "Lecture notes on higher mathematics", 005, p. 0 (extremum of a function of one variable); 4) Bortakovsky A.S., Panteleev A.V. "Linear algebra in examples and problems", 005, p. 6. 1

2 Brief presentation of the solution to the problem. Theorem (sufficient condition for extremum). Let there be a function f at a stationary point 0, 0 in the neighborhood. Let us calculate the values ​​of A f, B f C f at the point 0, 0. Let us denote by some its continuous partial derivatives up to the second order inclusive. A B AC B B C, then: 1) if 0, then the function f, at the point, minimum, if A 0 ;) if 0, then the function f, at the point 0, 0, In the case of 0, the extremum is at the research point. 0 0 has an extremum: maximum if A 0 ; has no extremum. maybe, maybe not. Additional 1) Examine the function z 50 0, 0, z z 0 for extremum Find stationary points: z 0 z Found stationary point P 5 ;. Let us check whether the sufficient conditions for the presence of an extremum at a given point are satisfied. 50 A z B z 1 0 C z 40 3

3 At point P 5 ; : A 450 B 1 C at point P 5 ; minimum, because A 0 0 Answer: the function has a minimum z ;. The value of the function at this point z ;) Find the extrema of the function z 1 z z z z 0 z Note that at 0 the solution is the set of points with coordinates R (in space 0 is a straight line parallel to the O axis). Consequently, among the points with 0 there are no stationary points. z 0 Excluding points with 0 from consideration, we obtain: P ; P; 4 1 3

4 Two stationary points found P 1 01 ; and P 1 1 ; ; Let's check them for compliance with the sufficient condition 4 for the presence of extrema at these points. A z B z 43 C z 6 At point P 1 01 ; : A B 1 A0, B1, C 0, 10 B C 1 0 Since 0, there is no extremum at this point. At point P 1 1 ; 4 A10, B14, C 38, A B B C A 0 Since 0, then at this point the function has a maximum z 1; Answer: the function has a maximum z 1 1 ; Let's make an illustration in Mathcad 14: - the found maximum is marked with a red dot; the straight line of black straight lines occurs at point P 1 01 ;. 0 z 0 is highlighted in burgundy; intersection 4

5 3 3) Investigate the function for the extremum z z z Find stationary points: z 0 z the circle at the intersection with the hyperbola will give four points: Four stationary points are found P1; 1, P 1;, P 3 1 ;, P 4 ; fulfillment of sufficient conditions for the presence of an extremum at these points. A z B z C z 61 6 At point P ; 1 1: A 1 0 B 6 C 1 A B B C Let's check 5

6 A maximum, because 0 - at point P; At point P; 1: A 60 B 1 C 6 A B B C at point P; 1 there is no extremum, because 0. At point P 3 1 ; : A 6 0 B 1 C 6 A B B C at point P 3 1 ; there is no extremum, because 0. At point P 4 1 ; : A 1 0 B 6 C 1 A B B C 6 1 P 4 - at a point; A 0 1 minimum, because 0 z Value of the function at this point;. The value of the function at this point z; Answer: the function has a minimum z 1; 8 and maximum; z Literature: 1) Written by D.T. "Lecture notes on higher mathematics", 005, page (extremum of a function of two variables). Solving the problem using the Hessian matrix. 4) Find the extrema of the function of two variables, 3 z 3 61 Let us determine the stationary points from the condition z 0 z 0 (a necessary condition for the existence of an extremum) 6

7 z z Points P1 1 ; 1 and P; stationary points; Let's check them for compliance with the sufficient condition for the presence of an extremum. To do this, we construct the Hessian matrix from the second derivatives of the function: z z H z z z 6 z H 6 1 z 1 Continuation of the solution through the analysis of the angular minors of the Hessian matrix Let us consider the behavior of the Hessian matrix at the found stationary points. 6 6 P11; 1: HP1 6 1 ; angular minors: M1 6 0, M Since M 0, there is no extremum at point P 1. P; angular minors: M1 6 0, M 4; : HP M1 0 Since M 0, then at point P the function has a local minimum z;

8 Theorem (sufficient conditions for an extremum). If at some point the necessary conditions for an extremum are satisfied and all partial derivatives of the th order are continuous, then the existence of an extremum at this point is determined by the values ​​of the angular minors of the matrix of second derivatives (Hessian matrix): M1 0, M 0 - local minimum; M1 0, M 0 - local maximum; M 0 - there is no extremum. If M1 0 or M 0 there may or may not be an extremum at the point under study, additional research is necessary. u u, z is considered When studying the local extremum of a function of three variables, the matrix u u u z u u u z uz uz u zz and its angular minors are studied. Literature: 1) Malugin V.A. "Linear algebra. Problems and exercises", 006, p. 149 (local extremum of the function);) Written by D.T. "Lecture notes on higher mathematics", 005, page (extremum of a function of two variables); 3) Pyatkova V.B., Ruzakov V.Ya., Turova O.E. "Mathematics, 3rd semester", training manual of Ural State State University (mining institute, Yekaterinburg), 005, p. 3 (scheme for studying the function of two variables on extrema). Continuation of the solution through the analysis of the eigenvalues ​​of the Hessian matrix Let's find the eigenvalues ​​of the Hessian matrix at each stationary point. 6 6 P11; 1: HP From equation 0 we find 1, Since the eigenvalues ​​of the 6 1 Hessian matrix are of different signs, then there is no extremum at point P 1. P From Eq.; : HP we find 1, Since all the eigenvalues ​​of the Hessian matrix are positive, then at point P there is a local minimum z; 4 3. Find the eigenvalues ​​of the Hessian matrix at each of the stationary points If all eigenvalues ​​* are positive: i 0, i 1,..., n, then there is a local minimum at the point; negative: i 0, i 1,..., n, then at the point non-negative: i 0, i 1,..., n, then at the point non-positive: i 0, i 1,..., n, then at point * local maximum; * functions. * there may be a local minimum; * there may be a local maximum; * different signs, then there is no extremum at the point; zero: i 0, i 1,..., n, then additional research is required. 8

9 Literature: 1) Bortakovsky A.S., Panteleev A.V. "Linear algebra in examples and problems", 005, p. 530, p. 531 (example 9.8). 5) Find the extremum points of the function z Let us determine the stationary points of the function of two variables z, from the condition z 0 z 0 (a necessary condition for the existence of the extremum) z z Three stationary points are found P 1 00 ;, P 0 ; 1 40, P 3 ; compliance with the sufficient condition for the presence of an extremum; let's check them on When studying the local extremum of a function of two variables z z, the quadratic form of the function with respect to differentials d, d is the Hessian matrix z z z z and is considered at each stationary point P i. If this quadratic form turns out to be definite, then the function z z, extremum: a) minimum, if the quadratic form is positive definite; b) maximum if the quadratic form is negative definite. If the quadratic form turns out to be indefinite, then a matrix is ​​compiled at the point P i has P i there is no extremum. In cases of non-negative definiteness or non-positive definiteness of the quadratic form, additional research is required - there may be an extremum. 9

10 Hessian matrix: H z z z z F F F F F F F (Note that 80 1). So, H Let us establish the definite sign of the quadratic form using the Sylvester criterion. In order for a quadratic form in n variables to be positive definite, it is necessary and sufficient that all angular minors of its matrix A be positive. In order for a quadratic form of n variables to be negative definite, it is necessary and sufficient that the signs of the angular minors of the matrix A of the quadratic form alternate, starting with the minus sign. For uncertainty (alternating sign) of a quadratic form, it is sufficient that at least one major minor of an even order is negative, or two major minors of an odd order have different signs (a sufficient sign of the uncertainty of a quadratic form). At a stationary point P 1 00 ; : H P angular minors: M1 410, 41 1 M 0110, the quadratic form is of indefinite sign, therefore, there is no extremum at point P 1. At a stationary point P; H P: angular minors: M1 410, 41 1 M 0110, the quadratic form is of indefinite sign, therefore, there is no extremum at point P. 10

11 At a stationary point P; H P: angular minors: M1 410, M 0 0, the quadratic form is positive definite, therefore, at point P 3 the function has a local minimum z; ,Answer: the function has a local minimum; z Let's find the minimum of a function in Mathematica 7 (for this you will have to indicate the area of ​​its dislocation): References: 1) Aksyonov A.P. "Mathematics. Mathematical analysis", part, 005, p. 193 (examples 13, 14);) Bortakovsky A.S., Panteleev A.V. "Linear algebra in examples and problems", 005, p. 530, p. 531 (example 9.8); 3) Bortakovsky A.S., Panteleev A.V. "Workshop on linear algebra and analytical geometry", 007, page) Malugin V.A. "Linear algebra. Course of lectures", 006, pp. 157, 164; 5) Baranova E.S., Vasilyeva N.V., Fedotov V.P. "A practical guide to higher mathematics. Typical calculations", 008, p. 301 (example 10.35). 6) Investigate the function, F z 4 3 z z for the presence of unconditional extrema using first and second order derivatives, analyzing stationary points. Let us determine the stationary points of the function of three variables F, z from the condition F 0 F 0 F 0 z (a necessary condition for the existence of an extremum) F 8z F 6 Fz z 8z ​​z 0 z 0 Point P 000 ; ; - stationary point; Let's check it for compliance with the sufficient condition for the presence of an extremum. eleven

12 When studying the local extremum of a function of three variables u u, z, the quadratic form of the function with respect to the differentials d, d, dz - the Hessian matrix u u u z u u u z uz uz u zz and is considered at each stationary point. If this quadratic form turns out to be defined, then the function z z, P i. extremum: a) minimum, if the quadratic form is positive definite; b) maximum if the quadratic form is negative definite. If the quadratic form turns out to be indefinite, then a matrix is ​​compiled at the point P i has P i there is no extremum. In cases of non-negative definiteness or non-positive definiteness of the quadratic form, additional research is required - there may be an extremum. Let's write the Hessian matrix: F F F z F F F H z F F F z z z F F F F 8z 8 1 z F F F z F z F z z F F (Note that, So, 8 1 H F F z z 1, F F z z 0). Analysis of the sign of a quadratic form. Sylvester criterion. Let us establish the definite sign of the quadratic form using the Sylvester criterion. 1

13 In order for a quadratic form in n variables to be positive definite, it is necessary and sufficient that all angular minors of its matrix A be positive. In order for a quadratic form of n variables to be negative definite, it is necessary and sufficient that the signs of the angular minors of the matrix A of the quadratic form alternate, starting with the minus sign. For uncertainty (alternating sign) of a quadratic form, it is sufficient that at least one major minor of an even order is negative, or two major minors of an odd order have different signs (a sufficient sign of the uncertainty of a quadratic form). In the event that one or more angular minors are equal to zero, but one of the conditions of sign definiteness could be satisfied, the quadratic form is non-negative definite or non-positive definite (this condition was written down to complete the picture; requires proof). At the found stationary point: 8 1 P000 ; ; : HP 6 0 ; 1 0 angular minors: M1 80, 8 M , M M1 0 Since M 0 M 3 0, then at point P the function has a local minimum F; ; Eigenvalues ​​of the Hessian matrix. Let us establish the definite sign of the quadratic form by analyzing the eigenvalues ​​of the Hessian matrix. * Find the eigenvalues ​​of the Hessian matrix at each of the stationary points of the function. If all eigenvalues ​​are positive: i 0, i 1,..., n, then the quadratic form is positive definite; negative: i 0, i 1,..., n, then the quadratic form is negative definite; non-negative: i 0, i 1,..., n, then the quadratic form is non-negative definite; non-positive: i 0, i 1,..., n, then the quadratic form is non-positive definite; different signs, then the quadratic form is indefinite; zero: i 0, i 1,..., n, then the quadratic form is non-negative definite or non-positive definite [, p. 530]. Let's find the eigenvalues ​​of the Hessian matrix at the found stationary point. 8 1 P000 ; ; : HP 6 0 ;

14 8 1 From the equation or we find, 4, 855 9, as the roots of a polynomial in Mathcad: or graphically in Mathcad: Since all the eigenvalues ​​of the Hessian matrix are positive, then at point P there is a minimum. Second differential function. Let us establish the definite sign of the quadratic form directly. The quadratic form with respect to differentials is the second differential of the function. Let us transform the expression of the second differential of the function so as to explicitly establish the fact that its sign is definite. This approach to studying the sign-determinacy of a quadratic form can be used as a method of additional research when the Sylvester criterion or analysis of the eigenvalues ​​of a matrix of a quadratic form does not produce results. Sufficient conditions for the extremum of a function of n variables If M,..., n is a stationary point of a twice differentiable function f,..., n 1 and if in some neighborhood of this point the second differential n f d f M 0 M 0did j i, j1 i j retains sign for any values d i and d j not equal to zero at the same time, then the function at point M 0 has an extremum: minimum at maximum at d f M0 0 ; d f M0 0 . 14

15 The second differential of the function Ф is equal to Ф Ф Ф d Ф, d d dd or, in the case of a function of three variables Ф z, Ф Ф Ф Ф Ф Ф Ф d Ф, z d d dz dd ddz ddz z z z Let's calculate the second order partial derivatives: F F F F F F 8z 8 1 z z F F F F z z F F z z z and write the second differential of the function at point P 000 ; ; : d F d d dz F z F F F F F F d d dz d d d dz d dz z z z 8d 6d dz d d 0d dz 1d dz 8 d 6 d dz 4d d d dz Select the complete squares; for brevity of notation, we redesignate d as, etc.: 8 6 z 4 z z z z z z i.e. at point P000 ; ; : d F d dz d d d (with d, d, dz not equal to zero at the same time) - therefore, point P000; ; is the minimum point. 15

16 Literature: 1) Aksyonov A.P. "Mathematics. Mathematical analysis", part, 005, p. 193 (examples 13, 14);) Bortakovsky A.S., Panteleev A.V. "Linear algebra in examples and problems", 005, p. 530, p. 531 (example 9.8); 3) Bortakovsky A.S., Panteleev A.V. "Workshop on linear algebra and analytical geometry", 007, page) Malugin V.A. "Linear algebra. Course of lectures", 006, pp. 157, 164; 5) Baranova E.S., Vasilyeva N.V., Fedotov V.P. "A practical guide to higher mathematics. Typical calculations", 008, p. 301 (example 10.35). Let's check the presence of this minimum in Mathematica 7: 16

Functions of several variables (FNP). Local extremum. 1) Investigate the function z z e for a local extremum; a) x variables b) 3 variables 3 3 3 u u z z 17 48 z. a) z e e e e 1 1 z e e Find

LECTURES Lecture 1 Section I. OPTIMIZATION THEORY 1. GENERAL FORMULATION OF THE OPTIMIZATION PROBLEM AND BASIC PROVISIONS The formulation of the problem of finding the minimum of functions contains: objective function f (x), where x = (x1,..., x

7 QUADRATIC FORMS 7 DEFINITION OF QUADRATIC FORM The quadratic form of variables is an expression of the form q a, 7 in which the coefficients a, not all equal to zero, satisfy the conditions of symmetry

Nonlinear optimization problem. Koltsov S.N. 2014 www.linis.ru Unconditional optimization problem The optimization problem is formulated as follows: set X (admissible set of the problem) and function are given

Mathematical analysis Section: Function of several variables Topic: Taylor's formula for the FNP. Extrema of the FNP Lecturer Rozhkova S.V. 1 18. Taylor formula for FNP If y = times differentiable in the neighborhood

Lecture 3 Extremum of a function of several variables Let a function of several variables u = f (x, x) be defined in the domain D, and the point x (x, x) = belongs to this domain The function u = f (x, x) has

Lecture 9 EXTREMUM OF A FUNCTION OF MANY VARIABLES The concept of an extremum of a function of many variables Some information about quadratic forms 3 Sufficient conditions for an extremum The concept of an extremum of a function of many variables

Chapter 1 Finite-dimensional problems 1 Finite-dimensional smooth problems without restrictions This section gives necessary and sufficient conditions for the extremum of functions of one and several variables. 1.1 Problem statement

Mathematical analysis 2.5 Lecture: Extrema of a function of several variables Associate Professor of the Department of VMMF Vladimir Feliksovich Zalmezh Consider the function w = f (x), defined in the domain D R n. The point x 0 D is called

Practical lesson 5 Extremum of a function of many variables 5 Definition and necessary conditions for an extremum 5 Some information about quadratic forms 53 Sufficient conditions for an extremum 5 Definition and necessary

LECTURE Extremum of a function of several variables Extremum of a function of several variables Necessary and sufficient conditions for the existence of an extremum Point M, 0) is called the point of minimum maximum) of the function

RYAZAN STATE RADIO TECHNICAL UNIVERSITY SV Bogatova, KV Bukhensky, IP Karasev, GS Lukyanova RESEARCH OF FUNCTIONS AND CONSTRUCTION OF GRAPHS IN THE MATHCAD ENVIRONMENT Workshop Ryazan Preface General

6 () We get that HP =. Therefore, using the theorem, it is impossible to answer the question about the extremum. In this case, the stationary point is P (); is a point of local mi- Δz > P O & P: z = z =. δ

1) Find the largest and smallest values ​​of the function 1 1 on the segment 6. To find the largest and smallest values ​​of the function on the segment, you need to: a) find stationary points located on this segment,

Moscow State Technical University named after N.E. Bauman Faculty of Fundamental Sciences Department of Mathematical Modeling A.N. Kasikov,

LECTURE 16 PROBLEM ON THE STABILITY OF THE EQUILIBRIUM POSITION IN A CONSERVATIVE SYSTEM 1. Lagrange’s theorem on the stability of the equilibrium position of a conservative system Let there be n degrees of freedom. q 1, q 2,

LECTURE N. Scalar field. Directional derivative. Gradient. Tangent plane and normal to the surface. Extrema of a function of several variables. Conditional extremum. Scalar field. Derivative with respect to

10 Study of functions and construction of graphs 10 STUDY OF FUNCTIONS AND CONSTRUCTION OF GRAPHS 1 Increasing and decreasing function 1 x (1 1 DEFINITION The function y = f (x) is called increasing (non-decreasing)

) Determine the eigenvalues ​​and eigenvectors of a 3rd order matrix 6 8 2 5 2 8 3 4 A nonzero vector p is called an eigenvector of a square matrix A if a linear transformation with the matrix

Appendix 3 to the working curriculum of the discipline METHODOLOGICAL INSTRUCTIONS FOR STUDENTS IN THE DISCIPLINE "MATHEMATICS". Practical lesson 1 Topic: “Installation. Fire safety and technology briefings

1) Bring the equation of the second-order curve x 4x y 0 to canonical form and find its intersection points with the straight line x y 0. Provide a graphical illustration of the resulting solution. x 4x y 0 x x 1 y 0 x 1 y 4

7. Extrema of functions of several variables 7.. Local extrema Let the function f(x,..., x n) be defined on some open set D R n. Point M D is called the point of local maximum (local

Sufficient conditions for the existence of a solution to the problem of a conditional extremum by the Lagrange method VV Kolybasova, NCh Krutitskaya VV Kolybasova, N Ch Krutitskaya Sufficient conditions for the existence of a solution to the problem of a conditional

1 SA Lavrenchenko Lecture 9 Extrema 1 Definitions and examples Definition 11 They say that a function has (or reaches) an absolute maximum at a point if for all from the domain of definition the Value is called

Mathematics (BkPl-100, BkK-100) M.P. Kharlamov 2009/2010 academic year, 2nd semester Lecture 5. Study of functions using derivatives 1 1. The concept of higher order derivatives Def. Let the function f(x) be given

17. Conditional extremum 17.1. Let us turn to the consideration of finding a conditional (they also say relative) extremum. The task of finding a conditional extremum is to search for local maxima and minima

MINISTRY OF EDUCATION AND SCIENCE OF THE RUSSIAN FEDERATION Federal State Autonomous Educational Institution of Higher Education "SOUTHERN FEDERAL UNIVERSITY" AV Abanin, DA Polyakova LOCAL

Department of Mathematics and Computer Science Mathematical analysis Educational and methodological complex for higher education students studying using distance technologies Module 4 Derivative applications Compiled by: Associate Professor

Materials for the orientation lecture Question 10. Quadratic forms. Law of inertia. Conditions for sign definiteness of quadratic forms. 1 Reducing a quadratic form to canonical form using the Lagrange method. Notation.

Chapter Extrema of a function of two variables Extrema of a function of two variables When solving many economic problems, it is necessary to calculate the largest and smallest values. As an example, consider the problem

6. Implicit functions 6.1 Definitions, preliminary information The dependence of one variable on another (or others) cannot necessarily be expressed using the so-called explicit representation, when

1 SA Lavrenchenko Lecture 10 Study of a function using derivatives 1 Study of a function using the first derivative By interval we will mean either a finite interval or one of the following

Department of Mathematics and Computer Science Elements of Higher Mathematics Educational and methodological complex for secondary vocational education students studying using distance technologies Module Differential calculus Compiled by:

Method of Lagrange multipliers Consider an extremal problem with constraints in the form of equalities: find a subject to the condition that) =) on the set of admissible values ​​described by the system of equations where R: R R: R

Lecture 11. CONDITIONAL EXTREMUM 1. The concept of conditional extremum.. Methods for finding conditional extremum.. The largest and smallest values ​​of a function of two variables in a closed area. 1. The concept of conditional

APPLYING THE DERIVATIVE FUNCTION Tangent equation Consider the following problem: we need to create an equation for the tangent l drawn to the graph of the function at the point. According to the geometric meaning of the derivative

CHAPTER 7 Differential calculus of functions of several variables 1 Partial derivatives and total differential of functions of several variables Def711 Let M (, y), : O(M,) Consider the function 1 = 1 ()=

Federal Agency for Education MOSCOW STATE UNIVERSITY OF GEODESY AND CARTOGRAPHY (MIIGAiK) O. V. Isakova L. A. Saykova TUTORIAL FOR STUDENTS FOR INDEPENDENT STUDY OF THE SECTION

APPLIED OPTIMIZATION METHODS V.V. Kornev V.V. Kurdyumov V.S. Rykhlov 2 Contents Introduction 5 1 Nonlinear optimization 9 1.1 Statement of the optimization problem. Basic concepts and definitions................

Ministry of Education and Science of the Russian Federation Federal State Budgetary Educational Institution of Higher Professional Education "Siberian State Industrial University"

~ 1 ~ FUNCTION OF MANY VARIABLES 3 Function of two variables, domain of definition, methods of definition and geometric meaning. Definition: z f, is called a function of two variables, if each pair of values,

Lecture Study of a function and construction of its graph Abstract: The function is studied for monotonicity, extremum, convexity-concavity, the existence of asymptotes An example of the study of a function is given, construction

1) Find all additional minors of the determinant 1 9 11 0 0 0 56 18 2. Let a square matrix of order n be given. An additional minor a of a matrix is ​​the determinant per unit of the smaller element M ij

Moscow State Technical University named after N.E. Bauman Faculty of Fundamental Sciences Department of Mathematical Modeling A.N. Kasikov,

1. Increasing and decreasing function. In order for a function f differentiable on the interval (ab,) to be increasing on this interval, it is necessary and sufficient that the condition be satisfied. Similarly, the condition

Ministry of Education and Science of the Russian Federation Federal State autonomous educational institution of higher professional education National

Introduction Home tests in mathematical analysis are one of the main forms of ongoing monitoring of students’ independent work. The approximate time required to complete the DCR is

Topic 8 DIFFERENTIAL CALCULUS OF FUNCTIONS OF SEVERAL VARIABLES Lecture 8.1. Functions of several variables. Partial derivatives Plan 1. The concept of a function of two and several variables. Limit and continuity

Federal Agency of Railway Transport Ural State Transport University E E Popovsky P P Skachkov FUNCTIONS OF SEVERAL VARIABLES Typical calculation Ekaterinburg 1 Federal

APPLICATION OF THE DERIVATIVE TO THE STUDY OF FUNCTIONS Study of the behavior of a function using derivatives Monotonicity intervals. Extremes Definition. The intervals over which the function f (x) increases (decreases)

I) Given linear subspaces U and W, generated by the vector systems: a ; ; 3; a a b b 3 ; ; ; ; ; ; ; ; ; 3; 3; ; Find the bases of the subspaces U a) Basis of the subspace U W. W and U W. The set of all

Modulus and derivative V.V. Silvestrov When solving some problems, it is necessary to find the derivative of a function containing one or more modules. Such tasks are also possible on the unified state exam.

MINISTRY OF EDUCATION AND SCIENCE MOSCOW STATE TECHNICAL UNIVERSITY "MAMI" Department of "Higher Mathematics" MA Bodunov, SI Borodina, VV Pokazeev, BE Teush OI Tkachenko, DIFFERENTIAL CALCULUS

Studying functions and constructing graphs.) Investigate the function f using differential calculus methods and construct its graph. Function domain: Dy R\. General function: y y y Critical

CONTENTS Section I. CONDITIONS FOR EXTREMUM OF FUNCTIONS... 6 Chapter. General formulation of the optimization problem and main provisions... 6 Chapter. Necessary and sufficient conditions for an unconditional extremum... Ch. Required

Differential calculus Basic concepts and formulas Definition 1 The derivative of a function at a point is the limit of the ratio of the increment of the function to the increment of the argument, provided that the increment of the argument

BELARUSIAN STATE UNIVERSITY FACULTY OF MECHANICS AND MATHEMATICS Department of Nonlinear Analysis and Analytical Economics V. I. BAKHTIN, I. A. IVANISHKO, A. V. LEBEDEV, O. I. PINDRIK METHOD OF MULTIPLIERS

Moscow State Technical University named after NE Bauman Dubogray IV Skudneva OV Levina A I Functions of several variables guidelines for preparing for certification Moscow Publishing House

Chapter CALCULUS OF VARIATIONS Lecture 9 Introduction In this chapter we will consider the problems of finding extrema (maxima or minima) of functionals. We immediately note that such problems are among

2 Finite-dimensional smooth problems with equalities This section gives necessary and sufficient conditions for an extremum in a smooth finite-dimensional problem with equality-type constraints. 2.1 Statement of the problem Let

St. Petersburg State University Department of Mathematical Analysis METHODOLOGICAL INSTRUCTIONS for conducting practical classes in mathematical analysis Part Problem on the local extremum of a function

Questions to prepare for the exam Topic. Linear algebra 1. What is a determinant? Under what transformations does the value of the determinant not change? 2. In what cases is the determinant equal to zero? What follows

PRACTICAL LESSON 11 Extremum of a function of two variables The maximum or minimum of a function is called its extremum The point M 0 at which the function has an extremum is called the extremum point If the differentiable

Lecture 1 6 Sufficient conditions for an extremum in a problem with fixed ends Let's return to the problem with fixed ends: find the minimum of the functional (,) V = F x x provided that = A, = B Necessary

FUNCTIONS OF SEVERAL VARIABLES The functions of one independent variable do not cover all the dependencies that exist in nature. Therefore, it is natural to expand the well-known concept of functional dependence and introduce

Rules for entering functions:

A twice continuously differentiable function f(x) is convex (concave) if and only if Hessian matrix the function f(x) with respect to x is positive (negative) semidefinite for all x (see points of local extrema of a function of several variables).

Function critical points:

• if the Hessian is positive definite, then x 0 is the local minimum point of the function f(x),
• if the Hessian is negative definite, then x 0 is the local maximum point of the function f(x),
• if the Hessian is not sign-definite (takes both positive and negative values) and is non-degenerate (det G(f) ≠ 0), then x 0 is the saddle point of the function f(x).

Criteria for the definiteness of a matrix (Sylvester's theorem)

• all diagonal elements of the matrix must be positive;
• all leading main qualifiers must be positive.

Positive semi-definiteness:

• all diagonal elements are non-negative;
• all main determinants are non-negative.

A square symmetric matrix of order n, the elements of which are the partial derivatives of the second-order objective function, called the Hessian matrix and is designated:

In order for a symmetric matrix to be positive definite, it is necessary and sufficient that all its diagonal minors are positive, i.e.


for the matrix A = (a ij) are positive.

Negative certainty.
In order for a symmetric matrix to be negative definite, it is necessary and sufficient that the following inequalities take place:
(-1) k D k > 0, k=1,.., n.
In other words, in order for the quadratic form to be negative definite, it is necessary and sufficient that the signs of the angular minors of a matrix of quadratic form alternate, starting with the minus sign. For example, for two variables, D 1< 0, D 2 > 0.

If the Hessian is semidefinite, then this may also be an inflection point. Additional research is needed, which can be carried out using one of the following options:

1. Decreasing order. A change of variables is made. For example, for a function of two variables it is y=x, as a result we get a function of one variable x. Next, we examine the behavior of the function on the lines y=x and y=-x. If in the first case the function at the point under study will have a minimum, and in the other case a maximum (or vice versa), then the point under study is a saddle point.
2. Finding the eigenvalues ​​of the Hessian. If all values ​​are positive, the function at the point under study has a minimum, if all values ​​are negative, there is a maximum.
3. Study of the function f(x) in the neighborhood of the point ε. Variables x are replaced by x 0 +ε. Next, it is necessary to prove that the function f(x 0 +ε) of one variable ε is either greater than zero (then x 0 is the minimum point) or less than zero (then x 0 is the maximum point).

Note. To find inverse Hessian it is enough to find the inverse matrix.

Example No. 1. Which of the following functions are convex or concave: f(x) = 8x 1 2 +4x 1 x 2 +5x 2 2 .
Solution. 1. Let's find partial derivatives.


2. Let's solve the system of equations.
-4x 1 +4x 2 +2 = 0
4x 1 -6x 2 +6 = 0
We get:
a) From the first equation we express x 1 and substitute it into the second equation:
x 2 = x 2 + 1/2
-2x 2 +8 = 0
Where x 2 = 4
We substitute these values ​​x 2 into the expression for x 1. We get: x 1 = 9 / 2
The number of critical points is 1.
M 1 (9 / 2 ;4)
3. Let's find the second order partial derivatives.



4. Let us calculate the value of these second-order partial derivatives at the critical points M(x 0 ;y 0).
We calculate the values ​​for point M 1 (9 / 2 ;4)



We build the Hessian matrix:

D 1 = a 11< 0, D 2 = 8 > 0
Since the diagonal minors have different signs, nothing can be said about the convexity or concavity of the function.

Parameters with small values ​​of second derivatives are reset to zero. Sensitivity analysis is computationally complex and requires a lot of additional memory.  

Relations (1.4) and (1.6) determine the signs of the main minors of the Hessian matrix for our function and thus are a sufficient condition for the non-positive definiteness of the corresponding quadratic form (1.3). Therefore, for concavity of linearly homogeneous functions with two resources, condition (1.4) is sufficient.  

The matrix I, as already mentioned, is called the Hessian matrix (or Hessian).  

In a more consistent approach, information about the second-order derivatives of the residual function can be used to improve the learning process. The corresponding optimization methods are called quadratic. All this information is collected in the Hessian matrix H, which has dimensions Nw x Nw, where Nw is the number of weights. This matrix contains information about how the gradient changes with small displacements in different directions in weight space. Direct matrix calculation requires a lot of time, so methods have been developed to avoid matrix calculation and storage (conjugate gradient descent, scaled conjugate gradient method (see), RBa kProp (see), quasi-Newtonian method, Levenberg-Marquard method ).  

The first equation (4.17) shows how output will change when the price of the firm's products increases. Since the Hess matrix H is negative definite, the matrix H"1 is also negative definite, therefore  

Note that from the fact of the existence of the function Q due to the symmetry of the matrix of second derivatives (Hessian matrix) for a twice differentiable function of several variables, equalities follow that relate the sensitivity of estimates to changes in resource reserves.  

In addition, the Hessian matrix of the second derivatives of this function with respect to C must be negative definite at C = 0.  

Let us consider the change in the Hessian matrix of the function /(C) during its monotonic transformation. Let us first write down the components of the gradient at the point  

For the function FQ() to be convex, it is sufficient that the matrix T = Tij be negative definite. The first terms in (9.108) differ from the elements 7 j of the Hessian matrix of the original problem by a non-negative factor, since the function FQ is monotonically increasing. If the second terms in these expressions are equal to zero, then the concave reachability function of the original problem will correspond to concavity and FQ().  

Thus, the Hessian matrix for the reachability function of the transformed problem is the sum  

The first of them represents n equations for the components of the vector A, and the second is the condition of negative definiteness of the quadratic form, which is checked using the Sylvester criterion in relation to the Hessian matrix of the function R.  

Here and below, R f0 and R i denote the partial derivatives of R with respect to the corresponding variables. The conditions of negative definiteness must be satisfied by the Hessian matrix of the function R with elements (see (9.125))  

The second part forms the theoretical core of the book. It is entirely devoted to a rigorous presentation of the theory of differentials and the foundations of analysis, formulated in the language of differentials. The concepts of first and second differentials are introduced, and an identification rule for the Jacobi and Hessian matrices is given. The chapter ends with a paragraph devoted to the theory of optimization in the presence of constraints, presented in terms of differentials.  

The fourth part, on inequalities, arose from our belief that econometricians should be comfortable with inequalities such as the Cauchy-Bunyakovsky (Schwartz) inequality, the Minkowski inequality and their generalizations, as well as powerful results such as Poincaré's separability theorem. To some extent, the chapter is also a story of our disappointment. When we started writing this book, we had an ambitious idea - to derive all inequalities using matrix differential calculus. After all, each inequality can be represented as a solution to some optimization problem. However, this idea turned out to be an illusion, since the Hessian matrix in most cases turns out to be singular at the extremum point.  

Notation. In this book we use mostly standard notation, except that vectors are denoted in plain (not bold) italics. Special symbols are used to denote the derivative (matrix) D and the Hessian matrix H. The differentiation operator is denoted as d. A complete list of all symbols used in the text is contained in the Notation Index at the end of the book.  

This chapter covers the concepts of second derivatives, twice differentiability, and second differential. Particular attention is paid to the connection between twice differentiability and second-order approximation. We define the Hessian matrix (for vector functions) and find conditions for its (column) symmetry. We also obtain the chain rule for Hessian matrices and its analogue for second differentials. Taylor's theorem is proved for real functions. Finally, higher order differentials are discussed very briefly and it is shown how the analysis of vector functions can be extended to matrix functions.  

Previously, we defined a matrix that contains all first-order partial derivatives. This was the Jacobian matrix. Now let's define a matrix (called the Hessian matrix) that contains all the second-order partial derivatives. Let us define this matrix first for real and then for vector functions.  

Let / S -> Rm, S With Rn is

Matrix G(X) dimension( n x n) is considered positive definite if all its eigenvalues ​​m 1 , m 2 ,…, m n are positive, i.e. m j> 0 for all j = 1, 2,…, n.

Matrix G(X) is considered negative definite if the eigenvalues ​​are negative, i.e. m j< 0 для всех j = 1, 2,…, n.

If among the eigenvalues G If both positive and negative ones occur, then the matrix is ​​sign-alternating, and the function under study is non-convex.

To determine the eigenvalues, it is necessary to solve the characteristic equation:

Where I– square identity matrix; det is the sign of the determinant.

The matrix differs from the Hessian matrix in that terms of the form are located along the diagonal.

So for a two-dimensional function f(x 1 , x 2) the characteristic equation will have the form:

(4.10)

The eigenvalues ​​m 1 and m 2 are the roots of the ordinary quadratic equation m 2 + b m +c= 0, are formed after expansion of the determinant.

For example, let's take functions of two variables:

f(x)= 2 – 2x 1 –2x 2 +x 1 2 +x 2 2 – x 1 x 2

Extreme point coordinates x* determined by solving the system of equations

And equal x 1 * = 2, x 2 * = 2

Hessian . After solving the characteristic equation , i.e. quadratic equation (2 – m) 2 – 1 = 0, the eigenvalues ​​m 1 = 3, m 2 = 1 are obtained, i.e. matrix G is positive definite. Therefore, the function f(x) is convex and at the extreme point X* = (2,2) takes the minimum value f(x*) = –2.

Both methods of checking sufficient and necessary conditions for a second-order extremum are given in Table 4.2.

Example 4.4. Find the extremum of a function on a set E 2 .

Solution. 1. Let us write down the necessary conditions for a first-order extremum:

;

x* = (0,0).

2. Let’s check whether the sufficient conditions for the extremum are met.

First way: The Hessian matrix has the form .Since M 1 = 2 > 0, , then at the point x* local minimum (line 1 in Table 4.2).

Second way: Let's find the eigenvalues ​​of the Hessian matrix using (4.10):

Hence . Since all eigenvalues ​​are positive, then at the point x* local minimum (line 1 in Table 4.2). From Example 3.3 it follows that the function is strictly convex on the set E 2. Therefore, the local minimum point is also a global minimum point (according to paragraph 3, statement 3.1).

3. Calculate the value of the function at the global minimum point: f(x*) = 0.

Example 4.5. Find the extremum of the function on the set E 2.

Solution. 1. Let us write down the necessary conditions of the first order:

; .

As a result of solving the system, we obtain a stationary point x* = (0,0).

2. Let us check the fulfillment of sufficient conditions for the extremum and necessary conditions of the second order.

First way: The Hessian matrix has the form . Since M 1 = 2 > 0, , then the sufficient conditions for the extremum are not met (lines 1 and 2 in Table 4.2). Let us check the fulfillment of the necessary second-order conditions.

Principal minors of the first order ( m= 1) are obtained from M 2 as a result of deleting n–m=2 – 1 = 1 rows and columns, with the same numbers: – 2, 2. Second order major minor ( m = 2) obtained from M 2 as a result of deleting n – m= 0 rows and columns, i.e. coincides with M 2: -4. It follows that the necessary conditions for a second-order extremum are not satisfied (lines 3 and 4 in Table 4.2). Since the Hessian matrix is ​​not zero, we can conclude that at the point X* no extremum (line 6 in Table 2.1).

Table 4.2

A criterion for checking sufficient and necessary conditions of the second order in the problem of searching for an unconditional extremum

Describing the behavior of a function in second order.

For function texvc , twice differentiable at the point Unable to parse expression (Executable file texvc not found; See math/README for setup help.): x\in \R^n

Where Unable to parse expression (Executable file texvc not found; See math/README - help with setup.): a_(ij)=\partial^2 f/\partial x_i \partial x_j(or Unable to parse expression (Executable file texvc not found; See math/README - help with setup.): a_(ij)=\partial^2 f/\partial z_i \partial \overline(z)_j) and function Unable to parse expression (Executable file texvc not found; See math/README for setup help.): f set to Unable to parse expression (Executable file texvc not found; See math/README for setup help.): n-dimensional real space Unable to parse expression (Executable file texvc not found; See math/README for setup help.): \mathbb(R)^n(or complex space Unable to parse expression (Executable file texvc not found; See math/README for setup help.): \mathbb(C)^n) with coordinates Unable to parse expression (Executable file texvc not found; See math/README - help with setup.): x_1,\ldots,x_n(or Unable to parse expression (Executable file texvc not found; See math/README - help with setup.): z_1,\ldots,z_n). In both cases, the Hessian is a quadratic form defined on the tangent space, which does not change under linear transformations of the variables. Hessian also often called the determinant of a matrix Unable to parse expression (Executable file texvc not found; See math/README - help with setup.): (a_(ij)), see below.

Hessian matrix

The matrix of this quadratic form is formed by the second partial derivatives of the function. If all derivatives exist, then

Hessian matrices are used in optimization problems by Newton's method. Complete calculation of the Hessian matrix can be difficult, so quasi-Newton algorithms have been developed based on approximate expressions for the Hessian matrix. The most famous of them is the Broyden-Fletcher-Goldfarb-Shanno algorithm.

Symmetry of the Hessian matrix

Mixed derivatives functions f- these are elements of the Hessian matrix that are not on the main diagonal. If they are continuous, then the order of differentiation is not important:

This can also be written as

In this case, the Hessian matrix is ​​symmetric.

Critical points of a function

Story

see also

• Sylvester criterion - criterion for positive/negative definiteness of a square matrix

Write a review about the article "Hessian functions"

Notes

Links

• Kamynin L.I. Mathematical analysis. T. 1, 2. - 2001.
• Kudryavtsev L.D. “Short course in mathematical analysis. T.2. Differential and integral calculus of functions of several variables. Harmonic analysis", FIZMATLIT, 2002, - 424 p. - ISBN 5-9221-0185-4. Or any other publication.
• Golubitsky M., Guillemin V. Stable mappings and their features, - M.: Mir, 1977.

An excerpt characterizing the Hessian functions