Brainstorming method. Brainstorming method: description, technology and reviews

Page 1


Brainstorming(MA) is based on the hypothesis that among large number There are at least a few good ideas that are useful for solving the problem that need to be identified. The essence of the method is that a group of experts generates alternative solutions, possible scenarios about the problem posed, offering everything that comes to mind. All ideas are written down on cards, similar solutions are grouped and these solutions are analyzed by another group of experts who make a final decision. In the first group, criticism of ideas is not allowed; in the second, discussion of ideas is possible. Methods of this type are also known as collective idea generation, idea conferences, and the method of exchanging opinions.  

Brainstorming is a free, unstructured process of generating any ideas on a selected topic that are spontaneously expressed by meeting participants. As a rule, not only specialists in a given problem are accepted as experts, but also people who are specialists in other fields of knowledge. The discussion is based on a pre-developed scenario.  

Brainstorming - a team working on a proposed title (usually five people) reviews all the information collected and refines the data with additional questions. After this, the company conducts a brainstorming session. During the meetings, any, even the most irrational idea or project is listened to. It is known that some initially unsuccessful ideas eventually become better ones.  

Brainstorming (brainstorming) must have a clear goal and include the following stages: silent generation of ideas, unordered listing of ideas, clarification of ideas, voting and ranking the importance of ideas to achieve the goal.  

Brainstorming (or brainstorming) is a way of collectively generating ideas at a meeting of experts, which is carried out according to a specially designed scheme. Direct brainstorming is based on the hypothesis that among the large number of ideas expressed by experts, there are at least a few good ones.  

Brainstorming (or brainstorming) is a way of collectively generating ideas at a meeting of experts, which is carried out according to a specially designed scheme. Direct brainstorming is based on the hypothesis that among the large number of ideas expressed by experts, there are at least a few good ones. Peculiarity this method is that the period of free creative generation of ideas, proposals and hypotheses is clearly separated from the stage of critical assessment of the information received, and this assessment itself is made in such a form that it does not connect, but stimulates further creative discussion of the issues under consideration.  

Brainstorming sessions are based on the principle that discussion possible ways solving the problem in the future is implemented in a mode of clear separation of the stages of generating ideas and their evaluation.  

Brainstorming (brainstorming) should have a clear goal and consist of the following stages: silent generation of ideas, unordered listing of ideas, clarification of ideas, voting and ranking the importance of ideas to achieve the goal. Types of brainstorming: direct reverse (starts with criticism of ideas), double (the number of participants is two or three times the optimal number with a corresponding increase in the duration of the event), conference of ideas (usually for 4 - 12 people for 2 - 3 days), individual brainstorming.  

Brainstorm - used when exists high degree uncertainty of the situation. The method is used to clarify the main tasks facing the organization and possible options to resolve it. According to this technique, attack participants put forward as many ideas as possible, which are then grouped.  

Brainstorming (brainstorming) as a method of group generation large quantity ideas in a relatively short period of time were proposed in the pre-war period.  

A brainstorming session lasts only a few minutes, approximately 5 - 7, since ideas should come to the participants’ heads spontaneously, and not through special thinking. This difficult task, since a whole series of ideas, about 10 - 20, have already been written down. Now it is necessary to cross out those of them that are not feasible at all or at the moment, and bring the rest into the system. In no case should criticism, which was prohibited at the first stage, be allowed at this time, since many may, because of this, abandon this method of work in the future.  

A reverse brainstorming is carried out if it is necessary to identify shortcomings and contradictions in a technical object that needs to be improved. In reverse brainstorming, in contrast to direct brainstorming, the main attention is paid to critical comments, and the choice is made not of a general, but of a purely specific technical (or technological) problem.  

Any problem can be considered using the brainstorming method if it is formulated simply and clearly enough. This method can be used at any stage of design, both at the beginning, when the problem has not yet been fully defined, and later, when complex sub-problems have already been identified.  

The concept of brainstorming is, of course, not an invention of our century.  

The brainstorming method is characterized by the open expression of the opinions of specialists (on ci on solving a specific problem. In this case, two conditions must be met: firstly, sanpei judgments; secondly, it is proposed to express any ideas for solving this issue without) value or possibility of implementation. All ideas expressed are recorded after the discussion. At the same time, rational points in each of the proposals made are identified and a solution is formulated. The advantage of this method is the ability to make a decision within a short period of time.  

The brainstorming method is a group decision creative problem, provided and facilitated nearby special techniques. Brain attack was proposed in the late 30s as a method aimed at activating creative thought; for this purpose, means are used that reduce a person’s criticality and self-criticism, thereby increasing his self-confidence and demonstrating the mechanisms of the creative act. As you know, the creative effectiveness of most people is determined not only by their talent, but also

the possibility of maximizing the realization of one’s creative potential, therefore, the basis of the brainstorming method is the assumption that reducing a person’s criticality towards one’s capabilities optimizes the conditions for creativity. In the initial period of creativity, many inventors and scientists spend considerable effort trying to drown out the voice of the inner critic (while a work of creative thought is still “in an embryonic” state, it may look unattractive even in the eyes of its creator).

Reducing process criticality brainstorming achieved in two ways. The first is a direct instruction: be free, creative, original, suppress criticism of yourself and your ideas, and do not be afraid of the assessment of others. The purpose of the instruction is to change the internal position, the attitude of the individual in relation to his abilities. The second way is to create favorable external conditions: sympathy, support and approval of partners. The presenter makes special efforts to create a special inviting atmosphere. In such an environment, internal control weakens and inclusion in creative process. After all, sometimes one critical remark is enough for an interesting, but risky proposal to be replaced on the fly by another - proven, but uninteresting. In a brainstorming session, not only does it make it easier to overcome internal barriers for individual group members, its advantage is that it opens up the possibility of switching to someone else’s logic - the logic of a neighbor, thus, the creative potentials of all participants in the attack are, as it were, summed up.

During the training, participants acquire the ability to argue kindly, listen, ask questions, encourage, and criticize. Very often people cannot separate what they actually see from what they are determined to see under the pressure of their prejudice, so we need to teach a person to observe with an open mind and as objectively as possible. Along with the development of observation, the ability for self-observation improves and at the same time the attitude towards oneself becomes more objective.

In a brainstorming session, each participant freely puts forward his proposals for solving the problem under consideration, while criticism is completely prohibited.

The method not only helps to overcome thought patterns, but also removes the social and subordination prohibitions that each person imposes on their statements during normal conversation! When working in a group, it is easier to see flaws in attack partners' ideas than in your own. The member of the group who is formulating the current proposal, whose attention is fully occupied, may not notice or appreciate the hint of a solution contained as a minor detail in his proposal. Another, watching from the side, finds himself in a more favorable conditions. For him, these minor details act as a hint to the desired solution, and he can use it when analyzing the quality of the proposal and improving it.

Since the basic rules of brainstorming exclude any criticism, each participant is convinced that any idea can be expressed without fear of being considered funny or untenable. During the work, the leader asks questions and in every possible way encourages the unbridled association of group members. The facilitator's questions should be phrased in a way that breaks the ice and encourages participants to start talking, for example: “Do you fully agree with this idea?” The presenter asks the participants to reformulate their statements in such a way as to turn them from evaluative into meaningful: “This is not just good, but good because...” The more wild (unlikely) the idea is proposed, the more encouragement it receives from the presenter. The number of ideas should be as large as possible; during the attack, everyone is allowed to combine, modify and improve the ideas expressed by other participants in any way they like. Usually, before a participant begins to present an amendment, addition or development of the previous comrade’s idea, the facilitator recommends briefly repeating his idea and asking whether he was understood correctly. Mutual encouragement contributes to the birth of many proposals; their interaction often gives rise to new ideas that none of the participants would have thought of on their own.

For efficiency teamwork A group is influenced not only by its quantitative composition, but also by the experience, work style and profession of each of its members. Psychological barrier individual person is easier to overcome if

the group is more heterogeneous in composition. The group form of work makes the internal barriers of individual group members more vulnerable and less stable. Having different life and professional experiences, different attitudes and personal taboos, they ask each other questions that they could not ask themselves, being limited by their own internal barriers and attitudes. So, in the conditions of a group attack, contradictions in reasoning and logical errors its individual participants.

The lesson is conducted as follows. The placement of participants in the brainstorming session is deliberate, as it has a significant impact on their activity, unity and integrity in the work of the group. For those sitting at the back or on the edge, it is more difficult to join in the general conversation, so it is advisable to position the participants facing each other. The facilitator then poses a problem to the group and asks group members to propose as many possible solutions as possible without pre-thinking in a short period of time. The attack time ranges from several minutes to an hour. Not a single one of the proposed options is criticized, but, on the contrary, is encouraged in every possible way, and the promotion of unusual and even completely unrealistic ideas is stimulated. The speaking time for each participant is, as a rule, no more than 1–2 minutes; you can speak many times, but preferably not in a row. All speeches are recorded as accurately as possible, all proposals, including the most valuable ideas, are the fruits of collective labor and are not personalized. Brainstorming usually ends when the flow of suggestions dries up.

In the classroom, they also use special techniques for activating thinking: lists test questions, dissection, presentation of the problem to a non-specialist. Using a list, the search is guided by leading questions. For each special area, a list of various questions is compiled, each participant in the attack asks himself sequentially in the process of solving the problem, which activates his thought, allowing him to turn and consider the question with different sides. Answering questions from the list sometimes allows you to find a way out of an impasse. Here are typical questions: “What if we do the opposite? What if we replace this task with another? What if you change the shape of an object? What if we take another material?

Why else can this product (unit, material) be used exactly in the form in which it is now? What about changes (if you make it bigger, smaller, stronger, weaker, heavier, lighter, etc.)? In combination with something else? Is it possible to rearrange, combine, replace?”

Dismemberment includes four successive steps. First, all the components of the structure to be improved are written down on separate cards. Then, on each one, the maximum number of characteristic features of the corresponding part is sequentially listed. After this, it is necessary to evaluate the meaning and role of each feature for the functions of a given part (should they remain unchanged from the point of view of the implementation of their functions) and highlight in different colors those features that cannot be changed at all, those that can be changed within specified limits, and those that which can be changed within any limits. Finally, all cards are laid out on the table at the same time and analyzed as a common field of effort. The essence of the dissection technique lies, from our point of view, in the simultaneous visibility of the entire set of elements to be transformed, that is, in the activation of not only the analytical capabilities of the left hemisphere of the brain, but also the synthetic ones of the right.

When solving a new problem, it can be helpful to seek the opinions of others. The very act of presenting a difficult problem to someone often helps to crystallize thoughts and bring the solution closer. However, if the problem is discussed with specialists, then many details are omitted as understandable in themselves, so it is useful to present the problem to a non-specialist in the field, which forces it to be simplified. A simple statement of the problem clarifies the problem for the author and thereby brings closer the solution, which at first is obscured by technical details.

The attack process encourages the creation of unexpected associations. To do this, they suggest straining your memory and imagining possible connections between the details of this task and other tasks of the same plan, then relax and link the problem being solved with what first comes to mind. Sometimes it seems that the thought that has arisen has absolutely nothing to do with solving a given problem, and only later does it become clear that it is precisely this thought that contains the desired answer.

The conditions of the problem to be solved must necessarily be freed from special terminology and presented in the most generalized form possible, since the terms impose old and unchanged ideas about the object (we have already pointed out the benefit of reformulating the problem in the section on thinking). If in the conditions of the problem we are talking, for example, about increasing the speed of an icebreaker, then the term “icebreaker” immediately limits the range of ideas under consideration: it is necessary to chop, break, destroy the ice. The simple idea that it is not a matter of destroying the ice at all and that the main thing is to move through the ice and not break it, in this case turns out to be beyond the psychological barrier.

During the lesson, the leader presents the problem and asks each group member to express their thoughts on how to solve it, without being embarrassed by putting forward the most incredible assumptions. The leader does not allow any discussion of the merits and demerits of the ideas expressed until the flow of new ideas stops. The group is confident that any idea expressed, no matter how far from the solution and stupid it may turn out to be, can make a certain contribution to clarifying the problem, which, in turn, will bring the solution of the problem closer. It is useful for the brainstorming leader to have a few appropriate cues ready to guide the group, such as: “Please, now you try. Who else would like to add and complement something, further define it?” It should demonstrate confidence in success, instill optimism in the participants and maintain a relaxed atmosphere. Once the group has exhausted its ideas, a discussion opens to combine and develop the proposed ideas into a coherent whole - a practical solution to the problem at hand.

The brainstorming method is used not only for learning, but also as a practical technique for solving complex and creative tasks. For this purpose it is sometimes modified. One of the modifications is the shuttle method. As you know, some people are more inclined to generate ideas, others - to critically analyze them. For example, the famous physicist P. Ehrenfest constantly suffered from

that his critical abilities were ahead of his constructive ones. Such increased criticality did not allow even his own ideas. In ordinary discussions of problems, creators and critics, when they find themselves together, interfere with each other. In a shuttle brainstorming session, this incompatibility is eliminated by selecting two groups of participants taking into account each person's abilities - for generating ideas and for criticizing. These groups work in different rooms. The brainstorming session begins in the idea generation group, the leader outlines the problem, asks everyone to make suggestions, writes down all the suggestions received, announces a break in this group and passes them on to the criticism group. Critics select the most interesting and promising proposals and, based on them, further define the task, which, after a break, is again proposed to the idea generation group. The work is repeated cyclically until an acceptable result is obtained. A group of just six people can come up with up to 150 ideas in 30 minutes during an attack. Group working conventional methods, would never have thought that the problem she was considering had such a variety of aspects.

The “synectics” technique, close to brainstorming, is a way to stimulate the imagination. Literally, synectics is the joining together of dissimilar elements. A synectics group usually consists of specialists from different fields. The collision of the most unexpected opinions and incredible analogies leads to the expansion of the field of ideas, the birth of new approaches to solving the problem and allows one to go beyond narrow professional capabilities; analogies from other fields of knowledge or fantastic analogies are more often used, in which the problem is solved mentally, as in a fairy tale.

A group working using the synectics method uses various analogies to promote spontaneous thinking: direct, subjective, symbolic and fantastic. Direct analogies are often found in biological systems that solve similar problems. For example, observing a carpenter worm drilling a tubular channel in wood led Brunel to think about the caisson method of constructing underwater structures.

Subjective analogies force you to imagine how you can use your body to achieve the desired result or what a person will feel if he imagines himself as a given detail. With symbolic analogies, the characteristics of one object are identified with the characteristics of another, and fantastic analogies require us to imagine things the way we would like to see them. It is permissible to ignore physical laws, for example the use of anti-gravity. Synectics excites and uses analogies as a means of shifting the process from the level of conscious thinking to the level of subconscious activity.

The brainstorming method is widely used in the USSR. Considerable experience has been accumulated in its use in universities, industry and research organizations. Brainstorming is used both as a method of problem solving and as a method of assimilation of knowledge, since the knowledge and experience of all participants in the discussion become accessible to everyone and can be effectively absorbed during the discussion. As they gain experience in group discussion of problems, participants acquire useful skills such as the ability to briefly and accurately state their position, correctly perceive someone else’s, and the ability to obey given rules discussions

A fairly common method of expert assessment is “brainstorming” or “brainstorming”. The basis of the method is to develop a solution based on joint solving of the problem by experts. As a rule, not only specialists in a given problem are accepted as experts, but also people who are specialists in other fields of knowledge. The discussion is based on a pre-developed scenario.

The brainstorming method appeared in the United States of America in the late 30s, and finally took shape and became known to a wide circle specialists with the publication of A. Osborne’s book “Controlled Imagination” in 1953, which revealed the principles and procedures of creative thinking.

Brainstorming methods can be classified based on the presence or absence of feedback between the leader and the brainstorming participants in the process of solving some problem. problematic situation.

The current situation required the development of a “brainstorming” method—destructive referenced evaluation (DRA), capable of evaluating options efficiently and quickly enough, without limiting their number.
The essence of this method is to actualize the creative potential of specialists during a “brainstorming” of a problem situation, which first involves the generation of ideas and the subsequent destruction (destruction, criticism) of these ideas with the formation of counter-ideas.

Structurally, the method is quite simple. It represents a two-stage procedure for solving a problem: at the first stage, ideas are put forward, and at the second they are specified and developed.

Osborne was faced with an ordinary situation that most citizens do not perceive as a problem. Many acute problems facing enterprises are not solved for a long time, despite the obviously high intellectual potential enterprise workers. Is it only the lack of resources and material incentives that is to blame? Let us follow A. Osborne and ask the same question: why is the creative potential of the country’s citizens used so little to solve the problems it faces? After all creativity All people have it. The answer was found by Osborne during a detailed examination of the procedure for including a “newbie” in solving a problem. As a rule, problems are formulated by specialists in professional language using special terms, based on knowledge of deep effects. It is not easy to thoroughly understand such a problem in order to engage in its discussion. And to top it all off, ideas are expressed by non-professionals without taking into account restrictions, often in an ‘Incorrect, lax form. All this leads to negative reaction professionals, a wave of criticism aimed at the form of expression. Judgments of incompetence very quickly develop into conclusions about the inability to use this person for creative work.

So, in order for an idea to be accepted by specialists, it must be put forward formalized “according to all the rules” - this is a widely held opinion.

The most important element of the method proposed by Osborne is the removal of this limitation. “Why not divide each problem so that one part of the experienced experts takes care of finding the facts about the legal judgment, while the creative consultants focus only on putting forward one idea after another,” writes A. Osborne.

This division of the process of searching for an idea into constructive stages and the selection of people to carry out each stage is the basis of the proposed method. A. Osborne points to the emergence of a new approach to solving problems, an approach that he called “imagination”. “You give free rein to your imagination and then “imagine” it down to earth.” The development of this idea led to the emergence of a rather complex sequence of actions. The most important premise on which Osborne relied is the idea that every person has two the most important aspects brain function: creative mind And analytical thinking. Their alternation, according to Osborne, forms the basis of all processes of creative work.

1. Think through all aspects of the problem. The most important ones are often so complex that identifying them requires the imagination.

2. Select sub-problems to “attack”. Refer to the list of various aspects of the problem, carefully analyze them, highlight several goals.

3. Consider what data might be useful. We have formulated the problem, now we need very specific information. But first, let's give ourselves over to creativity to come up with all kinds of data that can help best.

4. Select your preferred sources of information. Answering the question about the types necessary information, let's move on to deciding which sources should be studied first.

5. Come up with all kinds of ideas - “keys” to the problem. This part of the thinking process certainly requires freedom of imagination, unaccompanied or interrupted by critical thinking.

6. Select ideas that are most likely to lead to a solution. This process is mainly associated with logical thinking. The emphasis here is on comparative analysis.

7. Come up with all sorts of ways to check. Here again we need creative thinking. It is often possible to discover completely new verification methods.

8. Select the most thorough verification methods. When deciding how best to check, we will be strict and consistent. We will select those methods that seem most convincing.

9. Imagine all possible applications. Even if our final solution is confirmed experimentally, we must have an idea of ​​what might happen as a result of its use in various fields. For example, each military strategy is finally formed on the basis of ideas about what the enemy can do.

10. Give a final answer.

Here you can clearly see the alternation of creative, synthesizing stages and analytical, rational ones. This alternation of expansion and contraction of the search field is inherent in all developed methods search. A shorter sequence of actions, also described in the book " Practical imagination” and part of the essence of the brainstorming method. The method includes two main stages:

— Stage of putting forward (generating) ideas.

— Stage of analysis of the proposed ideas.

Work within these stages must be carried out subject to a number of basic rules. At the generation stage there are three of them:

3. Encouragement of all ideas put forward, including unrealistic and fantastic ones.

At the analysis stage, the basic rule is:

4. Identification of the rational basis in each analyzed idea.

The method proposed by A. Osborne was called (“brainstorming”).

Methods of this type are also known as brainstorming, idea conferences, and collective idea generation (CGI). Usually, when conducting a brainstorming session, or CGI sessions, they try to perform certain rules, the essence of which is to ensure as much as possible more freedom the thinking of OIG participants and their expression of new ideas; To do this, it is recommended to welcome any ideas, even if they seem dubious or absurd at first (discussion and evaluation of ideas is carried out later), criticism is not allowed, an idea is not declared false, and discussion of no idea is stopped. It is required to express as many ideas as possible (preferably non-trivial ones), try to create, as it were, chain reactions ideas.

Working with the DOO method involves the implementation of the following six stages.

The first stage is the formation of a group of brainstorming participants (in terms of size and composition). The optimal size of a group of participants is determined empirically: groups of 10–15 people are recognized as the most productive. The composition of the group of participants involves their targeted selection:

1) from persons of approximately the same rank, if the participants know each other;

2) from persons of different ranks, if the participants do not know each other (in this case, each participant should be leveled by assigning him a number and then addressing the participant by number).

The second stage is drawing up a problem note from a brainstorming participant. It is compiled by the problem situation analysis group and includes a description of the ECE method and a description of the problem situation.

The third stage is the generation of ideas. The duration of brainstorming is recommended to be at least 20 minutes and no more than 1 hour, depending on the activity of the participants. It is advisable to record the ideas expressed on a tape recorder so as not to “miss” any idea and to be able to systematize them for the next stage.

The fourth stage is the systematization of ideas expressed at the generation stage. The problem situation analysis group carries out systematization of ideas in the following sequence: a nomenclature list of all expressed ideas is compiled; each of the ideas is formulated in commonly used terms; duplicate and complementary ideas are identified; duplicate and (or) complementary ideas are combined and formed into one complex idea; signs are identified according to which ideas can be combined; ideas are combined into groups according to selected characteristics; a list of ideas is compiled into groups (in each group, ideas are written down in order of their generality from more general to specific, complementing or developing more general ideas).

The fifth stage is the destruction (destruction) of systematized ideas (a specialized procedure for assessing ideas for practical feasibility in the process of a brainstorming session, when each of them is subjected to comprehensive criticism by the brainstorming participants).

The basic rule of the destruction stage is to consider each of the systematized ideas only from the point of view of obstacles to its implementation, that is, the participants in the attack put forward conclusions that reject the systematized idea. Particularly valuable is the fact that in the process of destruction a counter-idea can be generated that formulates existing restrictions and suggests the possibility of removing these restrictions.

The sixth step is to evaluate the criticisms and compile a list of practical ideas.

The method of collective idea generation has been tested in practice and allows us to find group decision when determining possible options for the development of a forecast object, excluding the path of compromise, when a single opinion cannot be considered the result of an impartial analysis of the problem.

Depending on the adopted rules and the rigidity of their implementation, there are direct brainstorming, the method of exchanging opinions, methods such as commissions, courts (when one group makes as many proposals as possible, and the second tries to criticize them as much as possible), etc. Lately, brainstorming is sometimes carried out in the form business games.

In practice, similarities to OIG sessions are various kinds meetings - design meetings, meetings of scientists and scientific councils, specially created temporary commissions.

In real conditions, it is quite difficult to ensure strict implementation of the required rules, to create an “atmosphere of brainstorming”; the influence of the official structure of the organization interferes with the design teams and councils: it is difficult to gather specialists on interdepartmental commissions. Therefore, it is desirable to use methods of attracting competent specialists that do not require their mandatory presence in a specific place and at a specific time and verbal statements their opinions.

2. "DELPHI" METHOD. ESSENCE AND FEATURES OF APPLICATION.

One of the most popular expert methods is the Delphi method.

Among the varieties of expert methods is the Delphi method. In 1970 – 1980 Separate methods have been created that allow, to a certain extent, to organize statistical processing of the opinions of expert experts and achieve a more or less agreed upon opinion. The Delphi method is one of the most common methods expert assessment of the future, i.e. expert forecasting. This method was developed by the American research corporation RAND and is used to determine and assess the likelihood of certain events occurring.

The Delphi method, or the “Delphic oracle” method, was originally proposed by O. Helmer and his colleagues as an iterative procedure during brainstorming, which would help reduce the influence of psychological factors when repeating meetings and increase the objectivity of the results. However, almost simultaneously, Delphi procedures became a means of increasing the objectivity of expert surveys using quantitative estimates when assessing the “goal tree” and developing “scenarios”.

The specificity of this method lies in the fact that the generalization of the research results is carried out through an individual written survey of experts in several rounds according to a specially developed research procedure.

The reliability of the Delphi method is considered high when forecasting for a period of 1 to 3 years, as well as for a longer period of time. Depending on the purpose of the forecast, from 10 to 150 experts can be involved in obtaining expert assessments.

The Delphi method is based on following principle: in inexact sciences, expert opinions and subjective judgments, by necessity, must replace the exact laws of causality reflected by the natural sciences.

The expert survey procedure using the Delphi method is built in several stages.

Stage 1. Formation of a working group

Task working group consists of organizing an expert survey procedure.

Stage 2. Formation of an expert group

In accordance with the Delphi method, a group of experts should include 10–15 specialists in the field. The competence of experts is determined by questionnaires, analysis of the level of abstracting (the number of references to the work of a given specialist), and the use of self-assessment sheets.

Stage 3. Formulation of questions

The wording of questions should be clear and unambiguously interpreted, suggesting unambiguous answers.

Stage 4. carrying out the examination

The Delphi method involves repeating several steps of conducting a survey. Based on the results of the first survey, extreme, so-called “heretical” opinions are identified, and the authors of these opinions justify their point of view with subsequent discussion. This allows, on the one hand, all experts to take into account the arguments of supporters extreme points on the other hand, it gives the latter the opportunity to once again think over their point of view and either further substantiate it or abandon it. After the discussion, the survey is conducted again to allow the experts to take into account the results of the discussion. And this is repeated 4–5 times until the experts’ points of view come closer.

Stage 5. summing up the survey results

According to the Delphi method, the median is taken as the final opinion of experts, that is, the average value in an ordered series of opinions. If a series ordered by the size of the answers (for example, answers to a question about the price of an innovative product) includes n values: P1, P2,..., Pn, then the final assessment based on the survey results is the opinion of M, defined as follows:

M = Pk, if n = 2k-1

M = (Рк + Рк+1)/2, if n = 2к,

where k = 1, 2, 3,…

The Delphi method allows you to summarize the opinions of individual experts into a consensus group opinion. It has all the shortcomings of forecasts based on expert assessments. However, the work carried out by the RAND Corporation to improve this system has significantly increased the flexibility, speed and accuracy of forecasting. The Delphi method is characterized by three features that distinguish it from conventional methods of group interaction between experts. These features include:

a) anonymity of experts;

b) using the results of the previous round of the survey;

c) statistical characteristics of the group response.

Anonymity lies in the fact that during the procedure of expert assessment of the predicted phenomenon or object, the participants of the expert group are unknown to each other. In this case, the interaction of group members when filling out questionnaires is completely eliminated. As a result of such a statement, the author of the answer may change his opinion without publicly announcing it.

The statistical characteristic of a group response involves processing the results obtained using the following measurement methods: ranking, paired comparison, sequential comparison and direct assessment.

In the development of the Delphi method, cross-correction is used. A future event is represented as a huge number of connected and transforming paths of development. When cross-correlation is introduced, the value of each event, due to the entered certain connections, will change either in a positive or negative direction, thereby adjusting the probabilities of the events in question. For the purpose of future compliance of the model with real conditions, elements of randomness can be introduced into the model.

The main means of increasing the objectivity of the results when using the Delphi method are the use of feedback, familiarization of experts with the results of the previous round of the survey and taking these results into account when assessing the significance of expert opinions.

In specific techniques that implement the Delphi procedure, this tool is used in varying degrees. Thus, in a simplified form, a sequence of iterative brainstorming cycles is organized. In a more complex version, a program of sequential individual surveys is developed using questionnaires that exclude contacts between experts, but provide for familiarizing them with each other’s opinions between rounds. Questionnaires may be updated from round to round. To reduce factors such as suggestion or adaptation to the opinion of the majority, experts are sometimes required to justify their point of view, but this does not always lead to the desired result, but on the contrary, can enhance the effect of adaptation. In the most developed techniques experts are assigned weighting coefficients of the significance of their opinions, calculated on the basis of previous surveys, refined from round to round and taken into account when obtaining generalized assessment results.

Due to the complexity of processing the results and significant time expenditure, the initially envisaged Delphi techniques cannot always be implemented in practice. Recently, the Delphi procedure in one form or another usually accompanies any other methods of system modeling - morphological, network, etc. In particular, a very promising idea for the development of expert assessment methods, proposed at one time by V.M. Glushkov, is to combine a targeted multi-stage survey with a “development” of the problem in time, which becomes quite feasible in the conditions of algorithmization of such a (rather complex) procedure and the use of computer technology.

To increase the effectiveness of surveys and activate experts, they sometimes combine the Delphi procedure with elements of a business game: the expert is asked to conduct a self-assessment, putting himself in the place of the designer who is actually tasked with carrying out the project, or in the place of a management employee, a manager at the corresponding level of the system organizational management etc.

The disadvantage of this method is that the problem of correlating scientific and technological changes is very complex, since in real life the magnitude of the correlation is very difficult to measure, correlations unclear and vary widely depending on the achievements in question.

REFERENCES

    Agapova T. Modern economic theory: methodological basis and models // Russian Economic Journal. – 1995. – No. 10.

    Beshelev S.D., Gurvich F.G. Expert assessments in making planning decisions. M.: Economics, 1976.

    Golubkov E.P. Marketing research: theory, methodology and practice. M.: Finpress, 1998.

    Glass J., Stanley J.. Statistical methods in forecasting. M.: Progress, 1976.

    Research on general systems theory: Collection of translations. General ed. and entry article by V.N. Sadovsky and E.G. Yudin. M., 1969. P. 106-125.

    Evlanov L.G., Kutuzov V.A. Expert assessments in management. M.: Economics, 1978.

    Eliseeva I.I., Yuzbashev M.M. General theory Statistics / Ed. I.I. Eliseeva. M.: Finance and Statistics, 2004.

Brainstorming

The brainstorming method is a group solution to a creative problem, provided and facilitated by a number of special techniques. Brain attack was proposed in the late 30s as a method aimed at activating creative thought; for this purpose, means are used that reduce a person’s criticality and self-criticism, thereby increasing his self-confidence and demonstrating the mechanisms of the creative act. As you know, the creative effectiveness of most people is determined not only by their talent, but also by the possibility of maximizing their creative potential, therefore the brainstorming method is based on the assumption that reducing a person’s criticality towards their capabilities optimizes the conditions for creativity. IN initial period creativity, many inventors and scientists spend considerable effort trying to drown out the voice of the inner critic (while a work of creative thought is still “in an embryonic” state, it may look unattractive even in the eyes of its creator).

Reducing criticality during the brainstorming process is achieved in two ways. The first is a direct instruction: be free, creative, original, suppress criticism of yourself and your ideas, and do not be afraid of the assessment of others. The purpose of the instruction is to change the internal position, the attitude of the individual in relation to his abilities. The second way is to create favorable external conditions: sympathy, support and approval of partners. The presenter makes special efforts to create a special inviting atmosphere. In such an environment, internal control weakens and inclusion in the creative process becomes easier. After all, sometimes one critical remark is enough for an interesting, but risky proposal to be replaced on the fly by another - proven, but uninteresting. Brainstorming not only makes it easier to overcome internal barriers for individual group members, its advantage is that it opens up the possibility of switching to someone else’s logic - the logic of a neighbor, thus creative potentials all participants in the attack are, as it were, summed up.

During the training, participants acquire the ability to argue kindly, listen, ask questions, encourage, and criticize. Very often people cannot separate what they actually see from what they are determined to see under the pressure of their prejudice, so we need to teach a person to observe with an open mind and as objectively as possible. Along with the development of observation, the ability to self-observe also improves and at the same time the attitude towards oneself becomes more objective.

In a brainstorming session, each participant freely puts forward his proposals for solving the problem under consideration, while criticism is completely prohibited.

The method not only helps to overcome thought patterns, but also removes social and subordination prohibitions that each person imposes on their statements during normal conversation! When working in a group, it is easier to see flaws in attack partners' ideas than in your own. The member of the group who is formulating the current proposal, whose attention is fully occupied, may not notice or appreciate the hint of a solution contained as a minor detail in his proposal. The other, watching from the outside, finds himself in more favorable conditions. For him, these minor details act as a hint of the right solution, and he can use it when analyzing the quality of the proposal and improving it.

Since the basic rules of brainstorming exclude any criticism, each participant is convinced that any idea can be expressed without fear of being considered funny or untenable. During the work, the leader asks questions and in every possible way encourages the unbridled association of group members. The facilitator's questions should be phrased in a way that breaks the ice and encourages participants to start talking, for example: “Do you fully agree with this idea?” The presenter asks the participants to reformulate their statements in such a way as to turn them from evaluative into meaningful: “This is not just good, but good because...” The more wild (unlikely) the idea is proposed, the more encouragement it receives from the presenter. The number of ideas should be as large as possible; during the attack, everyone is allowed to combine, modify and improve the ideas expressed by other participants in any way they like. Usually, before a participant begins to present an amendment, addition, or development of the previous comrade’s idea, the facilitator recommends briefly repeating his idea and asking whether he was understood correctly. Mutual encouragement contributes to the birth of many proposals; their interaction often gives rise to new ideas that none of the participants would have thought of on their own.

The effectiveness of a group’s collective work is influenced not only by its quantitative composition, but also by the experience, work style and profession of each of its members. The psychological barrier of an individual can be overcome more easily if the group is more heterogeneous in composition. The group form of work makes the internal barriers of individual group members more vulnerable and less stable. Having different life and professional experiences, different attitudes and personal taboos, they ask each other questions that they could not ask themselves, being limited by their own internal barriers and attitudes. Thus, in the conditions of a group attack, contradictions in reasoning and logical errors of individual participants are quickly discovered and overcome.

The lesson is conducted as follows. The placement of participants in the brainstorming session is deliberate, as it has a significant impact on their activity, unity and integrity in the work of the group. For those sitting at the back or on the edge, it is more difficult to join in the general conversation, so it is advisable to position the participants facing each other. The facilitator then poses a problem to the group and asks group members to propose as many possible solutions as possible without pre-thinking in a short period of time. The attack time ranges from several minutes to an hour. Not a single one of the proposed options is criticized, but, on the contrary, is encouraged in every possible way, and the promotion of unusual and even completely unrealistic ideas is stimulated. The speaking time for each participant is usually no more than 1–2 minutes; you can speak many times, but preferably not in a row. All speeches are recorded as accurately as possible, all proposals, including the most valuable ideas, are the fruits of collective labor and are not personalized. Brainstorming usually ends when the flow of suggestions dries up.

During classes, special techniques for activating thinking are also used: checklists, dissection, presentation of the problem to a non-specialist. Using a list, the search is guided by leading questions. For each special area, a list of various questions is compiled, each participant in the attack asks himself them sequentially in the process of solving the problem, which activates his thought, allowing him to turn and consider the issue from different angles. Answering questions from the list sometimes allows you to find a way out of an impasse. Here are typical questions: “What if we do the opposite? What if we replace this task with another? What if you change the shape of an object? What if we take another material?

Why else can this product (unit, material) be used exactly in the form in which it is now? What about changes (if you make it bigger, smaller, stronger, weaker, heavier, lighter, etc.)? In combination with something else? Is it possible to rearrange, combine, replace?”

Dissection involves four sequential steps. First, all the components of the structure to be improved are written down on separate cards. Then on each one they list sequentially maximum quantity characteristic features of the corresponding part. After this, it is necessary to evaluate the meaning and role of each feature for the functions of a given part (should they remain unchanged from the point of view of the implementation of their functions) and highlight in different colors those features that cannot be changed at all, those that can be changed within specified limits, and those that which can be changed within any limits. Finally, all cards are laid out on the table at the same time and analyzed as a common field of effort. The essence of the dissection technique lies, from our point of view, in the simultaneous visibility of the entire set of elements to be transformed, that is, in the activation of not only the analytical capabilities of the left hemisphere of the brain, but also the synthetic ones of the right.

When deciding new task It may be helpful to seek the opinions of others. The very act of presenting a difficult problem to someone often helps to crystallize thoughts and bring the solution closer. However, if the problem is discussed with specialists, then many details are omitted as understandable in themselves, so it is useful to present the problem to a non-specialist in the field, which forces it to be simplified. A simple statement of the problem clarifies the problem for the author and thereby brings closer the solution, which at first is obscured by technical details.

The attack process encourages the creation of unexpected associations. To do this, they suggest straining your memory and imagining possible connections between the details of this task and other tasks of the same plan, then relax and link the problem being solved with what first comes to mind. Sometimes it seems that the thought that has arisen has absolutely nothing to do with solving a given problem, and only later does it become clear that it is precisely this thought that contains the desired answer.

The conditions of the problem to be solved must necessarily be freed from special terminology and presented in the most generalized form possible, since the terms impose old and unchanged ideas about the object (we have already pointed out the benefit of reformulating the problem in the section on thinking). If in the conditions of the problem we are talking, for example, about increasing the speed of an icebreaker, then the term “icebreaker” immediately limits the range of ideas under consideration: it is necessary to chop, break, destroy the ice. The simple idea that the point is not at all about destroying the ice and that the main thing is to move through the ice and not break it, in this case turns out to be beyond the psychological barrier.

During the lesson, the leader presents the problem and asks each group member to express their thoughts on how to solve it, without being embarrassed by putting forward the most incredible assumptions. The manager does not allow any discussion of the merits and demerits of the ideas expressed until the flow of new ideas stops. The group is confident that any idea expressed, no matter how far from the solution and stupid it may turn out to be, can make a certain contribution to clarifying the problem, which, in turn, will bring the solution of the problem closer. It is helpful for the brainstorming leader to have a few appropriate cues ready to guide the group, such as: “Please, now you try. Who else would like to add and complement something, to further define it?” It should demonstrate confidence in success, instill optimism in the participants and maintain a relaxed atmosphere. When the group has exhausted its supply of ideas, a discussion opens to combine and develop the proposed considerations into a coherent whole - a practical solution to the problem at hand.

Brainstorming is used not only for learning, but also as a practical technique for solving complex and creative problems. For this purpose it is sometimes modified. One of the modifications is shuttle method. As you know, some people are more inclined to generate ideas, others - to critically analyze them. For example, the famous physicist P. Ehrenfest constantly suffered from the fact that his critical abilities ahead of constructive ones. Such increased criticality did not allow even his own ideas to mature and strengthen. In ordinary discussions of problems, creators and critics, when they find themselves together, interfere with each other. In a shuttle brainstorming session, this incompatibility is eliminated by selecting two groups of participants taking into account each person's abilities - for generating ideas and for criticizing. These groups work in different rooms. The brainstorming session begins in the idea generation group, the leader outlines the problem, asks everyone to make suggestions, writes down all the suggestions received, announces a break in this group and passes them on to the criticism group. Critics select the most interesting and promising proposals and, based on them, further define the task, which, after a break, is again proposed to the idea generation group. The work is repeated cyclically until an acceptable result is obtained. A group of just six people can come up with up to 150 ideas in 30 minutes during an attack. A group working by conventional methods would never have come to the idea that the problem it was considering had such a variety of aspects.

The “synectics” technique, close to brainstorming, is a way to stimulate the imagination. Literally, synectics is the joining together of dissimilar elements. The synectics group usually consists of specialists different areas. The collision of the most unexpected opinions and incredible analogies leads to the expansion of the field of ideas, the birth of new approaches to solving the problem and allows one to go beyond narrow professional capabilities; analogies from other fields of knowledge or fantastic analogies are more often used, in which the problem is solved mentally, as in a fairy tale.

A group working using the synectics method uses various analogies to promote spontaneous thinking: direct, subjective, symbolic and fantastic. Direct analogies are often found in biological systems that solve similar problems. For example, observing a carpenter worm drilling a tubular channel in wood led Brunel to think about the caisson method of constructing underwater structures.

Subjective analogies make you imagine how you can use your body to achieve the desired result or what a person will feel if he imagines himself as a given detail. With symbolic analogies, the characteristics of one object are identified with the characteristics of another, and fantastic analogies require us to imagine things the way we would like to see them. It is permissible to ignore physical laws, for example the use of anti-gravity. Synectics excites and uses analogies as a means of shifting the process from the level of conscious thinking to the level of subconscious activity.

The brainstorming method is widely used in the USSR. Considerable experience has been accumulated in its use in universities, industry and research organizations. Brainstorming is used both as a method of problem solving and as a method of assimilation of knowledge, since the knowledge and experience of all participants in the discussion become accessible to everyone and can be effectively absorbed during the discussion. As they gain experience in group discussion of problems, participants acquire useful skills such as the ability to briefly and accurately state their position, correctly perceive someone else’s, and the ability to obey given rules of discussion.

From the book Super Thinking by Buzan Tony

Chapter 6 BRAIN ATTACK KEY WORDS This chapter introduces the reader to the information processing system that underlies radiant thinking. Through practical exercises, you will gain an understanding of the enormous potential that your brain has in

From the book Self-Inquiry - the Key to the Higher Self. Understanding yourself. author Pint Alexander Alexandrovich

Chapter 7 BRAINATTACK KEY IMAGES This chapter discusses research findings that have excited scientific world. Along with the practical exercises offered below, the knowledge you gain will allow you to access enormous potential

From the book Brain and Soul [How nervous activity shapes our inner world] by Frith Chris

Earth's Brain Cell - No. I wouldn't say that. Let's try to look at this from a holistic perspective. What is integrity? Let's take the integrity that is closest to us - our body. The body represents integrity. It consists of many cells. These cells

From the book Tough Negotiations: You Can't Win, You Can't Lose author Kozlov Vladimir

From the book Language and Consciousness author Luria Alexander Romanovich

Stage 3. Attack During a relaxed conversation with the interlocutor, in the “by the way, I remembered that I wanted to ask a question...” mode, it is asked additional question– detailing based on information from the previous stage of the conversation. The question should be tied to as specific a detail as possible,

From the book Brain Plasticity [Stunning facts about how thoughts can change the structure and function of our brain] by Doidge Norman

Lecture VII. Inner speech and its cerebral organization We have traced the first stages of the formation of the regulatory function of the word, as a result of which the child gradually develops the ability to subordinate his actions to the speech instructions of an adult. We have seen that at these stages

From the book The Oxford Manual of Psychiatry by Gelder Michael

Brain organization of the regulatory function of speech What are the brain mechanisms that provide the regulatory role of first external and then internal speech? What are the brain mechanisms underlying consciousness? act of will human? Hardly psychology

From the book Basics of Personal Security author Samoilov Dmitry

Lecture XV. Brain organization speech activity. Pathology of speech utterance Above, we covered in detail the main issues of the psychology of speech activity. We focused on the structure of words and phrases, on the origin of these basic constituent units of language, on

From the book Make Your Brain Work. How to Maximize Your Efficiency by Brann Amy

Brain organization motivational basis and programming of speech utterances It was already mentioned above that speech utterance begins with the presence of a known motive - to convey something to another, to ask for something, or to understand some idea. If this

From the author's book

Brain organization of the syntagmatic structure of an utterance It is known that the presence of a motive for an utterance, the need to communicate something, demand something or understand some content require the creation of a certain scheme that ensures the verbal formulation of this

From the author's book

Cerebral organization of the paradigmatic structure of speech processes Until now, we have been talking about those disorders of speech activity that arise when certain stages of the transition from thought to a detailed statement are affected. Described speech impairments

Attack Most modern martial arts and self-defense systems focus specifically on countering the attack itself, missing the three stages preceding it, at which the attack itself could have been prevented. The disadvantages of this approach are obvious: 1. Self defense on this

From the author's book

Attack on the Hippocampus Glucocorticoids (a type of steroid hormone) that the body releases during emotional outbursts and in conditions chronic stress, destroy hippocampal neurons. When Jessie is stressed, she gravitates toward something close and familiar. U

The second common method of conducting examinations is brainstorming method. This method, also known as “brainstorming”, “conference of ideas”, was proposed by the American scientist A. Osborne in 1955.

The main focus of the method is to identify new ideas and solutions. For this purpose, the organizers of the examination create an atmosphere most conducive to the generation of ideas (benevolence, support), freeing the expert from unnecessary constraint. The problem being discussed must be clearly formulated.

The brainstorming method is characterized by open expression of the opinions of specialists (at a special meeting) on ​​solving a specific problem. In this case, two conditions must be met: firstly, criticism of other people’s opinions is prohibited; secondly, it is expected to express any ideas to resolve this issue without taking into account immediate value or feasibility. All ideas expressed are recorded and, after discussion, are worked out in detail. At the same time, rational points in each of the assumptions made are identified and solutions are formulated based on their generalization. The advantage of the brainstorming method is the ability to make decisions in a relatively short time.

When implementing the method, the Pareto principle can be applied. After registering ideas from their entire pool, each expert selects 20% of the ideas that, from their point of view, deserve most attention. This selection is also recorded. Next, those who received greatest number points.

In the brainstorming method, a significant role belongs to the organizer who conducted the examination. He knows about ultimate goal examination, directing the discussion in the appropriate direction, but if the leader highlights only promising ideas from his point of view, the result of the examination will be less significant.

The use of this method eliminates the effect of conformity, i.e. opportunism, allows you to get productive results for short time, involve all experts in an active creative process.

The best results of the brainstorming method are achieved when developing new products, improving products and existing methods work, in promoting marketing and sales, improving technical designs, and in building a goal tree.

The brainstorming method is based on the following principles:

1. Two groups of people are involved in solving the problem - idea generators and experts. Idea generators bring together people with creative thinking, imagination and knowledge of science, technology and economics. Experts are usually people with a lot of knowledge and a critical mind. Experts play the role of analysts.


2. There should be no restrictions when generating. Any ideas are expressed, including obviously erroneous, humorous ones, without any evidence or feasibility study. The ideas expressed are usually recorded in a protocol, on a computer, on a dictaphone, etc. Thus, the basis of the method is the separation of the process of integrating ideas from the process of evaluating them.

3. Philosophical basis method - the theory of S. Freud, according to which the human consciousness is a muddy and fragile layer over the subconscious. Under normal conditions, a person’s thinking and behavior are determined by the basic consciousness, in which control and order reign: consciousness is “programmed” by habitual ideas and prohibitions. But through the thin crust of consciousness, dark elemental forces and instincts, raging in the subconscious, break through every now and then. These forces push a person to illogical actions, violation of prohibitions, and all sorts of irrational thoughts. The inventor has to overcome all psychological complexes and prohibitions caused by ideas about the possible and impossible.

The main advantage of the brainstorming method is the prohibition of criticism. But the ban on criticism is also a weakness of the method. To develop an idea, it is necessary to identify its shortcomings, and for this we need criticism.

The entire “attack” process can be divided into six stages:

1. A group of experts is formed. Usually its number is 10-15 people.

The composition of the group involves their targeted selection:

a) from persons of approximately the same rank, if the participants know each other;

b) from persons of different ranks, if the participants are unfamiliar with each other (in this case, each participant is assigned a number and subsequently addressed by number);

c) the group may include specialists from other fields of knowledge who have high level erudition and understanding of the meaning of the problem situation.

2. A problem note is drawn up.

To prepare it, a problem situation analysis group is preliminarily formed. The note may contain following information: composition of the causes of the problem situation, analysis of the causes and possible consequences problem situation, analysis of world experience in resolving similar problems (if any), classification (systematization) of possible ways to resolve the situation, formulation of the problem situation in the form of a central question with a hierarchy of sub-questions.

3. Generation of ideas.

The facilitator reveals the contents of the problem note, reminds the participants of the brainstorming session about the following:

a) statements must be clear and concise;

b) skeptical remarks and criticism of previous speakers are prohibited;

c) each participant can perform multiple times, but not in a row;

d) it is not allowed to read out a list of ideas prepared by the participant in advance.

One of the main tasks of the presenter is to induce the mental receptivity of the participants, their will to goal-oriented thinking. The leader’s work is supposed to be active only at the beginning of the “assault”. Soon enough, the excitement of the participants reaches a critical point and the development of new ideas becomes spontaneous.

After this, the role of the presenter comes down to the following:

a) focus the participants’ attention on the problem situation;

b) not to announce, condemn or stop the research of any project;

c) support and encourage participants who need it;

d) create a relaxed atmosphere, thereby facilitating active work experts.

The duration of a brainstorming session can be 20-60 minutes (depending on the activity of the participants). It is advisable to record the ideas expressed so as not to forget and be able to systematize them later.

4. Systematization of ideas expressed at the third stage (generation).

This work is assigned to the problem situation analysis group.

At this stage:

a) a nomenclature list of all ideas expressed is compiled;

b) duplicate and additional ideas are identified, then they are combined with the main idea;

c) signs are identified by which ideas can be combined;

d) ideas are combined into groups according to the selected characteristics;

e) a list of ideas is compiled into groups; in each group, ideas are written according to the rule from general to specific.

5. Destruction (destruction) of systematized ideas.

Each of the systematized ideas is studied for the possibility of its implementation. Participants in the assault put forward arguments that refute the systematized idea. In the process of destruction, a counter-idea may appear. The process of destruction continues until the systematized idea is criticized.

6. Evaluating criticisms and compiling a list of actionable ideas.

At this stage, a summary table is compiled. The first column of the table is the stages of systematization of ideas, the second is critical comments refuting ideas, the third is indicators of the practical applicability of ideas, the fourth is counter-ideas.

Each criticism and counter-idea is then evaluated:

a) is deleted from the table if it is refuted by at least one indicator of practical applicability;

b) is not crossed out if not refuted by any indicator.

A final list of ideas is compiled. Only ideas that are not refuted by criticism or counter-ideas are transferred to the list.

The brainstorming method is often used when developing a tree of goals. With the help of experts, the tree itself is obtained, as well as the coefficient of the relative importance of the goals.



Did you like the article? Share with your friends!