DirectX Raytracing: Real-time ray tracing. Ray Tracing Methods

Preface

Moore's law, which speaks of the exponential growth of computing power over time, gives reason to believe that sooner or later, ray tracing-based methods used to create highly realistic images in three-dimensional editors can be used in real time in computer games.

But in reality, laws passed by legislators, the tastes of voters, that is, users, and scientific and technological advances in distant fields will influence the prospects of ray tracing to a much greater extent.

Introduction

Let's briefly highlight the essence of the (inverse) ray tracing method. In the rasterization method used in modern real-time graphics, to draw an object, there is a projection onto the screen plane of the triangles that make up the object. And they are drawn pixel by pixel, filling the depth buffer, that is, the distance to the screen plane. The depth buffer is required so that the triangles closest to the observer draw the ones farther away, and not vice versa. And all other effects are done based on rasterization.

In the reverse ray tracing method, on the contrary, the image is constructed from screen pixels, and not from objects. An imaginary ray is drawn through each point of the screen in the direction away from the observer. It simulates a ray of light coming to the observer from a given direction. And for each ray it looks at which object it intersects first. And the color of the object area corresponding to the intersection point will set the color of this pixel. But then the fun begins. After crossing the object, the beam begins its journey across the scene. Rays are drawn in the direction of light sources to check whether a given point of this object is obscured; a reflected ray can be drawn if the object has specular properties; a refracted ray can be drawn if the object is semi-transparent.

In this case, the point of the object is directly illuminated by only one light source, the second is obscured by another object.

Thus, some simulation of light propagation occurs. The method has many complex modifications, but they are based on “ray tracing,” that is, finding the intersection of a ray (light) with scene objects.

Problem

Although the tracing method allows you to render a scene with lighting effects, transparency and reflections, it is computationally extremely expensive. The operation of finding the intersection of an arbitrary ray with objects in a complex scene is very non-trivial. And it cannot be accelerated as easily by special (fairly simple) “accelerators” as the mathematically simple operation of triangle rasterization. Therefore, in game graphics, a rasterization method is used, which allows you to quickly draw geometry, that is, object shapes and textures with all kinds of shaders. And the lighting of almost the entire scene is static. Only for individual moving models private methods of drawing shadows are used. They are also based on rasterization: shadows are essentially just drawn.

The simplest example: the silhouette of an object is drawn into a separate buffer, from the point of view of the light source, and then the contents of this buffer, like a texture, are superimposed on the surface under the object. You get these dynamic running shadows. They can be seen in many computer games for a long time. The method can be improved; this silhouette can be projected onto walls and onto curved surfaces. The texture of this silhouette can be blurred, thus creating a pattern with shades of gray rather than just a black and white crisp silhouette. And then, when applied, you will get a soft transition from darkness to light, the so-called soft shadow. It won't be a completely physical shadow, but it will look similar.

A soft shadow constructed using ray tracing will be more realistic, but it will be much more computationally expensive to draw. And the very first question is, will a gamer, in the excitement of, for example, a computer shooter, notice the difference between a very roughly drawn shadow and a more physically correct one? Here we come to the issue of subjective perception by people, that is, gamers, of graphics. After all, the picture on the monitor screen only roughly approximates reality. And if you use different criteria, the measure of this approximation will change.

It turned out, and this is obvious, that for the majority the decisive criterion for approximation is geometric detail. Next, by some margin, is high-quality texturing. In terms of texturing, the ray tracing method has approximately parity with the rasterization method; we will not particularly consider the issues of texturing and material shaders.

But it is not profitable to draw the geometry of the scene using ray tracing, although it depends on the scene. It is more efficient to draw scenes of a certain type using tracing, but scenes from modern games are far from this class.

Later in the article we will take a closer look at various projects in the field of tracing, but, for example, at one time Intel demonstrated rendering levels from Quake III using tracing. Low-poly, low-resolution levels were slowly drawn on a very expensive and advanced system, far from the consumer market. The trick was that you can draw dynamic shadows and complex reflections.

But human vision and perception are designed in such a way that it is very adaptive to lighting. Actually, all sorts of shadows only prevent the human eye from identifying the objects it needs. For example, when hunting, a typical activity of our ancestors, prey could be hidden in the shadows of trees. It is necessary to remove shadows from the virtual image formed in the brain.

Another point is that the actual lighting of one scene can be incredibly diverse, depending on the reflective properties of surfaces and the properties of the environment, air in particular, as well as the properties of the light source. This does not mean specular reflections, but the scattering of light by objects. As we see, in the very corner it is darker than closer to the window, because the darkest corner ultimately receives fewer photons of light traveling around the room. The air itself can also scatter light in different ways. And for an approximate simplified lighting model used in many games, you can select realistic parameters for the reflective properties of surfaces, the properties of air, and the light source in order to approximately reproduce the lighting of game scenes in reality.

Games often also use pre-rendered lighting for the scene, which is pre-calculated using the same ray tracing method and written into the material textures of objects. Okay, most of the time we see static light in real life. The sun slowly moves across the sky, and when we enter the room, we turn on the light if it is not already on. Then we take the machine gun and shoot the light bulbs, the light turns off. All this can be calculated in advance and placed in special textures called lightmaps (to save space, they are of lower resolution than material textures, since the lighting changes smoothly and can be qualitatively interpolated for each point using small-sized textures). Or calculate the lighting for each vertex of the triangles of a highly detailed scene, and draw shadows from moving models approximately using one of the private methods.

Over the past few years, ray tracing seems to have become the number one dream in the real-time 3D graphics world. Interest in this rendering technology peaked when young researcher Daniel Pohl announced his project on this technology back in 2004.

The reason for the general public's interest in the work was largely due to Pohl's focus on the famous id Software games Quake III, Quake IV and the shooter franchise Quake Wars 3D. The researcher attracted a lot of attention from the press, and gamers began to dream of a bright future when their favorite games would be rendered using ray tracing and get rid of rasterization.

Intel quickly took notice of the project, and it seemed to the company an ideal way to justify increasing the number of cores in processors. The company quickly launched its own research program, and today Intel never misses an opportunity to emphasize that ray tracing is the future of real-time 3D gaming. But is this really so? What technological realities lie behind the marketing hype? What are the real benefits of ray tracing? Can we expect ray tracing to replace rasterization? We will try to answer these questions.

Basic principles

The basic idea of ​​ray tracing is very simple: for each pixel on the display, the rendering engine draws a direct ray from the observer's eye (camera) to an element of the rendered scene. The first intersection is used to determine the color of the pixel as a function of the element's surface being intersected.

But this alone is not enough to produce a realistic scene. The illumination of the pixel must be determined, which requires drawing secondary rays (as opposed to primary rays, which determine the visibility of the various objects that make up the scene). To calculate the effects of scene lighting, secondary rays are drawn from intersection points to different light sources. If these rays are blocked by an object, then that point is in the shadow cast by the light source in question. Otherwise, the light source affects the lighting. The sum of all the secondary rays that reach the light source determines the quality of illumination that hits our scene element.

But that's not all. To achieve the most realistic rendering, the reflection and refraction characteristics of the material must be taken into account. In other words, you need to know how much light is reflected at the point where the primary beam intersects, as well as the amount of light that passes through the material at that point. Again, reflection and refraction rays must be drawn to calculate the final pixel color.

As a result, we get several types of rays. Primary rays are used to determine the visibility of an object and are similar to a kind of Z-buffer used in rasterization. And the secondary rays are divided into the following:

• rays of shadow/light;
• reflection rays;
• refraction rays.

This ray tracing algorithm is the result of the work of Turner Whitted, the researcher who invented the algorithm 30 years ago. Until then, the ray tracing algorithm only worked with primary rays. And the improvements Whitted made turned out to be a giant step towards realism in scene rendering.

If you are familiar with physics, then you probably noticed that the ray tracing algorithm works “in the opposite direction” from phenomena occurring in the real world. Contrary to the popular belief in the Middle Ages, our eyes do not emit rays of light; on the contrary, they receive rays of light from light sources that are reflected on various objects around us. In principle, this is how the very first ray tracing algorithms worked.

But the main disadvantage of the first algorithms was that they imposed a huge computational load. For each light source, you need to cast thousands of rays, many of which will not affect the rendered scene at all (since they do not intersect the imaging plane). Modern ray tracing algorithms are optimizations of the basic algorithms, and they use so-called reverse ray tracing, since rays are drawn in the opposite direction compared to reality.

Benefits of Ray Tracing

As you have already seen, the main advantage of the ray tracing method is its simplicity and elegance. The algorithm uses only one primitive object to display effects that often require a non-trivial approach and complex stimulation technologies when using a standard rasterization method.

Reflections are one area where ray tracing excels. Today, in 3D engines of modern games, reflections are calculated using environment maps. This technology gives a good approximation to reflections of objects located "at infinity" or in the environment (as the name suggests), but for closely spaced objects the approach shows its limitations.

Racing game developers, in particular, have created tricks to simulate the reflections of nearby objects using so-called dynamic cube maps. The camera is positioned at the level of the gamer's machine, after which rendering is carried out in the main directions. The rendering results are then stored in cubemaps, which are used to display reflections.

Of course, dynamic cubemaps also have their drawbacks. It is quite expensive in terms of computing power to calculate several rendering results, and to prevent performance from dropping too much, cube maps are not recalculated as many times as the main image. This may cause a slight delay in reflections. To reduce the load on the fill rate, rendering is performed at a lower resolution, which can lead to pixelation in reflections. Finally, this technology is often limited to the gamer's machine, with all other objects using simpler (spherical) environment maps.

With the ray tracing method, reflections are displayed perfectly, and without complex algorithms, since everything is calculated by the main rendering algorithm. Another advantage can be considered the display of reflections of parts of an object on each other, for example, the reflection of a side-view mirror on the body of a car, which is very difficult to obtain using the rasterization method - but here this reflection is obtained in the same way as others.

Another indisputable advantage of the ray tracing method is the high-quality processing of transparency effects. It is extremely difficult to render transparency effects using the rasterization algorithm, since the calculation of transparency directly depends on the rendering order. To get good results, you need to sort the transparent polygons in order of furthest from the camera to closest, and then render.

But in practice, this task is too heavy from a computational point of view, and transparency errors are also possible, since the sorting is done on polygons, not pixels. There are several technologies that allow you to bypass scene polygon sorting (such as depth peeling and A-buffers), but at the moment none of them can be called saving. At the same time, the ray tracing algorithm allows you to elegantly handle transparency effects.

Another important advantage is the calculation of shadows. In the world of rasterization, shadow mapping technology has become the standard. But it has several problems, such as “ladders” on the circuits and the amount of memory used. The ray tracing algorithm solves the shadow problem very elegantly, without resorting to complex algorithms, using the same primitive object and without requiring additional memory.

Finally, another strong advantage of the ray tracing method is its native ability to work with curved surfaces. Modern GPUs have had support for curved surfaces for several years now (it appears and disappears as new drivers and new architectures are released). But if rasterizers have to do an initial tessellation pass to create the triangles (which is the only primitive the rasterization engine can work with), then a ray tracing engine can simply work with the intersection of rays, without a precise mathematical definition of the surface.

Myths about ray tracing

But ray tracing still shouldn't be idealized, so it's time to break down some of the myths surrounding this algorithm.

Let's start with the fact that many gamers consider the ray tracing algorithm to be fundamentally better than rasterization, since it is used in films. This is wrong. Most synthetic/hand-drawn films (like all Pixar films) use an algorithm called REYES, which is based on rasterization. Pixar only added ray tracing to its RenderMan rendering engine later, during the production of Cars. But even for this film, ray tracing was used selectively so as not to overwhelm existing computing power. Prior to this project, Pixar used a plug-in to make limited use of ray tracing, such as ambient occlusion (AO) shading effects.

The second common myth among ray tracing advocates concerns the complexity of the scenes that ray tracing and rasterization can render. To understand, we need to take a closer look at each algorithm.

Below is how the rasterization algorithm works on each triangle in the scene.

• A set of pixels is determined that covers each triangle;
• For each involved pixel, its depth is compared with the depth of the neighboring pixel.

The main limitation of the rasterization method concerns the number of triangles. The algorithm has complexity O(n), where n is the number of triangles. The algorithm in this case has linear complexity depending on the number of triangles, since for each frame a list of triangles that need to be processed is compiled, one after the other.

In contrast, the ray tracing algorithm works as follows.

For each pixel of the frame:

• a ray is drawn to determine which triangle is the closest;
• For each triangle, the distance from the triangle to the image output plane is calculated.

As you can see, the processing sequence has become reversed. In the first case, we took each polygon and looked at which pixels it covered. And in the second case, we took each pixel and looked at which polygon corresponded to it. Therefore, you might think that the ray tracing method is less dependent on the number of polygons than the rasterization method, since the number of polygons does not affect the main loop. But in practice this is not the case. In fact, to determine which triangle will intersect the ray, we need to process all the triangles in the scene. Here, of course, defenders of the ray tracing method will say that it is not necessary to process all the triangles of the scene with each ray. If we use the appropriate type of data structure, it is very easy to organize the triangles so that only a small percentage of them are tested with each ray, which means that the ray tracing method has a complexity of O(log n), where n is the number of polygons.

Yes, the arguments can be considered true. But ray tracing advocates are a little disingenuous in that the same is true for rasterization. Game engines have been using BSP trees (binary space partitioning) and other techniques for years to limit the number of polygons that need to be calculated for each frame. Another controversial point is that such structures are most effective for static data. All we need is to calculate the data once, and then simply provide access to it, and this gives very good results. But what to do with dynamic data? In this case, the data will have to be recalculated for each image, and there are no miraculous formulas for this. You still have to study each polygon.

Simple algorithm?

The last myth concerns the natural simplicity and elegance of the ray tracing algorithm. Of course, a ray tracing algorithm can be written in a few lines of code (some algorithms fit on one side of a business card), but a high-performance ray tracing algorithm is a completely different matter.

David Luebke, an engineer at nVidia, made the following comment that perfectly reflects the reality: "Rasterization is fast, but you need to think carefully about how to do complex visual effects. Ray tracing supports complex visual effects, but you need to think carefully about how make it fast."

All you need to do is read a few articles about the optimizations that need to be made to the ray tracing algorithm to appreciate what Luebke said. For example, the most powerful ray tracing algorithms do not process rays independently; they use what are called ray sets, which allows performance optimization with rays that have the same origin and direction. This optimization is great for single instruction many data (SIMD) function blocks inside CPUs and GPUs, and is also very effective for core rays with some degree of coherence or shadow rays. But, on the other hand, the optimization is no longer suitable for refraction or reflection rays.

Moreover, as Daniel Pohl points out in in his article about Quake Wars RT, using ray sets can become problematic with transparent textures (the famous alpha textures used for trees), because if all the rays in a set do not behave the same way (some hitting the surface, others passing through it), then the additional overhead can become much higher greater than the benefits of optimizations provided by using ray sets.

Finally, as we already mentioned, ray tracing requires a suitable data structure to store the different elements of the scene, and it is this structure that will play a determining role in the final performance. But choosing and then working with such a data structure is not as simple as it seems at first glance. Some structures perform better for static data, while others can be updated more quickly for dynamic data or take up less memory. As usual, it all comes down to finding an acceptable compromise. There are no miracles.

Therefore, as we see, the ray tracing algorithm cannot always be called the ideal of simplicity and elegance, as some believe. To get good performance from a ray tracing algorithm, you need to find no less non-trivial programming solutions than in the case of obtaining complex visual effects using the rasterization method.

Now that we've cleared up some of the myths surrounding ray tracing, let's look at the real issues that come with this technology.

And we'll start with the main problem associated with this rendering algorithm: its slowness. Of course, some enthusiasts will say that this is no longer a problem, since the ray tracing algorithm is highly parallelizable and the number of processor cores is increasing every year, so we should see a linear increase in ray tracing performance. Additionally, research into optimizations that can be applied to ray tracing is still in its infancy. If you look at the first 3D accelerators and compare them with what is available today, there is indeed reason for optimism.

However, this viewpoint misses an important point: the most interesting thing about ray tracing is the secondary rays. In practice, calculating an image with only primary rays will not provide much improvement in image quality compared to the classic algorithm with a Z-buffer. But the problem with secondary beams is that they have absolutely no coherence (codirectionality). When moving from one pixel to another, completely different data must be calculated, which negates all the usual caching techniques that are very important for good performance. This means that the calculation of secondary rays is very dependent on the memory subsystem, in particular, on its delays. This is the worst-case scenario possible, since of all memory characteristics, latency has improved the least in recent years, and there is no reason to believe that the situation will improve in the foreseeable future. It's quite easy to increase memory bandwidth by using multiple chips in parallel, but the latencies will still remain the same.

For video cards, memory latency (latency) decreases much more slowly than bandwidth (bandwidth) increases. If the latter improves by a factor of 10, then the delays improve only by a factor of two.

The reason for the popularity of GPUs lies in the fact that creating hardware that specializes in rasterization has proven to be a very effective solution. With rasterization, memory accesses are performed coherently (in parallel), regardless of whether we are working with pixels, texels, or vertices. Therefore, small caches, complemented by serious memory bandwidth, are the ideal solution for excellent performance. Of course, increasing throughput is very expensive, but this solution is quite suitable if it pays off. In contrast, today there are no solutions to speed up memory access when computing multiple rays. This is the reason why ray tracing will never be as efficient as rasterization.

Another common problem with the ray tracing method concerns anti-aliasing (AA). The rays are drawn as a simple mathematical abstraction, and they do not take into account the real size. The triangle intersection test is a simple Boolean function that gives a yes or no answer, but doesn't give details like "the ray intersects the triangle 40% of the way." A direct consequence of this effect will be the appearance of “ladders”.

To solve this problem, several technologies have been proposed, such as beam tracing and cone tracing, which take into account the thickness of the rays, but their complexity prevents efficient implementation. And the only technology that can give good results is the calculation of more rays than there are pixels, that is, supersampling (rendering at a higher resolution). It’s hardly worth mentioning once again that this technology is much more computationally expensive than the multisampling used in modern GPUs.

Hybrid rendering engine?

If you have read the entire article up to this point, you are probably already thinking that ray tracing cannot yet replace rasterization, but perhaps it is worth mixing the two technologies together? And at first glance it seems that the two technologies complement each other. It's easy to imagine rasterizing triangles to define what's visible, taking advantage of the technology's excellent performance, and then applying ray tracing to just some surfaces, adding realism where it's needed, such as adding shadows or getting good reflections and transparency. Actually, Pixar used this approach for the cartoon "Cars". Geometric models are created using REYES, and ray tracing is used "on demand" where certain effects need to be simulated.

Unfortunately, although this method sounds promising, hybrid solutions are not so easy to implement. As we have already seen, one of the important disadvantages of the ray tracing method is the need to organize the data structure in such a way as to limit the number of checks for the intersection of rays and objects. And using a hybrid model instead of pure ray tracing doesn't change that. It is necessary to organize the data structure along with all the disadvantages that come with it. For example, suppose that the ray tracing method is based on static data, but the dynamic data is rendered through rasterization. But in this case we lose all the benefits of ray tracing. Since there is no dynamic data for ray tracing, you won't be able to force objects to cast shadows or see reflections.

Moreover, when it comes to performance, the biggest problem is the memory access associated with secondary rays, and these rays are exactly what we need in our hybrid rendering engine. So the performance gain won't be as big as you might initially expect. Since most of the rendering time will be spent on secondary rays, the gain from not rendering the main rays will be negligible.

In other words, by trying to combine the advantages of both methods, we inevitably combine their disadvantages, losing the elegance of the ray tracing method and the high performance of rasterization.

As we've mentioned several times in our article, there are many challenges to overcome before ray tracing can become a viable alternative to rasterization in real-time rendering. And if you think about it, will this method become a panacea for all ills? The benefits of ray tracing are not so revolutionary as to justify the significant performance penalty. The algorithm's strengths relate to reflections and transparency, since these two effects are the most difficult to achieve with existing rasterization algorithms. But, again, is this such a serious drawback? The world around us does not consist entirely of very transparent or shining objects, so our vision may well be satisfied with a rough approximation.

If you look at the latest car simulators, for example, Gran Turismo and Forza, you can clearly see quite satisfactory rendering quality, even if the reflections on the body are completely false. And the exact reflection of the rear view mirror on the paint can hardly be considered sufficient to recognize another step towards photorealism.

Most enthusiasts believe that ray tracing inherently produces a better image than rasterization - but they often base their opinion on the image produced by an offline, non-real-time engine. However, the results of such engines are much better than the capabilities of modern games. Additionally, there is some confusion around ray tracing. Enthusiasts often compare photorealistic images, which are produced by a combination of several techniques, to rasterization, such as ray tracing for direct reflections, radiosity for diffuse reflection, photon mapping for caustics, etc. All these technologies combine to provide the most photorealistic quality possible.

In its basic version, ray tracing, when considering existing real-time implementations, is only suitable for perfect reflections and hard (sharp) shadows. Doom 3 proved a few years ago that it was possible to create a reliable 3D engine that could handle dynamic shadows and rasterization perfectly, but in retrospect, the game also showed that hard shadows were not realistic.

To create soft shadows or diffuse reflections (the kind you see on textured metal, for example), more advanced ray tracing techniques such as path tracing and or distributed ray tracing are required. But such techniques require a significantly larger number of rays, so they are still poorly suited for real time.

Some users believe that sooner or later so much processing power will be available that the performance advantage of rasterization will no longer be a decisive factor. Due to the law of diminishing returns, the performance gains from rasterization will quickly be forgotten in favor of the elegance of ray tracing. In much the same way as before, the productivity benefits of assembly language coding were forgotten, which turned out to be insufficient to outweigh the benefits of high-level languages.

However, this is unlikely to convince us. In any case, we are still far from the time when we can sacrifice performance for elegance and simplicity. Just look at what has happened in the last 10 years in the world of offline rendering. While one frame of Toy Story took an average of two hours to render, one frame of Ratatouille took six and a half hours, despite the increased processing power between the two films. more than 400 times. In other words, the more computing power and resources you provide to computer artists, the faster they consume them.

If even a company like Pixar, which can afford to dedicate several hours of computation to creating a single frame, decides to use ray tracing only occasionally due to the negative impact on performance, this means that the time before we will have sufficient processing power in 3D -real-time games to perform all rendering using ray tracing are very, very far away. And in the future, enthusiasts will certainly have somewhere to spend such computing power.

Ray tracing techniques are considered to be the most powerful methods for creating realistic images today. The versatility of tracing methods is largely due to the fact that they are based on simple and clear concepts that reflect our experience of perceiving the world around us.

Let's look at how an image is formed. The image is produced by light entering the camera. Let's release many rays from the light sources. Let's call them primary rays. Some of these rays will fly away into free space, and some will hit objects. The rays can be refracted and reflected on them. In this case, part of the beam energy will be absorbed. Refracted and reflected rays form many secondary rays. Then these rays will again be refracted and reflected and form a new generation of rays. Eventually, some of the rays will hit the camera and form an image.

There are algorithms that work according to this algorithm. But they are extremely ineffective, since most of the rays emanating from the source do not reach the camera. But an acceptable picture is obtained if you trace a large number of rays, which will take a very long time. This algorithm is called direct ray tracing.

The reverse ray tracing method can significantly reduce the search for light rays. This method was developed in the 1980s by Whitted and Kaye. In this method, rays are tracked not from sources, but from the camera. Thus, a certain number of rays are traced, equal to the resolution of the image.

Let's assume that we have a camera and a screen located at a distance h from it. Let's divide the screen into squares. Next, we will take turns drawing rays from the camera to the center of each square (primary rays). Let's find the intersection of each such ray with scene objects and select the one closest to the camera among all intersections. Next, by applying the desired lighting model, you can obtain an image of the scene. This is the simplest ray tracing method. It only allows you to cut off invisible edges.

But we can go further. If we want to simulate phenomena such as reflection and refraction, we need to launch secondary rays from the closest intersection. For example, if a surface reflects light and it is perfectly flat, then it is necessary to reflect the primary ray from the surface and send a secondary ray in this direction. If the surface is uneven, then it is necessary to launch many secondary rays. This is not done in the program, as this will greatly slow down the tracing.

If the object is transparent, then it is necessary to construct a secondary ray such that when refracted it produces the original ray. Some bodies may have the property of diffuse refraction. In this case, not one, but many refracted rays are formed. As with reflection, I neglect this.

Thus, the primary ray, having found an intersection with the object, is generally divided into two rays (reflected and refracted). Then these two rays are divided into two more and so on.

The main reverse ray tracing procedure in my program is the Ray procedure. It has the following structure:

If the beam generation is equal to the maximum recursion depth, then we return the average brightness over all components. If not, then move on

We determine the nearest triangle with which the ray intersects.

If there is no such triangle, return the background color; if there is, move on.

If the surface with which the intersection was found is reflective, then we form a reflected ray and call the Ray procedure recursively with the ray generation increased by 1.

If the surface with which the intersection was found refracts, then we form a refracted ray and call the Ray procedure recursively with the ray generation increased by 1.

We determine the final illumination of the pixel, taking into account the location of the sources, the properties of the material, as well as the intensity of the reflected and refracted beam.

I've already discussed a number of limitations of the tracing method when we talked about diffuse refraction and uneven mirrors. Let's look at some others.

Only special objects - light sources - can illuminate the scene. They are point-like and cannot absorb, refract or reflect light.

The properties of a reflective surface consist of two components - diffuse and specular.

With diffuse reflection, only rays from light sources are taken into account. If the source illuminates a point through a mirror (with a bunny), then it is considered that the point is not illuminated.

Specularity is also divided into two components.

reflection - takes into account reflection from other objects (not light sources)

specular - takes into account glare from light sources

The tracing does not take into account the dependences on the wavelength of light:

refractive index

absorption coefficient

reflection coefficient

Since I'm not modeling diffuse reflection and refraction, I won't be able to get backlighting. Therefore, we introduce minimum background illumination. Often it simply allows you to significantly improve image quality.

The tracing algorithm allows you to draw very high-quality shadows. This will not require much reworking of the algorithm. You'll have to add something to it. When calculating the illumination of the points, it is necessary to place a “Shadow Front” in each of the light sources. The "shadow front" is a ray that checks whether there is anything between the point and the source. If there is an opaque object between them, then the point is in the shadow. This means that this source does not contribute to the final illumination of the point. If a transparent object is lying, then the intensity of the source decreases. Drawing shadows is very time-consuming. So, in some situations they are disabled.

My program has the ability to enable image smoothing. Antialiasing is about determining the color of a pixel. Not one ray, but four, is launched and the average color value of these rays is determined. If it is necessary to find the color of a pixel (i,j), then 4 rays are sent to points on the screen plane with coordinates (i-0.25,j-0.25), (i-0.25,j+0.25), (i+0.25,j-0.25) , (i+0.25,j+0.25).

Ministry of Education of the Russian Federation

Moscow State Institute of Electronics and Mathematics

(Technical University)

Department of Information and Communication

technologies

Coursework on the topic:

“Analysis of the prospects of using the ray tracing method in 3D modeling”

Completed:

Guliyan Boris

Podzorov Ivan

Group C -35

Moscow 2010

1. 3D graphics. Introduction

3. Ray tracing algorithms

4. The main advantages and disadvantages of ray tracing

5. Application of ray tracing method

6. Experiment.

Task: "Analysis of the prospects of using the ray tracing method in 3D modeling"

Formulation of the problem

Familiarize yourself with the ray tracing method and its use in the field of 3D graphics, conduct an experiment using one of the ray tracing algorithms.

In our experiment we consider:
1) the performance of the ray tracing algorithm depending on the number of polygons of the model (3 balls are taken as a model: matte, transparent and mirror).

2)Analysis of the resulting images with and without ray tracing.

Blender software is used as the environment for the experiment.

3D graphics. Introduction.

Three-dimensional graphics is a section of computer graphics, a set of techniques and tools intended for depicting three-dimensional objects. It is most used to create images on the plane of a screen or sheet of printed material in architectural visualization, the entertainment industry, printed materials, as well as in science, industry and augmented reality technology.

Any 3D image is determined by the following parameters and objects:

· Geometry (constructed by medels)

· Materials (information about the visual properties of the model)

· Light sources (direction, power, lighting spectrum settings)

· Virtual cameras (selection of point and projection angle)

· Forces and impacts (settings for dynamic distortions of objects, used mainly in animation)

· Additional effects (objects simulating atmospheric phenomena: light in fog, clouds, flames, etc.)

The task of three-dimensional modeling is to describe these objects and place them on the scene using geometric transformations in accordance with the requirements for the future image.

The main problem of 3D graphics and modeling is obtaining the most photorealistic image with minimal expenditure of computer resources and time for processing the scene. Since there are different needs in different areas, different ideas and algorithms are created to solve a specific problem. One such idea is ray tracing, which we will look at in our work.

Forward and backward ray tracing

Ray tracing is a method of processing 3D models to produce a photorealistic image, which takes into account the relative position of objects, as well as such physical properties of objects as reflective and refractive ability.

There are 2 ray tracing methods: forward and backward

Direct ray tracing considers all rays from light sources that strike objects and ultimately reach the observer's eye. But this method is not rational from a performance point of view, because it has to process all the rays of the environment (outgoing and refracted) in all directions, including those that do not fall on the scene visible to the observer.

In reverse ray tracing, all rays originate from the observer's eye, thereby defining a scene with objects that will be processed later. This method allows you to avoid processing objects that do not fall within the visible area, which significantly reduces the amount of necessary calculations.

All ray tracing algorithms are based on the inverse ray tracing method.

Ray Tracing Algorithms

Let's consider the basic tracing algorithm (Fig. 1). Let's take the sphere as an object.

1. For each pixel on the screen, a beam is fired from the observer's eye.

2. After the ray intersects the object, it is determined:

· Transparency/opacity of the object. If the object is transparent, then a refraction ray is emitted from the intersection; if it is opaque, it is not emitted.

· Light/shade. From the point where the ray intersects the spheres, a ray is emitted to the light source (or alternately for each light source, if there are several of them). If this ray does not intersect other opaque objects or surfaces, then the light source directly affects the illumination of a given point. If there are several light sources, then the effect determined by the RGB value of a given point is calculated from the influence of all rays.

· Reflectivity. If an object is capable of reflecting rays, then from the point where the ray intersects the sphere, a reflected ray is emitted to the objects that will be reflected in the sphere.

As a result, we get several types of rays. Primary rays are used to determine the visibility of an object, and secondary rays are divided into the following:

· refraction rays;

· rays of shadow/light;

Reflection rays.

Rice. 1 Ray tracing algorithm diagram

All other algorithms are based on the algorithm shown above and are designed to optimize the calculations.

kd-tree

The algorithm for constructing a kd-tree can be presented as follows (we will call a rectangular parallelepiped the English word “box”).

1. "Add" all primitives to the bounding box. That is, build a box that limits all primitives, which will correspond to the root node of the tree.

2. If there are few primitives in a node or the tree depth limit has been reached, complete the construction.

3. Select the split plane that divides the given node into two child nodes. We will call them the right and left nodes of the tree.

4. Add primitives intersecting with the left node box to the left node, primitives intersecting with the right node box to the right one.

5. For each node, perform this algorithm recursively starting from step 2.

Regular grid

The entire 3D space is divided into a fine regular grid consisting of N*N*N cubes. The idea is that you can only run through those cubes through which the beam went.

The method is not used in practice.

Dadvantages and disadvantages

In addition to the fact that the ray tracing method gives the most photorealistic picture, it has a number of other advantages:

1. The ability to render smooth objects without interpolating them with polygonal surfaces (for example, triangles).

2. The computational complexity of the method weakly depends on the complexity of the scene.

3. High algorithmic parallelizability of calculations - you can trace two or more rays in parallel and independently.

4. With the ray tracing method, reflections are displayed perfectly (Fig. 2), and without complex algorithms, since everything is calculated by the main rendering algorithm.

font-size:14.0pt"> Fig. 2 Reflections of two mirror balls in each other

The ray tracing method has disadvantages observed in all algorithms that determine the scope of use of this method.

1. The main disadvantage of this rendering algorithm is its slowness. However, the ray tracing algorithm is highly parallelizable and the number of processor cores is increasing every year, so we should see linear growth in ray tracing performance. But this approach does not take into account secondary rays (reflections, refractions and shading detection), and rendering with primary rays provides virtually no improvement in image quality compared to the classic algorithm.

2. The problem with secondary rays is that they have absolutely no coherence (codirectionality). When moving from one pixel to another, completely different data must be calculated, which negates all the usual caching techniques that are very important for good performance. This means that the calculation of secondary rays is very dependent on memory delays.

3. Lack of hardware support for the method (all GPUs specialize in rasterization).

4. Another common problem with the ray tracing method concerns anti-aliasing (AA). The rays are drawn as a simple mathematical abstraction, and they do not take into account real size. The triangle intersection test is a simple Boolean function that gives a yes or no answer, but doesn't give details like "the ray intersects the triangle 40% of the way." A direct consequence of this effect will be the appearance of “ladders” (Fig. 3).

Rice. 3 shadow smoothing

And the only technology that can give good results is the calculation of more rays than there are pixels, that is, supersampling (Oversampling or Anti-Aliasing) (rendering at a higher resolution).

You should also remember that the rendering speed and quality using ray tracing is highly dependent on code optimization.

Using Ray Tracing Method

Due to its features (photorealistic image, slow calculations), this method is used in areas where the quality of the image is important, and not the time of its rendering (in this case, combined rendering methods are most often used, which improves performance). These areas include:

· 3D animation;

· Film industry special effects;

· Realistic photo rendering;

· Cad systems.

Special terms:

A polygonal mesh is a collection of vertices and polygons that defines the shape of the displayed object.

Rendering - (English rendering - “visualization”) - the process of obtaining an image from a model.

Here, a model is a description of any objects or phenomena in a strictly defined language or in the form of a data structure. Such a description may contain geometric data, the position of the observer’s point, information about lighting, the degree of presence of some substance, etc.

Figure 4. polygonal mesh

Experiment.

We chose the Blender 3D editor as the software for conducting the experiment.

It is quite easy to learn and contains all the necessary functions:

· Rendering an image with the ability to connect and disconnect the tracer.

· Oversampling (anti-aliasing or anti-aliasing)

We measured the time required to render 3 different spheres (glass, mirror and matte) on different Multeris Equals (each level increases the number of polygons by 4 times). When the level increased, the time was counted from 0.

0 " style="margin-left:48.35pt;border-collapse:collapse">

Lv. Multeris

Rendering time for each level. from 0

Without RayT [c]

With RayT [c]

0,53

3,36

0,46

0,54

2,84

0,55

3,02

0,61

3,85

0,96

5,96

10,64

29,12

43,9

Table 1.

Rendering was done with maximum parameters to increase the difference in processing speed.

As a result, we see that the time spent processing three spheres with level 4 (256 polygons on each sphere) is less than the time spent processing spheres with level 2 (16 polygons each).

Figure 5. Polygonal meshes for different levels

Bottom line

From the experiment performed, it can be seen that the time spent on rendering 3 balls using ray tracing is significantly greater than the time spent on rendering without using ray tracing. But during the experiment, an interesting observation was noticed: the time for processing 3, 4 and 5 level models is less than the time for processing a two-level model.

Analysis of the resulting images:
1) In the picture obtained without using ray tracing (hereinafter A), it is clear that the transparent sphere does not give the effect of a lens (using an alpha channel), while in the picture using ray tracing (hereinafter B) the transparent ball magnifies objects beyond him (Fig. 6).

Rice. 6 transparent spheres (alpha channel on the left, ray tracing on the right)

2) In picture A there is no mirror ball, because obtaining a reflection on it is based on ray tracing (Fig. 7).

Figure 7. Experiment model (alpha channel above, ray tracing below).

3) Figure 8 shows that when rendering without using ray tracing, internal cavities are illuminated, where, logically, light should not penetrate.

Fig. 8 Incidence of light on a concavity in a ball (A on the left, B on the right)

From this analysis it is clear that the quality of images using ray tracing is significantly better than images obtained without it, which justifies the use of this method in areas where the quality of the resulting image, rather than the processing time, is important.

Ray tracing and rasterization - what's the difference?

We are not sure that all our readers know or remember what it is ray tracing, how different rendering methods differ in principle and what their advantages and disadvantages are. Therefore, first of all, let’s try to talk about it very briefly, without complex mathematics and more or less clearly. Before we move on to ray tracing, we need to remember the basics of the classic algorithm rasterization with Z-buffer.

In the rasterization method, now generally accepted in modern real-time graphics, to draw each object, there is a projection onto the screen plane from the geometric primitives (polygons, most often triangles) that make up the object. Triangles are drawn pixel by pixel using a depth buffer, which contains the distance to the screen plane and is necessary to ensure that the triangles closest to the camera overlap those farther during rendering.

In addition to the vertices and the polygons connecting them, it also stores information about the color, texture coordinates, and normals needed to define the front and back portions of each surface. Pixel color is determined through complex calculations in vertex and pixel shaders, and effects such as shadows are rendered using additional passes, but also using rasterization.

Shading process ( shading) consists of calculating the amount of lighting for a pixel, taking into account the overlay of one or more textures on the pixel, which determines its final color. All this requires a lot of calculations, because the scenes of modern games contain several million polygons and several million pixels on high-resolution screens, and information on the screen must be updated at a rate of at least 30 frames per second, and better yet, 60 FPS. Not to mention virtual reality helmets, where you need to simultaneously draw images for both eyes at a frequency of 90 FPS.

But since GPUs operate at very high clock speeds and have a large number of hardware units specialized in certain calculations, and rasterization lends itself very well to parallelization, there are no particular problems with rendering performance, and the vast majority of 3D computer games use rasterization. In reality, things are a little more complicated, since many additional optimizations are used in order to avoid drawing a lot of invisible triangles, but this is the essence of rasterization in general.

During the development of GPUs throughout their development, a huge amount of time has been spent optimizing the work of discarding invisible geometry and reducing the computational load. First, they discarded the rendering of objects out of sight, then objects obscured by others, closer to the camera, etc. The developed optimizations for rasterization are quite effective; in modern games, invisible objects consume almost no computing resources, significantly reducing the amount of work required to rasterize the scene. You will understand further why we touch on the topic of invisible objects.

To calculate global illumination, draw shadows and other effects, you have to use cunning hacks based on the same rasterization. As a result, over the years, GPUs have become very sophisticated, learning to speed up geometry processing in vertex shaders, render pixels efficiently using pixel shaders, and even use universal compute shaders to calculate physics, post-effects, and many other calculations. But the basis of the GPU operation remained the same all the time.

Ray tracing has a completely different basic idea, but in theory it’s almost simpler. Tracing simulates the propagation of light rays across a 3D scene. Ray tracing can be performed in two directions: from light sources or from each pixel in the opposite direction, then usually several reflections from scene objects are determined in the direction of the camera or light source, respectively. Calculating rays for each pixel in the scene is less computationally demanding, and projecting rays from light sources gives higher rendering quality.

Backtrace was first described in 1969 by an IBM employee in the work "Some Techniques for Shading Machine Renderings of Solids" and this technique calculates the path of the light beam for each pixel on the screen depending on the 3D models in the scene. 10 years later, there was another breakthrough in technology, when researcher Turner Whitted (now working at Nvidia Research, by the way) published a paper "An Improved Illumination Model for Shaded Display", which showed how to calculate shadows, reflection and refraction of light when tracing.

A couple of other papers in the 1980s further described the fundamentals of ray tracing for computer graphics, which led to a revolution in synthetic imaging in the film industry. So, in 1984, several employees Lucasfilm described how to use ray tracing to create effects such as motion blur, depth of field, soft shadows, blurred reflections and refractions. A couple of years later, Caltech professor Jim Kajiya in his work "The Rendering Equation" described a more precise way of scattering light in a scene. And since then, ray tracing has been used literally everywhere in the film industry.

So, in the common method of inverse ray tracing, for each pixel on the screen, an imaginary ray is drawn from the camera to an object in the scene. In this way, a ray of light entering the camera from a light source in this direction is simulated, and the first intersection with the object is used to determine the color of the pixel. Primary rays determine the visibility of objects (like a Z-buffer in rasterization), and to determine color, you need to draw further secondary rays from the intersection points to different light sources (if the rays are blocked by an object, then the light source will not affect the illumination of the pixel), and the set of secondary rays is determined by the illumination falling on the pixel.

But all the most interesting things happen even further - to achieve photorealism, you need to take into account the characteristics of materials in the form of the amount of light reflected and refracted by them, and to calculate the color of a pixel, you need to draw more reflection and refraction rays. They are not indicated in the figure above, but they can be mentally imagined as rays reflected from the surface of the ball and refracted by it. This improved ray tracing algorithm was invented several decades ago, and these additions were a big step towards increasing the realism of a synthetic image. To date, the method has acquired many modifications, but they are always based on finding the intersection of light rays with scene objects.

The first practical experiments in implementing real-time ray tracing began quite a long time ago, at a well-known conference SIGGraph Similar developments appeared frequently. Demonstrations of real-time tracing date back to the late 1980s and achieved speeds of several frames per second using highly optimized techniques and multiple shared memory computing systems. Since then, many developments have appeared designed to speed up tracing for work, including on one PC.

Not to mention the numerous 3D engine enthusiasts in the demo scene in the late 90s and beyond who were inspired by the capabilities and fundamental simplicity of the method, bringing many useful optimizations to ray tracing. At one time, we published on our website a whole series of materials devoted to one of the ray tracing software engines, very specific and with a lot of serious limitations that did not allow us to create serious game projects based on it:

Hardware manufacturers did not lag behind, and for a long time they had been showing experimental prototypes of tracing accelerators and demo programs optimized for them at exhibitions. So, in June 2008 the company Intel showed a special version of the game Enemy Territory: Quake Wars (Quake Wars: Ray Traced), which uses ray tracing when rendering at a resolution of 1280x720 at a speed of 15-30 frames per second, which is already considered real time. That demonstration did not use hardware accelerators, but ran on 16 Xeon cores at a frequency of 3 GHz.

The Intel project demonstrated the benefits of ray-traced rendering, demonstrating realistic water, object shadows through transparent surfaces, and reflections. The development of the demonstration became the project Wolfenstein: Ray Traced, and various enthusiasts often take the engine of the series Quake to add tracing - so at the suggestion of modders in Quake 2 realistic reflections appeared, which were spoiled by very strong noise and the highest system requirements.

And the company showed prototypes of hardware tracing accelerators for several years (from 2012 to 2016). Imagination Technologies, which even offers an open API for ray tracing - OpenRL. It was stated that the company's hardware development accelerator is capable of running Autodesk Maya and providing real-time ray tracing. However, the company did not have enough funds to promote hardware acceleration of ray tracing for success, as well as the “weight” of this company in the graphics market to be its locomotive. And the demo programs were not the most impressive, to be honest, although they showed some of the advantages of tracing:

The company did much better Nvidia, which announced the technology back at SIGGraph 2009 OptiX, designed for real-time ray tracing on their GPUs. The new API has opened access to ray tracing in professional applications with the necessary flexibility, in particular - bidirectional path tracing and other algorithms.

Renderers based on OptiX technology already exist for numerous professional software, such as Adobe AfterEffects, Bunkspeed Shot, Autodesk Maya, 3ds max and other applications, and are used by professionals in their work. This can be attributed to real-time rendering only with certain assumptions, because at a high frame rate the result was a very noisy picture. Only a few years later the industry came close to using hardware acceleration of ray tracing in games.

Pros and cons of ray tracing

The ray tracing rendering technique is highly realistic compared to rasterization, as it simulates the propagation of light rays very similar to how they occur in reality (of course, still not 100% accurate). Tracing can produce highly realistic shadows, reflections and refractions of light, and as such has long been valued in architectural applications and industrial design. The technology helps specialists in this field, long before physical implementation, understand how materials will look under different lighting conditions in the real world.

The obvious advantages of tracing can also include the fact that the computational complexity of the method depends little on the geometric complexity of the scene, and the calculations are perfectly parallelized - you can easily and independently trace several rays at the same time, dividing the screen surface into zones for tracing them on different computing cores. It is also very useful that cutting off invisible surfaces is a logical consequence of the algorithm.

But what is more important is that the method simulates the real propagation of light rays, obtaining a final image of higher quality compared to rasterization. Rasterization has obvious disadvantages - for example, an object not included in the scene will not be rendered on the GPU, but it can cast a visible shadow or should be visible in a reflective surface (mirror), and rasterization optimizations discarded it and did not take it into account. Not to mention, this invisible object can greatly affect the global illumination of the scene by reflecting light onto visible surfaces. These problems are partially solved, in particular, the use of shadow maps allows you to draw shadows from objects invisible in the scene, but the resulting picture is still far from ideal. And the point here is in principle, because rasterization works completely differently than human vision.

Effects such as reflections, refractions and shadows, which are quite difficult to implement well when rasterizing, are a natural result of the ray tracing algorithm. Take reflections - this is just one of the areas in which ray tracing is noticeably better than rasterization. In modern games, reflections are usually simulated using environment maps (static or dynamic) or reflections in screen space ( Screen-Space), which give a good simulation of reflections in most cases, but still have very large limitations, in particular, they are not suitable for closely located objects.

Calculating reflections in screen space makes it possible to obtain reflections that are more or less similar to the truth, subject to some restrictions, but with hardware acceleration on the GPU using rasterization. And with ray tracing, reflections are always rendered perfectly without the need for additional complex algorithms. Another important advantage of tracing is displaying reflections of parts of the same object on each other (for example, so that the handle of a teapot or its spout is reflected on itself), which is much more difficult to do using rasterization.

Another example of a clear advantage of ray tracing is the rendering of transparent objects. Using rasterization, it is very difficult to simulate transparency effects, since its calculation depends on the rendering order and for this you have to pre-sort transparent polygons, and even then visual artifacts may appear. Several hacks have been invented to bypass polygon sorting, but this all results in complications of the method and additional complications. But the ray tracing algorithm itself allows you to draw any transparency effects with ideal quality.

Well, the last (for starters) example is drawing shadows. When rasterizing in most cases, shadow maps are used ( shadow mapping), which are also based on rasterization, simply rendering is done from a different point in the scene and with different parameters. The silhouettes of the object are drawn into a separate buffer from the light source, the contents of the buffer are filtered and superimposed on the surface where the shadow should be cast. These methods have several problems, including the jagged edges you've all seen in games, as well as increased video memory consumption. Ray tracing allows you to solve the problem of shadows automatically, without requiring additional algorithms and memory. Moreover, in the case of a rasterization hack, the result will be a physically incorrect shadow in any case, but a soft shadow drawn using ray tracing will be realistic.

But ray tracing also has a disadvantage. One, but very important, is that drawing everything described above is several times more difficult from a computational point of view. Low performance on existing hardware is the main drawback of the tracing method, which for a long time negated all its advantages. Finding the intersection of rays with scene objects is not accelerated as easily as relatively simple operations when rasterizing triangles, for which special 3D accelerators have been used for many years, which is why in real-time graphics the rasterization method is still used, which allows you to draw a picture quite quickly, although somewhat inferior in quality to a full-fledged tracing, it is still quite realistic.

When tracing, you need to calculate thousands of rays for each light source, most of which will have little effect on the final image, so you need both additional optimizations for the ray tracing algorithm and new hardware that can speed up the corresponding operations. Plus, the use of tracing alone does not guarantee photorealism. If you use simple algorithms, the result will be good, but still not realistic enough, and to fully simulate reality you need to use additional techniques, such as photon mapping And path tracing, which more accurately simulate the propagation of light in the world.

On the other hand, since the ray tracing algorithm is well parallelized, it can be solved by the simplest technical method - increasing the number of computing cores of the (graphics) processor, the number of which increases every year. At the same time, a linear increase in performance during tracing is ensured. And given the apparent lack of optimization in both hardware and software for ray tracing on GPUs right now, hardware ray tracing capabilities can potentially grow rapidly.

But here smaller problems arise. Rendering only the primary rays in itself is not too complicated, but it will not provide a noticeable improvement in rendering quality compared to classical rasterization, and even with tricky hacks. But secondary rays are much more difficult to calculate because they do not have coherence - unidirectionality. For each pixel, completely new data has to be calculated, which is not very good for caching it, which is important for achieving high speed. Therefore, the calculation of secondary rays is highly dependent on memory delays, which are almost not decreasing, in contrast to memory bandwidth (MBB), which is growing at a rapid pace.

Ray tracing, although it seems like a fairly simple and elegant method that can be implemented in just a few lines of code, is a completely unoptimized algorithm, and high-performance ray tracing code is extremely difficult to make. If with rasterization the algorithm works quickly, but you have to come up with clever methods for complex visual effects, then ray tracing can draw them all initially, but forces you to very carefully optimize the code so that it runs fast enough for real time.

There are many methods for speeding up tracing; the most productive ray tracing algorithms do not process rays one at a time, but use sets of rays, which speeds up the process of processing rays of the same direction. These optimizations are great for running on modern SIMD CPUs and GPUs, they are effective for main co-rays and shadow rays, but still not suitable for refraction and reflection rays. Therefore, it is necessary to seriously limit the number of rays calculated for each pixel of the scene, and to remove the increased “noise” of the image using special filtering.

In addition, ray tracing requires a suitable data structure to store scene elements, and this can have a performance impact. Some structures are better suited for static data, others for dynamically changing ones. So ray tracing only at a superficial glance seems to be a simple and extremely elegant method, but to get the desired performance from it you will have to do a lot of optimization work - no less than simulating complex effects during rasterization. And this work has just begun, in fact.

There are several issues that need to be addressed before ray tracing becomes a viable alternative to rasterization for games. Right now, it appears that the benefits of tracing are not as great as the significant performance penalty associated with using it. Yes, tracing has very important advantages in the form of realistic reflections, shadows and processing of transparent objects, which is difficult to do with rasterization, but... are there enough such objects in games for the lack of realism for them to become serious? On the one hand, most objects in the world reflect light, on the other hand, games have proven that our eyes and brain are content with just approximating realism. In most modern games, reflections on objects, although not completely photorealistic, are often enough to trick our brain.

Yes, ray tracing can provide better quality than rasterization, but by what means? If you strive for complete realism, then full-fledged tracing with the calculation of many rays for lighting and reflections, as well as a combination of techniques like radiosity And photon mapping, will be super-demanding of computing power. Often, even offline renders that do not work in real time use simplifications. Of course, after some time, sufficiently high computing power will become available in order to gain an advantage over rasterization, including in terms of performance, but for now we are still very far from this point.

Even with offline rendering for the film industry, as computing power increases, rendering times do not decrease over time, as the appetites of artists grow even faster! And even leading companies in the production of animated films, like Pixar, try to optimize the rendering process by using ray tracing only for some of the effects - precisely because of the significant impact on performance. So you need to understand that the days of full-fledged tracing for the entire scene in real-time games are still very far away. And for full-fledged real-time rendering using ray tracing in games, the computing power is definitely not enough yet. This is a long way to go even with the development of GPUs that is still ongoing.

But in any case, ray tracing is the physically correct way that can solve many large and small problems of the existing approach. Using various hacks and tricks currently used in rasterization, you can achieve good results, but this certainly cannot be called a universal and ideal method for rendering 3D graphics. Soon enough, in the pursuit of realism, real-time 3D application developers will reach the limit of the existing rasterization method, and they will have to switch to a method with an advanced lighting model similar to what happens in reality. Most likely, it will be ray tracing. But since ray tracing is a very expensive method and is unlikely to be supported by even the most powerful systems, you should initially rely on hybrid rendering methods that combine rasterization performance and ray tracing quality.

Hybrid rendering for transition

Due to the demanding nature of ray tracing, even with a small number of rays to be calculated for each pixel, this method is unlikely to be used exclusively and will not yet replace rasterization. But there is an option to mix the two methods. For example, the underlying geometry can be rasterized with high performance, and then only soft shadows and reflections can be rendered using ray tracing. Although rasterization will continue to play a critical role in the coming years with the advent of hybrid rendering, the share of ray tracing algorithms in such engines will gradually increase based on the increasing computing capabilities of future GPUs.

This approach has long been used in the same cartoons of the company Pixar, despite the fact that their requirements seem to have no strict restrictions on rendering time. However, it is easier and faster to render geometry using the same micropolygons of the rendering system Reyes, and use tracing only where specific effects are needed. Almost all Pixar animated films previously used micropolygons and rasterization, and ray tracing to the rendering engine RenderMan added later for the cartoon "Cars", where it was used selectively - to calculate global occlusion (ambient occlusion) and render reflections.

But in reality, hybrid solutions are not so simple, because for effective ray tracing you need to organize the data structure in a special way to reduce the number of checks for the intersection of rays with scene objects. Therefore, even with hybrid rendering, you will have to create an optimized data structure. And on the performance side, a big issue is the memory access associated with the secondary rays that are needed for hybrid rendering. It turns out that when combining two rendering methods, many of their disadvantages are combined, in particular, the simplicity of the ray tracing method and high rasterization performance are lost.

But when the advantages still outweigh, such a hybrid approach makes sense. A combination of some rasterization and tracing capabilities is already available, including hardware-accelerated GPU lightmap preparation, rendering of dynamic lightmaps and partial shadows, rendering of reflections and translucent objects with refraction. This is already a great achievement, since this approach has been available only with offline rendering for many years. Back in the late 90s, hybrid rendering was used in animated films to improve efficiency, and now it is becoming available for real-time applications.

But this is just the beginning before the coming “Golden Era” of real-time rendering. In the future, this hybrid approach will evolve into something more, and instead of selective effects, it will be possible to use full-fledged techniques with advanced lighting, shading and various complex effects.

In much the same way that offline rendering went from "Bug's Life" to much more complex animated films, like "Coco", which already uses a full-fledged path tracing with tens, or even hundreds of calculated rays per pixel. Unlike previous years, there were no shadow maps or separate passes for calculating lighting, but only full-fledged tracing - this is what game developers are striving for, it’s just that their path will be a little longer, but the goal is the same.

And before the transition from rasterization to full tracing occurs, you need to use hybrid rendering and change your development approach in many ways. For example, outsource some of the work on preliminary preparation and “baking” some data in the GPU, remake your production pipeline and prepare rendering engines for the fact that an increasing part of the calculations will gradually switch to tracing. And partial benefits of tracing can be used now, albeit with an extremely small number of rays per pixel and with mandatory noise reduction.

But even with a gradual transition to tracing, there is no need to discard the need for optimizations that are not specific to rasterization. High-level optimizations like level of detail (LOD), occlusion culling, tiling, and streaming will also work great with ray tracing. Until the industry moves to full-fledged tracing, we need to continue to apply efficient techniques using screen space where high performance is required and quality is not critical.

Well, rendering using ray tracing needs to be optimized. For example, when rendering dynamic lightmaps using DXR, it is effective to cache the lighting in the lightmaps, and then use the accumulation of data over time for the next frame. The process is relatively fast and should be used because ray tracing in lightmap space provides better results compared to ray tracing in screen space. True, you will have to use noise suppression, since it will not be possible to calculate particularly many rays in real time.

Even ordinary filters for noise suppression with settings specifically for the specifics of ray tracing work well, and if you apply noise reduction using the capabilities of neural networks, which Nvidia has already demonstrated, and even hardware accelerated on the tensor cores of Volta architecture GPUs, then the future of hybrid rendering It seems pretty clear that at least some of the effects that can be easily added to existing engines (shadow calculations or global illumination and shading) that use rasterization will be added to games quite soon.

So, the obvious way to use hybrid rendering is to rasterize the scene and use ray tracing for only part of its lighting calculations, as well as for calculations of reflections with refractions. This approach provides rasterization speed and tracing quality in the form of accurate simulation of lighting, including global lighting, reflections and refractions of light rays, and drawing optically correct shadows. Moreover, simulating these effects using rasterization hacks and making them more complex will someday lead to the point where it becomes so resource-intensive that it will be easier to replace the calculations with real ray tracing. And in general, this is the only correct way if we look into the future of graphics development.

DirectX Raytracing - standard ray tracing API

So, over time, they learned to make rasterization very impressive by adding various algorithms and hacks, such as parallax mapping, which adds volume to not too complex surfaces, as well as using shadow maps. To improve graphics, it was only necessary to increase the speed of GPUs and make them a little more universal, leaving the basis in the form of rasterization practically untouched (not counting optimization methods in the form of breaking the frame into tiles, etc.).

Modern techniques like screen space reflections and global illumination simulation have pushed rasterization to its practical limits, as these algorithms require clever processing hacks and complex calculations, sometimes performed asynchronously with rendering. And in the near future, the complexity and resource intensity of such algorithms will continue to grow. Ray tracing allows you to do complex effects in a simple way, also opening the door to completely new techniques not previously possible in real-time rendering. But how can this be achieved if GPUs can only rasterize?

Current version DirectX 12 It only seems quite new, but in fact this graphics API was announced back at GDC 2014, and was released publicly as part of Windows 10 a year later. Until now, the use of this version is far from what was desired, and this happened for many reasons. Firstly, the development cycle for games and engines is quite long, and the fact that DirectX 12 only works on the latest version of Windows and has limited support on current generation consoles only reduces the arguments in favor of using it on PC. However, we have already seen the use of low-level APIs in several games, but what next? And then the DirectX development line took another sharp turn, introducing tools to support ray tracing.

As part of the Game Developers Conference GDC 2018 Microsoft has introduced a new addition to the DirectX API, in which many partners involved in software and hardware development have participated in one way or another. The addition is called DirectX Raytracing and its name suggests that it is a standard API for software and hardware support for ray tracing in DirectX applications, allowing developers to use algorithms and effects using the mentioned technique. DirectX Raytracing (DXR for short) provides a standardized approach for implementing ray tracing that is accelerated by GPUs. This extension combines with the capabilities of the existing DirectX 12 API, allowing you to use both traditional rasterization and ray tracing, as well as mix them in desired proportions.

All ray tracing-related work of the DXR API is controlled using lists of commands sent by the application. Ray tracing is tightly integrated with rasterization and compute commands and can be run multi-threaded. Ray tracing shaders (five new types of shaders!) are controlled similarly to compute shaders, allowing them to be processed in parallel on the GPU, controlling their execution at a relatively low level. The application is fully responsible for synchronizing the work of the GPU and the use of its resources, both during rasterization and calculations, which gives developers control over optimizing the execution of all types of work: rasterization, ray tracing, calculations, data transfer.

Different rendering types share all resources such as textures, buffers and constants without requiring conversion, transfer and duplication to be accessed from trace shaders. Resources storing ray tracing-specific data, such as acceleration structures (data structures used to speed up tracing - finding intersections between rays and geometry) and shader tables (describing the relationship between ray tracing shaders, resources and geometry) are entirely managed by the application, DXR itself The API does not do any data movement of its own accord. Shaders can be compiled individually or in batches, their compilation is completely controlled by the application and can be parallelized across multiple CPU threads.

At the highest level, DXR adds four new concepts to the DirectX 12 API:

1. Acceleration structure ( acceleration structure) is an object that represents a 3D scene in a format that is optimal for calculating rays on GPUs. Presented as a two-level hierarchy, this structure provides optimized ray rendering on the GPU and efficient modification of dynamic data.
2. New command list method ( command list) entitled DispatchRays is the basis for ray tracing in the scene. This is how the game transfers DXR workloads to the GPU.
3. A set of new ray shader types that define what DXR will calculate. When DispatchRays is called, the ray generation shader is launched. When using a new feature TraceRay in HLSL, the ray generation shader sends a ray into the scene, and depending on where the ray hits in the scene, one of several hit shaders can be called at the intersection point ( hit) or miss ( miss), which allows you to assign each object its own set of shaders and textures and create unique materials.
4. A tracing pipeline state added to existing graphics and compute pipeline states, translating ray tracing shaders and other states relevant to tracing workloads.

Thus, DXR does not add a new GPU engine to the existing graphics and compute engine in DirectX 12. The DXR workload can be run on existing engines, since DXR is a computational task at its core. DXR tasks are presented as compute workloads because GPUs are becoming more versatile anyway and are capable of performing almost any task that is not necessarily graphics related, and in the future most of the fixed functions of the GPU will likely be replaced by shader code.

When using DXR, the first step is to build acceleration structures at two levels. At the lowest level of the structure, the application defines a set of geometric data (vertex and index buffers) that define the objects in the scene. At the top level of the structure, a list of descriptions is defined that contains references to specific geometric data, as well as additional data such as transformation matrices, which are updated every frame, similar to how games do it to dynamically change objects. This ensures efficient traversal of a large number of complex geometries.

The second step when using DXR is to create a trace pipeline state. Modern games group draw calls ( draw calls) to increase the efficiency of their execution into special groups - packages ( batch), for example, drawing all metal objects in one batch, and all plastic objects in another. But when tracing, it is impossible to know in advance exactly what material a particular beam will hit, and batches cannot be applied. Instead, the routing pipeline state allows multiple sets of routing shaders and texture resources to be assigned. This way you can specify, for example, that all ray intersections with one object should use such and such a specific shader and such and such a texture, and intersections with another object should use a different shader and a different texture. This allows the application to use the correct shader code with the correct textures for the materials that are hit by the rays.

The last step in DXR is to call DispatchRays, which calls the shader to generate the ray. Inside it, the application makes calls to the TraceRay function, which causes the acceleration structure to be traversed and the corresponding shader executed on a hit or miss (two different types of shaders). TraceRay can also be called from within these two shaders when using ray recursion or effects with multiple bounces.

Why not use computational shaders already known to us from DirectX for ray tracing? Firstly, DXR allows you to run separate shaders when rays hit and miss, and secondly, the rendering process can be accelerated on GPUs (using Nvidia RTX or analogues from competitors), and thirdly, the new API allows you to bind resources using shader tables.

Nvidia RTX is a set of software and hardware algorithms that accelerate tracing on Nvidia solutions based on graphics architecture Volta. Why aren't previous architectures that aren't all that different from Volta supported? Perhaps this is partly a marketing ploy to attract buyers to new products, or perhaps there are some hardware optimizations in Volta that can seriously speed up ray tracing on the GPU, which we have not yet been told about. Yes, the only GPU with this architecture so far has tensor cores that accelerate artificial intelligence tasks, but if it can be used in rendering with ray tracing, then only in the process of noise reduction, and even then - according to available data, in existing noise reduction algorithms Such possibilities have not yet been applied.

DXR and RTX benefit from a powerful and flexible programming model, similar to Nvidia OptiX, that makes it relatively easy to write efficient ray tracing algorithms. To start developing applications using DXR ray tracing, hardware accelerated with RTX, you will need a graphics card based on the Volta architecture (currently only Titan V) and driver version 396 or higher, as well as the Windows 10 RS4 operating system and the Microsoft DXR developer kit containing everything you need. It will also be useful for debugging Microsoft PIX or NSight Graphics Nvidia companies that already have DXR API support.

For ease of development and debugging, Microsoft immediately released a new version of the utility PIX for Windows with support for DXR capabilities. This tool allows you to capture and analyze frames built using DXR so that developers understand exactly how DXR works with hardware, catch any errors and optimize their code. With PIX, programmers can examine API calls, view the state of objects and resources associated with tracing work, and view acceleration structures. All this helps a lot when developing DXR applications.

Ultimately, the DirectX Raytracing API complements the developer experience with specialized shaders and structures that are convenient for ray tracing, the ability to work simultaneously with the rest of the traditional graphics pipeline and compute shaders, etc. Conceptually, this is not much different from what Imagination Tech offered for several years back to OpenRL and its hardware solutions. Alas, ImgTec was too far ahead of its time with its PowerVR Wizard chips, but you need to have enough funds not only for initial development, but also for promoting your brainchild. DXR is an API of such a large and generally recognized company as Microsoft, and both manufacturers of gaming GPUs (Nvidia and AMD, and maybe Intel will soon be added to them, who knows) are already working together with Microsoft to optimize the new API for their hardware architectures .

Like all closed APIs, DXR also has a certain disadvantage in that the developer simply does not know how certain things work inside the API, what specific accelerating structures are used to ensure efficient parallel rendering on GPUs, and what are the advantages and disadvantages , what characteristics (memory consumption, latency, etc.), how the ray tracing scheduler works, is a balance achieved between memory usage, latency, register usage, etc., what part of the tracer’s work is performed in hardware on the GPU, and what in driver and API. All such solutions suffer from their closed nature, and DXR is no exception.

By the way, there is an alternative to using the DXR API - Nvidia employees are working on expanding the multi-platform Vulkan API, designed for ray tracing - VK_NV_raytracing. The development team collaborates with colleagues from Khronos to create a multi-platform open standard, and one of the main goals is to try to make ray tracing in DirectX and Vulkan work as similar as possible.

Games that use rasterization often look very believable and realistic, as their developers spent a lot of time adding all the necessary effects and algorithms that simulate the propagation of light rays in reality. And in the early years, DXR's capabilities will also be used to complement existing rendering techniques, such as screen-space reflections - to fill in data about hidden geometry not visible within the screen, which will lead to an increase in the quality of these effects. But over the next few years, you can expect to see an increase in the use of DXR for techniques that are not used in rasterization, such as full global illumination. In the future, ray tracing may completely replace rasterization when rendering 3D scenes, although rasterization will remain for a long time the carrier of the ideal balance between performance and quality.

At the moment, only Nvidia solutions of the Volta family (using RTX technology) have full hardware support for DirectX Raytracing, that is, today only the expensive Titan V, and on the previous GPUs of this company, as well as on AMD GPUs, ray tracing is fully performed using compute shaders - that is, only basic DXR support is available with lower performance. However, AMD has already stated that they are working together with Microsoft to implement hardware tracing acceleration and will soon provide a driver to support it, although for now it seems that existing AMD architectures are unlikely to be able to provide a high level of acceleration similar to Nvidia Volta. RTX hardware-accelerated ray tracing technology leverages the Volta architecture's yet-to-be-announced hardware ray tracing acceleration capabilities, and is expected to support gaming solutions later this fall.

Looking even further into the future, the emergence of APIs for accelerating rasterization goes somewhat counter to the general universalization of GPUs, which are becoming more and more similar to conventional processors designed for any type of computing. For many years there has been talk about completely removing from the GPU all blocks that perform fixed functions, although this has not worked out very well so far (you can remember the not very successful Intel Larrabee). But in general, the greater programmability of GPUs will make it even easier to mix rasterization and tracing, and full tracing may no longer require any APIs to support hardware acceleration. But this is looking too far ahead, for now we are dealing with DXR.

DirectX Raytracing and the support of this API extension by software and hardware developers provide the practical possibility of using ray tracing in combination with the familiar "rasterization" API. Why is this necessary, since modern GPUs are already capable of performing almost any calculation using compute shaders, and developers can perform ray tracing using them? The whole point is to standardize the capabilities of hardware acceleration of tracing on specialized units in the GPU, which will not happen if universal computational shaders are not intended for this purpose. Some new hardware capabilities of modern graphics architectures allow for faster ray tracing and this functionality cannot be exposed using the existing DirectX 12 API.

Microsoft remains true to itself - like the rasterization part of DirectX, the new API does not define exactly how the hardware should work, but allows GPU developers to accelerate only certain Microsoft standardized capabilities. Hardware developers are free to support executing DXR API commands the way they want, Microsoft does not tell them exactly how GPUs should do it. Microsoft introduces DXR as a compute task that can be run in parallel with the "rasterization" part, and DXR also brings several new types of shaders for ray processing, as well as an optimized structure for the 3D scene, convenient for ray tracing.

Since the new API is aimed at software developers, Microsoft is giving them a base level of ray tracing support in DXR that can use all existing hardware that supports DirectX 12. And the first experiments with DXR can be started on existing GPUs, although it will not be fast enough for use in real applications. All hardware with support for DirectX 12 will support ray tracing and some simple effects can be done even with the existing base of video cards in the hands of players. We will see some effects using DXR in games this year, but definitely in 2019 - at least as an early demonstration of the capabilities of new technologies.

It is likely that the initial performance of tracing on different GPUs will vary greatly. Solutions without native support, using a basic level of support through compute shaders, will be very slow, and GPUs with hardware tracing support will immediately speed up the process several times - just like in the good old days of the initial development of hardware rasterization support. Over time, more and more calculations during tracing will be performed more optimally and significantly more efficiently, but this will require new graphics solutions. The first of which should appear in the coming months.

A side-by-side comparison of rasterization and tracing

Let's try to look at specific examples of what ray tracing can provide. In fact, it is already used in games now, but in slightly different, more primitive forms. In particular, in algorithms using screen space or the voxel cone tracing algorithm when calculating global illumination, including the well-known algorithm Voxel Ambient Occlusion (VXAO) Nvidia company. But this is still not full-fledged ray tracing, but rather hacks with its use in one form or another during rasterization, and today we are talking about full-fledged ray tracing for the entire geometry of the scene.

Modern GPUs are already quite powerful and are capable of tracing light rays at high speed using software such as Arnold (Autodesk), V-Ray (Chaos Group) or Renderman (Pixar), and many architects and designers are already using hardware accelerated ray tracing to quickly create photorealistic renderings of their products, reducing costs in the overall development process. Nvidia has been involved in the development of hardware-accelerated ray tracing techniques in the professional world for over a decade, and now the time has come to bring these capabilities to games.

To help game developers implement ray tracing, Nvidia announced an upcoming addition to GameWorks SDK such features as specific noise reduction algorithms, high-quality global shading, shadows from area light sources ( area lights) and an algorithm for drawing high-quality reflections.

The best ray-traced renderings require a large number of samples (rays calculated per pixel) to achieve high quality—hundreds to thousands! It depends on the complexity of the scene, but even a few dozen rays are not suitable for real-time calculations, since even near-future GPUs with hardware tracing support will be able to provide acceptable performance with a much smaller number of rays per pixel - only a few. Is there any point in bothering?

Yes, if you further process the resulting image (we wanted to get away from rasterization hacks, but it looks like we’ll have to put up with others for now). In particular, performing tracing on a productive Volta architecture solution allows for real-time performance when calculating 1-2 samples per pixel with the mandatory use of noise reduction. There are already existing denoising algorithms that can significantly improve image quality after ray tracing, and these are just the first developments that are ongoing.

The requirements for real-time noise reduction algorithms are quite high; you need to be able to process very noisy input images with an extremely low number of rays per pixel (down to 1 sample), provide stable quality in motion using information from previous frames, and execute extremely quickly without spending more 1 ms GPU time. Existing Nvidia algorithms can achieve very good results when rendering reflections, soft shadows and global occlusion. For each effect, specific algorithms are used, which also use information about the 3D scene.

Ray tracing was used to render shadows with one sample per pixel and noise reduction enabled

To calculate global occlusion, we used two rays per pixel with noise reduction

And when rendering reflections, only one ray per pixel was calculated; noise reduction is also indispensable

Ray Tracing Denoiser as part of GameWorks SDK is a set of libraries for using several fast ray tracing techniques that use noise reduction, which is very important for tracing with a small number of rays per pixel, since the result is usually extremely noisy. The algorithms include rendering soft shadows from area light sources and algorithms for rendering reflections and global shading ambient occlusion. Using noise reduction allows you to achieve high speed with a small number of samples per pixel, but the image quality remains excellent - much better than the techniques currently used to simulate the propagation of light across the scene and use of screen space.

Let's talk about the benefits of ray tracing when rendering shadows. Using tracing, you can draw physically correct shadows with soft edges, much more realistic than the most sophisticated techniques available using shadow maps and filtering. Even for very large light sources, realistic soft shadows are obtained without the flaws encountered in rasterization.

Ray-traced shadows

Shadows obtained using rasterization and shadow maps

You can also use algorithms that are impossible or complex when simulating shadow maps, such as shadows from area light sources. And most importantly, this completely eliminates all possible visual artifacts: flickering pixels at the edges, jagged lines, etc. Yes, during the development of rasterization, many hacks have already been invented to suppress artifacts, but ray tracing does everything naturally.

To calculate global shading ( Ambient Occlusion) I would also like to use ray tracing, since it provides significantly higher quality compared to all existing screen space techniques (all of these SSAO, HBAO and even VXAO). Almost all algorithms used today simply add darkness to the corners found in a flat picture, only simulating the propagation of light, and the use of tracing allows this to be done in a physically correct manner.

Global occlusion using ray tracing

Global shading by simulating the effect using screen space

Moreover, all techniques using screen space ignore the impact of geometric objects outside the scene and behind the camera, and also add the same shading to completely different surfaces. In the example shown above, many of these problems are clearly visible - it is clear that this is just an attempt to simulate the propagation of light in a 3D scene, but tracing achieves a noticeably more photorealistic look.

When rendering reflections Tracing can also provide noticeably better quality than current methods that use screen space, which lack off-screen data (they are physically unable to draw in the reflection what is not visible on the screen) and which draw highlights on reflections incorrectly - from -due to the fact that the direct direction of view is used, and not the reflected one.

Reflections obtained using ray tracing

Reflections resulting from screen space rasterization

Nvidia's example may be over-exaggerated and too obvious about the problems with reflection techniques that use screen space, but the point is clear - physically correct reflections can only be rendered using ray tracing. Other methods for rendering reflections are not universal and provide inferior quality - for example, plane reflections only work on flat surfaces. But the tracing method also has a drawback - with a small number of samples, noise reduction will be required, since with one calculated ray per pixel the picture turns out to be extremely noisy.

It turns out that at the moment noise reduction should always be used, and the current version of specific techniques with Nvidia noise reduction has its own limitations and disadvantages. For example, a shadow rendering technique will produce degraded occluded shadows from two shadow-casting objects with large differences in distance from the surface being shadowed. The reflection rendering algorithm degrades in quality with increased surface roughness, and the global occlusion rendering algorithm may require not one, but two or even more calculated rays per pixel to render fine details.

But these are just the beginning versions of techniques using noise reduction filters, which will continue to improve in both quality and performance. In addition, in the future it is possible to use noise reduction using artificial intelligence technologies, which are already included in the Nvidia OptiX 5.0, but is not yet used when tracing with RTX. It is likely that in the future a single noise reduction will be used for all lighting components at once (rather than three separate ones, as is done now) to reduce memory costs and performance. There is also nothing stopping you from using a hybrid approach to rendering, using elements of screen space algorithms with additional ray tracing.

In addition to using ray tracing in real-time game engines, the power of GPU-accelerated DXR can also be used in content creation. For example, for high-quality lighting calculations, which are then placed in light maps, for creating pre-rendered scenes in a game engine, but with higher quality, etc. Moreover, you can use ray tracing not for rendering at all, but in sound engines for virtual reality ( Nvidia VRWorks Audio), in physical calculations or even in artificial intelligence algorithms.

Ray tracing is useful in the process of creating content: fine-tuning the characteristics of materials with high-quality and fast rendering, adding and adjusting the characteristics of light sources, debugging noise reduction algorithms, etc. You can also get an even higher-quality offline render using the same structures with relatively little effort and resources as the real-time engine. For example, this has already been done in Unreal Engine 4- Nvidia itself wrote the experimental Path Tracer immediately after integrating DXR capabilities into the engine, which, although it does not yet provide sufficient quality for full-fledged offline rendering, shows such a possibility.

We're not even talking about the possibility of quickly and efficiently preparing light maps - "baking" light into special lighting maps (lightmaps) for static objects in the scene. Such an engine can use the same code in the game and editor and provide the preparation of various types of light maps (2D, 3D) and cubic maps of the environment.

This is important not only because ray tracing will speed up the process of final generation of lightmaps, it will also provide a better preview of such light maps, allowing you to quickly change the location and characteristics of light sources and objects in the scene, immediately getting the result on the screen - almost the same what the final lighting will be like.

Finally, we suggest looking at all the advantages of ray tracing in dynamics. Nvidia has released a whole collection of technology demos showing the benefits of hardware-accelerated ray tracing using Nvidia RTX technology using the DXR API (only in the form of a Youtube video, alas).

The demo clearly shows the benefits of rendering traced shadows, including soft and colored ones, the difference in global occlusion quality when using rasterization and screen space compared to ray tracing, realistic reflections on various types of materials with multiple reflections, clever noise reduction systems and the use of tracing when preparing pre-rendered static light maps.

Demonstration of ray tracing capabilities

To demonstrate the capabilities of the DirectX Raytracing API and Nvidia RTX technology, several leading game engine and benchmark developers released their technology demos for GDC 2018, showing some of the capabilities of new technologies using ray tracing: 4A Games, Electronic Arts, Epic Games, Remedy Entertainment, Unity and others. Alas, for now they are only available in the form of screenshots, presentations and videos on Youtube.

Whereas previously similar demonstrations of real-time ray tracing were shown either in very simple scenes with simple effects or at low performance, the capabilities of future GPUs can make ray tracing real even in gaming conditions with acceptable performance. Developers Epic Games and Remedy Entertainment believe that DXR and RTX capabilities will bring better graphics to future games, and implementing basic support for the new API in their engines has proven to be relatively straightforward.

DirectX Raytracing tech demo (Futuremark)

For example, the company known to all 3D graphics enthusiasts for its test packages Futuremark showed a technology demo of DXR, made using a specially developed hybrid engine using ray tracing for high-quality reflections in real time.

We have already said that when using currently common methods, drawing realistic and physically correct reflections in a 3D scene is very difficult; in the process of creating algorithms, developers face a lot of difficulties, which are eventually overcome using different methods, but they remain far from the ideal reflection. Over the past few months, developers at Futuremark have been exploring the use of DXR in hybrid rendering and have achieved some pretty good results.

Using hardware-accelerated GPU ray tracing, they obtained physically correct reflections for all objects in the scene, including dynamic ones. Open the next few pictures at full size, as they are GIF animations that clearly show the difference between tracing and more conventional methods using screen space:

The difference is obvious. In addition to differences in reflection detail, using DXR tracing you can get reflections of objects that exist outside the screen space, i.e., not within the viewing range of the game camera, as can be seen in the comparative screenshots, and the reflection itself generally looks much more believable. Here is another example, perhaps less obvious, but quite giving an idea:

The use of ray tracing produces accurate, perspective-corrected reflections on all surfaces in the scene in real time. It is clearly visible that the tracing is much closer to realism than the more familiar screen-space reflections used in most modern games. Here's another comparison:

If you don’t look at the reflections obtained using DXR, then conventional methods may seem to give good quality, but they only seem so. Moreover, reflections are important not only for mirrors with a high reflectivity, but also for all other surfaces - they all become more realistic even if it is not immediately visible.

In its demo, Futuremark uses ray tracing capabilities only to solve problems that are difficult to combat with conventional methods, such as reflections of dynamic objects located outside the main screen space, reflections on non-planar surfaces, and perspective-corrected reflections for complex objects. Here are higher quality screenshots from the DXR demo:

Modern GPUs can already use hybrid rendering, using rasterization for most of the work and relatively little input from tracing to improve the quality of shadows, reflections and other effects that are difficult to handle using traditional rasterization techniques. And the Futuremark demo program just shows an example of such an approach; it works in real time on an existing GPU, albeit one of the most powerful.

The main thing is that according to the developers from Futuremark, it was quite easy for them to implement ray tracing support into the existing DirectX 12 engine from the benchmark 3DMark Time Spy, using models and textures from their tests. Along with the tech demo, well-known 3D test developers announced the use of DirectX Raytracing capabilities in their next 3DMark benchmark, which is planned to be released towards the end of this year.

Reflections Real-Time Ray Tracing Demo (Epic Games)

Company Epic Games together with ILMxLAB And Nvidia also showed her option to include real-time ray tracing capabilities in the engine Unreal Engine 4. The screening took place at the opening of GDC 2018, where three of these companies presented an experimental film-realistic demonstration on the theme of the film series "Star Wars" using characters from the series "The Force Awakens" And "The Last Jedi".

The Epic Games demo uses a modified version of Unreal Engine 4 and Nvidia RTX technology, the capabilities of which are revealed through the DirectX Raytracing API. To build the 3D scene, the developers used real resources from films Star Wars: The Last Jedi With Captain Phasma in shining armor and two stormtroopers with a scene in the ship's elevator First Order.

The tech demo in question features dynamically changing lighting that can be adjusted on the fly, as well as ray-traced effects, including high-quality soft shadows and photorealistic reflections - all rendered in real time and with very high quality. This kind of picture quality is simply not available without the use of ray tracing, and now the familiar Unreal Engine can provide it, which the founder and president of Epic Games was very impressed with Tim Sweeney.

Advanced techniques in the technology demo include: area lights including soft shadows rendered using ray tracing, as well as reflection and global occlusion rendering using ray tracing, noise reduction of the result of ray tracing from the Nvidia GameWorks package, as well as a high-quality depth of field effect ( does not use tracing, but is also cute).

The screenshots and video show the very high quality of all these effects, and what is especially impressive are the realistic reflections, of which there are a lot in the scene. All objects are reflected in all objects, which is very difficult, if not impossible, to render when rasterizing. The method of rendering reflections in screen space would only give an imitation of reality, in which everything that was not included in the frame would not be reflected, and the rest would be very difficult to render qualitatively.

In addition to reflections, one can note the softest shadows, which do not catch the eye with their torn and/or very sharp edges, as happens when using shadow maps. Well, the post-processing here is very high quality. In general, the developers did their best and this demonstration turned out to be perhaps one of the most impressive for hardware acceleration of ray tracing.

To create this demo, Epic Games worked closely with artists from ILMxLAB and engineers from Nvidia to demonstrate the capabilities of Nvidia RTX technology running through the DXR API. Unreal Engine demo running in real time on a workstation DGX Station Nvidia, which includes as many as four Volta architecture GPUs. Combining the power of the Unreal Engine, the DXR ray tracing graphics API, and Nvidia RTX technology running on the Volta family of GPUs, we get closer to real-time cinematic realism.

In addition to the technology demonstration, specialists from Epic Games held a large hour-long session at GDC "Cinematic Lighting in Unreal Engine", dedicated to the new features of their engine. And the demo itself is shown to everyone with the opportunity to watch the scene in various modes, including wireframe rendering. We can assume that all this will sooner or later be available in games, because the Unreal Engine is very popular. Epic Games promised to provide access to the capabilities of the DXR API this year - probably closer to the fall, when new Nvidia GPUs are released.

Support for DirectX Raytracing and Nvidia RTX opens the way for Unreal Engine 4 to a new class of techniques and algorithms that were not previously available with the dominance of rasterization. In the near future, game developers will be able to use a hybrid approach, using some high-quality ray tracing for some effects and high-performance rasterization for most of the work. This is a good foundation for the future, because the capabilities of GPUs related to effective acceleration of ray tracing will only grow.

Pica Pica — Real-time Raytracing Experiment (Electronic Arts/SEED)

The latest developer interested in ray tracing via DXR is the studio SEED from Electronic Arts, which created a special demo program Pica Pica, using an experimental engine Halcyon, which uses hybrid rendering like the previous demos. This demo is also interesting because it created a procedural world without any preliminary calculations.

Why did SEED researchers decide to use hybrid ray-traced rendering? They experimentally found that this method can produce a much more realistic image compared to rasterization, very close to full-fledged ray tracing (path tracing), which is overly resource-demanding or produces a too noisy image with a small number of calculated samples. All this can be clearly seen from the comparative screenshots:

Full tracing

Hybrid rendering

Rasterization

In modern games, various hacks are used to calculate reflections and lighting, including preliminary calculation of lighting (its static part, at least). All this requires additional work from level designers, who cleverly place fake light sources, start pre-calculation of lighting, which is then recorded in lightmaps. And using ray tracing for rendering tasks makes it possible to eliminate this additional work, because ray tracing allows you to naturally calculate everything you need, as we already described above.

And since full tracing is not yet possible, the Halcyon engine uses a hybrid approach. To calculate deferred shading, rasterization is used, to calculate direct shadows you can use either rasterization or ray tracing if necessary, for direct lighting you use compute shaders, for reflections you can also use both the traditional approach and tracing, for global illumination you always use tracing, and for To simulate ambient occlusion, you can either rely on conventional screen methods such as SSAO or also enable ray tracing. Only tracing is used to render transparent objects, and compute shaders are used for post-processing.

In particular, ray tracing is used to calculate shadows and reflections - much better and more natural than with currently common techniques. For example, such reflections in general cannot be made using algorithms for calculating reflections when rasterizing and using screen space:

Ray tracing when calculating reflections occurs at half resolution, that is, 0.25 rays/pixel is used for reflections and 0.25 rays/pixel for shadows. And here the problem of a small number of calculated rays appears in the form of an extremely noisy picture with reflections, when without special additional processing the ray tracing result looks too rough:

Therefore, after tracing, the picture is reconstructed to full rendering resolution in a special way - several very clever algorithms (details can be found in the presentation of the development team at GDC 2018), when the received data is filtered and information from previous frames is additionally collected and taken into account. The result is a completely acceptable result with realistic reflections, not much different from a full-fledged path tracing:

But maybe the usual methods in screen space will give no worse results and we simply don’t need “expensive” tracing? Check out this side-by-side comparison: On the left are screen-space reflections, in the middle is hybrid ray tracing, and on the right is a reference render with full ray tracing:

The difference is obvious. The screen space method is very approximate, unrealistic and only simulates reflections, although not bad in some places, but with obvious artifacts and problems of lack of resolution. There is no such problem with tracing, even taking into account the reduced resolution when rendering rays. In Pica Pica, ray tracing is also used to render transparent and translucent objects. The demo program calculates light refraction without the need for pre-sorting, as well as subsurface light scattering:

So far, the engine has not been fully developed and it has one drawback that is important for photorealism - it cannot yet draw shadows from translucent objects, but this is a matter of time. But the demo uses a global illumination algorithm that does not use preliminary calculations and supports both static and dynamic objects, minimizing the need for additional work on the part of artists:

Global Illumination Off

Global Illumination enabled

Global illumination significantly affects some objects in the scene, adding realism to their lighting. In the demo, you can also additionally use techniques to simulate global shading, giving additional shadows. Algorithms in screen space are also supported - Screen Space Ambient Occlusion (SSAO):

Perhaps it would have worked even better with something like VXAO, which Nvidia is promoting, but it already looks quite good. But the image will be even better and more realistic with a full calculation of global shading using ray tracing. Look at the comparison pictures, the difference is striking:

While SSAO gives only a semblance of global shadows, obscuring only the most obvious corners, then full tracing does everything perfectly, giving deep shadow where it should be, based on the laws of light propagation.

As for shadows from direct rays of light sources, with hard shadows when tracing everything is quite simple - rays are launched in the direction of the light sources and hits are checked. For soft shadows, the algorithm is similar, but the result with one sample per pixel is too “noisy” and has to be additionally filtered, after which the picture becomes more realistic:

Hard shadows, soft unfiltered and soft filtered shadows

The developers from the SEED studio specifically note that although their research into hybrid rendering is at an early stage, this approach allows replacing numerous hacks with a bunch of objective shortcomings with a unified ray tracing approach that provides better rendering quality. What is especially important is that now software developers have a single, generally accepted API for ray tracing, and only further refinement of the algorithms is required both to improve the quality of rendering and to optimize its performance, since ray tracing remains fairly demanding on hardware.

At the moment, the Pica Pica demo program calculates only 2.25 rays per pixel (in total, including all effects), and the result is a photorealistic image with quality close to full tracing, although with some limitations. And now - the fly in the ointment: as in the case of the Epic Games demo, to speed up the rendering process, so far we have to use the capabilities of several top GPUs simultaneously and transfer a minimum amount of data over the relatively slow PCI Express bus. But further development of hardware acceleration on GPUs should help rid us of such system requirements in the future.

Experiments with DirectX Raytracing in Northlight (Remedy Entertainment)

Another demo program to promote DXR and RTX, presented at GDC 2018, was experiments with the game engine Northlight Engine Finnish company Remedy Entertainment, known to the public for games such as Max Payne, Alan Wake and Quantum Break. The Northlight Engine is intensively developed by a company known for its interest in the latest graphics technologies. Therefore, it is no wonder that they became interested in hardware-accelerated ray tracing.

At GDC, the company showed off developments they were working on with Nvidia and Microsoft. Among several developers, Remedy received early access to Nvidia's RTX and DXR API capabilities, which were embodied in a special version of the Northlight engine. The chief graphics programmer of Tatu Aalto presented a speech at the conference "Experiments with DirectX Raytracing in Remedy's Northlight Engine", in which he spoke about the features of the hybrid approach they adopted.

This demo traditionally uses rasterization for speed and ray tracing for some effects that would be difficult to do otherwise. Quality improvements include physically based soft shadows, high-quality global occlusion and lighting, and realistic reflections. The video shows the result of the Northlight Engine with all effects enabled, calculated using ray tracing with an increased number of calculated rays:

To conduct experiments on introducing ray tracing, Remedy created a new scene that was in no way related to the company’s games. As we already said, the DXR API supports two levels of acceleration structures: lower and upper. The idea is that the bottom level structure is for storing geometry, and the top level contains the lower level structures. That is, each polygonal mesh is one lower-level structure, and each upper level contains several lower-level structures with possible geometric transformations (rotations, etc.).

The lower level structure is needed for the static parts of the scene; the red squares in the diagram are the boundaries of the lower level tree. For example, in a scene there are four examples of a small chair (small red squares) that have the same geometry but their own geometric transformations. Medium squares are small sofas, large squares are large round sofas. To create a ray tracing scene, you need to insert these lower-level structures into a top-level structure, for which the DXR API has a special function that accepts multiple instances of the lower-level structure with transformations.

Working with dynamically changing geometry is made a little more difficult, since the lower-level builder only accepts static buffers. But deformation is also possible here - with the help of a computational shader that accepts geometry and skinning matrices and writes the already changed geometry. Next, you can begin work on calculating the rays.

First, let's take ambient occlusion, a visibility-based algorithm that can be easily implemented using ray tracing. The following image was produced by rendering four rays per pixel, with a maximum length set to four meters, and the result certainly looks better than the SSAO method, which uses only screen space.

The traditional method of calculating global occlusion is indicated on the left half, and ray tracing is indicated on the right. While the SSAO technique does a decent job of capturing some edges, it clearly lacks geometric information about the scene—such algorithms don't know what's off-screen or behind surfaces visible to the camera. Therefore, the result is clearly not ideal, although it is clearly better than without shading at all.

Unfortunately, the performance of ray tracing is relatively low and it is much more expensive than screen space methods. According to Remedy, in their demo program, rendering one ray per pixel for global occlusion with a maximum length of 4 meters at Full HD resolution takes approximately 5 ms and performance scales almost linearly, so rendering 16 rays will take about 80 ms. With constant improvement of quality, of course:

These screenshots were taken with regular full-screen antialiasing, taking into account the time component (data from previous frames), and without tricky filtering, as is done in most of the other demos shown at GDC. With clever noise reduction, you can achieve acceptable quality with 1-2 rays per pixel.

In addition to global shading, the Remedy demo also uses ray tracing to render regular shadows, which now most often use cascaded shadow maps when rasterized ( cascaded shadow maps - CSM). The developers note that if the engine fills shadows from directional light sources before rendering the lighting, then it will be very easy to replace the cascaded shadow map shader with code using tracing, which will write the calculated data to the same buffer.

In this case, the difference in quality will clearly be in favor of the trace (shown on the right). A ray-traced image uses 8 rays per pixel without additional filtering, while the CSM technique uses 16 Percentage Closer Filtering (PCM) samples with a special filter applied to the buffer. However, you need to take into account that the developers clearly did not optimize the work of CSM in this case, because it was possible to adjust the resolution of shadow maps and their filtering to get better shadows, but this is just a shadow with the default settings of their engine.

But even taking into account this discount, the difference is obvious - with ray tracing the shadows are much more realistic, they have smooth edges without jagged edges, better blur at the edges, and even small details (chair legs) cast a physically correct shadow. The end result is decent shadows with soft and hard shadow edges exactly where they should be. You can also easily draw shadows from area light sources, which is extremely difficult to do with rasterization.

In terms of performance, this demo renders a single ray for Full HD resolution in less than 4 ms, which is slightly faster than global occlusion, even though the rays are longer. Implementing ray tracing into an existing DX12 shadow rendering engine will require several days of programmer work, but the result will be worth it if the performance is sufficient in the end.

It seems that Remedy added almost all the effects possible with tracing to their engine at the initial stage of DXR development. Including reflections rendered using ray tracing. At the same time, there is no such obvious use in the form of purely mirror surfaces, but rather a more subtle approach with reflections on all objects, but less obvious. The following screenshot shows a comparison of techniques using ray tracing (right) and screen space (left):

The traced image was obtained by rendering only one ray of reflections per pixel without filtering. Reflections in screen space are clearly less realistic and take into account only objects visible to the main camera, while ray tracing allows you to show them too, although it has its drawbacks in the form of significant pixel noise. But this can be solved in principle, as other demo programs show, and in the Finnish company’s version, noise reduction is simply not applied yet, with the exception of using pixel values ​​from previous frames for full-screen anti-aliasing.

The Northlight Engine already uses global illumination calculations ( G.I.) - specifically in the game Quantum Break, and this effect is enabled in the engine by default. To calculate the GI, voxels with a size of approximately 25 cm are used, which are combined with the result of the global shading technique SSAO, which uses screen space. As an experiment, Remedy replaced SSAO with a similar effect using ray tracing and the result was better.

It can be seen that the surfaces are not shaded as they should be and something is clearly wrong with them. The problem is solved by modifying the method of using GI volumetric data, with which most of the artifacts are eliminated:

Why do you need to calculate global illumination/shading at all, and is it possible to do without this extremely resource-intensive step? Look at the visual example of what the result of calculating direct lighting alone looks like:

Similar to Doom with its constant darkness and harsh stencil shadows. But in the next screenshot, indirect lighting has also been added to direct lighting - that is, rays of light reflected from other objects in the scene:

It has become much better, despite the noise, the scene has acquired volume and does not look as if all its objects are located in space with the presence of a single bright light source (the sun). And this is what the final image looks like, with color information superimposed, full lighting and post-processing:

The reflections and shading in the scene look very realistic in our opinion. In particular, the lamp reflects all objects, including a bright window invisible to the main camera. And the mug on the right reflects its own pen - this cannot be done using rasterization without clever hacks. The only obvious tracing problem here is a lot of pixel noise, which Remedy hasn't really tried to remove yet. But the same algorithm from Nvidia GameWorks could help a lot, not to mention noise reduction using artificial intelligence.

Of course, it would be very nice to use ray tracing wherever possible, but the optimal solution for hybrid rendering is optimization using shadow maps, which are used in the Remedy demo for most light sources except the sun. And this will initially be done in every application using ray tracing, because using it directly everywhere will be too expensive and it is not yet possible in real time, even using several GPUs at once.

It is important that the integration of DXR and RTX support into the Northlight engine was quite quick and painless. The Finnish developers were amazed at how quickly they were able to prototype improved lighting, shading and reflections using ray tracing - all with much better quality than traditional rasterization hacks. While the technologies shown are currently in early development and far from being included in games right now, it's a great start to future adoption.

Real-Time Ray Tracing in Metro Exodus (4A Games)

It is likely that in the coming years we will see more than one game that uses hybrid rendering with ray tracing to render some of the effects. In particular, the first (or one of the first, at least) should be the game Metro Exodus, which will use DXR ray tracing using Nvidia RTX technology to calculate global illumination and shading.

It is assumed that this calculation method G.I. will be available in the game as an alternative to more familiar algorithms SSAO And IBL(image based lighting, lighting based on the environment texture). Of course, this is still an extremely limited use of tracing, but the quality of global illumination/shading with ray tracing is much higher than even at VXAO, not to mention SSAO. Here is a visual comparison of screen space methods with tracing, taken by our German colleagues from the screen of an exhibition system (so we apologize in advance for the quality):

Textures were turned off during the demonstration so that the difference in scene lighting was clearly visible. And this is true, screen rasterization methods give a flat picture that only vaguely imitates shading in the corners between the edges of objects, and ray tracing gives physically correct global shading and lighting with dark shadows exactly where they should be - for example, look inside the barrel on entering the house through cracks - with SSAO it is not obscured inside at all, but with ray tracing in its depths it is dark, as it should be.

There is only one question here - if the video shows a static scene, without dynamic objects at all and their influence on global illumination, then what prevents you from first calculating everything offline and entering this data into static light maps? It seems to us that when dynamically calculating global illumination in real time, the scene for demonstrating the capabilities should have been chosen somehow more lively, at least with moving light sources, not to mention moving objects. Otherwise, one is left wondering why the players did not understand what exactly was shown to them and why it is impossible to do this using rasterization right now.

conclusions

Ray tracing provides much better image quality compared to rasterization, and has long been used where possible - in the film industry, advertising, design, etc. But for a long time it was simply not suitable for real-time rendering due to its enormous resource intensity - after all, for each pixel it is necessary to calculate several rays reflected from objects in the scene and refracted into them. For offline rendering, which does not require quick results, this approach was always the highest quality, but in real-time graphics we had to settle for rasterization - the simplest and fastest way to project a 3D scene onto a 2D screen. Naturally, high rasterization performance has the disadvantage of only approximate calculations of the color of pixels in the scene, which do not take into account many factors: reflection of light rays, some properties of materials, etc. Rasterization, even with a bunch of cunning hacks, only approximately reproduces the scene, and even the most complex pixel and computational shaders will not provide the quality of full-fledged ray tracing, simply based on the principle of their operation.

The announcement of the DXR API and Nvidia RTX technology has enabled developers to begin researching algorithms that use high-performance ray tracing - perhaps the most significant change in real-time graphics since programmable shaders were introduced. Interested developers have already shown the public some very impressive technology demonstrations using only a small number of samples per pixel during tracing, and the future of gaming is in their hands. And in the hands of GPU manufacturers, who should release new solutions that support hardware tracing, expected in several game projects late this year and early next year.

Naturally, the first attempts to use tracing will be hybrid and seriously limited in the quantity and quality of effects, and full-fledged tracing will have to wait for decades. All demo programs shown use 1-2 rays per pixel, or even less, while professional applications have hundreds of them! And to get the quality of offline renders in real time, you still have to wait a very long time. But now is the time to start working on introducing tracing into existing engines, and whoever is first in mastering the capabilities of DXR may gain a certain advantage in the future. In addition, ray tracing can make the development of virtual worlds easier, since it will eliminate many of the small tasks of manually modifying shadows, lightmaps and reflections, which has to be done with imperfect rasterization algorithms. Already now, hardware-accelerated tracing can be used in the development process itself - to speed up things like preliminary rendering of lightmaps, reflections and static shadow maps.

There are many options for optimizing hybrid rendering, and one of the most impressive features in the examples shown above seems to be the efficiency of noise reduction, which is extremely important when ray tracing with a small number of samples per pixel - this is known to anyone who has ever seen the work of offline tracers, which render the image gradually and at the very beginning it is extremely noisy. An approach with a small number of calculated rays and additional noise reduction makes it possible to obtain an acceptable final quality in a fraction of the time required for full scene tracing. And this despite the fact that the capabilities of artificial intelligence in noise reduction have not yet been used, although this can be done.

The global capabilities of ray tracing should not be judged solely by hastily released demo programs. They deliberately highlight the main effects, as they are technology demos made for a single purpose. The ray-traced picture becomes much more realistic in general, but users do not always understand where exactly to look, even if they feel that it has become more believable in general. Especially if the difference is not so great at first and the masses are willing to put up with artifacts inherent in algorithms for calculating reflections and global shading in screen space, as well as other rasterization hacks.

But with physically correct global illumination, shading and reflections calculated using ray tracing, the rendered picture becomes more realistic even without the presence of spectacular mirrors and other clearly reflective surfaces. Modern games almost always use physically based rendering, in which materials have roughness and reflectivity properties, as well as cubic maps of the environment, so reflections are always present, even if they are not visible to the naked eye. In such a game, you can quickly replace cube maps of the environment with tracing, offering this opportunity to owners of high-performance systems. Traced shadows also look better and solve the fundamental problems of shadow maps, although some of them are solved in tricky advanced algorithms, like Nvidia Hybrid Frustum Traced Shadows (HFTS), also using some form of tracing, but a unified approach is still best. And rendering very soft shadows from area light sources can produce ideal, ultra-realistic shadows in most cases.

The main difficulty of tracing is that not all first implementations will immediately look noticeably better than tricky screen-space methods, but we can say for sure that this is exactly the direction in which we need to move to achieve photorealism. Because screen space algorithms have fundamental limitations that cannot be jumped over. In many respects, the picture of even existing demo programs is quite good, even if it is rendered by several powerful GPUs and uses tricky noise reduction. For now we have to use a small number of rays per pixel and suppress noise, but in the future this will be solved with the help of primitive extensive development. These are just the very first tests with real-time ray tracing; in the future, the image quality will increase along with performance.

For now, in the next couple of years we will be able to include one or two new techniques that use ray tracing to complement rasterization or to replace only part of its work. This is always done at the beginning of the life of new technologies, when it is possible to disable new algorithms that are too heavy for the average gaming PC. But if you focus only on them, then there will simply be no progress. And Nvidia’s support for hardware tracing is important because they know how to help developers implement new technologies. And we are sure that Metro Exodus is far from the only game in which Nvidia is promoting tracing, because they are collaborating with game developers on several projects at once. Notorious Tim Sweeney from Epic Games predicted that in two years GPUs will gain sufficient performance for the widespread use of ray tracing in games, and you can believe it.

Developers closest to Microsoft began exploring the capabilities of DXR almost a year ago, and this is just the beginning of the new API. Moreover, there are simply no available graphics solutions on the market that support hardware tracing acceleration. The DXR announcement is intended to get hardware and software developers to begin work on understanding and optimizing ray tracing and begin the early stage of introducing new technologies into games. Interested developers have already begun experimenting with DXR and modern GPUs, and companies such as Epic Games, Futuremark, DICE, Unity and Electronic Arts have even announced plans to use DXR capabilities in future versions of game engines and games.

It's likely that enthusiasts will have to wait (that's our lot) for hardware-accelerated GPUs to become available to see even the very first ray-traced effects, as the basic level of support through compute shaders may be too slow for even simpler algorithms. Games that make meaningful use of DXR will require hardware support for tracing, which will initially only be available on Nvidia Volta, but threatens to be actively improved over time. It is also possible that relatively simple games with stylized graphics will emerge that will use ray tracing exclusively.

Another important point is that the current generation of game consoles does not support hardware acceleration of ray tracing, Microsoft has not said anything about DXR in Xbox One. Most likely, such support simply will not exist, which could become another obstacle to the active use of ray tracing capabilities in games. Although the Xbox One has almost full support for DirectX 12, it does not have any hardware units to accelerate tracing, so there is a good chance that, at least until the next generation of consoles, it will be limited to a couple of ray tracing effects in a few game projects supported by Nvidia , promoting its RTX technology. I would really like to be wrong, because computer graphics enthusiasts have already been waiting for such global improvements in real-time rendering.