What is the difference between Pipeline and Inline Ray Tracing?

One of the most frequent questions we get is: What is the difference between the pipeline and inline ray tracing modes in Evolve? In short, they are two different approaches to solving the same problem, each with trade-offs. To really answer this question, we will dive deep in this blog post on how they both work, how they differ, and how they perform on different GPUs.

In a nutshell: What is the difference between inline and pipeline Ray Tracing?

To start, let’s do a quick recap of what ray tracing is. Ray tracing allows us to choose a position in a 3D scene, choose a direction, and travel along a line in that direction until we hit something in that scene. We then find out what we hit, and how far the thing was that we hit. This lets us simulate how rays of light would traverse through a 3D scene.

For example, we can simulate a light photon by starting at a light source, traveling in the direction of that light into the scene, hitting an object, then tracing a second ray that travels to the camera, and then calculating how much energy is reflected from the light to the surface, to the camera.

All of these lighting computations are performed by small GPU programs called shaders. Rendering engines use shaders to determine object positions relative to the camera, carry out per-pixel calculations, or even perform more general-purpose computations. Before ray tracing, there were already five types of shaders for rasterization (three of those being optional), as well as one for general computations. Inline ray tracing lets programmers add ray tracing to these existing shaders. This has the benefit of being relatively easy to integrate into existing algorithms, but it limits the GPU in how it schedules the ray tracing operations.

Pipeline ray tracing takes a different approach by introducing five new shader types just for ray tracing. This model does allow for hardware designs that can take much more freedom in scheduling all the ray tracing operations efficiently, but it does come with the downside that it is not as easy to integrate into existing systems and can introduce overhead when the ray tracing operation needs to be performed from a different shader than the one where the result was originally needed.

Benchmarking Inline vs Pipeline Ray Tracing

When designing and building the Evolve benchmark, we wanted to be able to measure the performance difference between inline and pipeline ray tracing. Evolve implements all ray tracing algorithms using both the inline and pipeline-based ray tracing techniques, meaning that we can get as close to a direct comparison as possible with various workloads like shadows, reflection, refractions, and global illumination.

We ran Evolve on three GPUs to compare their inline and pipeline ray tracing score. The goal here is not to say which GPU is the better value, but rather to take a look at how different techniques perform per GPU.

We used these graphics cards:

• Sapphire Nitro+ AMD Radeon RX 9070 XT 16GB with driver 25.9.1
• MSI NVIDIA GeForce RTX 5070 Ti 16GB Vanguard SOC with driver 581.29
• Sparkle Intel Arc B580 TITAN AC 12GB with driver 32.0.101.7029

The rest of our test setup consisted of:

• CPU: AMD Ryzen 7 9800X3D
• Motherboard: ASUS ROG Strix B650E-F GAMING WIFI
• Memory: 32 GiB DDR5 Kingston Fury Beast @ 6000MT/s
• Power Supply: Corsair RM1000x
• CPU Cooler: Arctic Liquid Freezer III Pro 360
• Storage: WD Black SN770 1TB
• OS: Windows 11 24H2
• Benchmark: Evolve v1.0.10

Before we ran the test, we were expecting to get fairly similar scores for both inline and pipeline ray tracing, which was true for the NVIDIA GeForce RTX 5070 Ti and the Intel Arc B580. However, on the AMD Radeon RX 9070 XT, we see that pipeline ray tracing performs much better than inline! This result is surprising to us for two reasons.

The first reason has to do with the fact that Evolve uses a single material shader architecture. Game engines can choose to have a shader per material or one shader with a lot of parameters. Pipeline ray tracing is great for this many-material systems as more closest hit shaders can be added as needed, while inline ray tracing would require processing the results in separate passes. However, with Evolve’s single material system we expected to see very similar performance between inline and pipeline ray tracing.

The second reason is that, from what we can gather, AMD compiles all the pipeline ray tracing shaders into one big shader. This would suggest to us that AMD could use different strategies for compiling the shader code between inline and pipeline ray tracing, but we can’t be certain of that.

A Deep Dive

To give a more in-depth look into inline and pipeline ray tracing we will look at how a ray is traced in the first place, and what type of cases need to be handled by the game engine. At its core the GPU needs a ray and a scene acceleration structure to trace the ray in. It will then find the closest primitive in the scene that intersects with the provided ray.

This will do great for most 3D geometry, but not everything. For example, plants like grass are usually not modelled blade by blade, but instead as a bigger rectangle with a texture that uses transparency to only draw the blades of grass themselves. This means that in order for it to count as a hit, the ray doesn’t need to just intersect with one of the triangles; it also needs to check if that part of the texture is transparent or not. Both inline and pipeline ray tracing techniques support this, but in slightly different ways.

Another case that needs to be handled is non-triangle geometry since it’s not possible to handle all types of geometry in hardware. Although the vast majority of geometry used in games are triangles, not everything translates that well into triangles. For example, hair or volumetric objects don’t translate well into triangles at all. For this reason a procedural primitive was added. By adding bounding boxes surrounding the procedural primitives to the scene acceleration structure, the GPU will know when it could have potentially hit a piece of procedural geometry. When a ray hits one of these bounding boxes it will need to run some code to determine if it hit the shape inside or not. Just like with handling partially transparent triangles, both inline and pipeline ray tracing support this case but in different ways.

Let’s take a look at how inline ray tracing can be added into existing shaders and how these special cases can be handled.

Understanding Inline Ray Tracing

In inline ray tracing the main design is to able to tell the GPU to trace a ray, get a result back. Based on the result the shader might get told that a special case needs to be handled or if the ray is done tracing. Let’s first look at the simplest case, where we assume that there is no partially transparent geometry and no procedural geometry.

Note that the lines starting with // are comments explaining what lines do

// Set up the ray query for ray tracing; nothing actually happens here yet.
query := RayQuery
query.TraceInline(accelerationStructure, flags, instanceMask, ray)

// Perform the ray tracing
query.Proceed()

// Run different code based on the hit result.
status := query.CommitedStatus()
if status == COMMITTED_TRIANGLE_HIT
// Ray hit a triangle, add code for shading here
else if status == COMMITTED_NOTHING
// Ray missed the scene, add code for shading the sky here

The Proceed function returns a value that determines if the ray has finished tracing or if it found a candidate, but needs help deciding if the hit counts or not. If the proceed function tells us that it needs help we can check if the candidate is a non opaque triangle and handle that case accordingly.

// Set up the ray query for ray tracing; nothing actually happens here yet.
query := RayQuery
query.TraceInline(accelerationStructure, flags, instanceMask, ray)

// Perform the ray tracing until the GPU tells us we are done
while query.Proceed()
candidate := query.CandidateType()
if candidate == CANDIDATE_NON_OPAQUE_TRIANGLE
// Use code to determine if hit should count or not

// Run different code based on the hit result.
status := query.CommitedStatus()
if status == COMMITTED_TRIANGLE_HIT
// Ray hit a triangle, add code for shading here
else if status == COMMITTED_NOTHING
// Ray missed the scene, add code for shading the sky here

Handling procedural geometry can be handled just like non opaque geometry by checking for the candidate type and running some code to determine if it is a hit or not. After that, when the ray has completed tracing, there is committed status just for procedural geometry to use different shading code, but a game engine may also decide to use the same shading code as for triangles.

// Set up the ray query for ray tracing; nothing actually happens here yet.
query := RayQuery
query.TraceInline(accelerationStructure, flags, instanceMask, ray)

// Perform the ray tracing until the GPU tells us we are done
while query.Proceed()
candidate := query.CandidateType()
if candidate == CANDIDATE_NON_OPAQUE_TRIANGLE
// Use code to determine if hit should count or not
else if candidate == CANDIDATE_PROCEDRUAL_PRIMITIVE
// Use code to determine if ray hits procedural primitive


// Run different code based on the result.
status := query.CommitedStatus()
if status == COMMITTED_TRIANGLE_HIT
// Ray hit a triangle, perform shading
else if status == COMMITTED_NOTHING
// Ray missed the scene, calculate sky color
else if status == COMMITTED_PROCEDURAL_HIT
// Ray hit a procedural primitive, perform shading

Inline Ray Tracing makes it fairly simple to introduce new ray tracing code with just a handle full lines of code. The only downside is that because the logic needs to keep coming back to the same shader for special cases, the GPU doesn’t have as much freedom in deciding when to execute what code. To see how we can give the GPU more flexibility let’s take a look at pipeline ray tracing.

Exploring Pipeline Ray Tracing

Pipeline Ray Tracing takes a different approach by introducing five new types of shaders specifically designed for ray tracing. The new shader types are:

• Ray Generation Shader
• Miss Shader
• Closest Hit Shader
• Any-Hit Shader
• Intersection Shader

‍

We start with a Ray Generation Shader, where initial rays are generated and a TraceRay() function is called. We give TraceRay a scene acelleration structure to traverse and a ray. Additionally, a ray payload is also provided that can be read and written to from the other shaders while the ray is going through the pipeline. The GPU will then use the acceleration structure to find primitives that the ray intersects with and launch the other shader types as needed.

A diagram showing the control flow of the shades in the pipeline

Let’s say we have a ray that doesn’t hit any object in the scene. The GPU will launch a miss shader for that ray. In that miss shader, where for example the sky color can be computed.

When a ray does intersect with the scene and the GPU has determined which primitive is the closest one, it will invoke the closest hit shader. In the closest hit shader, we can calculate how the surface of that object should be shaded.

Just like with inline ray tracing, when a ray hits non-opaque geometry, it will need to know whether to count the hit or not. The GPU will dispatch the any hit shader where some code can determine if the ray hit a transparent section or not, and decide to ignore or keep the hit.

Pipeline ray tracing also supports handling procedural geometry. This is done by launching an intersection shader. In the intersection shader, custom shader code can determine if the ray hits the geometry or not. After that, the GPU continues tracing the ray and will end with calling the closest hit or miss shader.

One important aspect we haven’t touched so far is that in a ray tracing pipeline, there may be many closest hit, miss, any-hit, and intersection shaders. For example, a different closest hit shader may be needed depending on whether a material is made of glass or metal. In order for the GPU to know which shader needs to be launched, a Shader Binding Table (SBT) is used. Using an instance index and a geometry index, the GPU can find an entry in this table and launch the corresponding shader.

The big advantage that pipeline ray tracing brings is that if the GPU finds hits, or needs to run intersection shaders, it doesn’t need to execute that as it finds them. It can gather similar shaders that need to be run, and then run a bunch of them at the same which, which GPUs are generally better at. However, current hardware seems to do this only in very limited amounts. That does not mean that future hardware won’t get better at this in the future.

Conclusion

Inline and Pipeline Ray Tracing both offer advantages and disadvantages between each other. One is not necessarily better than the other, but they are different ways of doing the same thing. In our Evolve Benchmark, we are interested in finding out how different GPUs perform with these different approaches and how this changes throughout the years. That’s why we have implemented all our ray tracing-based graphics algorithms with both the Inline and the Pipeline Ray Tracing methods. From some benchmarks we ran with Evolve we don’t see any major performance differences between inline and pipeline on Nvidia and Intel, while on AMD Pipeline Ray Tracing is much faster. Only time will tell how this will change as better hardware and better drivers are released.

Sources

• https://www.intel.com/content/www/us/en/developer/articles/guide/real-time-ray-tracing-in-games.html
• https://gitlab.freedesktop.org/mesa/mesa/-/blob/a905ff3d544b94f5996e2eabb91a0137efd2a1cf/src/amd/vulkan/nir/radv_nir_rt_common.c
• https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna4-instruction-set-architecture.pdf
• https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
• https://vulkan.lunarg.com/doc/view/1.3.283.0/mac/1.3/vkspec.html
• https://microsoft.github.io/DirectX-Specs/d3d/Raytracing.html

‍

‍