One of the most exciting projects we had the chance to work on in 2024 was the creation of the Vulkan SC Emulation driver stack. This software stack enables all developers to prototype and develop Vulkan SC applications on consumer PC hardware running Windows or Linux without needing specialized hardware and software components that are typically required to target safety-critical systems.
The Vulkan SC 1.0 API specification has been out there since 2022 and compliant implementations are available from CoreAVI and NVIDIA on various platforms expected to be used in safety-critical environments, however, access for the general public to hardware and software platforms having native Vulkan SC support remains limited. This is why, in 2023, we proposed the Vulkan SC Working Group the idea of creating a Vulkan SC Emulation layer that enables running Vulkan SC applications on top of regular Vulkan implementations in order to significantly improve the accessibility of Vulkan SC as an API to developers around the globe.
Admittedly, NVIDIA did provide a public Vulkan SC implementation in their JetPack SDK available for Jetson-based devices and they also extended this in 2024 to cover desktop systems by including Vulkan SC implementations in all their desktop drivers. That, however, still only covers support for a single vendor’s hardware, so the Vulkan SC Emulation driver stack remained an important next step in the evolution of the Vulkan SC Ecosystem toolchain because it widens the coverage to all PCs (or any other compatible computer) running Windows or Linux. Furthermore, the Vulkan SC Emulation driver stack enables a whole set of additional possibilities to further aid the development of Vulkan SC applications, as covered later in this article.
Nomenclature
You can see the Vulkan SC Emulation driver stack being referred to as the Vulkan SC Emulation layer here and there, but it is worth clarifying that it is not an actual layer in the traditional Vulkan (SC) sense. Rather it is a standalone ICD (installable client driver) similar to actual native Vulkan and Vulkan SC drivers, more analogous to e.g. the Mesa Zink driver that enables the emulation of OpenGL on top of Vulkan.
It is also important to make it clear that the Vulkan SC Emulation driver stack is not an officially conformant and safety certifiable Vulkan SC solution on its own, irrespective of being able to pass the Vulkan SC conformance test. Even if itself would be, which was never a goal, it still relies on an underlying Vulkan implementation that itself is not designed for safety-critical use, by definition. Rather, the Vulkan SC Emulation driver stack is a developer tool that aims to enable developers to prototype and develop Vulkan SC applications on consumer hardware and therefore eliminate the high barrier of entry for developers interested in learning and using the Vulkan SC API.
As any other implementation of the Vulkan SC API, the driver stack comprises of two components:
- An installable client driver – the Vulkan SC Emulation ICD (vksconvk.dll / libvksconvk.so)
- An offline pipeline cache compiler – the Vulkan SC Emulation PCC (pcconvk)
For those interested, the origin of the binary names is that they provide access to the Vulkan SC API (vksc) and a corresponding Pipeline Cache Compiler (pcc) on top of the Vulkan API (vk).
How it works
The Vulkan SC Emulation ICD implements all the logic necessary to execute Vulkan SC applications on top of any compatible Vulkan implementation available on the system. First, this needs the system to have a GPU and corresponding Vulkan driver that fulfills the minimum requirements to emulate Vulkan SC on them. Vulkan SC 1.0 is based on Vulkan 1.2 and requires support for the Vulkan Memory Model, so any Vulkan physical device available on the system that meets these requirements will be exposed as Vulkan SC physical devices by the Emulation ICD. As otherwise it is just another Vulkan SC ICD with a set of physical devices, applications are able to gain access to all emulated devices as well as native Vulkan SC devices (such as the ones exposed by the NVIDIA desktop Vulkan SC driver) through the Vulkan SC loader.
Of course, the secret sauce is the efficient and transparent translation of the incoming Vulkan SC API command stream to a corresponding set of Vulkan API commands. Fortunately, the Vulkan SC API is very close to the Vulkan API, as one of the key design goals of Vulkan SC was to be able to reuse most existing software, source code, and developer mindshare with minimal deviations required to meet safety-critical requirements, so the Emulation ICD does not need to jump through flaming hoops to make that translation happen.
Beyond just doing the bare minimum needed for correct API translation, however, the Emulation ICD provides additional Vulkan SC specific and general functionalities that further aid developers in creating robust Vulkan SC applications, some of which go beyond what one may be able to achieve even when using a native Vulkan SC driver implementation. We’ll talk about these later in this article.
The emulation pipeline cache compiler
As any other Vulkan SC implementation, the Vulkan SC Emulation driver stack comes with an “offline” pipeline cache compiler. The role of the pipeline cache compiler in Vulkan SC is to offload pipeline cache compilation (and inherently shader compilation) into a preprocessing stage and ship Vulkan SC applications with pre-built pipeline cache binaries in order to eliminate the need for online pipeline and shader compilation, which is a key aspect of the special requirements of safety-critical systems to ensure predictability (although it may often be preferred even in case of regular Vulkan applications). This happens by issuing the pipeline cache compiler to produce the binaries from a set of shader files and pipeline JSON files, as described in detail in the Khronos Vulkan SC Overview and our last article about the Vulkan SC Ecosystem components.
The Vulkan SC Emulation PCC “cheats” a bit here in order to keep things simple and flexible…
An actual production pipeline cache compiler produces the final shader and pipeline binaries (built for a specific target hardware) as output in order to completely eliminate the need for online shader compilation. The Emulation PCC, however, currently only packages the pipeline and shader SPIR-V binaries into the pipeline cache binaries, practically only containing the “debug information”. This means that online pipeline compilation will still happen when running a Vulkan SC application against the Emulation ICD, but exactly reproducing how a Vulkan SC application would behave in a real safety-critical environment in terms of performance predictability is not entirely achievable when using consumer hardware and software anyway. There is room for more options on this front, as it is certainly possible to add support to the Emulation PCC to build and emit Vulkan pipeline cache binaries targeting specific devices exposed by the Emulation ICD, although even that may not be able to provide 100% guarantees on no online pipeline compilation due to the inherent behavior of the underlying Vulkan API used for the emulation.
Barring the drawbacks above, using this simple model for the output of the Emulation PCC has its own set of benefits. The most important one is that the produced pipeline cache binary is “portable”, i.e. it can be used with any physical device exposed by the Emulation ICD, even on a different system, because the produced binaries do not contain any hardware-specific or Vulkan implementation-specific data.
Advanced emulation features
The Emulation ICD comes with additional features that aim to help with specific aspects of Vulkan SC application development. In this section we will go through the most important ones.
Command pool memory tracking
The Emulation ICD tracks command pool memory consumption similarly to a native Vulkan SC driver and therefore reacts to the parameters specified through VkCommandPoolMemoryReservationCreateInfo. The actual command pool memory usage of individual command buffers and commands is emulated (unsurprisingly) based on calculations customizable in the source code generators of the ICD, as no such information can be extracted from the underlying Vulkan implementation.
Still, this can come useful to test the handling and resilience of Vulkan SC applications to command pool memory exhaustion that even triggers the corresponding faults like a native Vulkan SC driver would. This leads us to the next interesting feature…
Fault handling
The Emulation ICD is able to trigger Vulkan SC faults, an area that is typically difficult to test on native Vulkan SC implementations yet it is something that Vulkan SC applications are expected to gracefully handle, when possible. The Emulation ICD can and will trigger faults in various situations some of which can be used by Vulkan SC application developers to deterministically test fault handling behavior, e.g.:
VK_FAULT_TYPE_COMMAND_BUFFER_FULLfaults are reported when the application reserved command pool memory is exhaustedVK_FAULT_TYPE_PHYSICAL_DEVICEfaults are reported in case any applicable command detects aVK_ERROR_DEVICE_LOSTerror- In some cases the ICD detects invalid API usage conditions and will report
VK_FAULT_TYPE_INVALID_API_USAGEfaults, although the ICD remains largely reliant on valid API usage, per the usual Vulkan (SC) expectations, and the Vulkan SC Validation Layers should be used to test for valid API usage in the application
We particularly recommend developers to test their Vulkan SC applications early with command pool memory consumption related faults, because it enables basic test coverage for both fault handling and resiliency to command pool memory exhaustion.
Display emulation
At the time of writing, this is the most recent feature addition to the Vulkan SC Emulation driver stack. One particular difficulty when dealing with graphical Vulkan SC applications is that the window / display system support is limited and may even be esoteric on some safety-critical platforms and Vulkan SC does not support the traditional desktop window systems (the corresponding Vulkan extensions are not available in Vulkan SC). Our best bet for on-screen rendering is therefore the VK_KHR_display extension that enables direct-to-display access in both Vulkan and Vulkan SC. While support for this extension is available on some existing driver implementations, they often require additional system configuration and operations in order to perform direct-to-display presentation on systems with a running window system / compositor.
Our goal was to maximize portability, flexibility, and ease of use, so the Emulation ICD exposes support for VK_KHR_display by emulating displays as plain Win32/X11 windows. This only requires the underlying Vulkan implementation to support the VK_KHR_win32_surface (on Windows) / VK_KHR_xcb_surface (on Linux X11 or Wayland through XWayland) extension.
The display emulation support included in the Vulkan SC Emulation driver stack is also highly configurable. The number of emulated displays (the default is 4 displays) can be changed using the VKSC_EMULATION_DISPLAYS environment variable and even the names, geometry, etc. of the emulated displays can be customized by providing a simple JSON configuration file describing them through the VKSC_EMULATION_DISPLAY_CONFIG environment variable. A sample JSON display configuration file is available in the repository.
Of course, even with all these capabilities, the display emulation support has its limitations that may not enable Vulkan SC application developers to completely implement their display / window system interactions using the Emulation driver stack, as the target safety-critical platforms may have different capabilities / requirements and may not even expose display support through the VK_KHR_display extension but rather provide other extensions and APIs for the purpose. Nonetheless, having display emulation included in the stack should still enable Vulkan SC application developers to get pictures on screen and to cover most of their window system integration implementation on readily available consumer systems.
Test drive
The Vulkan SC Emulation driver stack is open-source and available on GitHub. The project can be easily built using CMake, as described in the build instructions. While CMake can be used to perform system-wide installs on Linux, it is more typical to use a custom install prefix when building from source, e.g.:
# Configure cmake -S <src-dir> -B <build-dir> -D CMAKE_INSTALL_PREFIX=<install-dir> -D CMAKE_BUILD_TYPE=Release -D UPDATE_DEPS=ON # Build & Install cmake --build <build-dir> --config Release --target install
Once built, in order to enable the Vulkan SC Loader to pick up the Vulkan SC Emulation driver stack, the driver location has to be included in the library loading path of the platform (PATH environment variable on Windows, LD_LIBRARY_PATH on Linux) and the VK_DRIVER_FILES or VK_ADD_DRIVER_FILES loader environment variable has to be set to point to the location of the driver’s JSON manifest file (this should be familiar to people messing around with custom Vulkan / Vulkan SC driver implementations):
# Windows set PATH=<install-dir>\bin;%PATH% set VK_DRIVER_FILES=<install-dir>\bin\vksconvk.json # Linux export LD_LIBRARY_PATH=<install-dir>/lib export VK_DRIVER_FILES=<install-dir>/share/vulkansc/icd.d/vksconvk.json
This configuration process and additional instructions related to the use of the Vulkan SC Emulation ICD can be found in the documentation.
The Vulkan SC version of the famous Cube Demo is now also available in the VulkanSC-Tools repository on GitHub. This builds out of the box with a portable pipeline cache binary embedded in the binary, compiled using the Emulation PCC, so executing the resulting vksccube program with the Vulkan SC Emulation driver stack configured should present you a familiar sight with a Vulkan SC twist to it:
Summary
The release of the Vulkan SC Emulation driver stack at the end of 2024 marked another milestone in the evolution of the Vulkan SC Ecosystem toolchain. The most important thing it brings to the table is that Vulkan SC application development is now accessible to developers on a wide range of consumer software and hardware environments. With the recent addition of display emulation, it is now also possible to get pretty pictures on screen without the need for special equipment or system configuration.
This stack can be of great value even for developers already using a vendor-specific Vulkan SC implementation and developer toolchain, as the nature and customizability of an emulation environment enables additional ways to exercise the API that can aid both in the development and the testing of Vulkan SC applications.
Another major benefit of the Vulkan SC Emulation driver stack is that, thanks to clever code generation, it can make new extensions promoted from Vulkan to Vulkan SC immediately available on compatible Vulkan implementations, therefore streamlining early adoption.
You can find the relevant components and related documentation in the following repositories:
- VulkanSC-Emulation – home of the Vulkan SC Emulation ICD (installable client driver) and PCC (pipeline cache compiler)
- VulkanSC-Tools – now also including the Vulkan SC Cube Demo
For further reading and materials, please check out our article Behind the scenes: the Vulkan SC Ecosystem, and the new official homepage of Vulkan SC.
