Compare commits

...
Author SHA1 Message Date
nmzik d8a4c83cc7 format src and tests with clang-format 2026-07-31 11:36:12 +02:00
nmzik 68be13345a fix(shader): support multisampled depth image loads 2026-07-31 11:36:12 +02:00
nmzik 167da0abe0 shader: fix readlane/writelane for inactive host lanes 2026-07-31 11:36:12 +02:00
nmzik 48c31d61ee Implement VideoDec2 2026-07-31 11:36:12 +02:00
nmzik 212282d693 fix(shader): preserve packed UINT16 MRT exports 2026-07-31 11:36:12 +02:00
nmzik 6bca35d1f5 renderer: broaden compatibility 2026-07-31 11:36:12 +02:00
M. AbdullahandGitHub e4ad5fc988 docs: add macOS build and run instructions (#137)
The README had macOS badges and an experimental-support note but no build,
run, or system-requirement information for the platform. Document the
Rosetta 2 / MoltenVK setup, the x86-64 configure invocation, the Qt
universal-build requirement, MoltenVK installation and signing, and the
SDL_VULKAN_LIBRARY variable needed at run time.
2026-07-31 05:32:29 +02:00
nmzik c0d3d261ea add TextToSpeech2 stubs 2026-07-31 04:05:50 +02:00
3b75a5659a shader: specialize cube image descriptors (#134)
* shader: specialize cube image descriptors

Track whether image descriptors refer to cube maps during resource specialization, and apply the coordinate offset conversion when sampling cube maps as 2D image arrays in SPIR-V emission.

* shader: fix cube array coordinate lowering

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-07-31 03:59:10 +02:00
M. AbdullahandGitHub d475387171 macOS: enable guest signal dispatch on the target thread (#136)
macos: enable guest signal dispatch on the target thread

The POSIX signal-dispatch path (pthread_kill based, added with the Linux
port) was compiled out on macOS, leaving KernelRaiseException to run the
guest handler on the calling thread. IL2CPP's garbage collector raises its
stop-the-world signal at every managed thread and each handler parks its
own thread until resume, so the collector parked itself and every Unity
title froze on the first collection.

Enable the same delivery path on macOS:
- translate between the Darwin mcontext (uc_mcontext->__ss) and the guest
  ucontext in CreateSignalUcontextFromHost/ApplySignalUcontextToHost
- use SIGUSR1 as the host dispatch signal (macOS has no realtime signals)
- block the dispatch signal inside the host fault handler so a suspend
  request cannot preempt fault resolution between the protection fix and
  the retry

Windows and Linux are unchanged.
2026-07-31 03:47:04 +02:00
nmzikandGitHub 2f5396c6a5 Rework guest memory tracking/virtual address space/direct and flexible memory (#135)
* Rework guest memory tracking

* add unknwon flag

* Fix macOS guest address-space reservation
2026-07-31 03:07:17 +02:00
ecb48f90bb Emulate SHA-NI and fix SSE4a EXTRQ/INSERTQ register form (#126)
* Emulate SHA-NI on illegal instruction faults

* Fix SSE4a EXTRQ/INSERTQ register form

* Fix SHA-NI memory operand emulation

* Revert "Fix SSE4a EXTRQ/INSERTQ register form"

This reverts commit ea2b54a4d0.

---------

Co-authored-by: neobugs1 <neobugs1@users.noreply.github.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-07-30 16:21:36 +02:00
nmzik 77aa28b27c update README 2026-07-30 05:16:37 +02:00
nmzikandGitHub d04938c88c Embedded fetch shader: Fix overlapping buffer loads (#133)
Fix overlapping buffer loads. Fixes many games
2026-07-30 05:08:38 +02:00
141 changed files with 25319 additions and 24349 deletions
+17 -3
View File
@@ -83,7 +83,12 @@ jobs:
- name: Build
shell: cmd
run: |
cmake --build _Build/windows --target launcher --parallel
cmake --build _Build/windows --target launcher virtual_memory_allocation_tests --parallel
- name: Test
shell: cmd
run: |
ctest --test-dir _Build/windows --output-on-failure -R "^virtual_memory_allocation$"
- name: Install
shell: cmd
@@ -153,7 +158,15 @@ jobs:
- name: Build
shell: bash
run: |
cmake --build _Build/macos --target launcher --parallel
cmake --build _Build/macos \
--target launcher virtual_memory_allocation_tests \
--parallel
- name: Test
shell: bash
run: |
ctest --test-dir _Build/macos --output-on-failure \
-R '^virtual_memory_allocation$'
- name: Install
shell: bash
@@ -284,13 +297,14 @@ jobs:
run: |
cmake --build _Build/linux \
--target launcher page_manager_tests memory_tracker_tests \
virtual_memory_allocation_tests \
--parallel
- name: Test
shell: bash
run: |
ctest --test-dir _Build/linux --output-on-failure \
-R '^(page_manager|memory_tracker)$'
-R '^(page_manager|memory_tracker|virtual_memory_allocation)$'
- name: Install
shell: bash
+67 -6
View File
@@ -27,8 +27,9 @@ Development is focused on compatibility and boot reliability.
Windows is the primary platform and receives the most testing. Linux builds and runs; see
[Building on Linux](#building-on-linux).
macOS support is experimental. Compatibility with the same games on Windows and macOS has not yet
been tested.
macOS support is experimental. The emulator is built for x86-64 and runs on Apple Silicon under
Rosetta 2, with Vulkan provided by MoltenVK. A small number of titles have been verified in-game
on Apple Silicon hardware; see [Building on macOS](#building-on-macos).
## Bugs and Issues
@@ -50,7 +51,7 @@ graphical glitches, low compatibility, and poor performance.
</tr>
<tr>
<td align="center">
<strong>Minecraft Legends</strong><br>
<strong>Neptunia ReVerse</strong><br>
<img src="docs/screenshots/ps5-04.png" width="300" alt="Minecraft Legends running in KytyPS5">
</td>
<td align="center">
@@ -58,8 +59,20 @@ graphical glitches, low compatibility, and poor performance.
<img src="docs/screenshots/ps5-05.png" width="300" alt="SILENT HILL: The Short Message running in KytyPS5">
</td>
</tr>
<tr>
<td align="center">
<strong>Hellboy</strong><br>
<img src="docs/screenshots/ps5-02.png" width="300" alt="Disgaea 6 running in KytyPS5">
</td>
<td align="center">
<strong>Paleo Pines</strong><br>
<img src="docs/screenshots/ps5-06.png" width="300" alt="Dreaming Sarah running in KytyPS5">
</td>
</tr>
</table>
<p align="center"><em>And many more...</em></p>
## Contributing
Testing games and submitting detailed bug reports are useful ways to contribute. Search existing
@@ -102,9 +115,10 @@ the Vulkan/SPIR-V validation rules.
### System requirements
- Windows 10 version 1803, or a current Linux distribution
- A 64-bit x86 processor
- A Vulkan 1.3-capable GPU with current drivers
- Windows 10 version 1803, a current Linux distribution, or macOS on Apple Silicon
- A 64-bit x86 processor (on macOS, an Apple Silicon processor with Rosetta 2)
- A Vulkan 1.3-capable GPU with current drivers (on macOS, Vulkan is provided by the bundled
MoltenVK)
### Build requirements (Windows)
@@ -176,6 +190,45 @@ time.
Note that the CMake source root is `src`, not the repository root.
### Building on macOS
macOS builds target x86-64 and run under Rosetta 2 on Apple Silicon, so the PS5's x86-64 game
code executes through the same translation layer as the emulator itself. Prebuilt archives are
attached to releases; the steps below are for building from source.
Requirements:
- An Apple Silicon Mac with Rosetta 2 installed (`softwareupdate --install-rosetta`)
- Xcode (or the Command Line Tools)
- Homebrew packages: `brew install cmake ninja glslang`
- Qt 6 (Concurrent, Network, Widgets) with x86-64 support. The official Qt installation is
universal and works; Homebrew's Qt is arm64-only and will not link
```bash
git submodule update --init --recursive
cmake -S src -B _Build/macos -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_OSX_ARCHITECTURES=x86_64 \
-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_PREFIX_PATH="$Qt6_DIR"
cmake --build _Build/macos --target launcher --parallel
cmake --install _Build/macos --prefix _Build/macos/install
```
The build re-signs `kyty_emulator` with the JIT entitlements it needs to execute translated
guest code; no manual signing step is required.
Vulkan comes from MoltenVK. Download `MoltenVK-macos.tar` from the
[MoltenVK releases](https://github.com/KhronosGroup/MoltenVK/releases), then copy
`MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib` next to `kyty_emulator` and ad-hoc sign it:
```bash
codesign --force --sign - _Build/macos/install/libMoltenVK.dylib
```
Release archives already include a signed `libMoltenVK.dylib`.
### Visual Studio Code
A ready-made Visual Studio Code setup is included in [`.vscode`](.vscode). It configures CMake
@@ -221,6 +274,14 @@ The emulator can also be started directly with a legally obtained game directory
./_Build/linux/install/kyty_emulator --game "/games/ExampleGame"
```
On macOS, point SDL at the MoltenVK library explicitly; the hardened runtime prevents it from
being picked up from the executable's directory:
```bash
cd _Build/macos/install
SDL_VULKAN_LIBRARY="$PWD/libMoltenVK.dylib" ./kyty_emulator --game "/games/ExampleGame"
```
Run `kyty_emulator --help` to see the available graphics, logging, validation, profiling, and
debugging options.
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.7 MiB

After

Width:  |  Height:  |  Size: 2.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.4 MiB

+16 -1
View File
@@ -314,11 +314,22 @@ function(add_kyty_full_emulator_test target source)
endif()
endfunction()
function(configure_macos_guest_address_space target)
if(APPLE AND (CMAKE_OSX_ARCHITECTURES STREQUAL "x86_64" OR
(NOT CMAKE_OSX_ARCHITECTURES AND CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64)$")))
target_sources(${target} PRIVATE kernel/macosGuestAddressSpace.cpp)
target_compile_definitions(${target} PRIVATE KYTY_LINKED_GUEST_ADDRESS_SPACE=1)
target_link_options(${target} PRIVATE
-Wl,-ld_classic,-no_pie,-no_fixup_chains,-no_huge,-pagezero_size,0x40000,-segaddr,SYSTEM_MANAGED,0x40000,-segaddr,SYSTEM_RESERVED,0x7ffffc000,-segaddr,USER_AREA,0x7000000000,-image_base,0x700000000000)
endif()
endfunction()
add_kyty_full_emulator_test(shader_cfg_tests ../tests/shaderCfgTests.cpp)
add_executable(scalar_provenance_tests EXCLUDE_FROM_ALL
../tests/ScalarProvenanceTests.cpp
graphics/host_gpu/hostMemory.cpp
graphics/shader/recompiler/ir/ReadLaneElimination.cpp
graphics/shader/recompiler/ir/ScalarProvenance.cpp
graphics/shader/recompiler/ir/SrtWalker.cpp
)
@@ -338,7 +349,6 @@ add_executable(memory_tracker_tests EXCLUDE_FROM_ALL
)
target_link_libraries(memory_tracker_tests fmt::fmt common)
target_include_directories(memory_tracker_tests PRIVATE ${inc_headers})
target_compile_definitions(memory_tracker_tests PRIVATE KYTY_MEMORY_TRACKER_TESTS=1)
add_executable(shader_vertex_metadata_tests EXCLUDE_FROM_ALL
../tests/ShaderVertexMetadataTests.cpp
@@ -421,6 +431,7 @@ target_sources(shader_recompiler_compute_tests PRIVATE
add_kyty_full_emulator_test(virtual_memory_allocation_tests ../tests/VirtualMemoryAllocationTests.cpp)
target_compile_definitions(virtual_memory_allocation_tests PRIVATE
KYTY_VIRTUAL_MEMORY_ALLOCATION_TESTS=1)
configure_macos_guest_address_space(virtual_memory_allocation_tests)
# These tests use exceptions.
if(NOT KYTY_CLANG_CL)
@@ -431,12 +442,15 @@ if(NOT KYTY_CLANG_CL)
endif()
if(BUILD_TESTING)
add_test(NAME scalar_provenance COMMAND $<TARGET_FILE:scalar_provenance_tests>)
add_test(NAME image_page_table COMMAND $<TARGET_FILE:image_page_table_tests>)
add_test(NAME memory_tracker COMMAND $<TARGET_FILE:memory_tracker_tests>)
add_test(NAME page_manager COMMAND $<TARGET_FILE:page_manager_tests>)
add_test(NAME resource_mutex COMMAND $<TARGET_FILE:resource_mutex_tests>)
add_test(NAME event_queue_lifetime COMMAND $<TARGET_FILE:event_queue_lifetime_tests>)
add_test(NAME shader_recompiler_compute COMMAND $<TARGET_FILE:shader_recompiler_compute_tests>)
add_test(NAME virtual_memory_allocation
COMMAND $<TARGET_FILE:virtual_memory_allocation_tests>)
add_test(NAME command_scheduler_timeline
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --scheduler-only)
add_test(NAME stream_buffer_ring
@@ -470,6 +484,7 @@ endif()
add_executable(kyty_emulator main.cpp ${kyty_emulator_src})
configure_macos_guest_address_space(kyty_emulator)
target_link_libraries(kyty_emulator ${kyty_emulator_link_libraries})
if (WIN32)
+10 -6
View File
@@ -175,9 +175,9 @@ static void SignalHandler(int sig, siginfo_t* si, void* uctx) {
}
g_in_exception_filter = true;
auto* uc = static_cast<ucontext_t*>(uctx);
const auto* mc = uc->uc_mcontext;
const auto& ss = mc->__ss;
auto* uc = static_cast<ucontext_t*>(uctx);
const auto* mc = uc->uc_mcontext;
const auto& ss = mc->__ss;
ExceptionInfo info {};
info.exception_address = ss.__rip;
@@ -214,7 +214,7 @@ static void SignalHandler(int sig, siginfo_t* si, void* uctx) {
FailFast("host exception callback is null");
}
const bool resolved = handler(info);
const bool resolved = handler(info);
g_in_exception_filter = false;
if (resolved) {
@@ -255,8 +255,8 @@ static void SignalHandler(int signal_number, siginfo_t* signal_info, void* nativ
info.native_context = context;
if (signal_number == SIGSEGV || signal_number == SIGBUS) {
info.type = ExceptionType::AccessViolation;
const auto error_code = static_cast<uint64_t>(gregs[REG_ERR]);
info.type = ExceptionType::AccessViolation;
const auto error_code = static_cast<uint64_t>(gregs[REG_ERR]);
if ((error_code & PAGE_FAULT_ERROR_INSTRUCTION) != 0) {
info.access_violation_type = AccessViolationType::Execute;
} else if ((error_code & PAGE_FAULT_ERROR_WRITE) != 0) {
@@ -324,6 +324,10 @@ bool InstallHandler(Handler handler) {
sa.sa_sigaction = SignalHandler;
sa.sa_flags = SA_SIGINFO;
sigemptyset(&sa.sa_mask);
// The guest signal-dispatch path (KernelRaiseException) interrupts threads with
// SIGUSR1; block it while a fault is being resolved so a stop-the-world request
// cannot preempt the handler between the protection fix and the retry.
sigaddset(&sa.sa_mask, SIGUSR1);
// macOS raises SIGBUS for protection faults on some paths and SIGSEGV on others;
// SIGILL covers instructions the host cannot execute (routed to the x64 emulator).
+7 -8
View File
@@ -19,10 +19,10 @@ class LeastRecentlyUsedCache {
public:
[[nodiscard]] size_t Insert(Object object, Tick tick) {
const auto id = Build();
const auto id = Build();
auto& item = m_items[id];
item.object = std::move(object);
item.tick = tick;
item.object = std::move(object);
item.tick = tick;
Attach(item);
return id;
}
@@ -49,8 +49,7 @@ public:
template <typename Function>
void ForEachItemBelow(Tick tick, Function&& function) {
constexpr bool ReturnsBool =
std::is_same_v<std::invoke_result_t<Function, Object>, bool>;
constexpr bool ReturnsBool = std::is_same_v<std::invoke_result_t<Function, Object>, bool>;
for (auto* item = m_first; item != nullptr;) {
if (item->tick > tick) {
return;
@@ -87,10 +86,10 @@ private:
m_last = &item;
return;
}
item.prev = m_last;
item.prev = m_last;
m_last->next = &item;
item.next = nullptr;
m_last = &item;
item.next = nullptr;
m_last = &item;
}
void Detach(Item& item) {
+3 -4
View File
@@ -31,10 +31,9 @@ static bool OnOwnStack() {
if (pthread_getattr_np(pthread_self(), &attr) != 0) {
return false;
}
void* base = nullptr;
size_t size = 0;
const bool ok =
pthread_attr_getstack(&attr, &base, &size) == 0 && base != nullptr && size != 0;
void* base = nullptr;
size_t size = 0;
const bool ok = pthread_attr_getstack(&attr, &base, &size) == 0 && base != nullptr && size != 0;
pthread_attr_destroy(&attr);
if (!ok) {
return false;
+3 -5
View File
@@ -172,8 +172,7 @@ sys_file_t* SysFileCreate(const std::filesystem::path& file_name) {
return ret;
}
sys_file_t* SysFileOpenR(const std::filesystem::path& file_name,
sys_file_cache_type_t cache_type) {
sys_file_t* SysFileOpenR(const std::filesystem::path& file_name, sys_file_cache_type_t cache_type) {
auto* ret = new sys_file_t;
ret->type = SYS_FILE_FILE;
@@ -218,8 +217,7 @@ sys_file_t* SysFileCreate() {
return ret;
}
sys_file_t* SysFileOpenW(const std::filesystem::path& file_name,
sys_file_cache_type_t cache_type) {
sys_file_t* SysFileOpenW(const std::filesystem::path& file_name, sys_file_cache_type_t cache_type) {
auto* ret = new sys_file_t;
auto real_name = get_internal_name(file_name);
@@ -241,7 +239,7 @@ sys_file_t* SysFileOpenW(const std::filesystem::path& file_name,
}
sys_file_t* SysFileOpenRw(const std::filesystem::path& file_name,
sys_file_cache_type_t cache_type) {
sys_file_cache_type_t cache_type) {
auto* ret = new sys_file_t;
auto real_name = get_internal_name(file_name);
+13 -13
View File
@@ -136,8 +136,8 @@ static void* map_anonymous(uintptr_t addr, size_t size, int protect, int flags)
break;
}
const auto hint = (top - step) & ~(LOW_ARENA_GRAIN - 1);
void* ptr = mmap(reinterpret_cast<void*>(hint), size, protect,
flags | MAP_FIXED_NOREPLACE, -1, 0); // NOLINT
void* ptr = mmap(reinterpret_cast<void*>(hint), size, protect, flags | MAP_FIXED_NOREPLACE,
-1, 0); // NOLINT
if (ptr != MAP_FAILED) {
return ptr;
}
@@ -161,8 +161,8 @@ uint64_t SysVirtualAlloc(uint64_t address, uint64_t size, VirtualMemory::Mode mo
if (ptr != MAP_FAILED) {
pthread_mutex_lock(&g_virtual_mutex);
record_alloc(ret_addr, size);
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
for (uintptr_t page = page_start; page <= page_end; page++) {
(*g_protects)[page] = protect;
}
@@ -194,8 +194,8 @@ uint64_t SysVirtualAllocAligned(uint64_t address, uint64_t size, VirtualMemory::
if (ptr != MAP_FAILED && ((ret_addr & (alignment - 1)) != 0)) {
munmap(ptr, size);
ptr = map_anonymous(addr, size + alignment, protect,
MAP_PRIVATE | MAP_ANON | MAP_NORESERVE);
ptr =
map_anonymous(addr, size + alignment, protect, MAP_PRIVATE | MAP_ANON | MAP_NORESERVE);
ret_addr = reinterpret_cast<uintptr_t>(ptr);
if (ptr != MAP_FAILED) {
#if defined(__APPLE__)
@@ -251,8 +251,8 @@ uint64_t SysVirtualAllocAligned(uint64_t address, uint64_t size, VirtualMemory::
pthread_mutex_lock(&g_virtual_mutex);
record_alloc(ret_addr, size);
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
for (uintptr_t page = page_start; page <= page_end; page++) {
(*g_protects)[page] = protect;
}
@@ -266,9 +266,9 @@ uint64_t SysVirtualAllocAligned(uint64_t address, uint64_t size, VirtualMemory::
// the first mapped region at or above `region_addr`; if it begins before the end of the
// requested range, the range overlaps an existing mapping.
static bool is_mapped(void* ptr, size_t length) {
auto query_addr = reinterpret_cast<mach_vm_address_t>(ptr);
mach_vm_address_t region_addr = query_addr;
mach_vm_size_t region_size = 0;
auto query_addr = reinterpret_cast<mach_vm_address_t>(ptr);
mach_vm_address_t region_addr = query_addr;
mach_vm_size_t region_size = 0;
vm_region_basic_info_data_64_t info {};
mach_msg_type_number_t count = VM_REGION_BASIC_INFO_COUNT_64;
mach_port_t object_name = MACH_PORT_NULL;
@@ -337,8 +337,8 @@ bool SysVirtualAllocFixed(uint64_t address, uint64_t size, VirtualMemory::Mode m
if (ptr != MAP_FAILED) {
pthread_mutex_lock(&g_virtual_mutex);
record_alloc(ret_addr, size);
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
for (uintptr_t page = page_start; page <= page_end; page++) {
(*g_protects)[page] = protect;
}
+1 -1
View File
@@ -5,9 +5,9 @@
#include <algorithm>
#include <atomic>
#include <cerrno>
#include <chrono> // IWYU pragma: keep
#include <condition_variable> // IWYU pragma: keep
#include <cerrno>
#include <mutex>
#include <vector>
+2 -4
View File
@@ -11,7 +11,7 @@ template <typename Result, typename... Args>
class UniqueFunction {
class CallableBase {
public:
virtual ~CallableBase() = default;
virtual ~CallableBase() = default;
virtual Result Invoke(Args&&... args) = 0;
};
@@ -20,9 +20,7 @@ class UniqueFunction {
public:
explicit Callable(Function function): m_function(std::move(function)) {}
Result Invoke(Args&&... args) override {
return m_function(std::forward<Args>(args)...);
}
Result Invoke(Args&&... args) override { return m_function(std::forward<Args>(args)...); }
private:
Function m_function;
-19
View File
@@ -58,25 +58,6 @@ bool FlushInstructionCache(uint64_t address, uint64_t size) {
return SysVirtualFlushInstructionCache(address, size);
}
bool PatchReplace(uint64_t vaddr, uint64_t value) {
Mode old_mode {};
Protect(vaddr, 8, Mode::ReadWrite, &old_mode);
auto* ptr = reinterpret_cast<uint64_t*>(vaddr);
bool ret = (*ptr != value);
*ptr = value;
Protect(vaddr, 8, old_mode);
if (IsExecute(old_mode)) {
FlushInstructionCache(vaddr, 8);
}
return ret;
}
} // namespace VirtualMemory
} // namespace Common
-1
View File
@@ -37,7 +37,6 @@ bool Free(uint64_t address);
bool FreeRange(uint64_t address, uint64_t size);
bool Protect(uint64_t address, uint64_t size, Mode mode, Mode* old_mode = nullptr);
bool FlushInstructionCache(uint64_t address, uint64_t size);
bool PatchReplace(uint64_t vaddr, uint64_t value);
} // namespace VirtualMemory
+13 -12
View File
@@ -105,7 +105,7 @@ static void ClearDebugTextureFolder() {
}
}
static void Init(const Config::ConfigOptions& cfg) {
static void Init(const Config::ConfigOptions& cfg, const std::filesystem::path& param_json) {
EXIT_IF(!Common::Thread::IsMainThread());
auto* slist = Common::SubsystemsList::Instance();
@@ -127,12 +127,21 @@ static void Init(const Config::ConfigOptions& cfg) {
slist->InitAll(true);
Config::Load(cfg);
slist->Add(log, {core, config});
slist->InitAll(true);
if (Common::File::IsFileExisting(param_json)) {
Loader::SystemContentLoadParamSfo(param_json);
if (const auto flexible_memory_size = Loader::SystemContentGetFlexibleMemorySize();
flexible_memory_size != 0) {
Libs::LibKernel::Memory::SetFlexibleMemorySize(flexible_memory_size);
}
}
slist->Add(audio, {core, log, pthread, memory});
slist->Add(controller, {core, log, config});
slist->Add(file_system, {core, log, pthread});
slist->Add(graphics, {core, log, pthread, memory, config, profiler, controller});
slist->Add(log, {core, config});
slist->Add(memory, {core, log});
slist->Add(network, {core, log, pthread});
slist->Add(profiler, {core, config});
@@ -180,7 +189,8 @@ void Run(const RunOptions& options) {
EXIT("ELF is required\n");
}
Init(options.config);
const auto param_json = options.app0_dir / "sce_sys" / "param.json";
Init(options.config, param_json);
ClearDebugTextureFolder();
@@ -192,15 +202,6 @@ void Run(const RunOptions& options) {
Libs::LibKernel::FileSystem::Mount(options.app0_dir, "/app0");
Libs::LibKernel::FileSystem::Mount(options.app0_dir, "/hostapp");
auto param_json = options.app0_dir / "sce_sys" / "param.json";
if (Common::File::IsFileExisting(param_json)) {
Loader::SystemContentLoadParamSfo(param_json);
if (auto flexible_memory_size = Loader::SystemContentGetFlexibleMemorySize();
flexible_memory_size != 0) {
Libs::LibKernel::Memory::SetFlexibleMemorySize(flexible_memory_size);
}
}
MountSandboxDirs();
auto* rt = Common::Singleton<Loader::RuntimeLinker>::Instance();
@@ -158,7 +158,7 @@ private:
void CheckBuffer() const { GetScheduler().CheckActive(); }
GpuResourceManager& GetGpuResources() const { return m_renderer.GetGpuResources(); }
RenderContext& m_renderer;
RenderContext& m_renderer;
HW::Context m_ctx;
HW::UserConfig m_ucfg;
HW::Shader m_sh_ctx;
@@ -170,9 +170,9 @@ private:
uint64_t m_dispatch_indirect_args_base_addr = 0;
uint32_t m_num_instances = 1;
uint32_t m_de_count = 0;
uint32_t m_ce_count = 0;
bool m_ce_complete = false;
uint32_t m_de_count = 0;
uint32_t m_ce_count = 0;
bool m_ce_complete = false;
bool m_readback_active = false;
uint32_t m_const_ram[0x3000] = {0};
+2
View File
@@ -374,6 +374,8 @@ enum class BufferFormat : uint32_t {
k32_32_32_32UInt = 75,
k32_32_32_32SInt = 76,
k32_32_32_32Float = 77,
k8Srgb = 128,
k8_8Srgb = 129,
k8_8_8_8Srgb = 130,
k9_9_9_5Float = 132,
k5_6_5UNorm = 133,
+2
View File
@@ -57,6 +57,8 @@ constexpr FormatInfo kFormatInfo[] = {
{GpuEnumValue(BufferFormat::k32_32_32_32UInt), 16, 0, 16, true, true},
{GpuEnumValue(BufferFormat::k32_32_32_32SInt), 16, 0, 16, false, false},
{GpuEnumValue(BufferFormat::k32_32_32_32Float), 16, 0, 16, true, false},
{GpuEnumValue(BufferFormat::k8Srgb), 1, 0, 0, true, false},
{GpuEnumValue(BufferFormat::k8_8Srgb), 2, 0, 0, true, false},
{GpuEnumValue(BufferFormat::k8_8_8_8Srgb), 4, 0, 4, true, false},
{GpuEnumValue(BufferFormat::k9_9_9_5Float), 4, 0, 0, true, false},
{GpuEnumValue(BufferFormat::k5_6_5UNorm), 2, 0, 2, true, false},
+11 -13
View File
@@ -962,9 +962,8 @@ void CommandProcessor::DrawIndexOffset(uint32_t index_offset, uint32_t index_cou
auto* index_addr = reinterpret_cast<const void*>(
m_index_base_addr + static_cast<uint64_t>(index_offset) * index_size);
m_renderer.GetRenderExecutor().DrawIndex(m_submit_id, CurrentBuffer(),
m_index_type_and_size, index_count, index_addr,
flags, 1, m_num_instances);
m_renderer.GetRenderExecutor().DrawIndex(m_submit_id, CurrentBuffer(), m_index_type_and_size,
index_count, index_addr, flags, 1, m_num_instances);
}
void CommandProcessor::DrawIndirect(uint32_t data_offset, uint32_t draw_initiator, bool indexed) {
@@ -1190,8 +1189,8 @@ void CommandProcessor::DispatchDirect(uint32_t thread_group_x, uint32_t thread_g
}
}
m_renderer.GetRenderExecutor().DispatchDirect(
m_submit_id, CurrentBuffer(), thread_group_x, thread_group_y, thread_group_z, mode);
m_renderer.GetRenderExecutor().DispatchDirect(m_submit_id, CurrentBuffer(), thread_group_x,
thread_group_y, thread_group_z, mode);
}
constexpr uint32_t DispatchInitiatorUseThreadDimensions = 1u << 5u;
@@ -1237,16 +1236,16 @@ void CommandProcessor::DrawIndexAuto(uint32_t index_count, uint32_t flags,
uint32_t first_vertex, uint32_t first_instance) {
CheckBuffer();
m_renderer.GetRenderExecutor().DrawAuto(
m_submit_id, CurrentBuffer(), index_count, flags, render_target_slice_offset,
instance_count, first_vertex, first_instance);
m_renderer.GetRenderExecutor().DrawAuto(m_submit_id, CurrentBuffer(), index_count, flags,
render_target_slice_offset, instance_count,
first_vertex, first_instance);
}
void CommandProcessor::WaitFlipDone(uint32_t video_out_handle, uint32_t display_buffer_index) {
BufferFlush();
m_renderer.GetVideoOut().WaitFlipDone(static_cast<int>(video_out_handle),
static_cast<int>(display_buffer_index));
static_cast<int>(display_buffer_index));
}
template <typename T>
@@ -1317,8 +1316,8 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
if (eop_event_type == 0x2f && cache_action == 0x00 && event_index == 0x06) {
auto* dst = static_cast<uint32_t*>(dst_gpu_addr);
SynchronizeGpu();
Sync::ReadGds(m_renderer.GetBufferCache().GetGdsBuffer(), dst,
value & 0xffffu, value >> 16u);
Sync::ReadGds(m_renderer.GetBufferCache().GetGdsBuffer(), dst, value & 0xffffu,
value >> 16u);
Sync::WriteAtEndOfPipeGds32(m_submit_id, CurrentBuffer(), dst, value & 0xffffu,
value >> 16u);
return;
@@ -1486,8 +1485,7 @@ void CommandProcessor::EmitGlobalBarrier() {
barrier.srcStageMask = vk::PipelineStageFlagBits2::eAllCommands;
barrier.srcAccessMask = vk::AccessFlagBits2::eMemoryWrite;
barrier.dstStageMask = vk::PipelineStageFlagBits2::eAllCommands;
barrier.dstAccessMask =
vk::AccessFlagBits2::eMemoryRead | vk::AccessFlagBits2::eMemoryWrite;
barrier.dstAccessMask = vk::AccessFlagBits2::eMemoryRead | vk::AccessFlagBits2::eMemoryWrite;
vk::DependencyInfo dependency {};
dependency.memoryBarrierCount = 1;
+11 -11
View File
@@ -65,41 +65,41 @@ struct TileVolumeLayout {
};
bool TileGetBlockLayout(TileBlockFamily family, uint32_t bytes_per_element,
TileBlockLayout& layout);
TileBlockLayout& layout);
bool TileGetBlockOffset(const TileBlockLayout& layout, uint32_t x, uint32_t y, uint32_t z,
uint32_t& byte_offset);
uint32_t& byte_offset);
bool TileGetBlockXor(const TileBlockLayout& layout, uint32_t block_x, uint32_t block_y,
uint32_t& byte_offset);
uint32_t& byte_offset);
bool TileGetBlockXor(const TileBlockLayout& layout, uint32_t block_x, uint32_t block_y,
uint32_t block_z, uint32_t& byte_offset);
uint32_t block_z, uint32_t& byte_offset);
bool TileIsStandard256BTextureSupported(uint32_t format);
bool TileIsStandard4KBTextureSupported(uint32_t format);
bool TileIsStandard64KBTextureSupported(uint32_t format);
bool TileGetTextureVolumeLayout(uint32_t format, uint32_t width, uint32_t height, uint32_t depth,
uint32_t levels, uint32_t tile, TileVolumeLayout& layout);
uint32_t levels, uint32_t tile, TileVolumeLayout& layout);
bool TileGetHtileSize(uint32_t width, uint32_t height, TileSizeAlign& htile_size);
bool TileGetDepthSize(uint32_t width, uint32_t height, uint32_t pitch, uint32_t z_format,
uint32_t stencil_format, bool htile, TileSizeAlign& stencil_size,
TileSizeAlign& htile_size, TileSizeAlign& depth_size,
uint32_t stencil_format, bool htile, TileSizeAlign& stencil_size,
TileSizeAlign& htile_size, TileSizeAlign& depth_size,
uint32_t num_fragments_log2 = 0);
uint32_t TileGetRenderTargetPitch(uint32_t width, uint32_t bytes_per_element,
uint32_t num_fragments_log2 = 0);
uint32_t TileGetDepthPitch(uint32_t width, uint32_t bytes_per_element,
uint32_t num_fragments_log2 = 0);
bool TileGetRenderTargetSize(uint32_t width, uint32_t height, uint32_t pitch,
uint32_t bytes_per_element, TileSizeAlign& total_size,
uint32_t bytes_per_element, TileSizeAlign& total_size,
uint32_t num_fragments_log2 = 0);
bool TileGetRenderTargetMipLayout(uint32_t width, uint32_t height, uint32_t pitch,
uint32_t bytes_per_element, uint32_t levels,
TileSizeAlign& total_size, TileSizeOffset* level_sizes,
uint32_t bytes_per_element, uint32_t levels,
TileSizeAlign& total_size, TileSizeOffset* level_sizes,
TilePaddedSize* padded_size);
void TileGetTextureSize(uint32_t format, uint32_t width, uint32_t height, uint32_t pitch,
uint32_t levels, uint32_t tile, TileSizeAlign* total_size,
TileSizeOffset* level_sizes, TilePaddedSize* padded_size);
void TileGetTextureTotalSize(uint32_t format, uint32_t width, uint32_t height, uint32_t depth,
uint32_t pitch, uint32_t levels, uint32_t tile, bool volume_texture,
TileSizeAlign& total_size);
TileSizeAlign& total_size);
uint32_t TileGetTexturePitch(uint32_t format, uint32_t width, uint32_t levels, uint32_t tile);
} // namespace Libs::Graphics
+12 -12
View File
@@ -60,19 +60,19 @@ struct VulkanImage {
VulkanImage() = default;
KYTY_CLASS_NO_COPY(VulkanImage);
vk::Format format = vk::Format::eUndefined;
vk::ImageType image_type = vk::ImageType::e2D;
vk::Extent3D extent = {1, 1, 1};
uint32_t guest_pitch = 0;
uint32_t layers = 1;
uint32_t mip_levels = 1;
uint32_t samples = 1;
vk::ImageUsageFlags usage = {};
vk::ImageCreateFlags flags = {};
vk::Image image = nullptr;
VulkanImageState state;
vk::Format format = vk::Format::eUndefined;
vk::ImageType image_type = vk::ImageType::e2D;
vk::Extent3D extent = {1, 1, 1};
uint32_t guest_pitch = 0;
uint32_t layers = 1;
uint32_t mip_levels = 1;
uint32_t samples = 1;
vk::ImageUsageFlags usage = {};
vk::ImageCreateFlags flags = {};
vk::Image image = nullptr;
VulkanImageState state;
std::vector<VulkanImageState> subresource_states;
Graphics::VulkanMemory memory;
Graphics::VulkanMemory memory;
};
struct VulkanBuffer {
+1 -1
View File
@@ -30,7 +30,7 @@ bool IsAccessible(DWORD protect, HostMemoryAccess access) {
} // namespace
bool HostMemoryQueryRange(uint64_t addr, uint64_t requested_size, HostMemoryAccess access,
uint64_t& accessible_size) {
uint64_t& accessible_size) {
accessible_size = 0;
if (addr == 0 || requested_size == 0) {
return false;
+1 -1
View File
@@ -8,7 +8,7 @@ namespace Libs::Graphics {
enum class HostMemoryAccess { Read, Mapped };
bool HostMemoryQueryRange(uint64_t addr, uint64_t requested_size, HostMemoryAccess access,
uint64_t& accessible_size);
uint64_t& accessible_size);
bool HostMemoryQueryReadable(uint64_t addr, uint64_t requested_size, uint64_t& readable_size);
bool HostMemoryIsReadable(uint64_t addr);
bool HostMemoryRangeIsReadable(uint64_t addr, uint64_t size);
-33
View File
@@ -4,16 +4,6 @@
namespace Libs::Graphics {
#if defined(KYTY_MEMORY_TRACKER_TESTS)
namespace {
std::atomic<MemoryTracker::UnmapContentionHook> g_unmap_contention_hook {nullptr};
}
void MemoryTracker::SetUnmapContentionHook(UnmapContentionHook hook) noexcept {
g_unmap_contention_hook.store(hook, std::memory_order_release);
}
#endif
static_assert(std::atomic<void*>::is_always_lock_free);
MemoryTracker::MemoryTracker(PageManager& page_manager, PageWatchMode gpu_watch_mode)
@@ -94,7 +84,6 @@ RegionManager* MemoryTracker::GetOrCreateRegion(uint64_t index) {
bool MemoryTracker::IsRegionCpuModified(uint64_t vaddr, uint64_t size) {
CheckNotInUploadCallback();
std::lock_guard access(m_access_mutex);
RequireMapped(vaddr, size);
return Iterate<true>(vaddr, size, [](RegionManager* manager, uint64_t offset, uint64_t bytes) {
std::scoped_lock lock(manager->lock);
return manager->IsModified<DirtySource::Cpu>(offset, bytes);
@@ -104,7 +93,6 @@ bool MemoryTracker::IsRegionCpuModified(uint64_t vaddr, uint64_t size) {
bool MemoryTracker::IsRegionGpuModified(uint64_t vaddr, uint64_t size) {
CheckNotInUploadCallback();
std::lock_guard access(m_access_mutex);
RequireMapped(vaddr, size);
return Iterate<false>(vaddr, size, [](RegionManager* manager, uint64_t offset, uint64_t bytes) {
std::scoped_lock lock(manager->lock);
return manager->IsModified<DirtySource::Gpu>(offset, bytes);
@@ -114,7 +102,6 @@ bool MemoryTracker::IsRegionGpuModified(uint64_t vaddr, uint64_t size) {
void MemoryTracker::MarkRegionAsCpuModified(uint64_t vaddr, uint64_t size) {
CheckNotInUploadCallback();
std::lock_guard access(m_access_mutex);
RequireMapped(vaddr, size);
Iterate<true>(vaddr, size, [](RegionManager* manager, uint64_t offset, uint64_t bytes) {
std::scoped_lock lock(manager->lock);
const auto changed =
@@ -126,7 +113,6 @@ void MemoryTracker::MarkRegionAsCpuModified(uint64_t vaddr, uint64_t size) {
void MemoryTracker::MarkRegionAsGpuModified(uint64_t vaddr, uint64_t size) {
CheckNotInUploadCallback();
std::lock_guard access(m_access_mutex);
RequireMapped(vaddr, size);
Iterate<true>(vaddr, size, [this](RegionManager* manager, uint64_t offset, uint64_t bytes) {
std::scoped_lock lock(manager->lock);
const auto changed =
@@ -138,7 +124,6 @@ void MemoryTracker::MarkRegionAsGpuModified(uint64_t vaddr, uint64_t size) {
void MemoryTracker::UnmarkRegionAsGpuModified(uint64_t vaddr, uint64_t size) {
CheckNotInUploadCallback();
std::lock_guard access(m_access_mutex);
RequireMapped(vaddr, size);
Iterate<true>(vaddr, size, [this](RegionManager* manager, uint64_t offset, uint64_t bytes) {
std::scoped_lock lock(manager->lock);
if (!manager->IsFullyModified<DirtySource::Gpu>(offset, bytes)) {
@@ -151,8 +136,6 @@ void MemoryTracker::UnmarkRegionAsGpuModified(uint64_t vaddr, uint64_t size) {
}
void MemoryTracker::UntrackMemoryLocked(uint64_t vaddr, uint64_t size) {
RequireMapped(vaddr, size);
std::vector<RegionManager*> managers;
managers.reserve((vaddr % TRACKER_REGION_SIZE + size + TRACKER_REGION_SIZE - 1) /
TRACKER_REGION_SIZE);
@@ -185,22 +168,6 @@ void MemoryTracker::UntrackMemory(uint64_t vaddr, uint64_t size) {
UntrackMemoryLocked(vaddr, size);
}
void MemoryTracker::UnmapMemory(uint64_t vaddr, uint64_t size) {
CheckNotInUploadCallback();
std::unique_lock access(m_access_mutex, std::try_to_lock);
if (!access.owns_lock()) {
#if defined(KYTY_MEMORY_TRACKER_TESTS)
if (const auto hook = g_unmap_contention_hook.load(std::memory_order_acquire);
hook != nullptr) {
hook();
}
#endif
access.lock();
}
UntrackMemoryLocked(vaddr, size);
m_page_manager.OnGpuUnmap(vaddr, size);
}
bool MemoryTracker::InvalidateRegion(uint64_t vaddr, uint64_t size, PageFaultPhase phase) noexcept {
switch (phase) {
case PageFaultPhase::Release: return true;
+3 -19
View File
@@ -30,7 +30,6 @@ public:
void MarkRegionAsGpuModified(uint64_t vaddr, uint64_t size);
void UnmarkRegionAsGpuModified(uint64_t vaddr, uint64_t size);
void UntrackMemory(uint64_t vaddr, uint64_t size);
void UnmapMemory(uint64_t vaddr, uint64_t size);
[[nodiscard]] CpuFaultAction
BeginCpuFault(uint64_t vaddr, uint64_t size,
PageFaultAccess access = PageFaultAccess::Write) noexcept;
@@ -91,8 +90,7 @@ public:
static_assert(std::is_nothrow_invocable_v<Preflight&, uint64_t, uint64_t>);
static_assert(std::is_nothrow_invocable_v<Func&, uint64_t, uint64_t>);
CheckNotInUploadCallback();
std::lock_guard access(m_access_mutex);
RequireMapped(vaddr, size);
std::lock_guard access(m_access_mutex);
std::vector<RegionManager*> managers;
Iterate<false>(vaddr, size, [&](RegionManager* manager, uint64_t, uint64_t) {
managers.push_back(manager);
@@ -132,11 +130,6 @@ public:
vaddr, size, [](uint64_t, uint64_t) noexcept {}, std::forward<Func>(func));
}
#if defined(KYTY_MEMORY_TRACKER_TESTS)
using UnmapContentionHook = void (*)() noexcept;
static void SetUnmapContentionHook(UnmapContentionHook hook) noexcept;
#endif
template <typename RangeFunc, typename UploadFunc>
void ForEachUploadRange(uint64_t vaddr, uint64_t size, bool is_written, RangeFunc&& range_func,
UploadFunc&& upload_func) {
@@ -144,7 +137,6 @@ public:
static_assert(std::is_nothrow_invocable_v<UploadFunc&>);
CheckNotInUploadCallback();
std::unique_lock access(m_access_mutex);
RequireMapped(vaddr, size);
Iterate<true>(vaddr, size, [](RegionManager*, uint64_t, uint64_t) {});
const auto* previous_upload_owner = std::exchange(s_upload_owner, this);
Iterate<false>(vaddr, size, [&](RegionManager* manager, uint64_t offset, uint64_t bytes) {
@@ -209,16 +201,8 @@ private:
return false;
}
static void ValidateRange(uint64_t vaddr, uint64_t size);
void UntrackMemoryLocked(uint64_t vaddr, uint64_t size);
void RequireMapped(uint64_t vaddr, uint64_t size) const {
ValidateRange(vaddr, size);
if (!m_page_manager.IsMapped(vaddr, size)) {
EXIT("memory tracker range [0x%llx, 0x%llx) is not mapped\n",
static_cast<unsigned long long>(vaddr),
static_cast<unsigned long long>(vaddr + size));
}
}
static void ValidateRange(uint64_t vaddr, uint64_t size);
void UntrackMemoryLocked(uint64_t vaddr, uint64_t size);
RegionManager* GetOrCreateRegion(uint64_t index);
std::unique_ptr<std::atomic<RegionManager*>[]> m_regions;
+43 -464
View File
@@ -1,6 +1,7 @@
#include "graphics/host_gpu/pageManager.h"
#include "graphics/host_gpu/regionDefinitions.h"
#include "kernel/memory.h"
#include <algorithm>
#include <array>
@@ -21,16 +22,11 @@
#undef min
#undef max
#elif defined(__APPLE__)
#include <mach/mach.h>
#include <mach/mach_vm.h>
#include <pthread.h>
#include <sys/mman.h>
#include <unistd.h>
#else
#include <cerrno>
#include <cstring>
#include <execinfo.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/syscall.h>
#include <unistd.h>
@@ -57,45 +53,8 @@ constexpr uint64_t REGION_PAGES = REGION_SIZE / PAGE_SIZE;
constexpr uint32_t NO_ACCESS_PROTECTION = PAGE_NOACCESS;
constexpr uint32_t READ_ONLY_PROTECTION = PAGE_READONLY;
constexpr uint32_t READ_WRITE_PROTECTION = PAGE_READWRITE;
#if defined(__APPLE__)
// Map the tracker's Win32-style protection tags to POSIX mprotect flags.
static int PageProtToPosix(uint32_t protection) {
switch (protection) {
case PAGE_NOACCESS: return PROT_NONE;
case PAGE_READONLY: return PROT_READ;
case PAGE_READWRITE: return PROT_READ | PROT_WRITE;
default: return PROT_NONE;
}
}
// Query the current protection of the page containing vaddr via the Mach VM map and
// collapse it to the tracker's read/write tags (execute is irrelevant to write tracking).
static uint32_t MachQueryPageProt(uint64_t vaddr) {
auto region_addr = static_cast<mach_vm_address_t>(vaddr);
mach_vm_size_t region_size = 0;
vm_region_basic_info_data_64_t info {};
mach_msg_type_number_t count = VM_REGION_BASIC_INFO_COUNT_64;
mach_port_t object_name = MACH_PORT_NULL;
kern_return_t kr =
mach_vm_region(mach_task_self(), &region_addr, &region_size, VM_REGION_BASIC_INFO_64,
reinterpret_cast<vm_region_info_t>(&info), &count, &object_name);
if (kr != KERN_SUCCESS || region_addr > vaddr) {
return PAGE_NOACCESS; // no region covering vaddr
}
if ((info.protection & VM_PROT_WRITE) != 0) {
return PAGE_READWRITE;
}
if ((info.protection & VM_PROT_READ) != 0) {
return PAGE_READONLY;
}
return PAGE_NOACCESS;
}
#elif defined(__linux__)
// Zero is the unknown protection sentinel.
constexpr uint32_t UNKNOWN_PROTECTION = 0;
#endif
thread_local bool g_in_fault_resolution = false;
@@ -136,6 +95,15 @@ thread_local bool g_in_fault_resolution = false;
std::_Exit(322);
}
Common::VirtualMemory::Mode ToMemoryMode(uint32_t protection) {
switch (protection) {
case NO_ACCESS_PROTECTION: return Common::VirtualMemory::Mode::NoAccess;
case READ_ONLY_PROTECTION: return Common::VirtualMemory::Mode::Read;
case READ_WRITE_PROTECTION: return Common::VirtualMemory::Mode::ReadWrite;
default: Fatal("unmappable protection 0x%08" PRIx32, protection);
}
}
uint32_t CurrentThread() noexcept {
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
return GetCurrentThreadId();
@@ -155,130 +123,6 @@ uint32_t CurrentThread() noexcept {
#endif
}
#if defined(__linux__)
int ToHostProtection(uint32_t protection) {
switch (protection) {
case NO_ACCESS_PROTECTION: return PROT_NONE;
case READ_ONLY_PROTECTION: return PROT_READ;
case READ_WRITE_PROTECTION: return PROT_READ | PROT_WRITE;
default: Fatal("unmappable protection 0x%08" PRIx32, protection);
}
}
struct HostMapping {
uint64_t end = 0;
uint32_t protection = UNKNOWN_PROTECTION;
};
// Async-signal-safe lookup in the address-ordered /proc/self/maps.
HostMapping QueryHostMapping(uint64_t vaddr) noexcept {
int fd = ::open("/proc/self/maps", O_RDONLY | O_CLOEXEC); // NOLINT
if (fd < 0) {
return {};
}
enum class Field { Start, End, Perms, Rest };
HostMapping result {};
auto field = Field::Start;
uint64_t start = 0;
uint64_t end = 0;
char perms[4] = {};
uint32_t perms_len = 0;
bool line_valid = true;
char buffer[8192];
for (bool done = false; !done;) {
const auto got = ::read(fd, buffer, sizeof(buffer));
if (got < 0) {
if (errno == EINTR) {
continue;
}
break;
}
if (got == 0) {
break;
}
for (ssize_t i = 0; i < got && !done; i++) {
const char c = buffer[i];
if (c == '\n') {
field = Field::Start;
start = 0;
end = 0;
perms_len = 0;
line_valid = true;
continue;
}
if (!line_valid) {
continue;
}
switch (field) {
case Field::Start:
case Field::End: {
uint64_t digit = 0;
if (c >= '0' && c <= '9') {
digit = static_cast<uint64_t>(c - '0');
} else if (c >= 'a' && c <= 'f') {
digit = static_cast<uint64_t>(c - 'a') + 10;
} else if (c == '-' && field == Field::Start) {
field = Field::End;
break;
} else if (c == ' ' && field == Field::End) {
field = Field::Perms;
perms_len = 0;
break;
} else {
line_valid = false;
break;
}
auto& value = (field == Field::Start ? start : end);
value = (value << 4u) | digit;
break;
}
case Field::Perms: {
if (c != ' ') {
if (perms_len < sizeof(perms)) {
perms[perms_len] = c;
}
perms_len++;
break;
}
if (vaddr < start) {
done = true;
} else if (vaddr < end && perms_len >= 2) {
result.end = end;
result.protection = perms[1] == 'w' ? READ_WRITE_PROTECTION
: perms[0] == 'r' ? READ_ONLY_PROTECTION
: NO_ACCESS_PROTECTION;
done = true;
} else {
field = Field::Rest;
}
break;
}
case Field::Rest: break;
}
}
}
::close(fd);
return result;
}
uint32_t QueryHostProtection(uint64_t vaddr) noexcept {
return QueryHostMapping(vaddr).protection;
}
#endif
class SpinGuard final {
public:
explicit SpinGuard(std::atomic_flag& lock): m_lock(lock) {
@@ -313,21 +157,16 @@ uint64_t PageEnd(uint64_t vaddr, uint64_t size) {
struct PageManager::Impl {
struct PageState {
std::atomic_flag lock = ATOMIC_FLAG_INIT;
uint32_t mappings = 0;
uint32_t gpu_read_mappings = 0;
uint32_t gpu_write_mappings = 0;
uint32_t write_watchers = 0;
uint32_t access_watchers = 0;
uint32_t original_protection = 0;
uint32_t backing_writer = 0;
#if defined(__linux__)
// Shadow the protection applied through Protect().
uint32_t current_protection = UNKNOWN_PROTECTION;
#endif
bool resolving = false;
bool resolving_read_write = false;
bool late_read_pending = false;
bool late_write_pending = false;
uint32_t current_protection = UNKNOWN_PROTECTION;
bool resolving = false;
bool resolving_read_write = false;
bool late_read_pending = false;
bool late_write_pending = false;
};
struct Region {
@@ -356,7 +195,7 @@ struct PageManager::Impl {
Impl(PageFaultHandler handler, void* context): fault_handler(handler), fault_context(context) {
if (fault_handler == nullptr) {
Fatal("null fault handler");
Fatal("null page-manager fault callback");
}
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
SYSTEM_INFO info {};
@@ -386,9 +225,8 @@ struct PageManager::Impl {
for (const auto& region: region_storage) {
for (auto& page: region->pages) {
SpinGuard lock(page.lock);
if (page.mappings != 0 || page.gpu_read_mappings != 0 ||
page.gpu_write_mappings != 0 || page.write_watchers != 0 ||
page.access_watchers != 0 || page.backing_writer != 0 || page.resolving) {
if (page.write_watchers != 0 || page.access_watchers != 0 ||
page.backing_writer != 0 || page.resolving) {
FailFast("PageManager destroyed with live page state");
}
}
@@ -441,179 +279,30 @@ struct PageManager::Impl {
}
}
static void ValidateInitialProtection(std::span<PageState*> pages, uint64_t vaddr) {
const auto end = vaddr + pages.size() * PAGE_SIZE;
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
for (auto address = vaddr; address < end;) {
MEMORY_BASIC_INFORMATION info {};
if (VirtualQuery(reinterpret_cast<const void*>(static_cast<uintptr_t>(address)), &info,
sizeof(info)) == 0 ||
info.State != MEM_COMMIT || info.Protect != PAGE_READWRITE) {
Fatal("basic path requires PAGE_READWRITE at 0x%016" PRIx64 " (state=0x%08" PRIx32
", protection=0x%08" PRIx32 ")",
address, static_cast<uint32_t>(info.State),
static_cast<uint32_t>(info.Protect));
}
const auto region_end = reinterpret_cast<uint64_t>(info.BaseAddress) + info.RegionSize;
if (region_end <= address) {
Fatal("VirtualQuery returned an invalid region at 0x%016" PRIx64, address);
}
address = std::min(end, region_end);
}
#elif defined(__APPLE__)
for (auto address = vaddr; address < end; address += PAGE_SIZE) {
const uint32_t protection = MachQueryPageProt(address);
if (protection != PAGE_READWRITE) {
Fatal("basic path requires PAGE_READWRITE at 0x%016" PRIx64
" (protection=0x%08" PRIx32 ")",
address, protection);
}
}
#else
for (auto address = vaddr; address < end;) {
const auto mapping = QueryHostMapping(address);
if (mapping.protection != READ_WRITE_PROTECTION || mapping.end <= address) {
Fatal("basic path requires a read/write mapping at 0x%016" PRIx64
" (protection=0x%08" PRIx32 ")",
address, mapping.protection);
}
address = std::min(end, mapping.end);
}
for (auto* page: pages) {
page->current_protection = READ_WRITE_PROTECTION;
}
#endif
static void InitializeProtection(std::span<PageState*> pages) {
for (auto* page: pages) {
page->original_protection = READ_WRITE_PROTECTION;
page->current_protection = READ_WRITE_PROTECTION;
}
}
static bool AllowsAccess([[maybe_unused]] const PageState& page, uint64_t vaddr,
static bool AllowsAccess(const PageState& page, [[maybe_unused]] uint64_t vaddr,
PageFaultAccess access) noexcept {
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
MEMORY_BASIC_INFORMATION info {};
if (VirtualQuery(reinterpret_cast<const void*>(static_cast<uintptr_t>(vaddr)), &info,
sizeof(info)) == 0 ||
info.State != MEM_COMMIT) {
return false;
}
switch (access) {
case PageFaultAccess::Read:
return info.Protect == PAGE_READONLY || info.Protect == PAGE_READWRITE;
case PageFaultAccess::Write: return info.Protect == PAGE_READWRITE;
return page.current_protection == READ_ONLY_PROTECTION ||
page.current_protection == READ_WRITE_PROTECTION;
case PageFaultAccess::Write: return page.current_protection == READ_WRITE_PROTECTION;
default: return false;
}
#elif defined(__APPLE__)
const uint32_t protection = MachQueryPageProt(vaddr);
switch (access) {
case PageFaultAccess::Read:
return protection == PAGE_READONLY || protection == PAGE_READWRITE;
case PageFaultAccess::Write: return protection == PAGE_READWRITE;
default: return false;
}
#else
const auto permitted = [](uint32_t protection, PageFaultAccess wanted) {
switch (wanted) {
case PageFaultAccess::Read:
return protection == READ_ONLY_PROTECTION ||
protection == READ_WRITE_PROTECTION;
case PageFaultAccess::Write: return protection == READ_WRITE_PROTECTION;
default: return false;
}
};
if (!permitted(page.current_protection, access)) {
return false;
}
return permitted(QueryHostProtection(vaddr), access);
#endif
}
static void ProtectRange(std::span<PageState*> pages, uint64_t vaddr, uint32_t protection,
std::span<const uint32_t> expected_old, bool fault_path) noexcept {
void ProtectRange(std::span<PageState*> pages, uint64_t vaddr, uint32_t protection,
std::span<const uint32_t> expected_old, bool fault_path) noexcept {
const auto size = pages.size() * PAGE_SIZE;
if (pages.size() != expected_old.size()) {
FailFast("protection range state size mismatch");
}
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
struct HostRange {
uint64_t begin = 0;
uint64_t end = 0;
};
std::vector<HostRange> host_ranges;
const auto end = vaddr + size;
for (auto address = vaddr; address < end;) {
MEMORY_BASIC_INFORMATION info {};
if (VirtualQuery(reinterpret_cast<const void*>(static_cast<uintptr_t>(address)), &info,
sizeof(info)) == 0 ||
info.State != MEM_COMMIT) {
if (fault_path) {
FailFast("VirtualProtect fault transition did not match expected protection");
}
Fatal("invalid protection transition at 0x%016" PRIx64 ", state=0x%08" PRIx32
", new=0x%08" PRIx32,
address, static_cast<uint32_t>(info.State), protection);
}
const auto region_end = reinterpret_cast<uint64_t>(info.BaseAddress) + info.RegionSize;
const auto query_end = std::min(end, region_end);
if (query_end <= address) {
if (fault_path) {
FailFast("VirtualQuery returned an invalid fault transition region");
}
Fatal("VirtualQuery returned an invalid region at 0x%016" PRIx64, address);
}
const auto first_page = static_cast<size_t>((address - vaddr) / PAGE_SIZE);
const auto last_page =
static_cast<size_t>((query_end - vaddr + PAGE_SIZE - 1) / PAGE_SIZE);
for (auto page = first_page; page < last_page; page++) {
if (info.Protect != expected_old[page]) {
if (fault_path) {
FailFast(
"VirtualProtect fault transition did not match expected protection");
}
Fatal("invalid protection transition at 0x%016" PRIx64 ", actual=0x%08" PRIx32
", expected=0x%08" PRIx32 ", new=0x%08" PRIx32,
vaddr + page * PAGE_SIZE, static_cast<uint32_t>(info.Protect),
expected_old[page], protection);
}
}
const auto allocation = reinterpret_cast<uint64_t>(info.AllocationBase);
if (host_ranges.empty() || allocation != host_ranges.back().begin) {
host_ranges.push_back({allocation, query_end});
} else {
host_ranges.back().end = query_end;
}
address = query_end;
}
for (auto range: host_ranges) {
range.begin = std::max(range.begin, vaddr);
DWORD old_protection = 0;
const auto first_page = static_cast<size_t>((range.begin - vaddr) / PAGE_SIZE);
if (VirtualProtect(reinterpret_cast<void*>(static_cast<uintptr_t>(range.begin)),
range.end - range.begin, protection, &old_protection) == 0 ||
old_protection != expected_old[first_page]) {
if (fault_path) {
FailFast("VirtualProtect fault transition did not match expected protection");
}
Fatal("invalid protection transition at 0x%016" PRIx64 ", old=0x%08" PRIx32
", expected=0x%08" PRIx32 ", new=0x%08" PRIx32,
range.begin, static_cast<uint32_t>(old_protection), expected_old[first_page],
protection);
}
}
#elif defined(__APPLE__)
// mprotect cannot report the previous protection, so the expected_old comparison
// is dropped; the tracker is the sole mutator of these pages and drives the
// transition from its own shadow state.
(void)expected_old;
if (mprotect(reinterpret_cast<void*>(static_cast<uintptr_t>(vaddr)), size,
PageProtToPosix(protection)) != 0) {
if (fault_path) {
FailFast("mprotect fault transition failed");
}
Fatal("mprotect failed at 0x%016" PRIx64 ", new=0x%08" PRIx32, vaddr, protection);
}
#else
for (size_t i = 0; i < pages.size(); i++) {
const auto actual = pages[i]->current_protection;
if (actual != UNKNOWN_PROTECTION && actual != expected_old[i]) {
@@ -625,22 +314,21 @@ struct PageManager::Impl {
vaddr + i * PAGE_SIZE, actual, expected_old[i], protection);
}
}
if (::mprotect(reinterpret_cast<void*>(static_cast<uintptr_t>(vaddr)), size,
ToHostProtection(protection)) != 0) {
if (!Libs::LibKernel::Memory::ProtectGuestHostMemory(vaddr, size,
ToMemoryMode(protection))) {
if (fault_path) {
FailFast("mprotect failed on the fault path");
FailFast("address-space fault protection transition failed");
}
Fatal("mprotect failed at 0x%016" PRIx64 ", new=0x%08" PRIx32 " (%s)", vaddr,
protection, std::strerror(errno));
Fatal("address-space protection failed at 0x%016" PRIx64 ", new=0x%08" PRIx32, vaddr,
protection);
}
for (auto* page: pages) {
page->current_protection = protection;
}
#endif
}
static void Protect(PageState& page, uint64_t vaddr, uint32_t protection, uint32_t expected_old,
bool fault_path) noexcept {
void Protect(PageState& page, uint64_t vaddr, uint32_t protection, uint32_t expected_old,
bool fault_path) noexcept {
PageState* pages[] = {&page};
uint32_t expected[] = {expected_old};
ProtectRange(pages, vaddr, protection, expected, fault_path);
@@ -680,50 +368,6 @@ bool PageManager::IsTracked(uint64_t vaddr) const noexcept {
return page.write_watchers != 0 || page.access_watchers != 0;
}
bool PageManager::IsMapped(uint64_t vaddr, uint64_t size) const noexcept {
if (vaddr == 0 || size == 0 || vaddr >= ADDRESS_SIZE || size > ADDRESS_SIZE - vaddr) {
return false;
}
const auto end = PageStart(vaddr + size - 1) + PAGE_SIZE;
for (auto page_vaddr = PageStart(vaddr); page_vaddr < end; page_vaddr += PAGE_SIZE) {
auto* region = m_impl->FindRegion(page_vaddr);
if (region == nullptr) {
return false;
}
auto& page = m_impl->GetPage(*region, page_vaddr);
SpinGuard lock(page.lock);
if (page.mappings == 0) {
return false;
}
}
return true;
}
bool PageManager::HasGpuAccess(uint64_t vaddr, uint64_t size, GpuAccess access) const noexcept {
if (access != GpuAccess::Read && access != GpuAccess::Write && access != GpuAccess::ReadWrite) {
FailFast("HasGpuAccess received an invalid GPU access mode");
}
const bool need_read = access == GpuAccess::Read || access == GpuAccess::ReadWrite;
const bool need_write = access == GpuAccess::Write || access == GpuAccess::ReadWrite;
if (vaddr == 0 || size == 0 || vaddr >= ADDRESS_SIZE || size > ADDRESS_SIZE - vaddr) {
return false;
}
const auto end = PageEnd(vaddr, size);
for (auto addr = PageStart(vaddr); addr < end; addr += PAGE_SIZE) {
auto* region = m_impl->FindRegion(addr);
if (region == nullptr) {
return false;
}
auto& page = m_impl->GetPage(*region, addr);
SpinGuard lock(page.lock);
if ((need_read && page.gpu_read_mappings == 0) ||
(need_write && page.gpu_write_mappings == 0)) {
return false;
}
}
return true;
}
void PageManager::UpdatePageWatchers(bool track, uint64_t vaddr, uint64_t size,
PageWatchMode mode) {
if (mode != PageWatchMode::Write && mode != PageWatchMode::ReadWrite) {
@@ -754,9 +398,6 @@ void PageManager::UpdatePageWatchers(bool track, uint64_t vaddr, uint64_t size,
if (page.resolving && track) {
FailFast("new page watcher raced active fault resolution");
}
if (page.mappings == 0) {
Fatal("watching unmapped page 0x%016" PRIx64, address);
}
auto& watchers =
(mode == PageWatchMode::ReadWrite ? page.access_watchers : page.write_watchers);
if (track) {
@@ -784,8 +425,7 @@ void PageManager::UpdatePageWatchers(bool track, uint64_t vaddr, uint64_t size,
last++;
}
if (first != last) {
Impl::ValidateInitialProtection(std::span {pages}.subspan(first, last - first),
chunk_begin + first * PAGE_SIZE);
Impl::InitializeProtection(std::span {pages}.subspan(first, last - first));
}
first = last;
}
@@ -831,9 +471,9 @@ void PageManager::UpdatePageWatchers(bool track, uint64_t vaddr, uint64_t size,
last = current + 1;
}
}
Impl::ProtectRange(std::span {pages}.subspan(first, last - first),
chunk_begin + first * PAGE_SIZE, protection,
std::span {old_protections}.subspan(first, last - first), false);
m_impl->ProtectRange(std::span {pages}.subspan(first, last - first),
chunk_begin + first * PAGE_SIZE, protection,
std::span {old_protections}.subspan(first, last - first), false);
first = current;
}
@@ -860,70 +500,9 @@ void PageManager::UpdatePageWatchers(bool track, uint64_t vaddr, uint64_t size,
}
}
void PageManager::OnGpuMap(uint64_t vaddr, uint64_t size, GpuAccess access) {
if (g_in_fault_resolution) {
FailFast("GPU mapping changed during fault resolution");
}
if (access != GpuAccess::Read && access != GpuAccess::Write && access != GpuAccess::ReadWrite) {
FailFast("GPU map received an invalid access mode");
}
const bool gpu_read = access == GpuAccess::Read || access == GpuAccess::ReadWrite;
const bool gpu_write = access == GpuAccess::Write || access == GpuAccess::ReadWrite;
const auto end = PageEnd(vaddr, size);
for (auto addr = PageStart(vaddr); addr < end; addr += PAGE_SIZE) {
auto& page = m_impl->GetPage(*m_impl->GetOrCreateRegion(addr), addr);
SpinGuard lock(page.lock);
if (page.resolving || page.mappings == std::numeric_limits<uint32_t>::max() ||
(gpu_read && page.gpu_read_mappings == std::numeric_limits<uint32_t>::max()) ||
(gpu_write && page.gpu_write_mappings == std::numeric_limits<uint32_t>::max())) {
Fatal("invalid map state at 0x%016" PRIx64, addr);
}
page.mappings++;
page.gpu_read_mappings += gpu_read ? 1u : 0u;
page.gpu_write_mappings += gpu_write ? 1u : 0u;
#if defined(__linux__)
// New guest mappings start read/write.
if (page.current_protection == UNKNOWN_PROTECTION) {
page.current_protection = READ_WRITE_PROTECTION;
}
#endif
}
}
void PageManager::OnGpuMap(uint64_t, uint64_t) {}
void PageManager::OnGpuUnmap(uint64_t vaddr, uint64_t size, GpuAccess access) {
if (g_in_fault_resolution) {
FailFast("GPU unmapping changed during fault resolution");
}
if (access != GpuAccess::Read && access != GpuAccess::Write && access != GpuAccess::ReadWrite) {
FailFast("GPU unmap received an invalid access mode");
}
const bool gpu_read = access == GpuAccess::Read || access == GpuAccess::ReadWrite;
const bool gpu_write = access == GpuAccess::Write || access == GpuAccess::ReadWrite;
const auto end = PageEnd(vaddr, size);
for (auto page_vaddr = PageStart(vaddr); page_vaddr < end; page_vaddr += PAGE_SIZE) {
auto* region = m_impl->FindRegion(page_vaddr);
if (region == nullptr) {
Fatal("unmapping unknown page 0x%016" PRIx64, page_vaddr);
}
auto& page = m_impl->GetPage(*region, page_vaddr);
SpinGuard lock(page.lock);
if (page.resolving || page.mappings == 0 || (gpu_read && page.gpu_read_mappings == 0) ||
(gpu_write && page.gpu_write_mappings == 0) ||
(page.mappings == 1 && (page.write_watchers != 0 || page.access_watchers != 0))) {
Fatal("invalid unmap state at 0x%016" PRIx64, page_vaddr);
}
page.mappings--;
page.gpu_read_mappings -= gpu_read ? 1u : 0u;
page.gpu_write_mappings -= gpu_write ? 1u : 0u;
if (page.mappings == 0) {
if (page.gpu_read_mappings != 0 || page.gpu_write_mappings != 0) {
FailFast("GPU unmap left nonzero GPU mapping counts");
}
page.late_read_pending = false;
page.late_write_pending = false;
}
}
}
void PageManager::OnGpuUnmap(uint64_t, uint64_t) {}
PageManager::BackingWrite::BackingWrite(PageManager& manager, uint64_t vaddr,
uint64_t size) noexcept
@@ -979,8 +558,7 @@ void PageManager::BeginBackingWrite(uint64_t vaddr, uint64_t size) noexcept {
}
auto& page = m_impl->GetPage(*region, address);
SpinGuard lock(page.lock);
if (page.mappings == 0 || page.resolving || page.backing_writer != 0 ||
page.access_watchers == 0) {
if (page.resolving || page.backing_writer != 0 || page.access_watchers == 0) {
Fatal("backing write races page resolution at 0x%016" PRIx64, address);
}
page.resolving = true;
@@ -1008,7 +586,7 @@ void PageManager::EndBackingWrite(uint64_t vaddr, uint64_t size) noexcept {
const auto old_protection = NO_ACCESS_PROTECTION;
const auto new_protection = Impl::WatcherProtection(page);
if (new_protection != old_protection) {
Impl::Protect(page, address, new_protection, old_protection, false);
m_impl->Protect(page, address, new_protection, old_protection, false);
}
Impl::PublishDelayedFaults(page, old_protection, new_protection);
if (page.write_watchers == 0 && page.access_watchers == 0) {
@@ -1109,7 +687,8 @@ bool PageManager::HandleFault(PageFaultAccess access, uint64_t fault_vaddr) noex
page.write_watchers = 0;
}
const auto restored_protection = Impl::WatcherProtection(page);
Impl::Protect(page, PageStart(fault_vaddr), restored_protection, old_protection, true);
m_impl->Protect(page, PageStart(fault_vaddr), restored_protection, old_protection,
true);
if (page.write_watchers == 0) {
page.original_protection = 0;
}
+2 -6
View File
@@ -13,11 +13,9 @@ namespace Libs::Graphics {
enum class PageFaultAccess { Read, Write, Execute, Unknown };
enum class PageFaultPhase { Invalidate, Complete, Release };
enum class PageWatchMode { Write, ReadWrite };
enum class GpuAccess { Read, Write, ReadWrite };
using PageFaultHandler = bool (*)(void* context, PageFaultAccess access, uint64_t vaddr,
uint64_t size, PageFaultPhase phase) noexcept;
class PageManager final {
public:
class BackingWrite final {
@@ -40,13 +38,11 @@ public:
[[nodiscard]] uint64_t GetPageSize() const;
[[nodiscard]] bool IsTracked(uint64_t vaddr) const noexcept;
[[nodiscard]] bool IsMapped(uint64_t vaddr, uint64_t size) const noexcept;
[[nodiscard]] bool HasGpuAccess(uint64_t vaddr, uint64_t size, GpuAccess access) const noexcept;
void UpdatePageWatchers(bool track, uint64_t vaddr, uint64_t size,
PageWatchMode mode = PageWatchMode::Write);
void OnGpuMap(uint64_t vaddr, uint64_t size, GpuAccess access = GpuAccess::ReadWrite);
void OnGpuUnmap(uint64_t vaddr, uint64_t size, GpuAccess access = GpuAccess::ReadWrite);
void OnGpuMap(uint64_t vaddr, uint64_t size);
void OnGpuUnmap(uint64_t vaddr, uint64_t size);
[[nodiscard]] bool HandleFault(PageFaultAccess access, uint64_t fault_vaddr) noexcept;
[[nodiscard]] std::vector<std::unique_ptr<BackingWrite>>
-21
View File
@@ -714,7 +714,6 @@ BufferBinding BufferCache::ObtainBuffer(CommandBuffer& command, uint64_t vaddr,
if (command.IsInvalid() || command.IsExecute()) {
EXIT("BufferCache: buffer request requires a recording command buffer\n");
}
ValidateGpuAccess(vaddr, size, is_read, is_written);
std::lock_guard transaction(m_resource_mutex);
(void)SynchronizeBacking(vaddr, size);
@@ -999,7 +998,6 @@ void BufferCache::FillBuffer(uint64_t vaddr, uint64_t size, uint32_t value, bool
if (vaddr == 0) {
EXIT("BufferCache: invalid fill memory address\n");
}
ValidateGpuAccess(vaddr, size, false, true);
(void)m_texture_cache.ClearMeta(vaddr);
{
std::lock_guard transaction(m_resource_mutex);
@@ -1041,12 +1039,6 @@ void BufferCache::CopyBuffer(uint64_t dst_vaddr, uint64_t src_vaddr, uint64_t si
(src_gds && (src_vaddr > m_gds_buffer.Size() || size > m_gds_buffer.Size() - src_vaddr))) {
EXIT("BufferCache: invalid or overlapping copy range\n");
}
if (src_memory) {
ValidateGpuAccess(src_vaddr, size, true, false);
}
if (dst_memory) {
ValidateGpuAccess(dst_vaddr, size, false, true);
}
if (src_memory || dst_memory) {
std::lock_guard transaction(m_resource_mutex);
if (src_memory) {
@@ -1203,19 +1195,6 @@ void BufferCache::PublishImageBuffer(uint64_t vaddr, uint64_t size) {
owner->second->tick_accessed_last = m_gc_tick;
}
void BufferCache::ValidateGpuAccess(uint64_t vaddr, uint64_t size, bool is_read,
bool is_written) const {
if ((!is_read && !is_written) || vaddr == 0 || size == 0 || size > UINT64_MAX - vaddr) {
EXIT("BufferCache: invalid GPU access request\n");
}
if (is_read && !m_page_manager.HasGpuAccess(vaddr, size, GpuAccess::Read)) {
EXIT("BufferCache: GPU-read access denied\n");
}
if (is_written && !m_page_manager.HasGpuAccess(vaddr, size, GpuAccess::Write)) {
EXIT("BufferCache: GPU-write access denied\n");
}
}
void BufferCache::RunGarbageCollector() {
std::lock_guard transaction(m_resource_mutex);
const auto tick = m_gc_tick++;
+1 -2
View File
@@ -77,8 +77,7 @@ public:
void CompleteBackingPublication(uint64_t vaddr, uint64_t size, uint64_t tick);
[[nodiscard]] bool SynchronizeBacking(uint64_t vaddr, uint64_t size);
void PublishImageBuffer(uint64_t vaddr, uint64_t size);
void ValidateGpuAccess(uint64_t vaddr, uint64_t size, bool is_read, bool is_written) const;
void RunGarbageCollector();
void RunGarbageCollector();
private:
friend struct BufferCacheTestAccess;
+6 -10
View File
@@ -4,7 +4,6 @@
#include "graphics/guest_gpu/command_processor/commandProcessor.h"
#include "graphics/guest_gpu/graphicsRun.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
namespace Libs::Graphics {
GpuResourceManager::GpuResourceManager(GraphicContext& graphics, CommandScheduler& scheduler)
@@ -115,22 +114,19 @@ bool GpuResourceManager::IsMapped(uint64_t vaddr, uint64_t size) const noexcept
return m_mapped_ranges.Contains(vaddr, size);
}
void GpuResourceManager::MapMemory(uint64_t vaddr, uint64_t size, GpuAccess access) {
void GpuResourceManager::MapMemory(uint64_t vaddr, uint64_t size) {
{
std::lock_guard lock(m_mapped_ranges_mutex);
m_mapped_ranges.Add(vaddr, size);
}
m_page_manager.OnGpuMap(vaddr, size, access);
m_page_manager.OnGpuMap(vaddr, size);
}
void GpuResourceManager::UnmapMemory(uint64_t vaddr, uint64_t size, GpuAccess access) {
if (!IsMapped(vaddr, size)) {
EXIT("cannot unmap an unmapped GPU resource range\n");
}
const auto unmap = [this, vaddr, size, access] {
m_texture_cache.UnmapMemory(vaddr, size);
void GpuResourceManager::UnmapMemory(uint64_t vaddr, uint64_t size) {
const auto unmap = [this, vaddr, size] {
m_buffer_cache.UnmapMemory(vaddr, size);
m_page_manager.OnGpuUnmap(vaddr, size, access);
m_texture_cache.UnmapMemory(vaddr, size);
m_page_manager.OnGpuUnmap(vaddr, size);
std::lock_guard lock(m_mapped_ranges_mutex);
m_mapped_ranges.Subtract(vaddr, size);
};
+2 -2
View File
@@ -29,8 +29,8 @@ public:
[[nodiscard]] bool HandleFault(PageFaultAccess access, uint64_t fault_vaddr) noexcept;
[[nodiscard]] bool InvalidateMemory(uint64_t vaddr, uint64_t size);
[[nodiscard]] bool IsMapped(uint64_t vaddr, uint64_t size) const noexcept;
void MapMemory(uint64_t vaddr, uint64_t size, GpuAccess access);
void UnmapMemory(uint64_t vaddr, uint64_t size, GpuAccess access);
void MapMemory(uint64_t vaddr, uint64_t size);
void UnmapMemory(uint64_t vaddr, uint64_t size);
void RunGarbageCollector();
private:
+10 -12
View File
@@ -135,8 +135,8 @@ void Buffer::Write(uint64_t offset, const void* source, uint64_t size) {
void Buffer::Flush(uint64_t offset, uint64_t size) {
EXIT_IF(m_mapped.empty() || offset > m_size || size > m_size - offset);
if (!m_is_coherent && size != 0) {
const auto result = vmaFlushAllocation(m_graphics->allocator, m_buffer->memory.allocation,
offset, size);
const auto result =
vmaFlushAllocation(m_graphics->allocator, m_buffer->memory.allocation, offset, size);
EXIT_NOT_IMPLEMENTED(static_cast<vk::Result>(result) != vk::Result::eSuccess);
}
}
@@ -144,8 +144,8 @@ void Buffer::Flush(uint64_t offset, uint64_t size) {
vk::BufferMemoryBarrier Buffer::Barrier(uint64_t offset, uint64_t size, vk::AccessFlags source,
vk::AccessFlags destination) const {
if (Handle() == nullptr || size == 0 || offset > m_size || size > m_size - offset) {
EXIT("Buffer: invalid DMA barrier, handle=%p offset=0x%016" PRIx64
" size=0x%016" PRIx64 " capacity=0x%016" PRIx64 "\n",
EXIT("Buffer: invalid DMA barrier, handle=%p offset=0x%016" PRIx64 " size=0x%016" PRIx64
" capacity=0x%016" PRIx64 "\n",
static_cast<const void*>(Handle()), offset, size, m_size);
}
vk::BufferMemoryBarrier barrier {};
@@ -175,10 +175,9 @@ void Buffer::CopyFrom(CommandBuffer& command, const Buffer& source, uint64_t sou
command.EndRendering();
const vk::BufferMemoryBarrier before[] = {
source.Barrier(source_offset, size, source_before, vk::AccessFlagBits::eTransferRead),
Barrier(destination_offset, size, destination_before,
vk::AccessFlagBits::eTransferWrite),
Barrier(destination_offset, size, destination_before, vk::AccessFlagBits::eTransferWrite),
};
const auto host_access = vk::AccessFlagBits::eHostRead | vk::AccessFlagBits::eHostWrite;
const auto host_access = vk::AccessFlagBits::eHostRead | vk::AccessFlagBits::eHostWrite;
auto before_stage = vk::PipelineStageFlags {vk::PipelineStageFlagBits::eAllCommands};
if (static_cast<bool>((source_before | destination_before) & host_access)) {
before_stage |= vk::PipelineStageFlagBits::eHost;
@@ -214,9 +213,8 @@ void Buffer::Fill(uint64_t offset, uint64_t size, uint32_t value) {
vk::PipelineStageFlagBits::eTransfer, vk::DependencyFlagBits::eByRegion,
0, nullptr, 1, &before, 0, nullptr);
native.fillBuffer(Handle(), offset, size, value);
const auto after =
Barrier(offset, size, vk::AccessFlagBits::eTransferWrite,
vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite);
const auto after = Barrier(offset, size, vk::AccessFlagBits::eTransferWrite,
vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite);
native.pipelineBarrier(vk::PipelineStageFlagBits::eTransfer,
vk::PipelineStageFlagBits::eAllCommands,
vk::DependencyFlagBits::eByRegion, 0, nullptr, 1, &after, 0, nullptr);
@@ -250,8 +248,8 @@ std::pair<uint8_t*, uint64_t> StreamBuffer::Map(uint64_t size, uint64_t alignmen
if (Mapped().empty()) {
return {nullptr, 0};
}
uint64_t mapped_size = size;
const auto atom = Graphics().physical_device_properties.limits.nonCoherentAtomSize;
uint64_t mapped_size = size;
const auto atom = Graphics().physical_device_properties.limits.nonCoherentAtomSize;
if (!NormalizeReservation(IsCoherent(), atom, mapped_size, alignment)) {
return {nullptr, 0};
}
+14 -15
View File
@@ -54,16 +54,15 @@ public:
[[nodiscard]] bool IsInBounds(uint64_t address, uint64_t size) const noexcept;
void Write(uint64_t offset, const void* source, uint64_t size);
void Flush(uint64_t offset, uint64_t size);
void CopyFrom(
CommandBuffer& command, const Buffer& source, uint64_t source_offset,
uint64_t destination_offset, uint64_t size,
vk::AccessFlags source_before = vk::AccessFlagBits::eMemoryWrite,
vk::AccessFlags destination_before =
vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite,
vk::AccessFlags source_after =
vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite,
vk::AccessFlags destination_after =
vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite);
void CopyFrom(CommandBuffer& command, const Buffer& source, uint64_t source_offset,
uint64_t destination_offset, uint64_t size,
vk::AccessFlags source_before = vk::AccessFlagBits::eMemoryWrite,
vk::AccessFlags destination_before = vk::AccessFlagBits::eMemoryRead |
vk::AccessFlagBits::eMemoryWrite,
vk::AccessFlags source_after = vk::AccessFlagBits::eMemoryRead |
vk::AccessFlagBits::eMemoryWrite,
vk::AccessFlags destination_after = vk::AccessFlagBits::eMemoryRead |
vk::AccessFlagBits::eMemoryWrite);
void Fill(uint64_t offset, uint64_t size, uint32_t value);
protected:
@@ -107,13 +106,13 @@ private:
uint64_t upper_bound = 0;
};
void ReserveWatches(std::vector<Watch>& watches, size_t grow_size);
void ReserveWatches(std::vector<Watch>& watches, size_t grow_size);
[[nodiscard]] static bool NormalizeReservation(bool coherent, uint64_t atom, uint64_t& size,
uint64_t& alignment);
[[nodiscard]] bool WaitPendingOperations(const std::vector<Watch>& watches,
std::optional<size_t> invalidation_mark,
uint64_t requested_upper_bound, bool allow_wait,
size_t& wait_cursor, uint64_t& wait_bound);
[[nodiscard]] bool WaitPendingOperations(const std::vector<Watch>& watches,
std::optional<size_t> invalidation_mark,
uint64_t requested_upper_bound, bool allow_wait,
size_t& wait_cursor, uint64_t& wait_bound);
uint64_t m_offset = 0;
uint64_t m_mapped_size = 0;
-1
View File
@@ -1377,7 +1377,6 @@ bool TextureCache::ClearImageFromBuffer(CommandBuffer& command, uint64_t address
if (command.IsInvalid() || !GuestRange {address, size}.Valid()) {
EXIT("TextureCache: invalid image clear\n");
}
m_buffer_cache.ValidateGpuAccess(address, size, false, true);
std::lock_guard transaction(m_resource_mutex);
CacheLock lock(*this, m_lock);
ImageId selected {};
@@ -7,8 +7,8 @@
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/guest_gpu/tile.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/renderer/image/textureCommon.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/image/textureCommon.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
@@ -23,10 +23,10 @@ static std::atomic<uint32_t> g_render_color_log_count = 0;
// NOLINTNEXTLINE(readability-function-cognitive-complexity)
void RenderExecutor::ResolveRenderColorTarget(uint64_t submit_id, RenderCommandBuffer& buffer,
RenderColorInfo& r,
uint32_t render_target_slice_offset,
uint32_t render_target_slot, bool ignore_target_mask,
bool exact_format) {
RenderColorInfo& r,
uint32_t render_target_slice_offset,
uint32_t render_target_slot, bool ignore_target_mask,
bool exact_format) {
KYTY_PROFILER_FUNCTION();
const auto& hw = buffer.GetRegisters();
@@ -79,10 +79,8 @@ void RenderExecutor::ResolveRenderColorTarget(uint64_t submit_id, RenderCommandB
const auto view = ResolveTargetViewInfo(
rt.view.base_array_slice_index, rt.view.last_array_slice_index, render_target_slice_offset);
switch (view.type) {
case TargetViewType::Image2D: break;
case TargetViewType::Image2DArray:
EXIT("layered render-target views are unsupported: base=%u count=%u\n", view.base_layer,
view.layer_count);
case TargetViewType::Image2D:
case TargetViewType::Image2DArray: break;
case TargetViewType::Unsupported:
EXIT("invalid render-target view: base=%u last=%u draw_offset=%u\n",
rt.view.base_array_slice_index, rt.view.last_array_slice_index,
@@ -241,12 +239,12 @@ void RenderExecutor::ResolveRenderColorTarget(uint64_t submit_id, RenderCommandB
}
TextureCache::ImageDesc desc {};
desc.type = TextureCache::BindingType::RenderTarget;
desc.info.data = {rt.base.addr, backing_size};
desc.info.pixel_format = target_format.format;
desc.info.guest_format = ImageOps::RenderTargetTransferFormat(bytes_per_element);
desc.info.type = Prospero::ImageType::kColor2D;
desc.info.extent = {width, height, 1};
desc.type = TextureCache::BindingType::RenderTarget;
desc.info.data = {rt.base.addr, backing_size};
desc.info.pixel_format = target_format.format;
desc.info.guest_format = ImageOps::RenderTargetTransferFormat(bytes_per_element);
desc.info.type = Prospero::ImageType::kColor2D;
desc.info.extent = {width, height, 1};
desc.info.resources = {levels, view.image_layers};
desc.info.pitch = pitch;
desc.info.bytes_per_block = bytes_per_element;
@@ -275,20 +273,20 @@ void RenderExecutor::ResolveRenderColorTarget(uint64_t submit_id, RenderCommandB
desc.view_info.base_layer = view.base_layer;
desc.view_info.layer_count = view.layer_count;
desc.view_info.usage = vk::ImageUsageFlagBits::eColorAttachment;
auto& texture_cache = m_context.GetTextureCache();
r.desc = std::move(desc);
r.image_id = texture_cache.FindImage(r.desc, exact_format);
r.type = RenderColorType::RenderTexture;
r.base_addr = rt.base.addr;
r.image_view = nullptr;
r.format = r.desc.view_info.format;
r.extent = view_extent;
r.base_mip_level = rt.view.current_mip_level;
r.buffer_size = backing_size;
r.samples = samples;
r.export_mapping = target_format.export_mapping;
r.color_clear_enable = false;
r.color_clear_value = {};
auto& texture_cache = m_context.GetTextureCache();
r.desc = std::move(desc);
r.image_id = texture_cache.FindImage(r.desc, exact_format);
r.type = RenderColorType::RenderTexture;
r.base_addr = rt.base.addr;
r.image_view = nullptr;
r.format = r.desc.view_info.format;
r.extent = view_extent;
r.base_mip_level = rt.view.current_mip_level;
r.buffer_size = backing_size;
r.samples = samples;
r.export_mapping = target_format.export_mapping;
r.color_clear_enable = false;
r.color_clear_value = {};
BindRenderTarget(r.image_id);
}
@@ -2,8 +2,8 @@
#define EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_COLORRENDERTARGET_H_
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/host_gpu/renderer/renderTarget.h"
#include "graphics/host_gpu/renderer/cache/textureCache.h"
#include "graphics/host_gpu/renderer/renderTarget.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include <cstdint>
@@ -44,13 +44,13 @@ CommandSlot* CommandScheduler::CommandPool::CreateSlot() {
allocate.commandPool = m_pool;
allocate.level = vk::CommandBufferLevel::ePrimary;
allocate.commandBufferCount = 1;
vk::CommandBuffer buffer = nullptr;
vk::CommandBuffer buffer = nullptr;
EXIT_IF(graphics.device.allocateCommandBuffers(&allocate, &buffer) != vk::Result::eSuccess);
vk::FenceCreateInfo fence_create {};
fence_create.sType = vk::StructureType::eFenceCreateInfo;
fence_create.flags = vk::FenceCreateFlagBits::eSignaled;
vk::Fence fence = nullptr;
vk::Fence fence = nullptr;
if (graphics.device.createFence(&fence_create, nullptr, &fence) != vk::Result::eSuccess) {
graphics.device.freeCommandBuffers(m_pool, 1, &buffer);
EXIT("failed to create command-buffer fence\n");
@@ -70,9 +70,9 @@ CommandSlot* CommandScheduler::CommandPool::Allocate(GraphicContext& graphics) {
Create(graphics);
}
EXIT_IF(m_graphics != &graphics);
auto found = std::ranges::find_if(m_slots, [](const auto& slot) { return !slot.busy; });
auto* slot = found != m_slots.end() ? &*found : CreateSlot();
slot->busy = true;
auto found = std::ranges::find_if(m_slots, [](const auto& slot) { return !slot.busy; });
auto* slot = found != m_slots.end() ? &*found : CreateSlot();
slot->busy = true;
slot->Reset();
return slot;
}
@@ -331,8 +331,7 @@ void CommandScheduler::WaitPriorityOperations(uint64_t tick) {
EXIT_IF(g_deferred_callback_scheduler == this);
std::unique_lock lock(m_operation_mutex);
m_operation_available.wait(lock, [this, tick] {
const bool active_before_or_at =
m_priority_active && m_priority_active_tick <= tick;
const bool active_before_or_at = m_priority_active && m_priority_active_tick <= tick;
const bool queued_before_or_at =
!m_priority_operations.empty() && m_priority_operations.front().tick <= tick;
return !active_before_or_at && !queued_before_or_at;
@@ -47,21 +47,21 @@ public:
void FinishCurrent();
// Deferred callbacks can observe an externally owned drain, but cannot initiate shutdown:
// the priority runner cannot join itself.
void Shutdown();
void Wait(uint64_t tick);
void PopPendingOperations();
void DrainPriorityOperations();
void WaitPriorityOperations(uint64_t tick);
void DeferOperation(Common::UniqueFunction<void>&& operation);
void DeferPriorityOperation(Common::UniqueFunction<void>&& operation);
void Shutdown();
void Wait(uint64_t tick);
void PopPendingOperations();
void DrainPriorityOperations();
void WaitPriorityOperations(uint64_t tick);
void DeferOperation(Common::UniqueFunction<void>&& operation);
void DeferPriorityOperation(Common::UniqueFunction<void>&& operation);
[[nodiscard]] static bool InDeferredOperation() noexcept;
[[nodiscard]] bool Active() const noexcept { return m_current >= 0; }
void CheckActive() const;
RenderCommandBuffer& Current() const;
[[nodiscard]] uint64_t CurrentTick() const noexcept { return m_master.CurrentTick(); }
[[nodiscard]] bool IsFree(uint64_t tick);
[[nodiscard]] RenderContext& Context() const noexcept { return m_context; }
[[nodiscard]] bool Active() const noexcept { return m_current >= 0; }
void CheckActive() const;
RenderCommandBuffer& Current() const;
[[nodiscard]] uint64_t CurrentTick() const noexcept { return m_master.CurrentTick(); }
[[nodiscard]] bool IsFree(uint64_t tick);
[[nodiscard]] RenderContext& Context() const noexcept { return m_context; }
[[nodiscard]] GraphicContext& Graphics() const noexcept { return m_graphics; }
private:
@@ -91,11 +91,11 @@ private:
uint64_t tick = 0;
};
void BindCurrent() const;
CommandBuffer& SubmitCurrent(SubmitInfo& submit);
void BeginNext();
void PriorityOperationsThread(std::stop_token stop);
void RunOperation(Common::UniqueFunction<void>&& operation);
void BindCurrent() const;
CommandBuffer& SubmitCurrent(SubmitInfo& submit);
void BeginNext();
void PriorityOperationsThread(std::stop_token stop);
void RunOperation(Common::UniqueFunction<void>&& operation);
[[nodiscard]] CommandSlot* AllocateCommandBuffer();
[[nodiscard]] uint64_t NextSubmitSequence() noexcept;
@@ -109,14 +109,14 @@ private:
std::mutex m_operation_mutex;
std::condition_variable m_operation_available;
std::jthread m_priority_thread;
bool m_priority_active = false;
bool m_priority_active = false;
uint64_t m_priority_active_tick = 0;
OperationState m_operation_state = OperationState::Open;
int m_current = -1;
bool m_recording = false;
HW::Context* m_registers = nullptr;
HW::UserConfig* m_user_config = nullptr;
HW::Shader* m_shaders = nullptr;
OperationState m_operation_state = OperationState::Open;
int m_current = -1;
bool m_recording = false;
HW::Context* m_registers = nullptr;
HW::UserConfig* m_user_config = nullptr;
HW::Shader* m_shaders = nullptr;
std::atomic<uint64_t> m_submit_sequence = 0;
friend class CommandBuffer;
+13 -13
View File
@@ -8,8 +8,8 @@
#include "graphics/host_gpu/renderer/colorRenderTarget.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/depthRenderTarget.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/image/imageView.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/host_gpu/vma.h"
@@ -270,30 +270,30 @@ void CommandBuffer::BeginRendering(const RenderState& state) const {
colors[i].sType = vk::StructureType::eRenderingAttachmentInfo;
colors[i].imageView = attachment.image_view;
colors[i].imageLayout = attachment.image_layout;
colors[i].loadOp = attachment.is_clear ? vk::AttachmentLoadOp::eClear
: vk::AttachmentLoadOp::eLoad;
colors[i].storeOp = vk::AttachmentStoreOp::eStore;
colors[i].clearValue.color.uint32 = attachment.clear_value;
colors[i].loadOp =
attachment.is_clear ? vk::AttachmentLoadOp::eClear : vk::AttachmentLoadOp::eLoad;
colors[i].storeOp = vk::AttachmentStoreOp::eStore;
colors[i].clearValue.color.uint32 = attachment.clear_value;
}
const auto& depth_stencil = state.depth_stencil_attachment;
const auto& depth_stencil = state.depth_stencil_attachment;
vk::RenderingAttachmentInfo depth {};
depth.sType = vk::StructureType::eRenderingAttachmentInfo;
depth.imageView = depth_stencil.image_view;
depth.imageLayout = depth_stencil.image_layout;
depth.loadOp = depth_stencil.depth_clear ? vk::AttachmentLoadOp::eClear
: vk::AttachmentLoadOp::eLoad;
depth.storeOp = vk::AttachmentStoreOp::eStore;
depth.loadOp =
depth_stencil.depth_clear ? vk::AttachmentLoadOp::eClear : vk::AttachmentLoadOp::eLoad;
depth.storeOp = vk::AttachmentStoreOp::eStore;
depth.clearValue.depthStencil.depth = std::bit_cast<float>(depth_stencil.clear_value[0]);
vk::RenderingAttachmentInfo stencil {};
stencil.sType = vk::StructureType::eRenderingAttachmentInfo;
stencil.imageView = depth_stencil.image_view;
stencil.imageLayout = depth_stencil.image_layout;
stencil.loadOp = depth_stencil.stencil_clear ? vk::AttachmentLoadOp::eClear
: vk::AttachmentLoadOp::eLoad;
stencil.storeOp = vk::AttachmentStoreOp::eStore;
stencil.clearValue.depthStencil.stencil = depth_stencil.clear_value[1];
stencil.loadOp =
depth_stencil.stencil_clear ? vk::AttachmentLoadOp::eClear : vk::AttachmentLoadOp::eLoad;
stencil.storeOp = vk::AttachmentStoreOp::eStore;
stencil.clearValue.depthStencil.stencil = depth_stencil.clear_value[1];
vk::RenderingInfo rendering {};
rendering.sType = vk::StructureType::eRenderingInfo;
-8
View File
@@ -548,14 +548,6 @@ static void ZCheck(const HW::DepthRenderTarget& z) {
EXIT_NOT_IMPLEMENTED(z.htile_surface.prefetch_height != 0x00000000);
EXIT_NOT_IMPLEMENTED(z.htile_surface.dst_outside_zero_to_one != 0x00000000);
if (z.depth_view.slice_start != 0x00000000 || z.depth_view.slice_max != 0x00000000) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1, std::memory_order_relaxed) < 16) {
LOGF("DepthTarget: temporary: ignoring PS5 array slice view start=0x%08" PRIx32
", max=0x%08" PRIx32 "\n",
z.depth_view.slice_start, z.depth_view.slice_max);
}
}
if (z.depth_view.current_mip_level != 0x00000000) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1, std::memory_order_relaxed) < 16) {
@@ -10,10 +10,10 @@
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/guest_gpu/tile.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/renderer/image/textureCommon.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/image/imageView.h"
#include "graphics/host_gpu/renderer/image/textureCommon.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/host_gpu/vulkanCommon.h"
@@ -150,10 +150,8 @@ void RenderExecutor::ResolveRenderDepthTarget(uint64_t submit_id, RenderCommandB
has_stencil, has_htile, z.stencil_info.htile_stencil_disabled);
const auto view = ResolveTargetViewInfo(z.depth_view.slice_start, z.depth_view.slice_max);
switch (view.type) {
case TargetViewType::Image2D: break;
case TargetViewType::Image2DArray:
DepthFatal("layered depth views are unsupported: base=%u count=%u", view.base_layer,
view.layer_count);
case TargetViewType::Image2D:
case TargetViewType::Image2DArray: break;
case TargetViewType::Unsupported:
DepthFatal("invalid depth view: base=%u last=%u", z.depth_view.slice_start,
z.depth_view.slice_max);
@@ -2,9 +2,9 @@
#define EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_DEPTHRENDERTARGET_H_
#include "common/assert.h"
#include "graphics/host_gpu/renderer/cache/textureCache.h"
#include "graphics/host_gpu/renderer/image/imageView.h"
#include "graphics/host_gpu/renderer/renderTarget.h"
#include "graphics/host_gpu/renderer/cache/textureCache.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include <cstdint>
@@ -182,8 +182,8 @@ void BlitHelper::ReinterpretColorAsMsDepth(Image& source, Image& destination) {
auto command = command_buffer.Handle();
source.Transit(vk::ImageLayout::eShaderReadOnlyOptimal, vk::AccessFlagBits2::eShaderRead, {},
command);
destination.Transit(ColorToMsDepthLayout,
vk::AccessFlagBits2::eDepthStencilAttachmentWrite, {}, command);
destination.Transit(ColorToMsDepthLayout, vk::AccessFlagBits2::eDepthStencilAttachmentWrite, {},
command);
vk::RenderingAttachmentInfo depth_attachment {};
depth_attachment.sType = vk::StructureType::eRenderingAttachmentInfo;
@@ -19,10 +19,9 @@ struct GuestRange {
uint64_t address = 0;
uint64_t size = 0;
[[nodiscard]] constexpr bool Empty() const noexcept { return address == 0 || size == 0; }
[[nodiscard]] constexpr bool Valid() const noexcept {
return !Empty() && address < TRACKER_ADDRESS_SIZE &&
size <= TRACKER_ADDRESS_SIZE - address;
[[nodiscard]] constexpr bool Empty() const noexcept { return address == 0 || size == 0; }
[[nodiscard]] constexpr bool Valid() const noexcept {
return !Empty() && address < TRACKER_ADDRESS_SIZE && size <= TRACKER_ADDRESS_SIZE - address;
}
[[nodiscard]] constexpr uint64_t End() const noexcept { return address + size; }
auto operator<=>(const GuestRange&) const = default;
@@ -47,10 +46,10 @@ struct ImageSubresources {
};
struct ImageSubresourceRange {
uint32_t base_level = 0;
uint32_t level_count = 1;
uint32_t base_layer = 0;
uint32_t layer_count = 1;
uint32_t base_level = 0;
uint32_t level_count = 1;
uint32_t base_layer = 0;
uint32_t layer_count = 1;
auto operator<=>(const ImageSubresourceRange&) const = default;
};
@@ -67,10 +66,10 @@ struct ImageInfo {
GuestRange stencil;
ImageMetadataInfo metadata;
uint32_t htile_clear_mask = UINT32_MAX;
vk::Format pixel_format = vk::Format::eUndefined;
uint32_t guest_format = 0;
Prospero::ImageType type = Prospero::ImageType::kColor2D;
vk::Extent3D extent = {1, 1, 1};
vk::Format pixel_format = vk::Format::eUndefined;
uint32_t guest_format = 0;
Prospero::ImageType type = Prospero::ImageType::kColor2D;
vk::Extent3D extent = {1, 1, 1};
ImageSubresources resources;
uint32_t pitch = 0;
uint32_t bytes_per_block = 0;
@@ -352,8 +351,7 @@ inline bool ImageInfo::IsDepth() const noexcept {
}
const auto transfer_bytes = DepthAspectTransferBytes(info.pixel_format);
return transfer_bytes == info.bytes_per_block ||
(info.bytes_per_block == sizeof(uint16_t) &&
transfer_bytes == sizeof(uint32_t));
(info.bytes_per_block == sizeof(uint16_t) && transfer_bytes == sizeof(uint32_t));
}
[[nodiscard]] inline VideoOutCompression
@@ -470,18 +468,13 @@ IsSupportedDisplayRenderTargetTileMode(uint32_t tile_mode) noexcept {
vk::ClearColorValue& clear) {
vk::ClearColorValue next {};
const auto unorm8 = [](uint32_t value) { return static_cast<float>(value & 0xffu) / 255.0f; };
const auto srgb8 = [](uint32_t value) {
const auto srgb8 = [](uint32_t value) {
const auto encoded = static_cast<float>(value & 0xffu) / 255.0f;
return encoded <= 0.04045f ? encoded / 12.92f
: std::pow((encoded + 0.055f) / 1.055f, 2.4f);
return encoded <= 0.04045f ? encoded / 12.92f : std::pow((encoded + 0.055f) / 1.055f, 2.4f);
};
switch (format) {
case vk::Format::eR32Uint:
next.uint32[0] = packed;
break;
case vk::Format::eR32Sint:
next.int32[0] = static_cast<int32_t>(packed);
break;
case vk::Format::eR32Uint: next.uint32[0] = packed; break;
case vk::Format::eR32Sint: next.int32[0] = static_cast<int32_t>(packed); break;
case vk::Format::eR8G8B8A8Srgb:
next.float32[0] = srgb8(packed);
next.float32[1] = srgb8(packed >> 8u);
@@ -70,15 +70,14 @@ namespace {
}
case vk::ImageType::e3D:
switch (info.type) {
case vk::ImageViewType::e3D:
return info.base_layer == 0 && info.layer_count == 1;
case vk::ImageViewType::e3D: return info.base_layer == 0 && info.layer_count == 1;
case vk::ImageViewType::e2D:
return static_cast<bool>(
image.flags & vk::ImageCreateFlagBits::e2DArrayCompatible) &&
return static_cast<bool>(image.flags &
vk::ImageCreateFlagBits::e2DArrayCompatible) &&
info.level_count == 1 && info.layer_count == 1;
case vk::ImageViewType::e2DArray:
return static_cast<bool>(
image.flags & vk::ImageCreateFlagBits::e2DArrayCompatible) &&
return static_cast<bool>(image.flags &
vk::ImageCreateFlagBits::e2DArrayCompatible) &&
info.level_count == 1;
default: return false;
}
@@ -325,11 +324,10 @@ bool FormatsCompatible(vk::Format base, vk::Format view) noexcept {
} // namespace ImageViewOps
vk::ImageView Image::FindView(const ImageViewInfo& view_info) {
const auto& image = backing;
const auto& image = backing;
auto normalized = view_info;
const bool is_storage =
static_cast<bool>(normalized.usage & vk::ImageUsageFlagBits::eStorage);
normalized.aspect = FullAspectMask(image.format);
const bool is_storage = static_cast<bool>(normalized.usage & vk::ImageUsageFlagBits::eStorage);
normalized.aspect = FullAspectMask(image.format);
if (normalized.aspect & vk::ImageAspectFlagBits::eDepth &&
IsDepthViewFormat(normalized.format)) {
normalized.format = image.format;
@@ -340,28 +338,26 @@ vk::ImageView Image::FindView(const ImageViewInfo& view_info) {
normalized.format = image.format;
normalized.aspect = vk::ImageAspectFlagBits::eStencil;
}
normalized.usage =
is_storage ? vk::ImageUsageFlagBits::eStorage : vk::ImageUsageFlags {};
normalized.usage = is_storage ? vk::ImageUsageFlagBits::eStorage : vk::ImageUsageFlags {};
const bool format_compatible = normalized.format != vk::Format::eUndefined &&
IsCompatibleViewFormat(image.format, normalized.format);
const bool slice_view = image.image_type == vk::ImageType::e3D &&
(normalized.type == vk::ImageViewType::e2D ||
normalized.type == vk::ImageViewType::e2DArray);
const bool slice_view =
image.image_type == vk::ImageType::e3D && (normalized.type == vk::ImageViewType::e2D ||
normalized.type == vk::ImageViewType::e2DArray);
const bool levels_valid = normalized.level_count != 0 &&
normalized.base_level < image.mip_levels &&
normalized.level_count <= image.mip_levels - normalized.base_level;
const auto view_layers = slice_view && levels_valid
? std::max(image.extent.depth >> normalized.base_level, 1u)
: image.layers;
const bool ranges_valid = levels_valid &&
normalized.layer_count != 0 && normalized.base_layer < view_layers &&
const auto view_layers = slice_view && levels_valid
? std::max(image.extent.depth >> normalized.base_level, 1u)
: image.layers;
const bool ranges_valid = levels_valid && normalized.layer_count != 0 &&
normalized.base_layer < view_layers &&
normalized.layer_count <= view_layers - normalized.base_layer;
const bool mapping_valid =
IsComponentSwizzle(normalized.mapping.r) && IsComponentSwizzle(normalized.mapping.g) &&
IsComponentSwizzle(normalized.mapping.b) && IsComponentSwizzle(normalized.mapping.a);
if (image.image == nullptr || !format_compatible || !ranges_valid || !mapping_valid ||
!IsValidViewType(image, normalized) ||
!IsValidAspect(image, normalized.aspect)) {
!IsValidViewType(image, normalized) || !IsValidAspect(image, normalized.aspect)) {
EXIT("invalid image view: image_format=%d view_format=%d type=%d aspect=0x%x "
"mip=%u+%u layer=%u+%u usage=0x%x image_levels=%u image_layers=%u\n",
static_cast<int>(image.format), static_cast<int>(normalized.format),
@@ -88,7 +88,9 @@ SelectSampledDepthView(vk::Format image_format, vk::Format view_format, uint32_t
IsSupportedSampledDepthResource(const ShaderRecompiler::IR::ImageResource& resource) noexcept {
return resource.kind == ShaderRecompiler::IR::ResourceKind::Image &&
(resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2D ||
resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2DArray) &&
resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2DArray ||
resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaa ||
resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaaArray) &&
resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::None && resource.read &&
!resource.written && !resource.atomic;
}
@@ -397,10 +397,10 @@ TextureUploadLayout TextureCalcUploadLayout(uint32_t fmt, uint64_t width, uint64
return layout;
}
std::vector<vk::BufferImageCopy>
TextureBuildImageCopies(const TextureUploadLayout& layout, uint32_t width, uint32_t height,
uint32_t depth, uint64_t levels, bool array_texture,
bool volume_texture) {
std::vector<vk::BufferImageCopy> TextureBuildImageCopies(const TextureUploadLayout& layout,
uint32_t width, uint32_t height,
uint32_t depth, uint64_t levels,
bool array_texture, bool volume_texture) {
uint32_t mip_width = width;
uint32_t mip_height = height;
uint32_t mip_pitch = volume_texture && static_cast<Prospero::TileMode>(layout.tile) !=
@@ -416,14 +416,13 @@ TextureBuildImageCopies(const TextureUploadLayout& layout, uint32_t width, uint3
const auto mip_depth = GetTextureLevelDepth(depth, i, volume_texture);
for (uint32_t z = 0; z < mip_depth; z++) {
const auto slice_offset = z * layout.slice_stride;
const auto slice_offset = z * layout.slice_stride;
vk::BufferImageCopy region {};
region.bufferOffset =
layout.level_sizes[i].offset + slice_offset;
region.imageSubresource = {vk::ImageAspectFlagBits::eColor, i,
array_texture ? z : 0, 1};
region.imageOffset.z = volume_texture ? static_cast<int>(z) : 0;
region.imageExtent = {mip_width, mip_height, 1};
region.bufferOffset = layout.level_sizes[i].offset + slice_offset;
region.imageSubresource = {vk::ImageAspectFlagBits::eColor, i, array_texture ? z : 0,
1};
region.imageOffset.z = volume_texture ? static_cast<int>(z) : 0;
region.imageExtent = {mip_width, mip_height, 1};
const bool linear =
static_cast<Prospero::TileMode>(layout.tile) == Prospero::TileMode::kLinear;
if (linear) {
@@ -433,9 +432,8 @@ TextureBuildImageCopies(const TextureUploadLayout& layout, uint32_t width, uint3
const auto align = [](uint32_t value, uint32_t block) {
return ((value + block - 1u) / block) * block;
};
const auto pitch = align(mip_pitch, layout.texel_block);
region.bufferRowLength =
pitch > align(mip_width, layout.texel_block) ? pitch : 0;
const auto pitch = align(mip_pitch, layout.texel_block);
region.bufferRowLength = pitch > align(mip_width, layout.texel_block) ? pitch : 0;
}
regions.push_back(region);
}
@@ -480,8 +478,7 @@ static bool SetGpuTileSize(uint64_t offset, uint64_t length, uint64_t capacity,
return true;
}
bool TextureBuildGpuTileInfos(uint64_t size,
const std::vector<vk::BufferImageCopy>& regions,
bool TextureBuildGpuTileInfos(uint64_t size, const std::vector<vk::BufferImageCopy>& regions,
const TextureUploadLayout& layout, uint32_t fmt, uint32_t depth,
uint64_t levels, std::vector<GpuTileInfo>& out_infos) {
if (size == 0 || levels == 0 || levels > 16 || depth == 0 ||
@@ -522,13 +519,12 @@ bool TextureBuildGpuTileInfos(uint64_t size,
for (uint32_t z = 0; z < mip_depth; z += block.block_depth) {
const uint32_t copy_depth = std::min(block.block_depth, mip_depth - z);
const auto& region = regions[region_base + z];
const auto pitch = region.bufferRowLength != 0
? region.bufferRowLength
: region.imageExtent.width;
const auto logical_height = region.bufferImageHeight != 0
? region.bufferImageHeight
: region.imageExtent.height;
GpuTileInfo info {};
const auto pitch =
region.bufferRowLength != 0 ? region.bufferRowLength : region.imageExtent.width;
const auto logical_height = region.bufferImageHeight != 0
? region.bufferImageHeight
: region.imageExtent.height;
GpuTileInfo info {};
info.family = block.family;
info.bytes_per_element = block.bytes_per_element;
info.linear_offset = region.bufferOffset;
@@ -544,20 +540,17 @@ bool TextureBuildGpuTileInfos(uint64_t size,
return false;
}
info.linear_slice_stride = linear_stride;
info.width = std::max(
(region.imageExtent.width + element.wide - 1u) / element.wide, 1u);
info.height = std::max(
(logical_height + element.tall - 1u) / element.tall, 1u);
info.depth = copy_depth;
info.surface_z = block.block_depth == 1
? static_cast<uint32_t>(region.imageOffset.z)
: 0;
info.pitch =
std::max((pitch + element.wide - 1u) / element.wide, 1u);
info.tail_x = tail ? volume.tail_x[level] : 0;
info.tail_y = tail ? volume.tail_y[level] : 0;
info.tail = tail;
info.tiled_width = volume.level_widths[level];
info.width =
std::max((region.imageExtent.width + element.wide - 1u) / element.wide, 1u);
info.height = std::max((logical_height + element.tall - 1u) / element.tall, 1u);
info.depth = copy_depth;
info.surface_z =
block.block_depth == 1 ? static_cast<uint32_t>(region.imageOffset.z) : 0;
info.pitch = std::max((pitch + element.wide - 1u) / element.wide, 1u);
info.tail_x = tail ? volume.tail_x[level] : 0;
info.tail_y = tail ? volume.tail_y[level] : 0;
info.tail = tail;
info.tiled_width = volume.level_widths[level];
info.tiled_height = volume.level_heights[level];
infos.push_back(info);
}
@@ -581,12 +574,11 @@ bool TextureBuildGpuTileInfos(uint64_t size,
const auto level_depth = GetTextureLevelDepth(depth, level, layout.volume_texture);
for (uint32_t z = 0; z < level_depth; z++) {
const auto& region = regions[region_index++];
const auto pitch = region.bufferRowLength != 0
? region.bufferRowLength
: region.imageExtent.width;
const auto logical_height = region.bufferImageHeight != 0
? region.bufferImageHeight
: region.imageExtent.height;
const auto pitch =
region.bufferRowLength != 0 ? region.bufferRowLength : region.imageExtent.width;
const auto logical_height = region.bufferImageHeight != 0
? region.bufferImageHeight
: region.imageExtent.height;
GpuTileInfo info {};
info.family = block.family;
info.bytes_per_element = block.bytes_per_element;
@@ -597,16 +589,14 @@ bool TextureBuildGpuTileInfos(uint64_t size,
info.tiled_size)) {
return false;
}
info.width = std::max(
(region.imageExtent.width + element.wide - 1u) / element.wide, 1u);
info.height = std::max(
(logical_height + element.tall - 1u) / element.tall, 1u);
info.width =
std::max((region.imageExtent.width + element.wide - 1u) / element.wide, 1u);
info.height = std::max((logical_height + element.tall - 1u) / element.tall, 1u);
info.surface_z = base_family == TileBlockFamily::RenderTarget64KB ||
base_family == TileBlockFamily::Depth64KB
? region.imageSubresource.baseArrayLayer
: 0;
info.pitch =
std::max((pitch + element.wide - 1u) / element.wide, 1u);
info.pitch = std::max((pitch + element.wide - 1u) / element.wide, 1u);
info.tail = tail;
info.tail_x = tail ? level_size.x : 0;
info.tail_y = tail ? level_size.y : 0;
@@ -32,20 +32,19 @@ struct TextureUploadLayout {
TilePaddedSize padded_sizes[16] = {};
};
vk::ComponentMapping TextureGetComponentMapping(uint32_t swizzle);
vk::ComponentMapping TextureGetComponentMapping(uint32_t swizzle);
vk::Format TextureGetFormat(uint32_t fmt);
RenderTargetFormatInfo TextureGetRenderTargetFormat(uint32_t layout, uint32_t type, uint32_t order);
TextureUploadLayout TextureCalcUploadLayout(uint32_t fmt, uint64_t width, uint64_t height,
uint64_t levels, uint32_t depth, uint64_t pitch,
uint64_t tile, uint64_t upload_size,
bool allow_depth_tile, bool volume_texture,
const char* owner);
std::vector<vk::BufferImageCopy>
TextureBuildImageCopies(const TextureUploadLayout& layout, uint32_t width, uint32_t height,
uint32_t depth, uint64_t levels, bool array_texture,
bool volume_texture);
bool TextureBuildGpuTileInfos(uint64_t size,
const std::vector<vk::BufferImageCopy>& regions,
TextureUploadLayout TextureCalcUploadLayout(uint32_t fmt, uint64_t width, uint64_t height,
uint64_t levels, uint32_t depth, uint64_t pitch,
uint64_t tile, uint64_t upload_size,
bool allow_depth_tile, bool volume_texture,
const char* owner);
std::vector<vk::BufferImageCopy> TextureBuildImageCopies(const TextureUploadLayout& layout,
uint32_t width, uint32_t height,
uint32_t depth, uint64_t levels,
bool array_texture, bool volume_texture);
bool TextureBuildGpuTileInfos(uint64_t size, const std::vector<vk::BufferImageCopy>& regions,
const TextureUploadLayout& layout, uint32_t fmt, uint32_t depth,
uint64_t levels, std::vector<GpuTileInfo>& infos);
@@ -14,9 +14,9 @@
#include "gpu_tiler_shaders/gpu_tiler_standard64_spv.h"
#include "gpu_tiler_shaders/gpu_tiler_swap_bgra16_spv.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/renderer/cache/streamBuffer.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
#include "graphics/host_gpu/renderer/image/image.h"
#include "graphics/host_gpu/renderer/cache/streamBuffer.h"
#include <algorithm>
#include <array>
@@ -26,8 +26,8 @@ MasterSemaphore::~MasterSemaphore() {
}
void MasterSemaphore::Refresh() {
uint64_t counter = 0;
const auto result = m_graphics.device.getSemaphoreCounterValue(m_semaphore, &counter);
uint64_t counter = 0;
const auto result = m_graphics.device.getSemaphoreCounterValue(m_semaphore, &counter);
EXIT_NOT_IMPLEMENTED(result != vk::Result::eSuccess);
auto known = m_gpu_tick.load(std::memory_order_acquire);
@@ -22,7 +22,7 @@ public:
[[nodiscard]] uint64_t KnownGpuTick() const noexcept {
return m_gpu_tick.load(std::memory_order_acquire);
}
[[nodiscard]] bool IsFree(uint64_t tick) const noexcept { return KnownGpuTick() >= tick; }
[[nodiscard]] bool IsFree(uint64_t tick) const noexcept { return KnownGpuTick() >= tick; }
[[nodiscard]] uint64_t NextTick() noexcept {
return m_current_tick.fetch_add(1, std::memory_order_release);
}
@@ -26,11 +26,15 @@ bool IsSampledImage(BindingKind kind) {
case BindingKind::Sampled1DArray:
case BindingKind::Sampled2D:
case BindingKind::Sampled2DArray:
case BindingKind::Sampled2DMsaa:
case BindingKind::Sampled2DMsaaArray:
case BindingKind::Sampled3D:
case BindingKind::SampledUint1D:
case BindingKind::SampledUint1DArray:
case BindingKind::SampledUint2D:
case BindingKind::SampledUint2DArray:
case BindingKind::SampledUint2DMsaa:
case BindingKind::SampledUint2DMsaaArray:
case BindingKind::SampledUint3D: return true;
default: return false;
}
@@ -95,7 +95,7 @@ private:
};
static vk::DescriptorImageInfo MakeImageInfo(const TextureBinding& texture);
void CreatePool();
void CreatePool();
VulkanDescriptorSet* Allocate(Stage stage, const ShaderRecompiler::IR::Program& program);
vk::DescriptorSetLayout
GetDescriptorSetLayoutInternal(Stage stage, const ShaderRecompiler::IR::Program& program);
@@ -73,6 +73,11 @@ static Prospero::ImageType TextureBaseType(Prospero::ImageType type) {
}
}
static bool IsMultisampledTexture(Prospero::ImageType type) {
return type == Prospero::ImageType::kColor2DMsaa ||
type == Prospero::ImageType::kColor2DMsaaArray;
}
static BufferView NativeStorageBuffer(RenderContext& context, CommandBuffer& command_buffer,
const ShaderBufferResource& descriptor,
const ShaderRecompiler::IR::BufferResource& resource,
@@ -159,6 +164,8 @@ static bool IsSupportedSampledColorResource(const ShaderRecompiler::IR::ImageRes
case ShaderRecompiler::Decoder::ImageDimension::Dim1DArray:
case ShaderRecompiler::Decoder::ImageDimension::Dim2D:
case ShaderRecompiler::Decoder::ImageDimension::Dim2DArray:
case ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaa:
case ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaaArray:
supported_dimension = true;
break;
default: break;
@@ -195,6 +202,22 @@ TargetTextureViewInfo ResolveTargetTextureView(const ShaderRecompiler::IR::Image
? TargetTextureViewInfo {vk::ImageViewType::e2DArray, base_layer,
image_layers - base_layer}
: TargetTextureViewInfo {};
case Prospero::ImageType::kColor2DMsaa:
return resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaa &&
base_layer == 0 && image_layers == 1
? TargetTextureViewInfo {vk::ImageViewType::e2D, 0, 1}
: TargetTextureViewInfo {};
case Prospero::ImageType::kColor2DMsaaArray:
if (resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaa &&
base_layer == 0 && image_layers == 1) {
return {vk::ImageViewType::e2D, 0, 1};
}
return resource.dimension ==
ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaaArray &&
base_layer < image_layers
? TargetTextureViewInfo {vk::ImageViewType::e2DArray, base_layer,
image_layers - base_layer}
: TargetTextureViewInfo {};
default: return {};
}
}
@@ -209,41 +232,64 @@ bool IsSupportedSampledVideoOutView(const ShaderRecompiler::IR::ImageResource& r
}
bool IsSupportedDepthTargetDescriptor(const ShaderTextureResource& descriptor, const Image& image) {
const auto width = static_cast<uint32_t>(descriptor.Width5()) + 1u;
const auto height = static_cast<uint32_t>(descriptor.Height5()) + 1u;
const auto pitch = TileGetTexturePitch(descriptor.Format(), width, 1, descriptor.TileMode());
const auto type = static_cast<Prospero::ImageType>(descriptor.Type());
const bool supported_single_layer =
image.info.resources.layers == 1 && descriptor.Depth() == 0 &&
descriptor.BaseArray5() == 0 &&
(type == Prospero::ImageType::kColor2D || type == Prospero::ImageType::kColor2DArray);
const auto width = static_cast<uint32_t>(descriptor.Width5()) + 1u;
const auto height = static_cast<uint32_t>(descriptor.Height5()) + 1u;
const auto type = static_cast<Prospero::ImageType>(descriptor.Type());
const bool multisampled = IsMultisampledTexture(type);
const auto samples = multisampled ? 1u << descriptor.LastLevel() : 1u;
const auto pitch =
multisampled ? TileGetDepthPitch(width, image.info.bytes_per_block, descriptor.LastLevel())
: TileGetTexturePitch(descriptor.Format(), width, 1, descriptor.TileMode());
const bool supported_2d = type == Prospero::ImageType::kColor2D &&
image.info.resources.layers == 1 && descriptor.Depth() == 0 &&
descriptor.BaseArray5() == 0;
const bool supported_array = type == Prospero::ImageType::kColor2DArray &&
descriptor.BaseArray5() <= descriptor.Depth() &&
descriptor.Depth() < image.info.resources.layers;
const bool supported_cube =
type == Prospero::ImageType::kCube && width == height && image.info.resources.layers >= 6 &&
image.info.resources.layers % 6u == 0 &&
static_cast<uint32_t>(descriptor.Depth()) + 1u == image.info.resources.layers &&
descriptor.BaseArray5() == 0;
const bool supported_msaa_2d = type == Prospero::ImageType::kColor2DMsaa &&
image.info.resources.layers == 1 && descriptor.Depth() == 0 &&
descriptor.BaseArray5() == 0;
const bool supported_msaa_array = type == Prospero::ImageType::kColor2DMsaaArray &&
descriptor.BaseArray5() <= descriptor.Depth() &&
descriptor.Depth() < image.info.resources.layers;
const bool levels_ok =
multisampled
? descriptor.BaseLevel() == 0 && descriptor.LastLevel() >= 1 &&
descriptor.LastLevel() <= 3 && descriptor.MaxMip() == descriptor.LastLevel() &&
image.info.resources.levels == 1 && image.info.samples == samples
: descriptor.BaseLevel() == 0 && descriptor.LastLevel() == 0 &&
descriptor.MaxMip() == 0 && image.info.samples == 1;
return image.info.IsDepth() && width == image.info.extent.width &&
height == image.info.extent.height && (supported_single_layer || supported_cube) &&
descriptor.BaseLevel() == 0 && descriptor.LastLevel() == 0 && descriptor.MaxMip() == 0 &&
descriptor.MinLod() == 0 && descriptor.BaseArray5() == 0 &&
height == image.info.extent.height &&
(supported_2d || supported_array || supported_cube || supported_msaa_2d ||
supported_msaa_array) &&
levels_ok && descriptor.MinLod() == 0 &&
descriptor.TileMode() == Prospero::GpuEnumValue(Prospero::TileMode::kDepth) &&
descriptor.BCSwizzle() == 0 && !descriptor.MsaaDepth() && pitch >= width &&
pitch == image.info.pitch;
descriptor.BCSwizzle() == 0 && descriptor.MsaaDepth() == multisampled &&
pitch >= width && pitch == image.info.pitch;
}
bool IsSupportedDepthTextureEncoding(const ShaderTextureResource& descriptor, const Image& image) {
constexpr uint32_t field1_reserved_mask = 0x200fff00u;
constexpr uint32_t field2_reserved_mask = 0xf0003000u;
constexpr uint32_t field3_common = 0x01800000u;
constexpr uint32_t field5_expected = 0x00700000u;
const uint32_t field3_expected =
(descriptor.Type() << 28u) | field3_common | descriptor.DstSelXYZW();
const uint32_t field4_expected = descriptor.Depth() | (descriptor.BaseArray5() << 16u);
const bool common = (descriptor.fields[1] & field1_reserved_mask) == 0 &&
(descriptor.fields[2] & field2_reserved_mask) == 0 &&
descriptor.fields[3] == field3_expected &&
descriptor.fields[4] == field4_expected &&
descriptor.fields[5] == field5_expected;
const uint32_t field3_expected = descriptor.DstSelXYZW() |
(static_cast<uint32_t>(descriptor.BaseLevel()) << 12u) |
(static_cast<uint32_t>(descriptor.LastLevel()) << 16u) |
(static_cast<uint32_t>(descriptor.TileMode()) << 20u) |
(static_cast<uint32_t>(descriptor.Type()) << 28u);
const uint32_t field4_expected = descriptor.Depth() | (descriptor.BaseArray5() << 16u);
const uint32_t field5_expected =
0x00700000u | (static_cast<uint32_t>(descriptor.MaxMip()) << 4u);
const bool common = (descriptor.fields[1] & field1_reserved_mask) == 0 &&
(descriptor.fields[2] & field2_reserved_mask) == 0 &&
descriptor.fields[3] == field3_expected &&
descriptor.fields[4] == field4_expected &&
descriptor.fields[5] == field5_expected;
if (!common || (descriptor.fields[6] == 0 && descriptor.fields[7] != 0)) {
return false;
}
@@ -251,8 +297,9 @@ bool IsSupportedDepthTextureEncoding(const ShaderTextureResource& descriptor, co
return true;
}
constexpr uint32_t htile_control = 0x00280000u;
const auto metadata_addr = descriptor.MetaAddr() << 8u;
return (descriptor.fields[6] & 0x00ffffffu) == htile_control && metadata_addr != 0 &&
const uint32_t expected_control = htile_control | (descriptor.MsaaDepth() ? (1u << 10u) : 0u);
const auto metadata_addr = descriptor.MetaAddr() << 8u;
return (descriptor.fields[6] & 0x00ffffffu) == expected_control && metadata_addr != 0 &&
descriptor.TileMode() == Prospero::GpuEnumValue(Prospero::TileMode::kDepth) &&
image.info.tile_mode == Prospero::GpuEnumValue(Prospero::TileMode::kDepth) &&
image.info.metadata.kind == ImageMetadataKind::Htile &&
@@ -518,6 +565,7 @@ static ImageViewInfo TextureViewInfo(const ShaderRecompiler::IR::ImageResource&
view.layer_count = 1;
break;
case ShaderRecompiler::Decoder::ImageDimension::Dim2DArray:
case ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaaArray:
view.type = vk::ImageViewType::e2DArray;
view.base_layer = descriptor.BaseArray5();
if (view.base_layer >= image_layers) {
@@ -526,6 +574,7 @@ static ImageViewInfo TextureViewInfo(const ShaderRecompiler::IR::ImageResource&
view.layer_count = image_layers - view.base_layer;
break;
case ShaderRecompiler::Decoder::ImageDimension::Dim2D:
case ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaa:
view.type = vk::ImageViewType::e2D;
view.base_layer = descriptor.BaseArray5();
if (view.base_layer >= image_layers) {
@@ -556,22 +605,23 @@ RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageResource& reso
return {id, nullptr, std::move(desc)};
}
const auto address = descriptor.Base40();
const auto width = static_cast<uint32_t>(descriptor.Width5()) + 1u;
const auto height = static_cast<uint32_t>(descriptor.Height5()) + 1u;
const auto base_level = descriptor.BaseLevel();
const auto last_level = descriptor.LastLevel();
const auto type = TextureType(descriptor);
const bool multisampled =
type == Prospero::ImageType::kColor2DMsaa || type == Prospero::ImageType::kColor2DMsaaArray;
const auto levels = multisampled ? 1u : static_cast<uint32_t>(descriptor.MaxMip()) + 1u;
const auto tile = descriptor.TileMode();
const bool msaa_tile = tile == Prospero::GpuEnumValue(Prospero::TileMode::kRenderTarget);
const auto address = descriptor.Base40();
const auto width = static_cast<uint32_t>(descriptor.Width5()) + 1u;
const auto height = static_cast<uint32_t>(descriptor.Height5()) + 1u;
const auto base_level = descriptor.BaseLevel();
const auto last_level = descriptor.LastLevel();
const auto type = TextureType(descriptor);
const bool multisampled = IsMultisampledTexture(type);
const auto levels = multisampled ? 1u : static_cast<uint32_t>(descriptor.MaxMip()) + 1u;
const auto tile = descriptor.TileMode();
const bool msaa_tile =
tile == Prospero::GpuEnumValue(descriptor.MsaaDepth() ? Prospero::TileMode::kDepth
: Prospero::TileMode::kRenderTarget);
const bool msaa_array = type == Prospero::ImageType::kColor2DMsaaArray;
if ((!multisampled && (base_level > last_level || last_level >= levels)) ||
(multisampled &&
(base_level != 0 || last_level == 0 || last_level > 3 ||
descriptor.MaxMip() != last_level || !msaa_tile || descriptor.MsaaDepth() ||
descriptor.MaxMip() != last_level || !msaa_tile ||
(!msaa_array && (descriptor.Depth() != 0 || descriptor.BaseArray5() != 0))))) {
EXIT("unsupported texture mip view: base=%u last=%u levels=%u\n", base_level, last_level,
levels);
@@ -616,8 +666,6 @@ RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageResource& reso
(address & (static_cast<uint64_t>(size.align) - 1u)) != 0);
if (storage) {
ValidateStorageTexture(resource, descriptor, size.size);
m_context.GetBufferCache().ValidateGpuAccess(address, size.size, resource.read,
resource.written);
}
const auto pixel_format = TextureGetFormat(format);
@@ -36,7 +36,7 @@ ResolveTargetTextureView(const ShaderRecompiler::IR::ImageResource& resource,
[[nodiscard]] bool IsSupportedDepthTargetDescriptor(const ShaderTextureResource& descriptor,
const Image& image);
[[nodiscard]] bool IsSupportedDepthTextureEncoding(const ShaderTextureResource& descriptor,
const Image& image);
const Image& image);
[[nodiscard]] bool
IsSupportedSampledVideoOutView(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderTextureResource& descriptor, const Image& image);
@@ -88,12 +88,12 @@ PipelineCache::GraphicsPipeline& PipelineCache::CreateGraphicsPipeline(
PipelineStaticParameters static_params {};
GraphicsPipeline p {};
p.ps_shader_id = ps_id;
p.vs_shader_id = vs_id;
p.ps_shader_id = ps_id;
p.vs_shader_id = vs_id;
static_params.color_count = color_count;
PipelineRenderingState rendering {};
rendering.color_count = color_count;
rendering.color_count = color_count;
uint32_t attachment_samples = 0;
for (uint32_t i = 0; i < color_count; i++) {
EXIT_IF(!colors[i].image_id || colors[i].format == vk::Format::eUndefined);
@@ -116,8 +116,8 @@ PipelineCache::GraphicsPipeline& PipelineCache::CreateGraphicsPipeline(
if (attachment_samples == 0) {
attachment_samples = depth.samples;
} else if (attachment_samples != depth.samples) {
EXIT("mixed color/depth sample counts are unsupported: %u and %u\n",
attachment_samples, depth.samples);
EXIT("mixed color/depth sample counts are unsupported: %u and %u\n", attachment_samples,
depth.samples);
}
}
EXIT_IF(attachment_samples == 0 ||
@@ -179,10 +179,10 @@ PipelineCache::GraphicsPipeline& PipelineCache::CreateGraphicsPipeline(
NormalizeStaticParamsForDynamicState(static_params);
GraphicsPipelineKey key {};
key.rendering = rendering;
key.vs_shader_id = p.vs_shader_id;
key.ps_shader_id = p.ps_shader_id;
key.static_params = static_params;
key.rendering = rendering;
key.vs_shader_id = p.vs_shader_id;
key.ps_shader_id = p.ps_shader_id;
key.static_params = static_params;
if (auto iter = m_graphics_pipelines.find(key); iter != m_graphics_pipelines.end()) {
return *iter->second;
@@ -203,9 +203,8 @@ PipelineCache::GraphicsPipeline& PipelineCache::CreateGraphicsPipeline(
LogPipelineTrace("CreatePipelineInternal begin", vs_id.hash0, vs_id.crc32, ps_id.hash0,
ps_id.crc32);
CreatePipelineInternal(m_graphics, m_descriptor_cache, *cached, rendering, vs_input_info,
vs_spirv, ps_input_info,
ps_spirv, static_params, vs_id.hash0, vs_id.crc32, ps_id.hash0,
ps_id.crc32, ps_active);
vs_spirv, ps_input_info, ps_spirv, static_params, vs_id.hash0,
vs_id.crc32, ps_id.hash0, ps_id.crc32, ps_active);
LogPipelineTrace("CreatePipelineInternal done", vs_id.hash0, vs_id.crc32, ps_id.hash0,
ps_id.crc32);
@@ -88,9 +88,9 @@ static_assert(sizeof(PipelineStaticParameters) ==
struct PipelineRenderingState {
std::array<vk::Format, RENDER_COLOR_ATTACHMENTS_MAX> color_formats {};
vk::Format depth_format = vk::Format::eUndefined;
vk::Format stencil_format = vk::Format::eUndefined;
uint32_t color_count = 0;
vk::Format depth_format = vk::Format::eUndefined;
vk::Format stencil_format = vk::Format::eUndefined;
uint32_t color_count = 0;
bool operator==(const PipelineRenderingState&) const = default;
};
@@ -118,11 +118,12 @@ public:
ShaderId cs_shader_id;
};
GraphicsPipeline& CreateGraphicsPipeline(
RenderColorInfo* colors, uint32_t color_count, RenderDepthInfo& depth,
ShaderVertexInputInfo& vs_input_info, RenderCommandBuffer& command,
ShaderPixelInputInfo* ps_input_info, vk::PrimitiveTopology topology, bool ps_active,
std::span<const uint32_t> vs_spirv, std::span<const uint32_t> ps_spirv);
GraphicsPipeline&
CreateGraphicsPipeline(RenderColorInfo* colors, uint32_t color_count, RenderDepthInfo& depth,
ShaderVertexInputInfo& vs_input_info, RenderCommandBuffer& command,
ShaderPixelInputInfo* ps_input_info, vk::PrimitiveTopology topology,
bool ps_active, std::span<const uint32_t> vs_spirv,
std::span<const uint32_t> ps_spirv);
ComputePipeline& CreateComputePipeline(ShaderComputeInputInfo& input_info,
const HW::ComputeShaderInfo& cs_regs,
std::span<const uint32_t> cs_spirv);
@@ -199,7 +200,7 @@ private:
}
};
GraphicContext& m_graphics;
GraphicContext& m_graphics;
DescriptorCache& m_descriptor_cache;
std::unordered_map<GraphicsPipelineKey, std::unique_ptr<GraphicsPipeline>,
GraphicsPipelineKeyHash>
@@ -211,16 +212,13 @@ private:
void LogPipelineTrace(const char* phase, uint32_t vs_hash0, uint32_t vs_crc32, uint32_t ps_hash0,
uint32_t ps_crc32);
void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descriptor_cache,
PipelineCache::GraphicsPipeline& pipeline,
const PipelineRenderingState& rendering,
const ShaderVertexInputInfo& vs_input_info,
std::span<const uint32_t> vs_shader,
const ShaderPixelInputInfo* ps_input_info,
std::span<const uint32_t> ps_shader,
const PipelineStaticParameters& static_params, uint32_t vs_hash0,
uint32_t vs_crc32, uint32_t ps_hash0, uint32_t ps_crc32,
bool ps_active);
void CreatePipelineInternal(
GraphicContext& graphics, DescriptorCache& descriptor_cache,
PipelineCache::GraphicsPipeline& pipeline, const PipelineRenderingState& rendering,
const ShaderVertexInputInfo& vs_input_info, std::span<const uint32_t> vs_shader,
const ShaderPixelInputInfo* ps_input_info, std::span<const uint32_t> ps_shader,
const PipelineStaticParameters& static_params, uint32_t vs_hash0, uint32_t vs_crc32,
uint32_t ps_hash0, uint32_t ps_crc32, bool ps_active);
void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descriptor_cache,
PipelineCache::ComputePipeline& pipeline,
const ShaderComputeInputInfo& input_info,
@@ -8,10 +8,10 @@
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/pipeline/pipelineCache.h"
#include "graphics/host_gpu/renderer/pipeline/shaderSubgroup.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/host_gpu/renderer/renderTarget.h"
#include "graphics/host_gpu/renderer/pipeline/shaderSubgroup.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/shader.h"
@@ -385,9 +385,8 @@ static vk::BlendOp GetBlendOp(uint32_t op) {
return vk::BlendOp::eAdd;
}
static void CreateLayout(DescriptorCache& descriptor_cache,
std::span<vk::DescriptorSetLayout> set_layouts,
uint32_t& set_layouts_num,
static void CreateLayout(DescriptorCache& descriptor_cache,
std::span<vk::DescriptorSetLayout> set_layouts, uint32_t& set_layouts_num,
std::span<vk::PushConstantRange> push_constant_info,
uint32_t& push_constant_info_num,
const ShaderRecompiler::IR::Program& program,
@@ -412,12 +411,11 @@ static void CreateLayout(DescriptorCache& descriptor_cache,
}
}
static void ConfigureSubgroupSize(const GraphicContext& graphics,
vk::ShaderStageFlagBits vk_stage,
static void ConfigureSubgroupSize(const GraphicContext& graphics, vk::ShaderStageFlagBits vk_stage,
const ShaderRecompiler::IR::Program& program,
vk::PipelineShaderStageRequiredSubgroupSizeCreateInfo& required,
vk::PipelineShaderStageCreateInfo& stage) {
const auto config =
const auto config =
ConfigureShaderSubgroup(ShaderSubgroupCapabilities {graphics}, vk_stage, program);
switch (config.mode) {
case ShaderSubgroupMode::Natural: return;
@@ -456,16 +454,13 @@ static void ConfigureSubgroupSize(const GraphicContext&
}
// NOLINTNEXTLINE(readability-function-cognitive-complexity)
void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descriptor_cache,
PipelineCache::GraphicsPipeline& pipeline,
const PipelineRenderingState& rendering,
const ShaderVertexInputInfo& vs_input_info,
std::span<const uint32_t> vs_shader,
const ShaderPixelInputInfo* ps_input_info,
std::span<const uint32_t> ps_shader,
const PipelineStaticParameters& static_params, uint32_t vs_hash0,
uint32_t vs_crc32, uint32_t ps_hash0, uint32_t ps_crc32,
bool ps_active) {
void CreatePipelineInternal(
GraphicContext& graphics, DescriptorCache& descriptor_cache,
PipelineCache::GraphicsPipeline& pipeline, const PipelineRenderingState& rendering,
const ShaderVertexInputInfo& vs_input_info, std::span<const uint32_t> vs_shader,
const ShaderPixelInputInfo* ps_input_info, std::span<const uint32_t> ps_shader,
const PipelineStaticParameters& static_params, uint32_t vs_hash0, uint32_t vs_crc32,
uint32_t ps_hash0, uint32_t ps_crc32, bool ps_active) {
EXIT_IF(ps_active && ps_input_info == nullptr);
vk::ShaderModule vert_shader_module = nullptr;
@@ -511,8 +506,7 @@ void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descripto
vert_shader_stage_info.pName = "main";
vert_shader_stage_info.pSpecializationInfo = nullptr;
EXIT_IF(!vs_input_info.stage);
ConfigureSubgroupSize(graphics, vk::ShaderStageFlagBits::eVertex,
*vs_input_info.stage.program,
ConfigureSubgroupSize(graphics, vk::ShaderStageFlagBits::eVertex, *vs_input_info.stage.program,
vert_subgroup_size, vert_shader_stage_info);
vk::PipelineShaderStageCreateInfo frag_shader_stage_info {};
@@ -527,8 +521,8 @@ void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descripto
if (ps_active) {
EXIT_IF(!ps_input_info->stage);
ConfigureSubgroupSize(graphics, vk::ShaderStageFlagBits::eFragment,
*ps_input_info->stage.program,
frag_subgroup_size, frag_shader_stage_info);
*ps_input_info->stage.program, frag_subgroup_size,
frag_shader_stage_info);
}
vk::PipelineShaderStageCreateInfo shader_stages[] = {vert_shader_stage_info,
@@ -728,13 +722,13 @@ void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descripto
clip_ext.depthClipEnable = static_params.depth_clip_enable ? VK_TRUE : VK_FALSE;
vk::PipelineRasterizationStateCreateInfo rasterizer {};
rasterizer.sType = vk::StructureType::ePipelineRasterizationStateCreateInfo;
rasterizer.sType = vk::StructureType::ePipelineRasterizationStateCreateInfo;
// MoltenVK lacks VK_EXT_depth_clip_enable; omit the depth-clip struct on macOS and accept
// Vulkan's default depth clipping (enabled) instead of the PS5's clamp behavior.
#if defined(__APPLE__)
rasterizer.pNext = nullptr;
rasterizer.pNext = nullptr;
#else
rasterizer.pNext = &clip_ext;
rasterizer.pNext = &clip_ext;
#endif
rasterizer.flags = {};
rasterizer.depthClampEnable = VK_FALSE;
@@ -812,13 +806,13 @@ void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descripto
color_write.pColorWriteEnables = color_write_enable;
vk::PipelineColorBlendStateCreateInfo color_blending {};
color_blending.sType = vk::StructureType::ePipelineColorBlendStateCreateInfo;
color_blending.sType = vk::StructureType::ePipelineColorBlendStateCreateInfo;
// MoltenVK lacks VK_EXT_color_write_enable; drop the dynamic color-write struct on macOS
// and rely on each attachment's static colorWriteMask (all channels enabled by default).
#if defined(__APPLE__)
color_blending.pNext = nullptr;
color_blending.pNext = nullptr;
#else
color_blending.pNext = &color_write;
color_blending.pNext = &color_write;
#endif
color_blending.flags = {};
color_blending.logicOpEnable = VK_FALSE;
@@ -838,15 +832,13 @@ void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descripto
EXIT_IF(!vs_input_info.stage);
CreateLayout(descriptor_cache, set_layouts, set_layouts_num, push_constant_info,
push_constant_info_num,
*vs_input_info.stage.program, vk::ShaderStageFlagBits::eVertex,
DescriptorCache::Stage::Vertex);
push_constant_info_num, *vs_input_info.stage.program,
vk::ShaderStageFlagBits::eVertex, DescriptorCache::Stage::Vertex);
if (ps_active) {
EXIT_IF(!ps_input_info->stage);
CreateLayout(descriptor_cache, set_layouts, set_layouts_num, push_constant_info,
push_constant_info_num,
*ps_input_info->stage.program, vk::ShaderStageFlagBits::eFragment,
DescriptorCache::Stage::Pixel);
push_constant_info_num, *ps_input_info->stage.program,
vk::ShaderStageFlagBits::eFragment, DescriptorCache::Stage::Pixel);
}
vk::PipelineLayoutCreateInfo pipeline_layout_info {};
@@ -923,32 +915,32 @@ void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descripto
dynamic_state.dynamicStateCount = dynamic_states_count;
dynamic_state.pDynamicStates = dynamic_states;
vk::GraphicsPipelineCreateInfo pipeline_info {};
vk::GraphicsPipelineCreateInfo pipeline_info {};
vk::PipelineRenderingCreateInfo rendering_info {};
rendering_info.sType = vk::StructureType::ePipelineRenderingCreateInfo;
rendering_info.colorAttachmentCount = rendering.color_count;
rendering_info.pColorAttachmentFormats = rendering.color_formats.data();
rendering_info.depthAttachmentFormat = rendering.depth_format;
rendering_info.stencilAttachmentFormat = rendering.stencil_format;
pipeline_info.sType = vk::StructureType::eGraphicsPipelineCreateInfo;
pipeline_info.pNext = &rendering_info;
pipeline_info.flags = {};
pipeline_info.stageCount = shader_stage_count;
pipeline_info.pStages = shader_stages;
pipeline_info.pVertexInputState = &vertex_input_info;
pipeline_info.pInputAssemblyState = &input_assembly;
pipeline_info.pTessellationState = nullptr;
pipeline_info.pViewportState = &viewport_state;
pipeline_info.pRasterizationState = &rasterizer;
pipeline_info.pMultisampleState = &multisampling;
pipeline_info.pDepthStencilState = (static_params.with_depth ? &depth_stencil_info : nullptr);
pipeline_info.pColorBlendState = &color_blending;
pipeline_info.pDynamicState = &dynamic_state;
pipeline_info.layout = pipeline.pipeline_layout;
pipeline_info.renderPass = nullptr;
pipeline_info.subpass = 0;
pipeline_info.basePipelineHandle = nullptr;
pipeline_info.basePipelineIndex = -1;
pipeline_info.sType = vk::StructureType::eGraphicsPipelineCreateInfo;
pipeline_info.pNext = &rendering_info;
pipeline_info.flags = {};
pipeline_info.stageCount = shader_stage_count;
pipeline_info.pStages = shader_stages;
pipeline_info.pVertexInputState = &vertex_input_info;
pipeline_info.pInputAssemblyState = &input_assembly;
pipeline_info.pTessellationState = nullptr;
pipeline_info.pViewportState = &viewport_state;
pipeline_info.pRasterizationState = &rasterizer;
pipeline_info.pMultisampleState = &multisampling;
pipeline_info.pDepthStencilState = (static_params.with_depth ? &depth_stencil_info : nullptr);
pipeline_info.pColorBlendState = &color_blending;
pipeline_info.pDynamicState = &dynamic_state;
pipeline_info.layout = pipeline.pipeline_layout;
pipeline_info.renderPass = nullptr;
pipeline_info.subpass = 0;
pipeline_info.basePipelineHandle = nullptr;
pipeline_info.basePipelineIndex = -1;
EXIT_IF(pipeline.pipeline != nullptr);
@@ -1012,8 +1004,7 @@ void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descripto
comp_shader_stage_info.pName = "main";
comp_shader_stage_info.pSpecializationInfo = nullptr;
EXIT_IF(!input_info.stage);
ConfigureSubgroupSize(graphics, vk::ShaderStageFlagBits::eCompute,
*input_info.stage.program,
ConfigureSubgroupSize(graphics, vk::ShaderStageFlagBits::eCompute, *input_info.stage.program,
comp_subgroup_size, comp_shader_stage_info);
vk::DescriptorSetLayout set_layouts[1] = {};
@@ -1024,9 +1015,8 @@ void CreatePipelineInternal(GraphicContext& graphics, DescriptorCache& descripto
EXIT_IF(!input_info.stage);
CreateLayout(descriptor_cache, set_layouts, set_layouts_num, push_constant_info,
push_constant_info_num,
*input_info.stage.program, vk::ShaderStageFlagBits::eCompute,
DescriptorCache::Stage::Compute);
push_constant_info_num, *input_info.stage.program,
vk::ShaderStageFlagBits::eCompute, DescriptorCache::Stage::Compute);
vk::PipelineLayoutCreateInfo pipeline_layout_info {};
pipeline_layout_info.sType = vk::StructureType::ePipelineLayoutCreateInfo;
@@ -10,14 +10,14 @@
#include "graphics/guest_gpu/graphicsRun.h"
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/renderer/image/imageInfo.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/pipeline/descriptors.h"
#include "graphics/host_gpu/renderer/image/imageInfo.h"
#include "graphics/host_gpu/renderer/pipeline/pipelineCache.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/host_gpu/renderer/pipeline/shaderResourceBarrier.h"
#include "graphics/host_gpu/renderer/pipeline/shaderSubgroup.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/recompiler/ir/ResourceMaterialization.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
@@ -14,8 +14,7 @@ namespace Libs::Graphics {
RenderContext::RenderContext(GraphicContext& graphics)
: m_graphics(graphics), m_render_executor(*this), m_command_scheduler(*this, graphics),
m_descriptor_cache(graphics), m_pipeline_cache(graphics, m_descriptor_cache),
m_sampler_cache(graphics),
m_gpu_resources(graphics, m_command_scheduler) {
m_sampler_cache(graphics), m_gpu_resources(graphics, m_command_scheduler) {
EXIT_NOT_IMPLEMENTED(!Common::Thread::IsMainThread());
}
@@ -27,7 +26,7 @@ RenderContext::~RenderContext() {
void RenderContext::InitializeGpu(VideoOut::VideoOutDriver* video_out) {
EXIT_IF(m_gpu != nullptr);
m_video_out = video_out;
m_gpu = std::make_unique<Gpu>(*this);
m_gpu = std::make_unique<Gpu>(*this);
m_gpu_resources.SetGpu(m_gpu.get());
}
@@ -99,8 +98,7 @@ void RenderContext::TriggerEopEvent(uint32_t context_id) {
registration.eq, static_cast<uintptr_t>(registration.id),
LibKernel::EventQueue::KERNEL_EVFILT_GRAPHICS,
reinterpret_cast<void*>(static_cast<uintptr_t>(context_id)));
if (result == LibKernel::KERNEL_ERROR_EBADF ||
result == LibKernel::KERNEL_ERROR_ENOENT) {
if (result == LibKernel::KERNEL_ERROR_EBADF || result == LibKernel::KERNEL_ERROR_ENOENT) {
DeleteEopEq(registration.eq, registration.id);
continue;
}
+17 -17
View File
@@ -6,12 +6,12 @@
#include "common/common.h"
#include "common/threads.h"
#include "graphics/host_gpu/renderer/cache/bufferCache.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/cache/gpuResourceManager.h"
#include "graphics/host_gpu/renderer/pipeline/pipelineCache.h"
#include "graphics/host_gpu/renderer/cache/samplerCache.h"
#include "graphics/host_gpu/renderer/cache/textureCache.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
#include "graphics/host_gpu/renderer/pipeline/descriptorCache.h"
#include "graphics/host_gpu/renderer/pipeline/pipelineCache.h"
#include "kernel/eventQueue.h"
#include <memory>
@@ -32,10 +32,10 @@ public:
~RenderContext();
KYTY_CLASS_NO_COPY(RenderContext);
[[nodiscard]] GraphicContext& GetGraphics() const noexcept { return m_graphics; }
void InitializeGpu(VideoOut::VideoOutDriver* video_out);
void ShutdownGpu();
[[nodiscard]] Gpu& GetGpu() const;
[[nodiscard]] GraphicContext& GetGraphics() const noexcept { return m_graphics; }
void InitializeGpu(VideoOut::VideoOutDriver* video_out);
void ShutdownGpu();
[[nodiscard]] Gpu& GetGpu() const;
[[nodiscard]] VideoOut::VideoOutDriver& GetVideoOut() const;
Common::Mutex& GetMutex() { return m_mutex; }
@@ -56,18 +56,18 @@ private:
struct EopEqRegistration {
LibKernel::EventQueue::KernelEqueue eq = LibKernel::EventQueue::KERNEL_EQUEUE_INVALID;
LibKernel::EventQueue::KernelEqueueRef queue;
int id = 0;
int id = 0;
};
GraphicContext& m_graphics;
Common::Mutex m_mutex;
RenderExecutor m_render_executor;
CommandScheduler m_command_scheduler;
DescriptorCache m_descriptor_cache;
PipelineCache m_pipeline_cache;
SamplerCache m_sampler_cache;
GpuResourceManager m_gpu_resources;
std::unique_ptr<Gpu> m_gpu;
GraphicContext& m_graphics;
Common::Mutex m_mutex;
RenderExecutor m_render_executor;
CommandScheduler m_command_scheduler;
DescriptorCache m_descriptor_cache;
PipelineCache m_pipeline_cache;
SamplerCache m_sampler_cache;
GpuResourceManager m_gpu_resources;
std::unique_ptr<Gpu> m_gpu;
VideoOut::VideoOutDriver* m_video_out = nullptr;
Common::Mutex m_eop_mutex;
@@ -12,13 +12,13 @@ namespace Libs::Graphics {
static constexpr uint32_t RENDER_COLOR_ATTACHMENTS_MAX = 8;
struct RenderAttachment {
vk::ImageView image_view = nullptr;
vk::ImageLayout image_layout = vk::ImageLayout::eUndefined;
std::array<uint32_t, 4> clear_value = {};
vk::ImageView image_view = nullptr;
vk::ImageLayout image_layout = vk::ImageLayout::eUndefined;
std::array<uint32_t, 4> clear_value = {};
bool is_clear = false;
bool has_depth = false;
bool depth_clear = false;
bool has_stencil = false;
bool has_depth = false;
bool depth_clear = false;
bool has_stencil = false;
bool stencil_clear = false;
bool operator==(const RenderAttachment&) const = default;
+3 -3
View File
@@ -251,9 +251,9 @@ uint64_t PrepareVideoOutFlip(CommandBuffer& buffer, int handle, int index, int f
int64_t flip_arg) {
for (;;) {
uint64_t request_id = 0;
auto& video_out = buffer.GetContext().GetVideoOut();
const auto result = video_out.SubmitFlipFromGpu(
buffer, handle, index, flip_mode, flip_arg, request_id);
auto& video_out = buffer.GetContext().GetVideoOut();
const auto result =
video_out.SubmitFlipFromGpu(buffer, handle, index, flip_mode, flip_arg, request_id);
if (result == OK) {
EXIT_IF(request_id == 0);
return request_id;
+6 -7
View File
@@ -122,9 +122,9 @@ uint64_t GraphicContext::GetDeviceMemoryUsage() const {
physical_device_properties.deviceType == vk::PhysicalDeviceType::eDiscreteGpu;
uint64_t usage = 0;
for (uint32_t heap = 0; heap < physical_device_memory_properties.memoryHeapCount; heap++) {
const bool device_local = static_cast<bool>(
physical_device_memory_properties.memoryHeaps[heap].flags &
vk::MemoryHeapFlagBits::eDeviceLocal);
const bool device_local =
static_cast<bool>(physical_device_memory_properties.memoryHeaps[heap].flags &
vk::MemoryHeapFlagBits::eDeviceLocal);
if (!discrete || device_local) {
usage += budgets[heap].usage;
}
@@ -144,7 +144,7 @@ uint64_t GraphicContext::GetTotalMemoryBudget() const {
uint64_t local = 0;
uint64_t usage = 0;
for (uint32_t heap = 0; heap < physical_device_memory_properties.memoryHeapCount; heap++) {
const auto& properties = physical_device_memory_properties.memoryHeaps[heap];
const auto& properties = physical_device_memory_properties.memoryHeaps[heap];
const bool device_local =
static_cast<bool>(properties.flags & vk::MemoryHeapFlagBits::eDeviceLocal);
if (device_local) {
@@ -159,9 +159,8 @@ uint64_t GraphicContext::GetTotalMemoryBudget() const {
return budget - std::min<uint64_t>(budget / 8, 1024ull * 1024 * 1024);
}
constexpr uint64_t system_reserve = 8ull * 1024 * 1024 * 1024;
const auto available = budget > usage ? budget - usage : uint64_t {0};
return std::max(local,
available > system_reserve ? available - system_reserve : uint64_t {0});
const auto available = budget > usage ? budget - usage : uint64_t {0};
return std::max(local, available > system_reserve ? available - system_reserve : uint64_t {0});
}
void GraphicContext::CreateBuffer(uint64_t size, VulkanBuffer& buffer) {
+4
View File
@@ -55,6 +55,10 @@ constexpr FormatMapping kFormatMappings[] = {
{Prospero::BufferFormat::k32_32_32_32UInt, vk::Format::eR32G32B32A32Uint},
{Prospero::BufferFormat::k32_32_32_32SInt, vk::Format::eR32G32B32A32Sint},
{Prospero::BufferFormat::k32_32_32_32Float, vk::Format::eR32G32B32A32Sfloat},
// Narrow-channel sRGB formats are optional in Vulkan. Keep a same-width fallback until
// sampler-aware sRGB emulation is available.
{Prospero::BufferFormat::k8Srgb, vk::Format::eR8Unorm},
{Prospero::BufferFormat::k8_8Srgb, vk::Format::eR8G8Unorm},
{Prospero::BufferFormat::k8_8_8_8Srgb, vk::Format::eR8G8B8A8Srgb},
{Prospero::BufferFormat::k9_9_9_5Float, vk::Format::eE5B9G9R9UfloatPack32},
{Prospero::BufferFormat::k5_6_5UNorm, vk::Format::eB5G6R5UnormPack16},
+7 -7
View File
@@ -20,14 +20,14 @@ public:
~Presenter();
KYTY_CLASS_NO_COPY(Presenter);
[[nodiscard]] Frame& PrepareFrame(CommandBuffer& command, const ImageInfo& info);
[[nodiscard]] Frame& PrepareBlankFrame(uint32_t width, uint32_t height, bool opaque,
CommandBuffer* producer = nullptr);
[[nodiscard]] Frame* PrepareLastFrame();
[[nodiscard]] bool IsGuestPaused() const noexcept;
[[nodiscard]] Frame& PrepareFrame(CommandBuffer& command, const ImageInfo& info);
[[nodiscard]] Frame& PrepareBlankFrame(uint32_t width, uint32_t height, bool opaque,
CommandBuffer* producer = nullptr);
[[nodiscard]] Frame* PrepareLastFrame();
[[nodiscard]] bool IsGuestPaused() const noexcept;
[[nodiscard]] RenderContext& Renderer() const noexcept;
void Present(Frame& frame, bool reuse = false);
void Discard(Frame& frame);
void Present(Frame& frame, bool reuse = false);
void Discard(Frame& frame);
private:
struct Impl;
+37 -41
View File
@@ -69,8 +69,8 @@ enum class FlipRequestSource { Cpu, GpuEop };
struct VideoOutEventState;
struct VideoOutEventRegistration {
EventQueue::KernelEqueue handle = EventQueue::KERNEL_EQUEUE_INVALID;
std::shared_ptr<VideoOutEventState> state;
EventQueue::KernelEqueue handle = EventQueue::KERNEL_EQUEUE_INVALID;
std::shared_ptr<VideoOutEventState> state;
uint64_t generation = 0;
VideoOutEventKind kind = VideoOutEventKind::Flip;
};
@@ -170,13 +170,13 @@ struct BufferAttributeGroup {
struct VideoOutConfig {
Common::Mutex mutex;
Common::CondVar vblank_cond;
std::shared_ptr<VideoOutEventState> events = std::make_shared<VideoOutEventState>();
uint32_t width = 0;
uint32_t height = 0;
uint64_t generation = 0;
bool opened = false;
bool closing = false;
int flip_rate = 0;
std::shared_ptr<VideoOutEventState> events = std::make_shared<VideoOutEventState>();
uint32_t width = 0;
uint32_t height = 0;
uint64_t generation = 0;
bool opened = false;
bool closing = false;
int flip_rate = 0;
uint64_t output_mode = VIDEO_OUT_OUTPUT_MODE_DEFAULT;
float gamma = 1.0f;
VideoOutFlipStatus flip_status;
@@ -250,8 +250,8 @@ public:
VideoOutConfig* Get(int handle, uint64_t& generation);
bool IsOpened(int handle);
void Init(uint32_t width, uint32_t height);
FlipQueue& GetFlipQueue() { return m_flip_queue; }
void Init(uint32_t width, uint32_t height);
FlipQueue& GetFlipQueue() { return m_flip_queue; }
Graphics::RenderContext& Renderer() const noexcept { return m_renderer; }
void VblankBegin();
@@ -259,12 +259,12 @@ public:
void PresentThread(std::stop_token token);
private:
Common::Mutex m_mutex;
VideoOutConfig m_video_out_ctx[VIDEO_OUT_NUM_MAX];
Common::Mutex m_mutex;
VideoOutConfig m_video_out_ctx[VIDEO_OUT_NUM_MAX];
Graphics::RenderContext& m_renderer;
Graphics::Presenter& m_presenter;
FlipQueue m_flip_queue;
std::jthread m_present_thread;
Graphics::Presenter& m_presenter;
FlipQueue m_flip_queue;
std::jthread m_present_thread;
};
static std::unique_ptr<VideoOutDriver> g_video_out_driver;
@@ -279,7 +279,7 @@ static uintptr_t VideoOutEventId(VideoOutEventKind kind) {
}
static VideoOutEventQueues& VideoOutEventQueuesFor(VideoOutEventState& state,
VideoOutEventKind kind) {
VideoOutEventKind kind) {
switch (kind) {
case VideoOutEventKind::Flip: return state.flip;
case VideoOutEventKind::Vblank: return state.vblank;
@@ -359,9 +359,9 @@ static void TriggerVideoOutEvents(VideoOutConfig& video_out, VideoOutEventKind k
if (!registration || registration->generation != video_out.generation) {
continue;
}
const auto result = EventQueue::KernelTriggerEvent(
registration->handle, VideoOutEventId(kind), EventQueue::KERNEL_EVFILT_VIDEO_OUT,
trigger_data);
const auto result =
EventQueue::KernelTriggerEvent(registration->handle, VideoOutEventId(kind),
EventQueue::KERNEL_EVFILT_VIDEO_OUT, trigger_data);
EXIT_NOT_IMPLEMENTED(result != OK && result != LibKernel::KERNEL_ERROR_EBADF &&
result != LibKernel::KERNEL_ERROR_ENOENT);
}
@@ -372,9 +372,8 @@ static void DeleteVideoOutEvents(const VideoOutEventQueues& queues, VideoOutEven
if (!registration) {
continue;
}
const auto result =
EventQueue::KernelDeleteEvent(registration->handle, VideoOutEventId(kind),
EventQueue::KERNEL_EVFILT_VIDEO_OUT);
const auto result = EventQueue::KernelDeleteEvent(
registration->handle, VideoOutEventId(kind), EventQueue::KERNEL_EVFILT_VIDEO_OUT);
EXIT_NOT_IMPLEMENTED(result != OK && result != LibKernel::KERNEL_ERROR_EBADF &&
result != LibKernel::KERNEL_ERROR_ENOENT);
}
@@ -383,7 +382,7 @@ static void DeleteVideoOutEvents(const VideoOutEventQueues& queues, VideoOutEven
static int RegisterVideoOutEvent(int handle, EventQueue::KernelEqueue eq, VideoOutEventKind kind,
void* udata) {
uint64_t generation = 0;
auto* video_out = DriverState().Get(handle, generation);
auto* video_out = DriverState().Get(handle, generation);
if (video_out == nullptr) {
return VIDEO_OUT_ERROR_INVALID_HANDLE;
}
@@ -425,27 +424,25 @@ static int RegisterVideoOutEvent(int handle, EventQueue::KernelEqueue eq, VideoO
bool add_queue = false;
{
Common::LockGuard event_lock(event_state->mutex);
const auto existing = std::find_if(queues.begin(), queues.end(), [&](const auto& candidate) {
return candidate->handle == eq && candidate->generation == generation;
});
const auto existing =
std::find_if(queues.begin(), queues.end(), [&](const auto& candidate) {
return candidate->handle == eq && candidate->generation == generation;
});
if (existing != queues.end()) {
registration = *existing;
} else {
registration = std::make_shared<VideoOutEventRegistration>(
VideoOutEventRegistration {.handle = eq,
.state = event_state,
.generation = generation,
.kind = kind});
registration = std::make_shared<VideoOutEventRegistration>(VideoOutEventRegistration {
.handle = eq, .state = event_state, .generation = generation, .kind = kind});
queues.push_back(registration);
add_queue = true;
}
}
event.filter.data = registration.get();
event.filter.owner = registration;
const int result = EventQueue::KernelAddEvent(eq, event);
const int result = EventQueue::KernelAddEvent(eq, event);
if (result != OK && add_queue) {
Common::LockGuard event_lock(event_state->mutex);
const auto added = std::find(queues.begin(), queues.end(), registration);
const auto added = std::find(queues.begin(), queues.end(), registration);
if (added != queues.end()) {
queues.erase(added);
}
@@ -455,7 +452,7 @@ static int RegisterVideoOutEvent(int handle, EventQueue::KernelEqueue eq, VideoO
static int DeleteVideoOutEvent(int handle, EventQueue::KernelEqueue eq, VideoOutEventKind kind) {
uint64_t generation = 0;
auto* video_out = DriverState().Get(handle, generation);
auto* video_out = DriverState().Get(handle, generation);
if (video_out == nullptr) {
return VIDEO_OUT_ERROR_INVALID_HANDLE;
}
@@ -814,8 +811,8 @@ void VideoOutDriver::Impl::PresentThread(std::stop_token token) {
m_presenter.Present(*frame, true);
}
const auto frame_end = Common::Timer::QueryPerformanceCounter();
total_wait += static_cast<int64_t>(period) -
static_cast<int64_t>(frame_end - frame_begin);
total_wait +=
static_cast<int64_t>(period) - static_cast<int64_t>(frame_end - frame_begin);
continue;
}
@@ -841,8 +838,7 @@ void VideoOutDriver::Impl::PresentThread(std::stop_token token) {
VblankEnd();
const auto frame_end = Common::Timer::QueryPerformanceCounter();
total_wait += static_cast<int64_t>(period) -
static_cast<int64_t>(frame_end - frame_begin);
total_wait += static_cast<int64_t>(period) - static_cast<int64_t>(frame_end - frame_begin);
}
}
@@ -1000,8 +996,8 @@ void FlipQueue::Prepare(uint64_t request_id, Graphics::CommandBuffer& buffer) {
}
Graphics::Presenter::Frame* frame = nullptr;
if (special) {
frame = &m_presenter.PrepareBlankFrame(width, height,
index == VIDEO_OUT_BUFFER_INDEX_BLACK, &buffer);
frame = &m_presenter.PrepareBlankFrame(width, height, index == VIDEO_OUT_BUFFER_INDEX_BLACK,
&buffer);
} else {
frame = &m_presenter.PrepareFrame(buffer, source_info);
}
+7 -7
View File
@@ -32,13 +32,13 @@ public:
~VideoOutDriver();
KYTY_CLASS_NO_COPY(VideoOutDriver);
int SubmitFlipFromGpu(Graphics::CommandBuffer& buffer, int handle, int index, int flip_mode,
int64_t flip_arg, uint64_t& request_id);
void PrepareFlip(uint64_t request_id, Graphics::CommandBuffer& buffer);
void CompleteFlip(uint64_t request_id);
void SubmitFlipPreparation(uint64_t request_id);
void WaitForSubmitSlot();
void WaitFlipDone(int handle, int index);
int SubmitFlipFromGpu(Graphics::CommandBuffer& buffer, int handle, int index, int flip_mode,
int64_t flip_arg, uint64_t& request_id);
void PrepareFlip(uint64_t request_id, Graphics::CommandBuffer& buffer);
void CompleteFlip(uint64_t request_id);
void SubmitFlipPreparation(uint64_t request_id);
void WaitForSubmitSlot();
void WaitFlipDone(int handle, int index);
[[nodiscard]] Impl& State() noexcept;
+61 -83
View File
@@ -61,7 +61,7 @@ namespace Libs::Graphics {
struct Presenter::Frame {
VulkanImage image;
std::unique_ptr<CommandBuffer> present_commands;
bool busy = false;
bool busy = false;
bool reusing_last = false;
void Configure(GraphicContext& graphics, vk::Extent2D extent, vk::Format format);
@@ -155,7 +155,7 @@ public:
EXIT("last submitted frame is not available for reuse\n");
}
m_free.erase(free);
m_last_frame = nullptr;
m_last_frame = nullptr;
frame->busy = true;
frame->reusing_last = true;
m_mutex.Unlock();
@@ -197,30 +197,27 @@ private:
}
}
WindowContext& m_window;
Common::Mutex m_mutex;
Common::CondVar m_available;
WindowContext& m_window;
Common::Mutex m_mutex;
Common::CondVar m_available;
std::vector<std::unique_ptr<Presenter::Frame>> m_frames;
std::deque<Presenter::Frame*> m_free;
Presenter::Frame* m_last_frame = nullptr;
vk::Format m_format = vk::Format::eUndefined;
vk::Format m_format = vk::Format::eUndefined;
};
void Presenter::Frame::Configure(GraphicContext& graphics, vk::Extent2D extent,
vk::Format format) {
void Presenter::Frame::Configure(GraphicContext& graphics, vk::Extent2D extent, vk::Format format) {
if (extent.width == 0 || extent.height == 0 || format == vk::Format::eUndefined) {
EXIT("unsupported prepared frame, extent=%ux%u format=%d\n", extent.width, extent.height,
static_cast<int>(format));
}
const auto features = graphics.GetFormatProperties(format).optimalTilingFeatures;
const auto required = vk::FormatFeatureFlagBits::eBlitSrc |
vk::FormatFeatureFlagBits::eSampledImageFilterLinear |
vk::FormatFeatureFlagBits::eTransferSrc |
vk::FormatFeatureFlagBits::eTransferDst;
const auto required =
vk::FormatFeatureFlagBits::eBlitSrc | vk::FormatFeatureFlagBits::eSampledImageFilterLinear |
vk::FormatFeatureFlagBits::eTransferSrc | vk::FormatFeatureFlagBits::eTransferDst;
if ((features & required) != required) {
EXIT("prepared presentation format lacks optimal blit support: format=%d features=0x%x\n",
static_cast<int>(format),
static_cast<vk::FormatFeatureFlags::MaskType>(features));
static_cast<int>(format), static_cast<vk::FormatFeatureFlags::MaskType>(features));
}
auto& dst = image;
@@ -234,11 +231,11 @@ void Presenter::Frame::Configure(GraphicContext& graphics, vk::Extent2D extent,
dst.memory = {};
}
dst.extent = {extent.width, extent.height, 1};
dst.format = format;
dst.layers = 1;
dst.mip_levels = 1;
dst.state = {};
dst.extent = {extent.width, extent.height, 1};
dst.format = format;
dst.layers = 1;
dst.mip_levels = 1;
dst.state = {};
dst.subresource_states.clear();
dst.memory.property = vk::MemoryPropertyFlagBits::eDeviceLocal;
@@ -262,13 +259,12 @@ void Presenter::Frame::Configure(GraphicContext& graphics, vk::Extent2D extent,
void Presenter::Frame::Transit(vk::CommandBuffer command, vk::ImageLayout layout,
vk::AccessFlags2 access) {
const auto stage = access == vk::AccessFlagBits2::eTransferRead ||
access == vk::AccessFlagBits2::eTransferWrite
? vk::PipelineStageFlagBits2::eTransfer
: vk::PipelineStageFlagBits2::eAllCommands;
const auto stage = access == vk::AccessFlagBits2::eTransferRead ||
access == vk::AccessFlagBits2::eTransferWrite
? vk::PipelineStageFlagBits2::eTransfer
: vk::PipelineStageFlagBits2::eAllCommands;
constexpr auto writes = vk::AccessFlagBits2::eTransferWrite |
vk::AccessFlagBits2::eShaderWrite |
vk::AccessFlagBits2::eMemoryWrite;
vk::AccessFlagBits2::eShaderWrite | vk::AccessFlagBits2::eMemoryWrite;
if (image.state.layout == layout && image.state.access_mask == access &&
!static_cast<bool>(image.state.access_mask & writes)) {
return;
@@ -299,35 +295,27 @@ void Presenter::Frame::Transit(vk::CommandBuffer command, vk::ImageLayout layout
void Presenter::Frame::CopyFrom(CommandBuffer& command_buffer, Image& source) {
command_buffer.EndRendering();
auto command = command_buffer.Handle();
source.Transit(vk::ImageLayout::eTransferSrcOptimal,
vk::AccessFlagBits2::eTransferRead, {}, command);
Transit(command, vk::ImageLayout::eTransferDstOptimal,
vk::AccessFlagBits2::eTransferWrite);
source.Transit(vk::ImageLayout::eTransferSrcOptimal, vk::AccessFlagBits2::eTransferRead, {},
command);
Transit(command, vk::ImageLayout::eTransferDstOptimal, vk::AccessFlagBits2::eTransferWrite);
vk::ImageCopy copy {};
copy.srcSubresource = {vk::ImageAspectFlagBits::eColor, 0, 0,
source.backing.layers};
copy.srcSubresource = {vk::ImageAspectFlagBits::eColor, 0, 0, source.backing.layers};
copy.dstSubresource = {vk::ImageAspectFlagBits::eColor, 0, 0, image.layers};
copy.extent = {std::min(source.backing.extent.width, image.extent.width),
std::min(source.backing.extent.height, image.extent.height), 1};
copy.extent = {std::min(source.backing.extent.width, image.extent.width),
std::min(source.backing.extent.height, image.extent.height), 1};
EXIT_IF(copy.srcSubresource.layerCount != copy.dstSubresource.layerCount);
command.copyImage(source.backing.image, vk::ImageLayout::eTransferSrcOptimal,
image.image, vk::ImageLayout::eTransferDstOptimal, copy);
Transit(command, vk::ImageLayout::eTransferSrcOptimal,
vk::AccessFlagBits2::eTransferRead);
command.copyImage(source.backing.image, vk::ImageLayout::eTransferSrcOptimal, image.image,
vk::ImageLayout::eTransferDstOptimal, copy);
Transit(command, vk::ImageLayout::eTransferSrcOptimal, vk::AccessFlagBits2::eTransferRead);
}
void Presenter::Frame::Clear(CommandBuffer& command_buffer,
const vk::ClearColorValue& color) {
void Presenter::Frame::Clear(CommandBuffer& command_buffer, const vk::ClearColorValue& color) {
command_buffer.EndRendering();
auto command = command_buffer.Handle();
Transit(command, vk::ImageLayout::eTransferDstOptimal,
vk::AccessFlagBits2::eTransferWrite);
const vk::ImageSubresourceRange range {
vk::ImageAspectFlagBits::eColor, 0, 1, 0, 1};
command.clearColorImage(image.image, vk::ImageLayout::eTransferDstOptimal, &color, 1,
&range);
Transit(command, vk::ImageLayout::eTransferSrcOptimal,
vk::AccessFlagBits2::eTransferRead);
Transit(command, vk::ImageLayout::eTransferDstOptimal, vk::AccessFlagBits2::eTransferWrite);
const vk::ImageSubresourceRange range {vk::ImageAspectFlagBits::eColor, 0, 1, 0, 1};
command.clearColorImage(image.image, vk::ImageLayout::eTransferDstOptimal, &color, 1, &range);
Transit(command, vk::ImageLayout::eTransferSrcOptimal, vk::AccessFlagBits2::eTransferRead);
}
class Swapchain final {
@@ -338,8 +326,8 @@ public:
~Swapchain();
KYTY_CLASS_NO_COPY(Swapchain);
void Create();
void Recreate(bool surface_lost = false);
void Create();
void Recreate(bool surface_lost = false);
[[nodiscard]] Status AcquireNextImage();
void RecordPresentCommands(CommandBuffer& command, VulkanImage& source);
void Submit(CommandBuffer& command);
@@ -395,17 +383,17 @@ struct Presenter::Impl {
desc.view_info.usage = vk::ImageUsageFlagBits::eTransferSrc;
desc.type = TextureCache::BindingType::VideoOut;
auto& cache = renderer.GetTextureCache();
auto& image = cache.GetImage(cache.FindImage(desc));
auto& cache = renderer.GetTextureCache();
auto& image = cache.GetImage(cache.FindImage(desc));
image.usage.video_out = true;
return image;
}
RenderContext& renderer;
WindowContext& window;
Swapchain swapchain;
RenderContext& renderer;
WindowContext& window;
Swapchain swapchain;
CommandScheduler present_scheduler;
FramePool frames;
FramePool frames;
};
void Swapchain::Create() {
@@ -441,25 +429,20 @@ void Swapchain::Create() {
? vk::CompositeAlphaFlagBitsKHR::eOpaque
: vk::CompositeAlphaFlagBitsKHR::eInherit;
vk::SurfaceFormatKHR format {vk::Format::eR8G8B8A8Unorm,
vk::ColorSpaceKHR::eSrgbNonlinear};
if (surface.formats.size() != 1 ||
surface.formats.front().format != vk::Format::eUndefined) {
vk::SurfaceFormatKHR format {vk::Format::eR8G8B8A8Unorm, vk::ColorSpaceKHR::eSrgbNonlinear};
if (surface.formats.size() != 1 || surface.formats.front().format != vk::Format::eUndefined) {
const auto it = std::find_if(surface.formats.begin(), surface.formats.end(),
[](const vk::SurfaceFormatKHR& candidate) {
return candidate.format ==
vk::Format::eB8G8R8A8Unorm ||
candidate.format ==
vk::Format::eR8G8B8A8Unorm;
return candidate.format == vk::Format::eB8G8R8A8Unorm ||
candidate.format == vk::Format::eR8G8B8A8Unorm;
});
if (it == surface.formats.end()) {
EXIT("no supported UNORM swapchain format\n");
}
format = *it;
}
m_format = format.format;
const auto swapchain_features =
graphics.GetFormatProperties(m_format).optimalTilingFeatures;
m_format = format.format;
const auto swapchain_features = graphics.GetFormatProperties(m_format).optimalTilingFeatures;
if (!static_cast<bool>(swapchain_features & vk::FormatFeatureFlagBits::eBlitDst)) {
EXIT("swapchain format cannot be a blit destination: format=%d\n",
static_cast<int>(m_format));
@@ -503,9 +486,8 @@ void Swapchain::Create() {
view.subresourceRange.baseMipLevel = 0;
view.subresourceRange.layerCount = 1;
view.subresourceRange.levelCount = 1;
RequireVulkanSuccess(
graphics.device.createImageView(&view, nullptr, &m_image_views[i]),
"vkCreateImageView");
RequireVulkanSuccess(graphics.device.createImageView(&view, nullptr, &m_image_views[i]),
"vkCreateImageView");
EXIT_IF(m_image_views[i] == nullptr);
}
@@ -600,7 +582,7 @@ void Swapchain::Recreate(bool surface_lost) {
Swapchain::Status Swapchain::AcquireNextImage() {
EXIT_IF(m_handle == nullptr || m_frame_index >= m_image_acquired.size());
m_image_index = static_cast<uint32_t>(-1);
m_image_index = static_cast<uint32_t>(-1);
const auto result = m_window.graphic_ctx.device.acquireNextImageKHR(
m_handle, std::numeric_limits<uint64_t>::max(), m_image_acquired[m_frame_index], nullptr,
&m_image_index);
@@ -683,10 +665,9 @@ void Swapchain::RecordPresentCommands(CommandBuffer& command, VulkanImage& sourc
to_present.subresourceRange.levelCount = 1;
to_present.subresourceRange.baseArrayLayer = 0;
to_present.subresourceRange.layerCount = 1;
vk_command.pipelineBarrier(vk::PipelineStageFlagBits::eAllCommands,
vk::PipelineStageFlagBits::eAllCommands,
vk::DependencyFlagBits::eByRegion, 0,
nullptr, 0, nullptr, 1, &to_present);
vk_command.pipelineBarrier(
vk::PipelineStageFlagBits::eAllCommands, vk::PipelineStageFlagBits::eAllCommands,
vk::DependencyFlagBits::eByRegion, 0, nullptr, 0, nullptr, 1, &to_present);
command.End();
}
@@ -700,7 +681,7 @@ void Swapchain::Submit(CommandBuffer& command) {
Swapchain::Status Swapchain::Present() {
EXIT_IF(m_image_index >= m_render_complete.size());
const auto ready = m_render_complete[m_image_index];
const auto ready = m_render_complete[m_image_index];
vk::PresentInfoKHR present {};
present.sType = vk::StructureType::ePresentInfoKHR;
present.swapchainCount = 1;
@@ -738,7 +719,7 @@ Presenter::~Presenter() = default;
Presenter::Frame& Presenter::PrepareFrame(CommandBuffer& buffer, const ImageInfo& info) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(buffer.IsInvalid());
auto* frame = m_impl->frames.Acquire();
auto* frame = m_impl->frames.Acquire();
Common::LockGuard render_lock(m_impl->renderer.GetMutex());
auto& image = m_impl->ResolveSurface(info);
if (image.backing.format == vk::Format::eUndefined) {
@@ -752,14 +733,13 @@ Presenter::Frame& Presenter::PrepareFrame(CommandBuffer& buffer, const ImageInfo
default: break;
}
frame->Configure(m_impl->window.graphic_ctx,
{image.backing.extent.width, image.backing.extent.height},
frame_format);
{image.backing.extent.width, image.backing.extent.height}, frame_format);
frame->CopyFrom(buffer, image);
return *frame;
}
Presenter::Frame& Presenter::PrepareBlankFrame(uint32_t width, uint32_t height, bool opaque,
CommandBuffer* producer) {
CommandBuffer* producer) {
KYTY_PROFILER_FUNCTION();
auto format = m_impl->frames.GetFormat();
auto* frame = m_impl->frames.Acquire();
@@ -772,8 +752,7 @@ Presenter::Frame& Presenter::PrepareBlankFrame(uint32_t width, uint32_t height,
frame->Clear(*producer, clear);
} else {
if (frame->present_commands == nullptr) {
frame->present_commands =
std::make_unique<CommandBuffer>(m_impl->present_scheduler);
frame->present_commands = std::make_unique<CommandBuffer>(m_impl->present_scheduler);
}
auto& command = *frame->present_commands;
command.WaitForFenceAndReset();
@@ -830,8 +809,7 @@ void Presenter::Present(Frame& frame, bool reuse) {
continue;
}
if (frame.present_commands == nullptr) {
frame.present_commands =
std::make_unique<CommandBuffer>(m_impl->present_scheduler);
frame.present_commands = std::make_unique<CommandBuffer>(m_impl->present_scheduler);
}
{
Common::LockGuard render_lock(m_impl->renderer.GetMutex());
@@ -32,11 +32,11 @@
#include "graphics/host_gpu/vma.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/presentation/presenter.h"
#include "kernel/memory.h"
#include "graphics/presentation/renderDoc.h"
#include "graphics/presentation/videoOut.h"
#include "graphics/presentation/window.h"
#include "graphics/presentation/window/windowInternal.h"
#include "kernel/memory.h"
#include "libs/controller.h"
#include "loader/systemContent.h"
@@ -475,9 +475,9 @@ static void VulkanInitSubgroupSizeControl(vk::PhysicalDevice physical_device,
}
static vk::Device VulkanCreateDevice(vk::PhysicalDevice physical_device, const VulkanExtensions& r,
uint32_t queue_family,
uint32_t queue_family,
const std::vector<const char*>& device_extensions,
GraphicContext& graphics) {
GraphicContext& graphics) {
EXIT_IF(physical_device == nullptr);
EXIT_IF(queue_family == static_cast<uint32_t>(-1));
@@ -551,19 +551,19 @@ static vk::Device VulkanCreateDevice(vk::PhysicalDevice physical_device, const V
features12.timelineSemaphore = VK_TRUE;
vk::PhysicalDeviceFeatures device_features {};
device_features.fragmentStoresAndAtomics = VK_TRUE;
device_features.samplerAnisotropy = VK_TRUE;
device_features.robustBufferAccess = VK_TRUE;
device_features.fragmentStoresAndAtomics = VK_TRUE;
device_features.samplerAnisotropy = VK_TRUE;
device_features.robustBufferAccess = VK_TRUE;
#if !defined(__APPLE__)
device_features.depthBounds = VK_TRUE; // unsupported by MoltenVK
device_features.depthBounds = VK_TRUE; // unsupported by MoltenVK
#endif
device_features.shaderStorageImageWriteWithoutFormat = VK_TRUE;
device_features.shaderStorageImageReadWithoutFormat = VK_TRUE;
device_features.shaderImageGatherExtended = VK_TRUE;
device_features.independentBlend = VK_TRUE;
device_features.tessellationShader = VK_TRUE;
device_features.sampleRateShading = VK_TRUE;
graphics.sample_rate_shading_enabled = true;
device_features.sampleRateShading = VK_TRUE;
graphics.sample_rate_shading_enabled = true;
device_features.vertexPipelineStoresAndAtomics =
supported_features2.features.vertexPipelineStoresAndAtomics;
@@ -909,10 +909,9 @@ void WindowContext::CreateVulkan() {
}
surface = native_surface;
std::vector<const char*> device_extensions = {VK_KHR_SWAPCHAIN_EXTENSION_NAME,
VK_EXT_DEPTH_CLIP_CONTROL_EXTENSION_NAME,
VK_KHR_PUSH_DESCRIPTOR_EXTENSION_NAME,
"VK_KHR_maintenance1"};
std::vector<const char*> device_extensions = {
VK_KHR_SWAPCHAIN_EXTENSION_NAME, VK_EXT_DEPTH_CLIP_CONTROL_EXTENSION_NAME,
VK_KHR_PUSH_DESCRIPTOR_EXTENSION_NAME, "VK_KHR_maintenance1"};
#if defined(__APPLE__)
// MoltenVK lacks VK_EXT_depth_clip_enable and VK_EXT_color_write_enable; the renderer
@@ -932,8 +931,8 @@ void WindowContext::CreateVulkan() {
uint32_t queue_family = static_cast<uint32_t>(-1);
VulkanFindPhysicalDevice(graphic_ctx.instance, surface, device_extensions,
surface_capabilities, graphic_ctx.physical_device, queue_family);
VulkanFindPhysicalDevice(graphic_ctx.instance, surface, device_extensions, surface_capabilities,
graphic_ctx.physical_device, queue_family);
if (graphic_ctx.physical_device == nullptr) {
EXIT("Could not find suitable device");
@@ -949,9 +948,8 @@ void WindowContext::CreateVulkan() {
auto available_extensions = EnumerateVulkan<vk::ExtensionProperties>(
"vkEnumerateDeviceExtensionProperties",
[&](uint32_t* count, vk::ExtensionProperties* values) {
return graphic_ctx.physical_device.enumerateDeviceExtensionProperties(nullptr,
count,
values);
return graphic_ctx.physical_device.enumerateDeviceExtensionProperties(
nullptr, count, values);
});
if (HasExtension(available_extensions, VK_EXT_MEMORY_BUDGET_EXTENSION_NAME)) {
@@ -985,7 +983,7 @@ void WindowContext::CreateVulkan() {
render_context = std::make_unique<RenderContext>(graphic_ctx);
LibKernel::Memory::InstallGpuResources(&render_context->GetGpuResources());
presenter = std::make_unique<Presenter>(*this);
presenter = std::make_unique<Presenter>(*this);
RenderDocSetActiveWindow(graphic_ctx.instance, window);
}
+23 -25
View File
@@ -1,7 +1,5 @@
#include "graphics/presentation/window.h"
#include <cstdlib>
#include "SDL.h"
#include "SDL_error.h"
#include "SDL_events.h"
@@ -40,6 +38,7 @@
#include <algorithm>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <memory>
#include <string>
@@ -59,7 +58,7 @@
namespace Libs::Graphics {
constexpr int KEYBOARD_CONTROLLER_ID = -1000;
constexpr int KEYBOARD_CONTROLLER_ID = -1000;
struct EventKeyboard {
bool down;
@@ -251,9 +250,7 @@ static void GameEventKeyboard(WindowLoopState& game, const EventKeyboard& key) {
if (key.down) {
switch (key.key_code) {
case SDLK_ESCAPE: game.need_exit = true; break;
case SDLK_SPACE:
SetPause(game, !game.paused.load(std::memory_order_acquire));
break;
case SDLK_SPACE: SetPause(game, !game.paused.load(std::memory_order_acquire)); break;
case SDLK_F1:
if (!key.repeat) {
RenderDocRequestCapture();
@@ -390,7 +387,9 @@ void WindowContext::Resize(uint32_t new_width, uint32_t new_height) {
void WindowContext::ProcessWindowEvent(const SDL_WindowEvent& event) {
const auto& window_event = event;
switch (window_event.event) {
case SDL_WINDOWEVENT_SHOWN: LOGF("Window %" PRIu32 " shown\n", window_event.windowID); break;
case SDL_WINDOWEVENT_SHOWN:
LOGF("Window %" PRIu32 " shown\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_HIDDEN:
LOGF("Window %" PRIu32 " hidden\n", window_event.windowID);
@@ -401,13 +400,13 @@ void WindowContext::ProcessWindowEvent(const SDL_WindowEvent& event) {
break;
case SDL_WINDOWEVENT_MOVED:
LOGF("Window %" PRIu32 " moved to %" PRId32 ",%" PRId32 "\n",
window_event.windowID, window_event.data1, window_event.data2);
LOGF("Window %" PRIu32 " moved to %" PRId32 ",%" PRId32 "\n", window_event.windowID,
window_event.data1, window_event.data2);
break;
case SDL_WINDOWEVENT_RESIZED:
LOGF("Window %" PRIu32 " resized to %" PRId32 "x%" PRId32 "\n",
window_event.windowID, window_event.data1, window_event.data2);
LOGF("Window %" PRIu32 " resized to %" PRId32 "x%" PRId32 "\n", window_event.windowID,
window_event.data1, window_event.data2);
LOGF("m: %d\n", static_cast<int>(SDL_ThreadID()));
Resize(window_event.data1, window_event.data2);
@@ -807,9 +806,8 @@ static void WindowCreate(WindowContext& context) {
window_flags |= static_cast<uint32_t>(SDL_WINDOW_BORDERLESS);
}
#endif
context.window =
SDL_CreateWindow(KYTY_SDL_WINDOW_CAPTION, KYTY_SDL_WINDOWPOS_CENTERED,
KYTY_SDL_WINDOWPOS_CENTERED, width, height, window_flags);
context.window = SDL_CreateWindow(KYTY_SDL_WINDOW_CAPTION, KYTY_SDL_WINDOWPOS_CENTERED,
KYTY_SDL_WINDOWPOS_CENTERED, width, height, window_flags);
context.window_hidden = true;
@@ -832,7 +830,7 @@ Presenter& WindowInit(uint32_t width, uint32_t height) {
WindowCreate(*window);
window->CreateVulkan();
auto& presenter = *window->presenter;
g_window = std::move(window);
g_window = std::move(window);
return presenter;
}
@@ -934,9 +932,9 @@ void WindowContext::UpdateTitle() {
Loader::SystemContentParamSfoGetString("TITLE_ID", title_id, sizeof(title_id));
static bool has_app_ver =
Loader::SystemContentParamSfoGetString("APP_VER", app_ver, sizeof(app_ver));
static uint64_t fps_start = Common::Timer::QueryPerformanceCounter();
static uint64_t frame_num = 0;
static uint64_t fps_frames = 0;
static uint64_t fps_start = Common::Timer::QueryPerformanceCounter();
static uint64_t frame_num = 0;
static uint64_t fps_frames = 0;
static double current_fps = 0.0;
const auto now = Common::Timer::QueryPerformanceCounter();
@@ -946,15 +944,15 @@ void WindowContext::UpdateTitle() {
if (now - fps_start >= frequency) {
current_fps = static_cast<double>(fps_frames) * static_cast<double>(frequency) /
static_cast<double>(now - fps_start);
fps_start = now;
fps_frames = 0;
fps_start = now;
fps_frames = 0;
}
auto fps = fmt::format("{}{}{}{}{}{}[{}] [{}], frame: {}, fps: {:f}", (has_title ? title : ""),
(has_title ? ", " : ""), (has_title_id ? title_id : ""),
(has_title_id ? ", " : ""), (has_app_ver ? app_ver : ""),
(has_app_ver ? " " : ""), device_name, processor_name,
frame_num, current_fps);
auto fps =
fmt::format("{}{}{}{}{}{}[{}] [{}], frame: {}, fps: {:f}", (has_title ? title : ""),
(has_title ? ", " : ""), (has_title_id ? title_id : ""),
(has_title_id ? ", " : ""), (has_app_ver ? app_ver : ""),
(has_app_ver ? " " : ""), device_name, processor_name, frame_num, current_fps);
#if defined(__APPLE__)
// AppKit traps on title changes off the main thread; fire-and-forget keeps present pacing.
@@ -28,8 +28,8 @@ struct SurfaceCapabilities {
};
struct WindowLoopState {
SDL_Event event {};
bool need_exit = false;
SDL_Event event {};
bool need_exit = false;
std::atomic_bool paused = false;
};
@@ -38,14 +38,13 @@ struct WindowContext {
~WindowContext();
KYTY_CLASS_NO_COPY(WindowContext);
[[nodiscard]] static vk::PhysicalDeviceVulkan13Features
RequiredVulkan13Features() noexcept;
void CreateVulkan();
void RecreateSurface();
void RefreshSurfaceCapabilities();
void UpdateIcon();
void UpdateTitle();
void Resize(uint32_t width, uint32_t height);
[[nodiscard]] static vk::PhysicalDeviceVulkan13Features RequiredVulkan13Features() noexcept;
void CreateVulkan();
void RecreateSurface();
void RefreshSurfaceCapabilities();
void UpdateIcon();
void UpdateTitle();
void Resize(uint32_t width, uint32_t height);
void ProcessWindowEvent(const SDL_WindowEvent& event);
void ProcessDisplayEvent(const SDL_DisplayEvent& event);
void ProcessEvent(double time_seconds);
@@ -59,14 +58,14 @@ struct WindowContext {
void DrainMainThreadTasks();
#endif
GraphicContext graphic_ctx;
SDL_Window* window = nullptr;
bool window_hidden = true;
vk::SurfaceKHR surface = nullptr;
SurfaceCapabilities surface_capabilities;
GraphicContext graphic_ctx;
SDL_Window* window = nullptr;
bool window_hidden = true;
vk::SurfaceKHR surface = nullptr;
SurfaceCapabilities surface_capabilities;
std::unique_ptr<RenderContext> render_context;
std::unique_ptr<Presenter> presenter;
WindowLoopState loop;
std::unique_ptr<Presenter> presenter;
WindowLoopState loop;
char device_name[VK_MAX_PHYSICAL_DEVICE_NAME_SIZE] = {0};
char processor_name[64] = {0};
@@ -76,7 +75,7 @@ struct WindowContext {
#if defined(__APPLE__)
Common::Mutex main_task_mutex;
Common::CondVar main_task_done;
std::vector<std::function<void()>> main_tasks; // guarded by main_task_mutex
std::vector<std::function<void()>> main_tasks; // guarded by main_task_mutex
uint64_t main_tasks_queued = 0; // guarded by main_task_mutex
uint64_t main_tasks_run = 0; // guarded by main_task_mutex
#endif
@@ -2,15 +2,16 @@
#include "common/assert.h"
#include "common/logging/log.h"
#include "graphics/shader/recompiler/cfg/ShaderCFG.h"
#include "graphics/shader/recompiler/decompiler/ShaderDecoder.h"
#include "graphics/shader/recompiler/emitter/SpirvEmitter.h"
#include "graphics/shader/recompiler/ir/BindingLayout.h"
#include "graphics/shader/recompiler/ir/ReadLaneElimination.h"
#include "graphics/shader/recompiler/ir/ResourceMaterialization.h"
#include "graphics/shader/recompiler/ir/ResourceTracking.h"
#include "graphics/shader/recompiler/ir/ScalarProvenance.h"
#include "graphics/shader/recompiler/cfg/ShaderCFG.h"
#include "graphics/shader/recompiler/decompiler/ShaderDecoder.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/ir/ShaderInfoCollection.h"
#include "graphics/shader/recompiler/emitter/SpirvEmitter.h"
#include "graphics/shader/recompiler/ir/SrtPatcher.h"
#include "graphics/shader/recompiler/ir/SrtWalker.h"
@@ -838,6 +839,11 @@ bool TryRecompile(std::span<const uint32_t> code, const CompileOptions& options,
if (!IR::AllocateBindings(ir, layout_options, error)) {
return false;
}
const auto read_lane_stats = IR::EliminateReadLane(ir);
if (read_lane_stats.rewritten_reads != 0) {
LOGF("%s read-lane elimination: reads=%" PRIu32 " shadow_writes=%" PRIu32 "\n",
GetDumpLabel(options), read_lane_stats.rewritten_reads, read_lane_stats.shadow_writes);
}
std::string ir_dump;
if (options.dump_ir) {
ir_dump = MakeIrDump(cfg, ir);
@@ -44,7 +44,7 @@ struct CompileResult {
};
bool TryRecompile(std::span<const uint32_t> code, const CompileOptions& options,
CompileResult& result, std::string* error);
CompileResult& result, std::string* error);
} // namespace Libs::Graphics::ShaderRecompiler
@@ -35,9 +35,9 @@ constexpr ImageDimension DecodeImageDimension(uint32_t dim) {
case 2u: return ImageDimension::Dim3D;
case 3u: return ImageDimension::Dim2DArray;
case 4u: return ImageDimension::Dim1DArray;
case 5u:
case 7u: return ImageDimension::Dim2DArray;
case 6u: return ImageDimension::Dim2D;
case 5u: return ImageDimension::Dim2DArray;
case 6u: return ImageDimension::Dim2DMsaa;
case 7u: return ImageDimension::Dim2DMsaaArray;
default: return ImageDimension::Unknown;
}
}
@@ -46,8 +46,10 @@ constexpr uint32_t ImageCoordComponents(ImageDimension dimension) {
switch (dimension) {
case ImageDimension::Dim1D: return 1u;
case ImageDimension::Dim1DArray: return 2u;
case ImageDimension::Dim2DMsaa:
case ImageDimension::Dim3D:
case ImageDimension::Dim2DArray: return 3u;
case ImageDimension::Dim2DMsaaArray: return 4u;
default: return 2u;
}
}
@@ -196,10 +196,10 @@ bool DecodeSopk(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index
case Opcode::SMovkI32: return DecodeScalarDestination(sdst, pc, inst.dst, error);
case Opcode::SWaitcnt: {
const uint32_t waitcnt = word & 0xffffu;
inst.dst.kind = OperandKind::Null;
inst.src0.signed_val = static_cast<int32_t>(waitcnt);
inst.src0.value = waitcnt;
inst.src_count = 1;
inst.dst.kind = OperandKind::Null;
inst.src0.signed_val = static_cast<int32_t>(waitcnt);
inst.src0.value = waitcnt;
inst.src_count = 1;
return true;
}
case Opcode::SSetregB32:
@@ -266,10 +266,10 @@ bool DecodeSopp(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index
inst.src0.value = simm;
inst.src0.signed_val = static_cast<int16_t>(simm);
inst.src_count = (inst.opcode == Opcode::SNop || inst.opcode == Opcode::SWaitcnt ||
inst.opcode == Opcode::SSleep || inst.opcode == Opcode::SSendmsg ||
inst.opcode == Opcode::STtraceData || inst.opcode == Opcode::SInstPrefetch)
? 1
: 0;
inst.opcode == Opcode::SSleep || inst.opcode == Opcode::SSendmsg ||
inst.opcode == Opcode::STtraceData || inst.opcode == Opcode::SInstPrefetch)
? 1
: 0;
inst.branch_offset = static_cast<int32_t>(static_cast<int16_t>(simm)) * 4;
inst.branch_target = pc + 4u + static_cast<uint32_t>(inst.branch_offset);
SetRawWords(inst, code, word_index, 1);
@@ -194,6 +194,8 @@ const char* ImageDimensionToString(ImageDimension dimension) {
case ImageDimension::Dim2D: return "2d";
case ImageDimension::Dim3D: return "3d";
case ImageDimension::Dim2DArray: return "2d_array";
case ImageDimension::Dim2DMsaa: return "2d_msaa";
case ImageDimension::Dim2DMsaaArray: return "2d_msaa_array";
default: return "unknown";
}
}
@@ -220,9 +222,9 @@ bool DecodeScalarSource(uint32_t code, uint32_t pc, Operand& operand, std::strin
}
if (code >= 240u && code <= 247u) {
constexpr float values[] = {0.5f, -0.5f, 1.0f, -1.0f, 2.0f, -2.0f, 4.0f, -4.0f};
operand.kind = OperandKind::FloatInlineConstant;
operand.float_val = values[code - 240u];
operand.value = FloatBits(operand.float_val);
operand.kind = OperandKind::FloatInlineConstant;
operand.float_val = values[code - 240u];
operand.value = FloatBits(operand.float_val);
return true;
}
if (code >= 256u && code <= 511u) {
@@ -285,7 +287,7 @@ bool DecodeVectorGpr(uint32_t reg, Operand& operand, std::string* error) {
SetError(error, "VGPR index is out of range");
return false;
}
operand = {};
operand = {};
operand.kind = OperandKind::Vgpr;
operand.reg = reg;
return true;
@@ -575,6 +575,8 @@ enum class ImageDimension : uint32_t {
Dim2D,
Dim3D,
Dim2DArray,
Dim2DMsaa,
Dim2DMsaaArray,
};
constexpr uint32_t MaxInstructionRawWords = 5u;
@@ -30,10 +30,16 @@ bool ImageBinding(const IR::ImageResource& image, IR::DescriptorBindingKind& kin
kind = integer ? Kind::SampledUint1DArray : Kind::Sampled1DArray;
return true;
case Dim::Dim2D: kind = integer ? Kind::SampledUint2D : Kind::Sampled2D; return true;
case Dim::Dim2DMsaa:
kind = integer ? Kind::SampledUint2DMsaa : Kind::Sampled2DMsaa;
return true;
case Dim::Dim3D: kind = integer ? Kind::SampledUint3D : Kind::Sampled3D; return true;
case Dim::Dim2DArray:
kind = integer ? Kind::SampledUint2DArray : Kind::Sampled2DArray;
return true;
case Dim::Dim2DMsaaArray:
kind = integer ? Kind::SampledUint2DMsaaArray : Kind::Sampled2DMsaaArray;
return true;
case Dim::Unknown: return false;
}
}
@@ -51,6 +57,8 @@ bool ImageBinding(const IR::ImageResource& image, IR::DescriptorBindingKind& kin
case Dim::Dim2DArray:
kind = uint_image ? Kind::StorageUint2DArray : Kind::Storage2DArray;
return true;
case Dim::Dim2DMsaa:
case Dim::Dim2DMsaaArray: return false;
case Dim::Unknown: return false;
}
return false;
@@ -12,9 +12,9 @@ namespace Libs::Graphics::ShaderRecompiler::Spirv {
bool ProgramRequiresExactSubgroupSize(const IR::Program& program);
bool EmitProgram(const IR::Program& program, const IR::ResourceSnapshot& resources,
const ShaderVertexInputInfo* vertex_input_info,
const ShaderPixelInputInfo* pixel_input_info,
const ShaderComputeInputInfo* compute_input_info, std::vector<uint32_t>& spirv,
const ShaderVertexInputInfo* vertex_input_info,
const ShaderPixelInputInfo* pixel_input_info,
const ShaderComputeInputInfo* compute_input_info, std::vector<uint32_t>& spirv,
std::string* error);
} // namespace Libs::Graphics::ShaderRecompiler::Spirv
@@ -129,7 +129,7 @@ uint32_t MaxCollectedVectorRegisterEnd(const std::vector<RegisterBinding>& regis
}
void CollectMoveRelSourceRegisters(const IR::Program& program,
std::vector<RegisterBinding>& registers) {
std::vector<RegisterBinding>& registers) {
const auto max_vector_end = MaxCollectedVectorRegisterEnd(registers);
for (const auto& block: program.blocks) {
for (const auto& inst: block.instructions) {
@@ -276,8 +276,7 @@ void CopyProgramInputsAndOutputs(EmitterState& state, const IR::Program& program
if (HasOutput(state.outputs, output.kind, output.index)) {
continue;
}
state.outputs.push_back(
{output.kind, output.index, output.location, 0, output.debug_name});
state.outputs.push_back({output.kind, output.index, output.location, 0, output.debug_name});
}
}
@@ -576,6 +575,8 @@ ImageViewKind ImageViewKindFromDimension(Decoder::ImageDimension dimension) {
case Decoder::ImageDimension::Dim1DArray: return ImageViewKind::Dim1DArray;
case Decoder::ImageDimension::Dim2DArray: return ImageViewKind::Dim2DArray;
case Decoder::ImageDimension::Dim3D: return ImageViewKind::Dim3D;
case Decoder::ImageDimension::Dim2DMsaa: return ImageViewKind::Dim2DMsaa;
case Decoder::ImageDimension::Dim2DMsaaArray: return ImageViewKind::Dim2DMsaaArray;
default: return ImageViewKind::Dim2D;
}
}
@@ -601,7 +602,9 @@ uint32_t ImageViewCoordinateComponents(ImageViewKind view) {
case ImageViewKind::Dim1DArray:
case ImageViewKind::Dim2D: return 2u;
case ImageViewKind::Dim2DArray:
case ImageViewKind::Dim2DMsaaArray:
case ImageViewKind::Dim3D: return 3u;
case ImageViewKind::Dim2DMsaa: return 2u;
default: return 0u;
}
}
@@ -611,7 +614,9 @@ uint32_t ImageViewSpatialComponents(ImageViewKind view) {
case ImageViewKind::Dim1D:
case ImageViewKind::Dim1DArray: return 1u;
case ImageViewKind::Dim2D:
case ImageViewKind::Dim2DArray: return 2u;
case ImageViewKind::Dim2DArray:
case ImageViewKind::Dim2DMsaa:
case ImageViewKind::Dim2DMsaaArray: return 2u;
case ImageViewKind::Dim3D: return 3u;
default: return 0u;
}
@@ -663,8 +668,7 @@ uint32_t LoadSampledImageDescriptor(EmitterState& state, const IR::MemoryInfo& m
uint32_t LoadSamplerDescriptor(EmitterState& state, uint32_t sampler, uint32_t use_pc) {
(void)use_pc;
const auto binding =
ResourceForDescriptor(state, IR::DescriptorBindingKind::Samplers, sampler);
const auto binding = ResourceForDescriptor(state, IR::DescriptorBindingKind::Samplers, sampler);
const auto pointer = DescriptorElementPointer(
state, state.ptr_uniform_sampler, state.sampler_variable, binding.array_index,
IR::DescriptorBindingKind::Samplers, sampler, "sampler descriptor array was not emitted");
@@ -1025,15 +1025,47 @@ void EmitDispatcherSwitch(EmitterState& state, const IR::Program& program) {
EmitDispatcherExit(state);
}
size_t BufferLoadGroupSize(const IR::BasicBlock& block, size_t first_index) {
const auto& first = block.instructions[first_index];
if (first.op != IR::Opcode::BufferLoadDword || first.memory.component_index != 0u ||
first.memory.component_count <= 1u) {
return 1u;
}
size_t count = 1u;
while (first_index + count < block.instructions.size() &&
count < first.memory.component_count) {
const auto& next = block.instructions[first_index + count];
if (next.op != IR::Opcode::BufferLoadDword || next.pc != first.pc ||
next.memory.component_index != count ||
next.memory.component_count != first.memory.component_count) {
break;
}
count++;
}
return count;
}
void EmitBlockInstructions(EmitterState& state, const IR::BasicBlock& block) {
for (size_t i = 0; i < block.instructions.size();) {
const auto count = BufferLoadGroupSize(block, i);
if (count > 1u) {
EmitBufferLoadDwordGroup(state, block.instructions.data() + i,
static_cast<uint32_t>(count));
} else {
EmitInstruction(state, block.instructions[i]);
}
i += count;
}
}
void EmitDispatcherBlocks(EmitterState& state, const IR::Program& program) {
for (const auto& block: program.blocks) {
if (block.id >= state.reachable_blocks.size() || !state.reachable_blocks[block.id]) {
continue;
}
state.builder.AddFunction({OpLabel, BlockLabel(state, block.id)});
for (const auto& inst: block.instructions) {
EmitInstruction(state, inst);
}
EmitBlockInstructions(state, block);
EmitDispatcherTerminator(state, block.terminator);
}
}
@@ -1083,9 +1115,7 @@ void EmitFunction(EmitterState& state, const IR::Program& program) {
continue;
}
state.builder.AddFunction({OpLabel, BlockLabel(state, block.id)});
for (const auto& inst: block.instructions) {
EmitInstruction(state, inst);
}
EmitBlockInstructions(state, block);
EmitTerminator(state, block.terminator);
}
@@ -24,7 +24,7 @@ uint32_t EmitExportVec4F32(EmitterState& state, const IR::Instruction& inst) {
const auto raw = EmitValueLoad(state, inst.src[pair_index]);
const auto unpacked = state.builder.AllocateId();
state.builder.AddFunction({OpExtInst, state.vec2_float_type, unpacked,
state.glsl_std450, GlslUnpackHalf2x16, raw});
state.glsl_std450, GlslUnpackHalf2x16, raw});
for (uint32_t lane = 0; lane < 2u; lane++) {
const auto component = pair_index * 2u + lane;
if (((inst.export_info.en >> component) & 1u) == 0) {
@@ -36,8 +36,8 @@ uint32_t EmitExportVec4F32(EmitterState& state, const IR::Instruction& inst) {
}
}
const auto vec = state.builder.AllocateId();
state.builder.AddFunction({OpCompositeConstruct, state.vec4_float_type, vec,
components[0], components[1], components[2], components[3]});
state.builder.AddFunction({OpCompositeConstruct, state.vec4_float_type, vec, components[0],
components[1], components[2], components[3]});
return vec;
}
@@ -50,7 +50,59 @@ uint32_t EmitExportVec4F32(EmitterState& state, const IR::Instruction& inst) {
return vec;
}
uint32_t ApplyMrtExportMapping(EmitterState& state, const IR::Instruction& inst, uint32_t value) {
uint32_t EmitExportComponentU32(EmitterState& state, const IR::Instruction& inst,
uint32_t component) {
const bool enabled = ((inst.export_info.en >> component) & 1u) != 0;
if (!enabled || component >= inst.src_count || component >= 4u) {
return ConstantU32(state, component == 3u ? 1u : 0u);
}
return EmitValueLoad(state, inst.src[component]);
}
uint32_t EmitExportVec4U32(EmitterState& state, const IR::Instruction& inst) {
uint32_t components[4] = {
ConstantU32(state, 0u),
ConstantU32(state, 0u),
ConstantU32(state, 0u),
ConstantU32(state, 1u),
};
if (inst.export_info.compr) {
for (uint32_t pair_index = 0; pair_index < 2u && pair_index < inst.src_count;
pair_index++) {
const auto raw = EmitValueLoad(state, inst.src[pair_index]);
for (uint32_t lane = 0; lane < 2u; lane++) {
const auto component = pair_index * 2u + lane;
if (((inst.export_info.en >> component) & 1u) == 0) {
continue;
}
components[component] = state.builder.AllocateId();
state.builder.AddFunction(
{OpBitFieldUExtract, state.uint_type, components[component], raw,
ConstantU32(state, lane * 16u), ConstantU32(state, 16u)});
}
}
} else {
for (uint32_t component = 0; component < 4u; component++) {
components[component] = EmitExportComponentU32(state, inst, component);
}
}
const auto vec = state.builder.AllocateId();
state.builder.AddFunction({OpCompositeConstruct, state.vec4_uint_type, vec, components[0],
components[1], components[2], components[3]});
return vec;
}
static bool MrtUsesUintOutput(const EmitterState& state, const IR::Instruction& inst) {
return inst.export_info.kind == IR::ExportTargetKind::Mrt &&
state.pixel_input_info != nullptr &&
inst.export_info.index < std::size(state.pixel_input_info->target_output_mode) &&
state.pixel_input_info->target_output_mode[inst.export_info.index] == 7u;
}
uint32_t ApplyMrtExportMapping(EmitterState& state, const IR::Instruction& inst, uint32_t value,
uint32_t vector_type) {
if (inst.export_info.kind != IR::ExportTargetKind::Mrt || state.pixel_input_info == nullptr ||
inst.export_info.index >= state.pixel_input_info->target_export_mapping.size()) {
return value;
@@ -62,8 +114,8 @@ uint32_t ApplyMrtExportMapping(EmitterState& state, const IR::Instruction& inst,
}
const auto mapped = state.builder.AllocateId();
state.builder.AddFunction({OpVectorShuffle, state.vec4_float_type, mapped, value, value,
mapping.Map(0), mapping.Map(1), mapping.Map(2), mapping.Map(3)});
state.builder.AddFunction({OpVectorShuffle, vector_type, mapped, value, value, mapping.Map(0),
mapping.Map(1), mapping.Map(2), mapping.Map(3)});
return mapped;
}
@@ -89,7 +141,7 @@ void EmitMrtZExport(EmitterState& state, const IR::Instruction& inst) {
const auto ptr = state.builder.AllocateId();
state.builder.AddFunction({OpBitcast, state.int_type, mask, raw});
state.builder.AddFunction({OpAccessChain, state.ptr_output_int, ptr,
state.sample_mask_variable, ConstantU32(state, 0)});
state.sample_mask_variable, ConstantU32(state, 0)});
state.builder.AddFunction({OpStore, ptr, mask});
}
}
@@ -114,11 +166,15 @@ void EmitExport(EmitterState& state, const IR::Instruction& inst) {
return;
}
const auto value = ApplyMrtExportMapping(state, inst, EmitExportVec4F32(state, inst));
const auto uint_output = MrtUsesUintOutput(state, inst);
const auto vector_type = uint_output ? state.vec4_uint_type : state.vec4_float_type;
const auto value = ApplyMrtExportMapping(
state, inst, uint_output ? EmitExportVec4U32(state, inst) : EmitExportVec4F32(state, inst),
vector_type);
if (inst.export_info.kind == IR::ExportTargetKind::Position) {
const auto pointer = state.builder.AllocateId();
state.builder.AddFunction({OpAccessChain, state.ptr_output_vec4_float, pointer, variable,
ConstantU32(state, 0)});
state.builder.AddFunction(
{OpAccessChain, state.ptr_output_vec4_float, pointer, variable, ConstantU32(state, 0)});
state.builder.AddFunction({OpStore, pointer, value});
return;
}
@@ -102,7 +102,7 @@ uint32_t EmitWqmLaneU32(EmitterState& state, uint32_t src) {
state.builder.AddFunction(
{OpINotEqual, state.bool_type, non_zero, masked, ConstantU32(state, 0)});
state.builder.AddFunction({OpSelect, state.uint_type, expanded, non_zero,
ConstantU32(state, mask), ConstantU32(state, 0)});
ConstantU32(state, mask), ConstantU32(state, 0)});
state.builder.AddFunction({OpBitwiseOr, state.uint_type, combined, ret, expanded});
ret = combined;
}
@@ -122,8 +122,8 @@ void EmitWqmB64(EmitterState& state, const IR::Instruction& inst) {
}
const auto ballot = state.builder.AllocateId();
state.builder.AddFunction({OpGroupNonUniformBallot, state.vec4_uint_type, ballot,
ConstantU32(state, ScopeSubgroup),
EmitLaneMaskOperandActiveBool(state, inst.src[0])});
ConstantU32(state, ScopeSubgroup),
EmitLaneMaskOperandActiveBool(state, inst.src[0])});
const auto low = state.builder.AllocateId();
const auto high = state.builder.AllocateId();
state.builder.AddFunction({OpCompositeExtract, state.uint_type, low, ballot, 0});
@@ -150,8 +150,8 @@ void EmitWqmB64(EmitterState& state, const IR::Instruction& inst) {
EmitPerInvocationMask(state, inst.dst, active);
} else {
const auto result = state.builder.AllocateId();
state.builder.AddFunction({OpSelect, state.uint_type, result, active,
ConstantU32(state, 1), ConstantU32(state, 0)});
state.builder.AddFunction({OpSelect, state.uint_type, result, active, ConstantU32(state, 1),
ConstantU32(state, 0)});
EmitStoreU32(state, inst.dst, result);
EmitStoreU32(state, OffsetRegisterOperand(inst.dst, 1), ConstantU32(state, 0));
}
@@ -205,8 +205,7 @@ void EmitSaveexecB32(EmitterState& state, const IR::Instruction& inst) {
const auto cond = state.builder.AllocateId();
const auto scc = state.builder.AllocateId();
state.builder.AddFunction(
{OpINotEqual, state.bool_type, cond, new_low, ConstantU32(state, 0)});
state.builder.AddFunction({OpINotEqual, state.bool_type, cond, new_low, ConstantU32(state, 0)});
state.builder.AddFunction(
{OpSelect, state.uint_type, scc, cond, ConstantU32(state, 1), ConstantU32(state, 0)});
EmitStoreU32(state, SccOperand(), scc);
@@ -269,18 +268,19 @@ void EmitReadFirstLaneU32(EmitterState& state, const IR::Instruction& inst) {
const auto first_lane = state.builder.AllocateId();
const auto first_value = state.builder.AllocateId();
state.builder.AddFunction({OpGroupNonUniformBallot, state.vec4_uint_type, ballot,
ConstantU32(state, ScopeSubgroup), active});
ConstantU32(state, ScopeSubgroup), active});
state.builder.AddFunction({OpGroupNonUniformBallotFindLSB, state.uint_type, first_lane,
ConstantU32(state, ScopeSubgroup), ballot});
ConstantU32(state, ScopeSubgroup), ballot});
state.builder.AddFunction({OpGroupNonUniformShuffle, state.uint_type, first_value,
ConstantU32(state, ScopeSubgroup), src, first_lane});
ConstantU32(state, ScopeSubgroup), src, first_lane});
EmitStoreU32(state, inst.dst, first_value);
}
uint32_t EmitLaneIndex(EmitterState& state, const IR::Operand& operand) {
const auto lane = state.builder.AllocateId();
const auto mask = state.wave_size == 32u ? 31u : 63u;
state.builder.AddFunction({OpBitwiseAnd, state.uint_type, lane, EmitValueLoad(state, operand),
ConstantU32(state, 63)});
ConstantU32(state, mask)});
return lane;
}
@@ -289,7 +289,7 @@ void EmitReadLaneU32(EmitterState& state, const IR::Instruction& inst) {
const auto lane = EmitLaneIndex(state, inst.src[1]);
const auto value = state.builder.AllocateId();
state.builder.AddFunction({OpGroupNonUniformShuffle, state.uint_type, value,
ConstantU32(state, ScopeSubgroup), src, lane});
ConstantU32(state, ScopeSubgroup), src, lane});
EmitStoreU32(state, inst.dst, value);
}
@@ -336,10 +336,8 @@ void EmitPermlaneB32(EmitterState& state, const IR::Instruction& inst, bool x16)
state.builder.AddFunction(
{OpBitwiseXor, state.uint_type, row_value, row, ConstantU32(state, 16)});
}
state.builder.AddFunction(
{OpBitwiseAnd, state.uint_type, lane, subid, ConstantU32(state, 15)});
state.builder.AddFunction(
{OpBitwiseAnd, state.uint_type, lane8, lane, ConstantU32(state, 7)});
state.builder.AddFunction({OpBitwiseAnd, state.uint_type, lane, subid, ConstantU32(state, 15)});
state.builder.AddFunction({OpBitwiseAnd, state.uint_type, lane8, lane, ConstantU32(state, 7)});
state.builder.AddFunction(
{OpShiftLeftLogical, state.uint_type, shift, lane8, ConstantU32(state, 2)});
state.builder.AddFunction(
@@ -350,7 +348,7 @@ void EmitPermlaneB32(EmitterState& state, const IR::Instruction& inst, bool x16)
{OpBitwiseAnd, state.uint_type, index1, index0, ConstantU32(state, 15)});
state.builder.AddFunction({OpBitwiseOr, state.uint_type, target, row_value, index1});
state.builder.AddFunction({OpGroupNonUniformShuffle, state.uint_type, shuffled,
ConstantU32(state, ScopeSubgroup), value, target});
ConstantU32(state, ScopeSubgroup), value, target});
uint32_t ret = shuffled;
if (!inst.dst.op_sel) {
const auto source_active = EmitLaneIndexActiveBool(state, target);
@@ -375,7 +373,7 @@ void EmitBarrier(EmitterState& state, const IR::Instruction& inst) {
(void)inst;
const auto semantics = MemorySemanticsAcquireRelease | MemorySemanticsWorkgroupMemory;
state.builder.AddFunction({OpControlBarrier, ConstantU32(state, ScopeWorkgroup),
ConstantU32(state, ScopeWorkgroup), ConstantU32(state, semantics)});
ConstantU32(state, ScopeWorkgroup), ConstantU32(state, semantics)});
}
} // namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter
@@ -2,6 +2,66 @@
#include "graphics/shader/recompiler/emitter/spirvEmitterInternal.h"
namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter {
namespace {
uint32_t EmitCubeAxisF32(EmitterState& state, uint32_t value) {
const auto normalized = state.builder.AllocateId();
state.builder.AddFunction(
{OpFSub, state.float_type, normalized, value, ConstantF32(state, 0x3f800000u)});
return normalized;
}
uint32_t EmitCubeLayerF32(EmitterState& state, uint32_t face_id) {
// Sampled RDNA2 cubemaps encode face_id as slice * 8 + face. The native
// 2D-array view stores six contiguous faces per slice, so remove the two
// reserved face IDs from every preceding slice.
const auto guest_layer = state.builder.AllocateId();
const auto slice = state.builder.AllocateId();
const auto padding = state.builder.AllocateId();
const auto host_layer = state.builder.AllocateId();
const auto result = state.builder.AllocateId();
state.builder.AddFunction({OpConvertFToU, state.uint_type, guest_layer, face_id});
state.builder.AddFunction(
{OpShiftRightLogical, state.uint_type, slice, guest_layer, ConstantU32(state, 3)});
state.builder.AddFunction(
{OpShiftLeftLogical, state.uint_type, padding, slice, ConstantU32(state, 1)});
state.builder.AddFunction({OpISub, state.uint_type, host_layer, guest_layer, padding});
state.builder.AddFunction({OpConvertUToF, state.float_type, result, host_layer});
return result;
}
uint32_t EmitImageCoordF32Impl(EmitterState& state, const IR::Instruction& inst,
const IR::Operand& address, uint32_t first_component,
uint32_t components) {
auto x = EmitImageAddressFloatLoad(state, inst, address, first_component);
if (components == 1u) {
return x;
}
auto y = inst.memory.image_address_components > first_component + 1u
? EmitImageAddressFloatLoad(state, inst, address, first_component + 1u)
: EmitZeroF32(state);
if (inst.memory.image_cube) {
// RDNA2 sampled cubemap S/T coordinates are biased by +1 relative to
// normalized 2D-array coordinates.
x = EmitCubeAxisF32(state, x);
y = EmitCubeAxisF32(state, y);
}
const auto coord = state.builder.AllocateId();
if (components == 3u) {
auto z = inst.memory.image_address_components > first_component + 2u
? EmitImageAddressFloatLoad(state, inst, address, first_component + 2u)
: EmitZeroF32(state);
if (inst.memory.image_cube) {
z = EmitCubeLayerF32(state, z);
}
state.builder.AddFunction({OpCompositeConstruct, state.vec3_float_type, coord, x, y, z});
} else {
state.builder.AddFunction({OpCompositeConstruct, state.vec2_float_type, coord, x, y});
}
return coord;
}
} // namespace
bool HasImageSampleFlag(const IR::Instruction& inst, uint32_t flag) {
return (inst.memory.image_sample_flags & flag) != 0;
@@ -21,7 +81,7 @@ ImageSampleLayout MakeImageSampleLayout(const IR::Instruction& inst, ImageViewKi
}
if (HasImageSampleFlag(inst, Decoder::ImageSampleFlagDerivative)) {
const auto components = ImageViewSpatialComponents(view);
layout.grad_x = cursor;
layout.grad_x = cursor;
cursor += components;
layout.grad_y = cursor;
cursor += components;
@@ -36,24 +96,8 @@ ImageSampleLayout MakeImageSampleLayout(const IR::Instruction& inst, ImageViewKi
uint32_t EmitImageCoordF32(EmitterState& state, const IR::Instruction& inst,
const ImageSampleLayout& layout, ImageViewKind view) {
const auto x = EmitImageAddressFloatLoad(state, inst, inst.src[0], layout.coord);
const auto components = ImageViewCoordinateComponents(view);
if (components == 1u) {
return x;
}
const auto y = inst.memory.image_address_components > layout.coord + 1u
? EmitImageAddressFloatLoad(state, inst, inst.src[0], layout.coord + 1u)
: EmitZeroF32(state);
const auto coord = state.builder.AllocateId();
if (components == 3u) {
const auto z = inst.memory.image_address_components > layout.coord + 2u
? EmitImageAddressFloatLoad(state, inst, inst.src[0], layout.coord + 2u)
: EmitZeroF32(state);
state.builder.AddFunction({OpCompositeConstruct, state.vec3_float_type, coord, x, y, z});
} else {
state.builder.AddFunction({OpCompositeConstruct, state.vec2_float_type, coord, x, y});
}
return coord;
return EmitImageCoordF32Impl(state, inst, inst.src[0], layout.coord,
ImageViewCoordinateComponents(view));
}
uint32_t EmitImageLodF32(EmitterState& state, const IR::Instruction& inst,
@@ -95,10 +139,10 @@ uint32_t EmitImageGradientF32(EmitterState& state, const IR::Instruction& inst,
: EmitZeroF32(state);
const auto grad = state.builder.AllocateId();
if (components == 3u) {
const auto z = inst.memory.image_address_components > first_component + 2u
? EmitImageAddressFloatLoad(state, inst, inst.src[0],
first_component + 2u)
: EmitZeroF32(state);
const auto z =
inst.memory.image_address_components > first_component + 2u
? EmitImageAddressFloatLoad(state, inst, inst.src[0], first_component + 2u)
: EmitZeroF32(state);
state.builder.AddFunction({OpCompositeConstruct, state.vec3_float_type, grad, x, y, z});
} else {
state.builder.AddFunction({OpCompositeConstruct, state.vec2_float_type, grad, x, y});
@@ -120,8 +164,7 @@ uint32_t EmitImagePackedOffsetI32(EmitterState& state, const IR::Instruction& in
state.builder.AddFunction(
{OpCompositeConstruct, state.vec3_int_type, ret, zero, zero, zero});
} else {
state.builder.AddFunction(
{OpCompositeConstruct, state.vec2_int_type, ret, zero, zero});
state.builder.AddFunction({OpCompositeConstruct, state.vec2_int_type, ret, zero, zero});
}
return ret;
}
@@ -131,18 +174,18 @@ uint32_t EmitImagePackedOffsetI32(EmitterState& state, const IR::Instruction& in
const auto offset_x = state.builder.AllocateId();
state.builder.AddFunction({OpBitcast, state.int_type, packed_i32, packed_bits});
state.builder.AddFunction({OpBitFieldSExtract, state.int_type, offset_x, packed_i32,
ConstantI32(state, 0), ConstantI32(state, 6)});
ConstantI32(state, 0), ConstantI32(state, 6)});
if (components == 1u) {
return offset_x;
}
const auto offset_y = state.builder.AllocateId();
const auto offset = state.builder.AllocateId();
state.builder.AddFunction({OpBitFieldSExtract, state.int_type, offset_y, packed_i32,
ConstantI32(state, 8), ConstantI32(state, 6)});
ConstantI32(state, 8), ConstantI32(state, 6)});
if (components == 3u) {
const auto offset_z = state.builder.AllocateId();
state.builder.AddFunction({OpBitFieldSExtract, state.int_type, offset_z, packed_i32,
ConstantI32(state, 16), ConstantI32(state, 6)});
ConstantI32(state, 16), ConstantI32(state, 6)});
state.builder.AddFunction(
{OpCompositeConstruct, state.vec3_int_type, offset, offset_x, offset_y, offset_z});
} else {
@@ -158,7 +201,7 @@ uint32_t EmitImageCoordU32(EmitterState& state, const IR::Instruction& inst, Ima
if (components == 1u) {
return x;
}
const auto y = inst.memory.image_address_components > 1u
const auto y = inst.memory.image_address_components > 1u
? EmitImageAddressValueLoad(state, inst, inst.src[1], 1)
: ConstantU32(state, 0);
const auto coord = state.builder.AllocateId();
@@ -180,7 +223,7 @@ uint32_t EmitImageLoadCoordU32(EmitterState& state, const IR::Instruction& inst,
if (components == 1u) {
return x;
}
const auto y = inst.memory.image_address_components > 1u
const auto y = inst.memory.image_address_components > 1u
? EmitImageAddressValueLoad(state, inst, inst.src[0], 1)
: ConstantU32(state, 0);
const auto coord = state.builder.AllocateId();
@@ -209,24 +252,8 @@ uint32_t EmitImageMipLodU32(EmitterState& state, const IR::Instruction& inst,
uint32_t EmitImageQueryCoordF32(EmitterState& state, const IR::Instruction& inst,
ImageViewKind view) {
const auto x = EmitImageAddressFloatLoad(state, inst, inst.src[0], 0);
const auto components = ImageViewCoordinateComponents(view);
if (components == 1u) {
return x;
}
const auto y = inst.memory.image_address_components > 1u
? EmitImageAddressFloatLoad(state, inst, inst.src[0], 1)
: EmitZeroF32(state);
const auto coord = state.builder.AllocateId();
if (components == 3u) {
const auto z = inst.memory.image_address_components > 2u
? EmitImageAddressFloatLoad(state, inst, inst.src[0], 2)
: EmitZeroF32(state);
state.builder.AddFunction({OpCompositeConstruct, state.vec3_float_type, coord, x, y, z});
} else {
state.builder.AddFunction({OpCompositeConstruct, state.vec2_float_type, coord, x, y});
}
return coord;
// OpImageQueryLod takes only the spatial coordinates, even for arrayed images.
return EmitImageCoordF32Impl(state, inst, inst.src[0], 0, ImageViewSpatialComponents(view));
}
uint32_t DmaskComponentIndex(uint32_t dmask, uint32_t component) {
@@ -35,9 +35,9 @@ uint32_t ConstantImageGatherHorizontalOffsets(EmitterState& state, ImageViewKind
uint32_t LoadStorageImageDescriptorAtIndex(EmitterState& state, uint32_t resource,
uint32_t array_index, bool uint_image,
ImageViewKind view) {
const auto kind = StorageBindingKind(uint_image, view);
const auto kind = StorageBindingKind(uint_image, view);
const auto& descriptors = state.storage_images[StorageImageIndex(uint_image, view)];
const auto pointer =
const auto pointer =
DescriptorElementPointer(state, descriptors.pointer_type, descriptors.variable, array_index,
kind, resource, "storage image descriptor array was not emitted");
const auto image = state.builder.AllocateId();
@@ -133,10 +133,18 @@ void EmitImageLoad(EmitterState& state, const IR::Instruction& inst) {
const bool integer = inst.memory.kind == IR::ResourceKind::ImageUint;
const auto color = state.builder.AllocateId();
state.builder.AddFunction({OpImageFetch, integer ? state.vec4_uint_type : state.vec4_float_type,
color, image, EmitImageLoadCoordU32(state, inst, view),
ImageOperandsLodMask,
EmitImageMipLodU32(state, inst, inst.src[0], view)});
const auto coord = EmitImageLoadCoordU32(state, inst, view);
if (ImageSpirvMultisampled(view) != 0) {
const auto sample = EmitImageAddressValueLoad(state, inst, inst.src[0],
ImageViewCoordinateComponents(view));
state.builder.AddFunction({OpImageFetch,
integer ? state.vec4_uint_type : state.vec4_float_type, color,
image, coord, ImageOperandsSampleMask, sample});
} else {
state.builder.AddFunction(
{OpImageFetch, integer ? state.vec4_uint_type : state.vec4_float_type, color, image,
coord, ImageOperandsLodMask, EmitImageMipLodU32(state, inst, inst.src[0], view)});
}
const auto dmask = inst.memory.dmask != 0 ? inst.memory.dmask : 1u;
uint32_t dst_index = 0;
@@ -158,8 +166,8 @@ void EmitImageLoad(EmitterState& state, const IR::Instruction& inst) {
void EmitImageStore(EmitterState& state, const IR::Instruction& inst) {
const auto uint_image = inst.memory.kind == IR::ResourceKind::StorageImageUint;
const auto view = StorageImageViewKind(state, inst.memory, uint_image, inst.pc);
const auto binding = ResourceForDescriptor(state, StorageBindingKind(uint_image, view),
inst.memory.resource);
const auto binding =
ResourceForDescriptor(state, StorageBindingKind(uint_image, view), inst.memory.resource);
const auto image = LoadStorageImageDescriptorAtIndex(state, inst.memory.resource,
binding.array_index, uint_image, view);
@@ -261,9 +269,9 @@ void EmitImageSample(EmitterState& state, const IR::Instruction& inst) {
} else if (integer) {
result_type = state.vec4_uint_type;
}
const auto explicit_lod = ImageSampleNeedsExplicitLod(state, inst);
const auto opcode = ImageSampleOpcode(state, inst);
std::vector<uint32_t> words = {opcode, result_type, sample, sampled_image, base_coord};
const auto explicit_lod = ImageSampleNeedsExplicitLod(state, inst);
const auto opcode = ImageSampleOpcode(state, inst);
std::vector<uint32_t> words = {opcode, result_type, sample, sampled_image, base_coord};
if (dref) {
words.push_back(EmitImageDrefF32(state, inst, layout));
}
@@ -3,11 +3,11 @@
#include "common/common.h"
#include "common/stringUtils.h"
#include "graphics/shader/recompiler/ir/BindingLayout.h"
#include "graphics/shader/recompiler/BufferFormat.h"
#include "graphics/shader/recompiler/emitter/SpirvBuilder.h"
#include "graphics/shader/recompiler/ir/BindingLayout.h"
#include "graphics/shader/recompiler/ir/ResourceMaterialization.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/emitter/SpirvBuilder.h"
#include <algorithm>
#include <array>
@@ -99,6 +99,7 @@ enum : uint32_t {
ImageOperandsGradMask = 0x00000004u,
ImageOperandsOffsetMask = 0x00000010u,
ImageOperandsConstOffsetsMask = 0x00000020u,
ImageOperandsSampleMask = 0x00000040u,
};
enum : uint32_t {
@@ -150,7 +151,6 @@ enum : uint32_t {
OpImageGather = 96,
OpImageDrefGather = 97,
OpImageWrite = 99,
OpImage = 100,
OpImageQuerySizeLod = 103,
OpImageQueryLod = 105,
OpImageQueryLevels = 106,
@@ -312,117 +312,117 @@ struct EmitterState {
EmitterState(const IR::Program& program_, const IR::ResourceSnapshot& resources_)
: program(program_), resources(resources_) {}
Builder builder;
const IR::Program& program;
const IR::ResourceSnapshot& resources;
const ShaderVertexInputInfo* vertex_input_info = nullptr;
const ShaderPixelInputInfo* pixel_input_info = nullptr;
const ShaderComputeInputInfo* compute_input_info = nullptr;
ShaderType stage = ShaderType::Unknown;
uint32_t wave_size = 64;
bool exact_subgroup_operations = false;
bool per_invocation_masks = false;
uint32_t void_type = 0;
uint32_t bool_type = 0;
uint32_t uint_type = 0;
uint32_t uint_pair_type = 0;
uint32_t int_pair_type = 0;
uint32_t int_type = 0;
uint32_t float_type = 0;
uint32_t vec2_uint_type = 0;
uint32_t vec3_uint_type = 0;
uint32_t vec4_uint_type = 0;
uint32_t vec2_int_type = 0;
uint32_t vec3_int_type = 0;
uint32_t vec4_int_type = 0;
uint32_t vec2_float_type = 0;
uint32_t vec3_float_type = 0;
uint32_t vec4_float_type = 0;
uint32_t ptr_func_uint = 0;
uint32_t ptr_input_float = 0;
uint32_t ptr_input_bool = 0;
uint32_t ptr_input_int = 0;
uint32_t ptr_input_uint = 0;
uint32_t ptr_input_vec2_float = 0;
uint32_t ptr_input_vec3_float = 0;
uint32_t ptr_input_vec2_int = 0;
uint32_t ptr_input_vec3_int = 0;
uint32_t ptr_input_vec4_int = 0;
uint32_t ptr_input_vec2_uint = 0;
uint32_t ptr_input_vec3_uint = 0;
uint32_t ptr_input_vec4_uint = 0;
uint32_t ptr_input_vec4_float = 0;
uint32_t sample_mask_array_type = 0;
uint32_t ptr_output_int = 0;
uint32_t ptr_output_sample_mask_array = 0;
uint32_t ptr_output_float = 0;
uint32_t ptr_output_vec4_float = 0;
uint32_t per_vertex_type = 0;
uint32_t ptr_output_per_vertex = 0;
uint32_t storage_runtime_array_type = 0;
uint32_t storage_buffer_type = 0;
uint32_t ptr_storage_buffer = 0;
uint32_t ptr_storage_buffer_uint = 0;
uint32_t storage_buffer_array_type = 0;
uint32_t ptr_storage_buffer_array = 0;
uint32_t storage_buffer_variable = 0;
Builder builder;
const IR::Program& program;
const IR::ResourceSnapshot& resources;
const ShaderVertexInputInfo* vertex_input_info = nullptr;
const ShaderPixelInputInfo* pixel_input_info = nullptr;
const ShaderComputeInputInfo* compute_input_info = nullptr;
ShaderType stage = ShaderType::Unknown;
uint32_t wave_size = 64;
bool exact_subgroup_operations = false;
bool per_invocation_masks = false;
uint32_t void_type = 0;
uint32_t bool_type = 0;
uint32_t uint_type = 0;
uint32_t uint_pair_type = 0;
uint32_t int_pair_type = 0;
uint32_t int_type = 0;
uint32_t float_type = 0;
uint32_t vec2_uint_type = 0;
uint32_t vec3_uint_type = 0;
uint32_t vec4_uint_type = 0;
uint32_t vec2_int_type = 0;
uint32_t vec3_int_type = 0;
uint32_t vec4_int_type = 0;
uint32_t vec2_float_type = 0;
uint32_t vec3_float_type = 0;
uint32_t vec4_float_type = 0;
uint32_t ptr_func_uint = 0;
uint32_t ptr_input_float = 0;
uint32_t ptr_input_bool = 0;
uint32_t ptr_input_int = 0;
uint32_t ptr_input_uint = 0;
uint32_t ptr_input_vec2_float = 0;
uint32_t ptr_input_vec3_float = 0;
uint32_t ptr_input_vec2_int = 0;
uint32_t ptr_input_vec3_int = 0;
uint32_t ptr_input_vec4_int = 0;
uint32_t ptr_input_vec2_uint = 0;
uint32_t ptr_input_vec3_uint = 0;
uint32_t ptr_input_vec4_uint = 0;
uint32_t ptr_input_vec4_float = 0;
uint32_t sample_mask_array_type = 0;
uint32_t ptr_output_int = 0;
uint32_t ptr_output_sample_mask_array = 0;
uint32_t ptr_output_float = 0;
uint32_t ptr_output_vec4_float = 0;
uint32_t per_vertex_type = 0;
uint32_t ptr_output_per_vertex = 0;
uint32_t storage_runtime_array_type = 0;
uint32_t storage_buffer_type = 0;
uint32_t ptr_storage_buffer = 0;
uint32_t ptr_storage_buffer_uint = 0;
uint32_t storage_buffer_array_type = 0;
uint32_t ptr_storage_buffer_array = 0;
uint32_t storage_buffer_variable = 0;
std::array<uint32_t, IR::ShaderInfo::MaxBuffers> storage_buffer_offsets {};
uint32_t address_memory_array_type = 0;
uint32_t ptr_address_memory_array = 0;
uint32_t address_memory_variable = 0;
uint32_t gds_variable = 0;
uint32_t push_constant_array_type = 0;
uint32_t push_constant_block_type = 0;
uint32_t ptr_push_constant_block = 0;
uint32_t ptr_push_constant_uint = 0;
uint32_t push_constant_variable = 0;
uint32_t vsharp_storage_variable = 0;
uint32_t flattened_srt_variable = 0;
uint32_t lds_array_type = 0;
uint32_t ptr_workgroup_array = 0;
uint32_t ptr_workgroup_uint = 0;
uint32_t lds_variable = 0;
std::array<SampledImageDescriptors, 10> sampled_images;
std::array<StorageImageDescriptors, 10> storage_images;
uint32_t sampler_type = 0;
uint32_t sampler_array_type = 0;
uint32_t ptr_uniform_sampler = 0;
uint32_t ptr_uniform_sampler_array = 0;
uint32_t sampler_variable = 0;
uint32_t ptr_image_uint = 0;
uint32_t func_type = 0;
uint32_t main_func = 0;
uint32_t entry_label = 0;
uint32_t pixel_valid_mask_variable = 0;
bool dispatcher_fallback = false;
uint32_t dispatch_pc_variable = 0;
uint32_t dispatch_header_label = 0;
uint32_t dispatch_select_label = 0;
uint32_t dispatch_default_label = 0;
uint32_t dispatch_after_switch_label = 0;
uint32_t dispatch_continue_label = 0;
uint32_t dispatch_merge_label = 0;
uint32_t glsl_std450 = 0;
uint32_t subgroup_local_invocation_id_variable = 0;
uint32_t per_vertex_variable = 0;
uint32_t depth_variable = 0;
uint32_t sample_mask_variable = 0;
bool needs_subgroup_ballot = false;
bool needs_subgroup_shuffle = false;
bool needs_subgroup_local_invocation_id = false;
bool needs_compute_derivatives = false;
bool needs_image_gather_extended = false;
bool needs_function_lds = false;
bool needs_pixel_valid_mask = false;
std::vector<RegisterBinding> registers;
std::vector<InputBinding> inputs;
std::vector<OutputBinding> outputs;
std::vector<uint32_t> interface_variables;
std::vector<bool> reachable_blocks;
std::map<uint32_t, uint32_t> block_labels;
std::map<uint32_t, uint32_t> constants;
std::map<uint32_t, uint32_t> signed_constants;
std::map<uint32_t, uint32_t> float_constants;
uint32_t address_memory_array_type = 0;
uint32_t ptr_address_memory_array = 0;
uint32_t address_memory_variable = 0;
uint32_t gds_variable = 0;
uint32_t push_constant_array_type = 0;
uint32_t push_constant_block_type = 0;
uint32_t ptr_push_constant_block = 0;
uint32_t ptr_push_constant_uint = 0;
uint32_t push_constant_variable = 0;
uint32_t vsharp_storage_variable = 0;
uint32_t flattened_srt_variable = 0;
uint32_t lds_array_type = 0;
uint32_t ptr_workgroup_array = 0;
uint32_t ptr_workgroup_uint = 0;
uint32_t lds_variable = 0;
std::array<SampledImageDescriptors, 14> sampled_images;
std::array<StorageImageDescriptors, 10> storage_images;
uint32_t sampler_type = 0;
uint32_t sampler_array_type = 0;
uint32_t ptr_uniform_sampler = 0;
uint32_t ptr_uniform_sampler_array = 0;
uint32_t sampler_variable = 0;
uint32_t ptr_image_uint = 0;
uint32_t func_type = 0;
uint32_t main_func = 0;
uint32_t entry_label = 0;
uint32_t pixel_valid_mask_variable = 0;
bool dispatcher_fallback = false;
uint32_t dispatch_pc_variable = 0;
uint32_t dispatch_header_label = 0;
uint32_t dispatch_select_label = 0;
uint32_t dispatch_default_label = 0;
uint32_t dispatch_after_switch_label = 0;
uint32_t dispatch_continue_label = 0;
uint32_t dispatch_merge_label = 0;
uint32_t glsl_std450 = 0;
uint32_t subgroup_local_invocation_id_variable = 0;
uint32_t per_vertex_variable = 0;
uint32_t depth_variable = 0;
uint32_t sample_mask_variable = 0;
bool needs_subgroup_ballot = false;
bool needs_subgroup_shuffle = false;
bool needs_subgroup_local_invocation_id = false;
bool needs_compute_derivatives = false;
bool needs_image_gather_extended = false;
bool needs_function_lds = false;
bool needs_pixel_valid_mask = false;
std::vector<RegisterBinding> registers;
std::vector<InputBinding> inputs;
std::vector<OutputBinding> outputs;
std::vector<uint32_t> interface_variables;
std::vector<bool> reachable_blocks;
std::map<uint32_t, uint32_t> block_labels;
std::map<uint32_t, uint32_t> constants;
std::map<uint32_t, uint32_t> signed_constants;
std::map<uint32_t, uint32_t> float_constants;
};
constexpr uint32_t PsInputOffsetMask = 0x0000001fu;
@@ -453,17 +453,20 @@ enum class ImageViewKind {
Dim2D,
Dim2DArray,
Dim3D,
Dim2DMsaa,
Dim2DMsaaArray,
Count,
};
constexpr uint32_t ImageViewKindCount = static_cast<uint32_t>(ImageViewKind::Count);
constexpr uint32_t SampledImageViewKindCount = static_cast<uint32_t>(ImageViewKind::Count);
constexpr uint32_t StorageImageViewKindCount = static_cast<uint32_t>(ImageViewKind::Dim2DMsaa);
constexpr uint32_t SampledImageIndex(bool integer, ImageViewKind view) {
return static_cast<uint32_t>(view) + (integer ? ImageViewKindCount : 0u);
return static_cast<uint32_t>(view) + (integer ? SampledImageViewKindCount : 0u);
}
constexpr uint32_t StorageImageIndex(bool integer, ImageViewKind view) {
return static_cast<uint32_t>(view) + (integer ? ImageViewKindCount : 0u);
return static_cast<uint32_t>(view) + (integer ? StorageImageViewKindCount : 0u);
}
constexpr IR::DescriptorBindingKind SampledBindingKind(bool integer, ImageViewKind view) {
@@ -474,6 +477,9 @@ constexpr IR::DescriptorBindingKind SampledBindingKind(bool integer, ImageViewKi
case ImageViewKind::Dim2D: return IR::DescriptorBindingKind::SampledUint2D;
case ImageViewKind::Dim2DArray: return IR::DescriptorBindingKind::SampledUint2DArray;
case ImageViewKind::Dim3D: return IR::DescriptorBindingKind::SampledUint3D;
case ImageViewKind::Dim2DMsaa: return IR::DescriptorBindingKind::SampledUint2DMsaa;
case ImageViewKind::Dim2DMsaaArray:
return IR::DescriptorBindingKind::SampledUint2DMsaaArray;
default: break;
}
}
@@ -483,6 +489,8 @@ constexpr IR::DescriptorBindingKind SampledBindingKind(bool integer, ImageViewKi
case ImageViewKind::Dim2D: return IR::DescriptorBindingKind::Sampled2D;
case ImageViewKind::Dim2DArray: return IR::DescriptorBindingKind::Sampled2DArray;
case ImageViewKind::Dim3D: return IR::DescriptorBindingKind::Sampled3D;
case ImageViewKind::Dim2DMsaa: return IR::DescriptorBindingKind::Sampled2DMsaa;
case ImageViewKind::Dim2DMsaaArray: return IR::DescriptorBindingKind::Sampled2DMsaaArray;
default: break;
}
return IR::DescriptorBindingKind::Count;
@@ -516,6 +524,8 @@ constexpr uint32_t ImageSpirvDimension(ImageViewKind view) {
case ImageViewKind::Dim1DArray: return Dim1D;
case ImageViewKind::Dim2D:
case ImageViewKind::Dim2DArray:
case ImageViewKind::Dim2DMsaa:
case ImageViewKind::Dim2DMsaaArray:
case ImageViewKind::Count: return Dim2D;
case ImageViewKind::Dim3D: return Dim3D;
}
@@ -523,7 +533,14 @@ constexpr uint32_t ImageSpirvDimension(ImageViewKind view) {
}
constexpr uint32_t ImageSpirvArrayed(ImageViewKind view) {
return view == ImageViewKind::Dim1DArray || view == ImageViewKind::Dim2DArray ? 1u : 0u;
return view == ImageViewKind::Dim1DArray || view == ImageViewKind::Dim2DArray ||
view == ImageViewKind::Dim2DMsaaArray
? 1u
: 0u;
}
constexpr uint32_t ImageSpirvMultisampled(ImageViewKind view) {
return view == ImageViewKind::Dim2DMsaa || view == ImageViewKind::Dim2DMsaaArray ? 1u : 0u;
}
struct AddCarryResult {
@@ -990,11 +1007,6 @@ uint32_t NormalizeFormatComponent(EmitterState& state, const Format::BufferForma
uint32_t UnpackTBufferFormat(EmitterState& state, const IR::Instruction& inst,
const Format::BufferFormatInfo& info);
bool EmitTypedTBufferLoad(EmitterState& state, const IR::Instruction& inst,
const Format::BufferFormatInfo& info);
bool EmitFormattedBufferLoad(EmitterState& state, const IR::Instruction& inst);
uint32_t FormattedBufferDwordStoreComponentCount(Prospero::BufferFormat format,
uint32_t opcode_components);
@@ -1020,6 +1032,9 @@ void EmitBufferLoadSshort(EmitterState& state, const IR::Instruction& inst);
void EmitBufferLoadDword(EmitterState& state, const IR::Instruction& inst);
void EmitBufferLoadDwordGroup(EmitterState& state, const IR::Instruction* instructions,
uint32_t count);
void EmitBufferStoreDword(EmitterState& state, const IR::Instruction& inst);
void EmitFlatLoadUbyte(EmitterState& state, const IR::Instruction& inst);
@@ -34,16 +34,15 @@ uint32_t EmitDppWriteActiveBool(EmitterState& state, const IR::Operand& dst) {
{OpShiftLeftLogical, state.uint_type, bank_bit, ConstantU32(state, 1), bank});
state.builder.AddFunction(
{OpShiftLeftLogical, state.uint_type, row_bit, ConstantU32(state, 1), row});
state.builder.AddFunction({OpBitwiseAnd, state.uint_type, bank_hit,
ConstantU32(state, dst.dpp_bank_mask), bank_bit});
state.builder.AddFunction(
{OpBitwiseAnd, state.uint_type, bank_hit, ConstantU32(state, dst.dpp_bank_mask), bank_bit});
state.builder.AddFunction(
{OpBitwiseAnd, state.uint_type, row_hit, ConstantU32(state, dst.dpp_row_mask), row_bit});
state.builder.AddFunction(
{OpINotEqual, state.bool_type, bank_active, bank_hit, ConstantU32(state, 0)});
state.builder.AddFunction(
{OpINotEqual, state.bool_type, row_active, row_hit, ConstantU32(state, 0)});
state.builder.AddFunction(
{OpLogicalAnd, state.bool_type, dpp_active, bank_active, row_active});
state.builder.AddFunction({OpLogicalAnd, state.bool_type, dpp_active, bank_active, row_active});
uint32_t write_active = dpp_active;
if (!dst.dpp_bound_ctrl) {
const auto target = EmitDppTargetLane(state, dst.dpp_ctrl);
@@ -102,7 +101,7 @@ void EmitStoreU32(EmitterState& state, const IR::Operand& dst, uint32_t value) {
const auto selected = state.builder.AllocateId();
state.builder.AddFunction({OpLoad, state.uint_type, old_value, pointer});
state.builder.AddFunction({OpSelect, state.uint_type, selected,
EmitDppWriteActiveBool(state, dst), wave_value, old_value});
EmitDppWriteActiveBool(state, dst), wave_value, old_value});
state.builder.AddFunction({OpStore, pointer, selected});
return;
}
@@ -131,8 +130,7 @@ uint32_t EmitNotEqualZeroBool(EmitterState& state, uint32_t value) {
uint32_t EmitSelectU32Value(EmitterState& state, uint32_t condition, uint32_t true_value,
uint32_t false_value) {
const auto ret = state.builder.AllocateId();
state.builder.AddFunction(
{OpSelect, state.uint_type, ret, condition, true_value, false_value});
state.builder.AddFunction({OpSelect, state.uint_type, ret, condition, true_value, false_value});
return ret;
}
@@ -212,12 +210,12 @@ bool IsStorageBufferMemoryKind(IR::ResourceKind kind) {
void EmitStorageBufferOffsets(EmitterState& state) {
for (uint32_t i = 0; i < state.program.bindings.buffer_offset_count; i++) {
const auto word = EmitShaderDataDwordLoad(
state, state.program.bindings.buffer_offset_dword + i / 4u);
const auto shift = ConstantU32(state, (i % 4u) * 8u + 2u);
const auto word =
EmitShaderDataDwordLoad(state, state.program.bindings.buffer_offset_dword + i / 4u);
const auto shift = ConstantU32(state, (i % 4u) * 8u + 2u);
state.storage_buffer_offsets[i] = EmitBinaryU32(
state, OpBitwiseAnd,
EmitBinaryU32(state, OpShiftRightLogical, word, shift), ConstantU32(state, 0x3fu));
state, OpBitwiseAnd, EmitBinaryU32(state, OpShiftRightLogical, word, shift),
ConstantU32(state, 0x3fu));
}
}
@@ -329,8 +327,7 @@ uint32_t EmitRelativeAddress(EmitterState& state, const IR::Instruction& inst, u
uint32_t EmitFlatVirtualAddress(EmitterState& state, const IR::Instruction& inst,
uint32_t first_src, uint32_t src_count) {
if (inst.memory.resource >= state.resources.addresses.size() ||
src_count < 2) {
if (inst.memory.resource >= state.resources.addresses.size() || src_count < 2) {
ExitDescriptorBindingFailure(state, IR::DescriptorBindingKind::AddressMemory,
inst.memory.resource, "flat address snapshot is missing");
}
@@ -429,20 +426,20 @@ uint32_t EmitStorageBufferObjectPointer(EmitterState& state, const IR::MemoryInf
ResourceForDescriptor(state, IR::DescriptorBindingKind::AddressMemory, mem.resource);
const auto pointer = state.builder.AllocateId();
state.builder.AddFunction({OpAccessChain, state.ptr_storage_buffer, pointer,
state.address_memory_variable,
ConstantU32(state, binding.array_index)});
state.address_memory_variable,
ConstantU32(state, binding.array_index)});
return pointer;
}
const auto binding = StorageBufferBindingForMemory(state, mem, use_pc);
const auto pointer = state.builder.AllocateId();
state.builder.AddFunction({OpAccessChain, state.ptr_storage_buffer, pointer,
state.storage_buffer_variable,
ConstantU32(state, binding.array_index)});
state.storage_buffer_variable,
ConstantU32(state, binding.array_index)});
return pointer;
}
uint32_t EmitStorageBufferElementInBounds(EmitterState& state, const IR::MemoryInfo& mem,
uint32_t index, uint32_t use_pc) {
uint32_t index, uint32_t use_pc) {
index = EmitStorageBufferIndex(state, mem, index, use_pc);
const auto object = EmitStorageBufferObjectPointer(state, mem, use_pc);
const auto length = state.builder.AllocateId();
@@ -453,7 +450,7 @@ uint32_t EmitStorageBufferElementInBounds(EmitterState& state, const IR::MemoryI
}
uint32_t EmitStorageBufferElementPointer(EmitterState& state, const IR::MemoryInfo& mem,
uint32_t index, uint32_t use_pc) {
uint32_t index, uint32_t use_pc) {
index = EmitStorageBufferIndex(state, mem, index, use_pc);
if (IsFlatMemoryKind(mem.kind)) {
if (state.address_memory_variable == 0) {
@@ -589,9 +586,9 @@ uint32_t EmitMemoryLoadSubDwordValueU32(EmitterState& state, const IR::Instructi
const auto left = state.builder.AllocateId();
const auto sign_shift = 32u - data_bits;
state.builder.AddFunction({OpShiftLeftLogical, state.uint_type, left, masked,
ConstantU32(state, sign_shift)});
ConstantU32(state, sign_shift)});
state.builder.AddFunction({OpShiftRightArithmetic, state.uint_type, value, left,
ConstantU32(state, sign_shift)});
ConstantU32(state, sign_shift)});
}
return value;
};
@@ -659,16 +656,15 @@ void EmitAtomicUpdateU32(EmitterState& state, uint32_t pointer, IR::ResourceKind
state.builder.AddFunction({OpBranch, preheader});
state.builder.AddFunction({OpLabel, preheader});
state.builder.AddFunction({OpAtomicLoad, state.uint_type, initial, pointer,
ConstantU32(state, scope),
ConstantU32(state, MemorySemanticsNone)});
ConstantU32(state, scope), ConstantU32(state, MemorySemanticsNone)});
state.builder.AddFunction({OpBranch, header});
state.builder.AddFunction({OpLabel, header});
state.builder.AddFunction(
{OpPhi, state.uint_type, observed, initial, preheader, exchanged, continue_label});
const auto desired = desired_value(observed);
state.builder.AddFunction({OpAtomicCompareExchange, state.uint_type, exchanged, pointer,
ConstantU32(state, scope), ConstantU32(state, MemorySemanticsNone),
ConstantU32(state, MemorySemanticsNone), desired, observed});
ConstantU32(state, scope), ConstantU32(state, MemorySemanticsNone),
ConstantU32(state, MemorySemanticsNone), desired, observed});
const auto success = state.builder.AllocateId();
state.builder.AddFunction({OpIEqual, state.bool_type, success, exchanged, observed});
state.builder.AddFunction({OpLoopMerge, merge, continue_label, LoopControlNone});
@@ -793,8 +789,7 @@ uint32_t EmitTBufferBitcastU32ToI32(EmitterState& state, uint32_t value) {
uint32_t EmitTBufferCompareU32Constant(EmitterState& state, uint32_t opcode, uint32_t value,
uint32_t constant) {
const auto ret = state.builder.AllocateId();
state.builder.AddFunction(
{opcode, state.bool_type, ret, value, ConstantU32(state, constant)});
state.builder.AddFunction({opcode, state.bool_type, ret, value, ConstantU32(state, constant)});
return ret;
}
@@ -821,7 +816,7 @@ uint32_t EmitExtractFormatFieldU32(EmitterState& state, uint32_t raw_word, uint3
const auto signed_word = EmitTBufferBitcastU32ToI32(state, raw_word);
const auto extracted = state.builder.AllocateId();
state.builder.AddFunction({OpBitFieldSExtract, state.int_type, extracted, signed_word,
ConstantU32(state, offset), ConstantU32(state, bits)});
ConstantU32(state, offset), ConstantU32(state, bits)});
const auto ret = state.builder.AllocateId();
state.builder.AddFunction({OpBitcast, state.uint_type, ret, extracted});
return ret;
@@ -829,7 +824,7 @@ uint32_t EmitExtractFormatFieldU32(EmitterState& state, uint32_t raw_word, uint3
const auto extracted = state.builder.AllocateId();
state.builder.AddFunction({OpBitFieldUExtract, state.uint_type, extracted, raw_word,
ConstantU32(state, offset), ConstantU32(state, bits)});
ConstantU32(state, offset), ConstantU32(state, bits)});
return extracted;
}
@@ -940,7 +935,7 @@ uint32_t NormalizeFormatComponent(EmitterState& state, const Format::BufferForma
state.builder.AddFunction(
{OpFDiv, state.float_type, normalized, value, ConstantF32Value(state, max_value)});
state.builder.AddFunction({OpExtInst, state.float_type, clamped, state.glsl_std450,
GlslFMax, normalized, ConstantF32Value(state, -1.0f)});
GlslFMax, normalized, ConstantF32Value(state, -1.0f)});
return EmitTBufferBitcastF32ToU32(state, clamped);
}
case Format::ComponentType::Float:
@@ -961,18 +956,8 @@ uint32_t UnpackTBufferFormat(EmitterState& state, const IR::Instruction& inst,
return NormalizeFormatComponent(state, info, inst.memory.component_index, raw);
}
bool EmitTypedTBufferLoad(EmitterState& state, const IR::Instruction& inst,
const Format::BufferFormatInfo& info) {
if (!Format::CanUseTypedBufferLoad(info.format)) {
return false;
}
const auto value = EmitMemoryLoadDwordValueU32(state, inst, IR::ResourceKind::Buffer, 0,
AddressSourceCount(inst, 0));
EmitStoreU32(state, inst.dst, value);
return true;
}
bool EmitFormattedBufferLoad(EmitterState& state, const IR::Instruction& inst) {
bool EmitFormattedBufferLoadValueU32(EmitterState& state, const IR::Instruction& inst,
uint32_t& value) {
if (!IsFormattedBufferComponent(inst)) {
return false;
}
@@ -984,18 +969,29 @@ bool EmitFormattedBufferLoad(EmitterState& state, const IR::Instruction& inst) {
const auto info = Format::GetFormatInfo(format);
if (inst.memory.component_index >= info.component_count) {
EmitStoreU32(state, inst.dst, ConstantU32(state, 0));
value = ConstantU32(state, 0);
return true;
}
if (EmitTypedTBufferLoad(state, inst, info)) {
if (Format::CanUseTypedBufferLoad(info.format)) {
value = EmitMemoryLoadDwordValueU32(state, inst, IR::ResourceKind::Buffer, 0,
AddressSourceCount(inst, 0));
return true;
}
EmitStoreU32(state, inst.dst, UnpackTBufferFormat(state, inst, info));
value = UnpackTBufferFormat(state, inst, info);
return true;
}
uint32_t EmitBufferLoadDwordValueU32(EmitterState& state, const IR::Instruction& inst) {
uint32_t value = 0;
if (EmitFormattedBufferLoadValueU32(state, inst, value)) {
return value;
}
return EmitMemoryLoadDwordValueU32(state, inst, IR::ResourceKind::Buffer, 0,
AddressSourceCount(inst, 0));
}
uint32_t FormattedBufferDwordStoreComponentCount(Prospero::BufferFormat format,
uint32_t opcode_components) {
switch (format) {
@@ -1117,8 +1113,8 @@ uint32_t EmitAtomicPointer(EmitterState& state, const IR::Instruction& inst) {
StorageImageDescriptorPointer(state, inst.memory.resource, true, inst.pc, view);
const auto pointer = state.builder.AllocateId();
state.builder.AddFunction({OpImageTexelPointer, state.ptr_image_uint, pointer,
image_pointer, EmitImageCoordU32(state, inst, view),
ConstantU32(state, 0)});
image_pointer, EmitImageCoordU32(state, inst, view),
ConstantU32(state, 0)});
return pointer;
}
default: return 0;
@@ -1142,8 +1138,8 @@ void EmitAtomicU32(EmitterState& state, const IR::Instruction& inst, uint32_t op
EmitStorageBufferElementPointer(state, inst.memory, index, inst.pc);
const auto result = state.builder.AllocateId();
state.builder.AddFunction({opcode, state.uint_type, result, pointer,
ConstantU32(state, ScopeDevice),
ConstantU32(state, MemorySemanticsNone), value});
ConstantU32(state, ScopeDevice),
ConstantU32(state, MemorySemanticsNone), value});
EmitDeviceAtomicMemoryBarrier(state);
return result;
});
@@ -1158,8 +1154,8 @@ void EmitAtomicU32(EmitterState& state, const IR::Instruction& inst, uint32_t op
const auto pointer = EmitGdsElementPointer(state, index);
const auto result = state.builder.AllocateId();
state.builder.AddFunction({opcode, state.uint_type, result, pointer,
ConstantU32(state, ScopeDevice),
ConstantU32(state, MemorySemanticsNone), value});
ConstantU32(state, ScopeDevice),
ConstantU32(state, MemorySemanticsNone), value});
EmitDeviceAtomicMemoryBarrier(state);
return result;
});
@@ -1177,7 +1173,7 @@ void EmitAtomicU32(EmitterState& state, const IR::Instruction& inst, uint32_t op
const auto old = state.builder.AllocateId();
const auto scope = inst.memory.kind == IR::ResourceKind::Lds ? ScopeWorkgroup : ScopeDevice;
state.builder.AddFunction({opcode, state.uint_type, old, pointer, ConstantU32(state, scope),
ConstantU32(state, MemorySemanticsNone), value});
ConstantU32(state, MemorySemanticsNone), value});
if (inst.memory.kind == IR::ResourceKind::StorageImageUint ||
inst.memory.kind == IR::ResourceKind::Gds) {
EmitDeviceAtomicMemoryBarrier(state);
@@ -1199,8 +1195,8 @@ void EmitSLoadDword(EmitterState& state, const IR::Instruction& inst) {
{OpShiftRightLogical, state.uint_type, index, address, ConstantU32(state, 2)});
const auto object = state.builder.AllocateId();
state.builder.AddFunction({OpAccessChain, state.ptr_storage_buffer, object,
state.address_memory_variable,
ConstantU32(state, binding.array_index)});
state.address_memory_variable,
ConstantU32(state, binding.array_index)});
const auto length = state.builder.AllocateId();
const auto in_bounds = state.builder.AllocateId();
state.builder.AddFunction({OpArrayLength, state.uint_type, length, object, 0});
@@ -1227,8 +1223,8 @@ void EmitLoadSrtDword(EmitterState& state, const IR::Instruction& inst) {
const auto pointer = state.builder.AllocateId();
const auto value = state.builder.AllocateId();
state.builder.AddFunction({OpAccessChain, state.ptr_storage_buffer_uint, pointer,
state.flattened_srt_variable, ConstantU32(state, 0),
ConstantU32(state, inst.src[0].imm)});
state.flattened_srt_variable, ConstantU32(state, 0),
ConstantU32(state, inst.src[0].imm)});
state.builder.AddFunction({OpLoad, state.uint_type, value, pointer});
EmitStoreU32(state, inst.dst, value);
}
@@ -1262,11 +1258,27 @@ void EmitBufferLoadSshort(EmitterState& state, const IR::Instruction& inst) {
}
void EmitBufferLoadDword(EmitterState& state, const IR::Instruction& inst) {
EmitGuardedByExec(
state, [&]() { EmitStoreU32(state, inst.dst, EmitBufferLoadDwordValueU32(state, inst)); });
}
void EmitBufferLoadDwordGroup(EmitterState& state, const IR::Instruction* instructions,
uint32_t count) {
if (instructions == nullptr || count == 0u) {
return;
}
EmitGuardedByExec(state, [&]() {
if (EmitFormattedBufferLoad(state, inst)) {
return;
// RDNA VMEM captures every VADDR component before making overlapping VDATA writes
// visible. Keep the split IR components instruction-atomic by deferring all stores.
std::vector<uint32_t> values;
values.reserve(count);
for (uint32_t i = 0; i < count; i++) {
values.push_back(EmitBufferLoadDwordValueU32(state, instructions[i]));
}
for (uint32_t i = 0; i < count; i++) {
EmitStoreU32(state, instructions[i].dst, values[i]);
}
EmitMemoryLoadU32(state, inst, IR::ResourceKind::Buffer, 0, AddressSourceCount(inst, 0));
});
}
@@ -1385,7 +1397,7 @@ DsCounterAddress EmitAppendConsumeAddress(EmitterState& state, const IR::Instruc
state.builder.AddFunction(
{OpShiftRightLogical, state.uint_type, index, address, ConstantU32(state, 2)});
state.builder.AddFunction({OpULessThan, state.bool_type, in_bounds,
ConstantU32(state, inst.memory.offset + 3u), size});
ConstantU32(state, inst.memory.offset + 3u), size});
return {index, size, in_bounds};
}
@@ -1400,7 +1412,7 @@ uint32_t EmitGdsElementInBounds(EmitterState& state, uint32_t index) {
uint32_t EmitGdsElementPointer(EmitterState& state, uint32_t index) {
const auto pointer = state.builder.AllocateId();
state.builder.AddFunction({OpAccessChain, state.ptr_storage_buffer_uint, pointer,
state.gds_variable, ConstantU32(state, 0), index});
state.gds_variable, ConstantU32(state, 0), index});
return pointer;
}
@@ -1431,7 +1443,7 @@ ExecMaskInfo EmitExecMaskInfo(EmitterState& state) {
if (state.per_invocation_masks) {
const auto ballot = state.builder.AllocateId();
state.builder.AddFunction({OpGroupNonUniformBallot, state.vec4_uint_type, ballot,
ConstantU32(state, ScopeSubgroup), EmitExecActiveBool(state)});
ConstantU32(state, ScopeSubgroup), EmitExecActiveBool(state)});
exec_lo = state.builder.AllocateId();
state.builder.AddFunction({OpCompositeExtract, state.uint_type, exec_lo, ballot, 0});
if (state.wave_size == 64u) {
@@ -1482,28 +1494,27 @@ void EmitDsAppendConsume(EmitterState& state, const IR::Instruction& inst, uint3
const auto do_atomic = state.builder.AllocateId();
state.builder.AddFunction({OpIEqual, state.bool_type, first_lane, subid, exec.first_lane});
const auto first_active = EmitLogicalAndBool(state, first_lane, exec.any_active);
state.builder.AddFunction(
{OpLogicalAnd, state.bool_type, do_atomic, first_active, in_bounds});
state.builder.AddFunction({OpLogicalAnd, state.bool_type, do_atomic, first_active, in_bounds});
const auto atomic_value = EmitValueOrZeroIfCondition(state, do_atomic, [&]() {
const auto pointer = gds ? EmitGdsElementPointer(state, address.index)
: EmitLdsElementPointer(state, address.index);
const auto result = state.builder.AllocateId();
state.builder.AddFunction({atomic_opcode, state.uint_type, result, pointer,
ConstantU32(state, gds ? ScopeDevice : ScopeWorkgroup),
ConstantU32(state, MemorySemanticsNone), exec.active_count});
ConstantU32(state, gds ? ScopeDevice : ScopeWorkgroup),
ConstantU32(state, MemorySemanticsNone), exec.active_count});
if (gds) {
EmitDeviceAtomicMemoryBarrier(state);
} else {
const auto semantics = MemorySemanticsAcquireRelease | MemorySemanticsWorkgroupMemory;
state.builder.AddFunction({OpMemoryBarrier, ConstantU32(state, ScopeWorkgroup),
ConstantU32(state, semantics)});
ConstantU32(state, semantics)});
}
return result;
});
const auto broadcast = state.builder.AllocateId();
state.builder.AddFunction({OpGroupNonUniformShuffle, state.uint_type, broadcast,
ConstantU32(state, ScopeSubgroup), atomic_value, exec.first_lane});
ConstantU32(state, ScopeSubgroup), atomic_value, exec.first_lane});
const auto value = EmitSelectU32Value(state, exec.any_active, broadcast, ConstantU32(state, 0));
EmitStoreU32(state, inst.dst, value);
}
@@ -1523,8 +1534,8 @@ void EmitDsFloatMinMaxF32(EmitterState& state, const IR::Instruction& inst, bool
const auto value_u32 = state.builder.AllocateId();
state.builder.AddFunction({OpBitcast, state.float_type, old_f32, old_u32});
state.builder.AddFunction({max_value ? OpFOrdGreaterThan : OpFOrdLessThan,
state.bool_type, store_src, max_value ? old_f32 : cmp_f32,
max_value ? cmp_f32 : old_f32});
state.bool_type, store_src, max_value ? old_f32 : cmp_f32,
max_value ? cmp_f32 : old_f32});
state.builder.AddFunction(
{OpSelect, state.float_type, value_f32, store_src, data_f32, old_f32});
state.builder.AddFunction({OpBitcast, state.uint_type, value_u32, value_f32});
@@ -1575,14 +1586,13 @@ uint32_t EmitDsSwizzleTargetLane(EmitterState& state, uint32_t subid, uint32_t c
const auto xored = state.builder.AllocateId();
const auto base = state.builder.AllocateId();
const auto target = state.builder.AllocateId();
state.builder.AddFunction(
{OpBitwiseAnd, state.uint_type, lane, subid, ConstantU32(state, 31)});
state.builder.AddFunction({OpBitwiseAnd, state.uint_type, lane, subid, ConstantU32(state, 31)});
state.builder.AddFunction(
{OpBitwiseAnd, state.uint_type, masked, lane, ConstantU32(state, control & 0x1fu)});
state.builder.AddFunction(
{OpBitwiseOr, state.uint_type, ored, masked, ConstantU32(state, (control >> 5u) & 0x1fu)});
state.builder.AddFunction({OpBitwiseXor, state.uint_type, xored, ored,
ConstantU32(state, (control >> 10u) & 0x1fu)});
state.builder.AddFunction(
{OpBitwiseXor, state.uint_type, xored, ored, ConstantU32(state, (control >> 10u) & 0x1fu)});
state.builder.AddFunction(
{OpBitwiseAnd, state.uint_type, base, subid, ConstantU32(state, 0xffffffe0u)});
state.builder.AddFunction({OpBitwiseOr, state.uint_type, target, base, xored});
@@ -1596,7 +1606,7 @@ void EmitDsSwizzleB32(EmitterState& state, const IR::Instruction& inst) {
const auto target = EmitDsSwizzleTargetLane(state, subid, control);
const auto value = state.builder.AllocateId();
state.builder.AddFunction({OpGroupNonUniformShuffle, state.uint_type, value,
ConstantU32(state, ScopeSubgroup), source, target});
ConstantU32(state, ScopeSubgroup), source, target});
const auto exec_active = EmitLaneIndexActiveBool(state, target);
const auto subgroup_active = EmitSubgroupLaneActiveBool(state, target);
const auto source_active = state.builder.AllocateId();
@@ -174,6 +174,12 @@ uint32_t VertexParameterInputPointerType(const EmitterState& state, VertexInputS
}
}
static bool MrtUsesUintOutput(const EmitterState& state, uint32_t index) {
return state.stage == ShaderType::Pixel && state.pixel_input_info != nullptr &&
index < std::size(state.pixel_input_info->target_output_mode) &&
state.pixel_input_info->target_output_mode[index] == 7u;
}
void AllocateInputVariables(EmitterState& state) {
for (auto& binding: state.inputs) {
binding.variable_id = state.builder.AllocateId();
@@ -323,23 +329,39 @@ void AddDescriptorAnnotationsAndNames(EmitterState& state) {
Decorate(state.address_memory_variable, "address_memory",
IR::DescriptorBindingKind::AddressMemory);
}
constexpr const char* SampledNames[] = {
"sampled_1d", "sampled_1d_array", "sampled_2d", "sampled_2d_array",
"sampled_3d", "sampled_uint_1d", "sampled_uint_1d_array",
"sampled_uint_2d", "sampled_uint_2d_array", "sampled_uint_3d"};
constexpr const char* SampledNames[] = {"sampled_1d",
"sampled_1d_array",
"sampled_2d",
"sampled_2d_array",
"sampled_3d",
"sampled_2d_msaa",
"sampled_2d_msaa_array",
"sampled_uint_1d",
"sampled_uint_1d_array",
"sampled_uint_2d",
"sampled_uint_2d_array",
"sampled_uint_3d",
"sampled_uint_2d_msaa",
"sampled_uint_2d_msaa_array"};
for (uint32_t i = 0; i < state.sampled_images.size(); i++) {
const auto view = static_cast<ImageViewKind>(i % ImageViewKindCount);
const auto view = static_cast<ImageViewKind>(i % SampledImageViewKindCount);
Decorate(state.sampled_images[i].variable, SampledNames[i],
SampledBindingKind(i >= ImageViewKindCount, view));
SampledBindingKind(i >= SampledImageViewKindCount, view));
}
constexpr const char* StorageNames[] = {
"storage_1d", "storage_1d_array", "storage_2d", "storage_2d_array",
"storage_3d", "storage_uint_1d", "storage_uint_1d_array",
"storage_uint_2d", "storage_uint_2d_array", "storage_uint_3d"};
constexpr const char* StorageNames[] = {"storage_1d",
"storage_1d_array",
"storage_2d",
"storage_2d_array",
"storage_3d",
"storage_uint_1d",
"storage_uint_1d_array",
"storage_uint_2d",
"storage_uint_2d_array",
"storage_uint_3d"};
for (uint32_t i = 0; i < state.storage_images.size(); i++) {
const auto view = static_cast<ImageViewKind>(i % ImageViewKindCount);
const auto view = static_cast<ImageViewKind>(i % StorageImageViewKindCount);
Decorate(state.storage_images[i].variable, StorageNames[i],
StorageBindingKind(i >= ImageViewKindCount, view));
StorageBindingKind(i >= StorageImageViewKindCount, view));
}
if (state.sampler_variable != 0) {
Decorate(state.sampler_variable, "samplers", IR::DescriptorBindingKind::Samplers);
@@ -409,6 +431,7 @@ void EmitHeaderAndTypes(EmitterState& state) {
state.ptr_output_sample_mask_array = state.builder.AllocateId();
state.ptr_output_float = state.builder.AllocateId();
state.ptr_output_vec4_float = state.builder.AllocateId();
const auto ptr_output_vec4_uint = state.builder.AllocateId();
state.per_vertex_type = state.builder.AllocateId();
state.ptr_output_per_vertex = state.builder.AllocateId();
state.storage_runtime_array_type = state.builder.AllocateId();
@@ -444,15 +467,15 @@ void EmitHeaderAndTypes(EmitterState& state) {
image.array_type = state.builder.AllocateId();
image.array_pointer_type = state.builder.AllocateId();
}
state.sampler_type = state.builder.AllocateId();
state.sampler_array_type = state.builder.AllocateId();
state.ptr_uniform_sampler = state.builder.AllocateId();
state.ptr_uniform_sampler_array = state.builder.AllocateId();
state.ptr_image_uint = state.builder.AllocateId();
state.func_type = state.builder.AllocateId();
state.main_func = state.builder.AllocateId();
state.entry_label = state.builder.AllocateId();
state.glsl_std450 = state.builder.AllocateId();
state.sampler_type = state.builder.AllocateId();
state.sampler_array_type = state.builder.AllocateId();
state.ptr_uniform_sampler = state.builder.AllocateId();
state.ptr_uniform_sampler_array = state.builder.AllocateId();
state.ptr_image_uint = state.builder.AllocateId();
state.func_type = state.builder.AllocateId();
state.main_func = state.builder.AllocateId();
state.entry_label = state.builder.AllocateId();
state.glsl_std450 = state.builder.AllocateId();
state.builder.AddCapability({CapabilityShader});
state.builder.AddCapability({CapabilitySampled1D});
@@ -462,7 +485,7 @@ void EmitHeaderAndTypes(EmitterState& state) {
state.builder.AddCapability({CapabilityImageGatherExtended});
}
if (std::any_of(state.storage_images.begin(),
state.storage_images.begin() + ImageViewKindCount,
state.storage_images.begin() + StorageImageViewKindCount,
[](const auto& image) { return image.variable != 0; })) {
state.builder.AddCapability({CapabilityStorageImageReadWithoutFormat});
state.builder.AddCapability({CapabilityStorageImageWriteWithoutFormat});
@@ -605,6 +628,8 @@ void EmitHeaderAndTypes(EmitterState& state) {
{OpTypePointer, state.ptr_output_int, StorageClassOutput, state.int_type});
state.builder.AddType(
{OpTypePointer, state.ptr_output_vec4_float, StorageClassOutput, state.vec4_float_type});
state.builder.AddType(
{OpTypePointer, ptr_output_vec4_uint, StorageClassOutput, state.vec4_uint_type});
if (state.per_vertex_variable != 0) {
state.builder.AddType({OpTypeStruct, state.per_vertex_type, state.vec4_float_type});
state.builder.AddType({OpTypePointer, state.ptr_output_per_vertex, StorageClassOutput,
@@ -615,8 +640,12 @@ void EmitHeaderAndTypes(EmitterState& state) {
for (const auto& binding: state.outputs) {
if (binding.kind == IR::StageOutputKind::Parameter ||
binding.kind == IR::StageOutputKind::Mrt) {
const auto pointer_type =
binding.kind == IR::StageOutputKind::Mrt && MrtUsesUintOutput(state, binding.index)
? ptr_output_vec4_uint
: state.ptr_output_vec4_float;
state.builder.AddType(
{OpVariable, state.ptr_output_vec4_float, binding.variable_id, StorageClassOutput});
{OpVariable, pointer_type, binding.variable_id, StorageClassOutput});
}
}
if (state.depth_variable != 0) {
@@ -700,11 +729,11 @@ void EmitHeaderAndTypes(EmitterState& state) {
}
for (uint32_t i = 0; i < state.sampled_images.size(); i++) {
auto& image = state.sampled_images[i];
const auto view = static_cast<ImageViewKind>(i % ImageViewKindCount);
const bool integer = i >= ImageViewKindCount;
const auto view = static_cast<ImageViewKind>(i % SampledImageViewKindCount);
const bool integer = i >= SampledImageViewKindCount;
const auto component = integer ? state.uint_type : state.float_type;
state.builder.AddType({OpTypeImage, image.image_type, component,
ImageSpirvDimension(view), 0, ImageSpirvArrayed(view), 0, 1,
state.builder.AddType({OpTypeImage, image.image_type, component, ImageSpirvDimension(view),
0, ImageSpirvArrayed(view), ImageSpirvMultisampled(view), 1,
ImageFormatUnknown});
state.builder.AddType({OpTypeSampledImage, image.sampled_image_type, image.image_type});
state.builder.AddType(
@@ -733,13 +762,12 @@ void EmitHeaderAndTypes(EmitterState& state) {
}
for (uint32_t i = 0; i < state.storage_images.size(); i++) {
auto& image = state.storage_images[i];
const auto view = static_cast<ImageViewKind>(i % ImageViewKindCount);
const bool integer = i >= ImageViewKindCount;
const auto view = static_cast<ImageViewKind>(i % StorageImageViewKindCount);
const bool integer = i >= StorageImageViewKindCount;
const auto component = integer ? state.uint_type : state.float_type;
const auto format = integer ? ImageFormatR32ui : ImageFormatUnknown;
state.builder.AddType({OpTypeImage, image.image_type, component,
ImageSpirvDimension(view), 0, ImageSpirvArrayed(view), 0, 2,
format});
state.builder.AddType({OpTypeImage, image.image_type, component, ImageSpirvDimension(view),
0, ImageSpirvArrayed(view), 0, 2, format});
state.builder.AddType(
{OpTypePointer, image.pointer_type, StorageClassUniformConstant, image.image_type});
if (image.variable != 0) {
@@ -786,15 +814,15 @@ void AllocateDescriptorVariables(EmitterState& state) {
state.flattened_srt_variable = state.builder.AllocateId();
}
for (uint32_t i = 0; i < state.sampled_images.size(); i++) {
const auto view = static_cast<ImageViewKind>(i % ImageViewKindCount);
if (DescriptorBinding(state, SampledBindingKind(i >= ImageViewKindCount, view)) !=
const auto view = static_cast<ImageViewKind>(i % SampledImageViewKindCount);
if (DescriptorBinding(state, SampledBindingKind(i >= SampledImageViewKindCount, view)) !=
nullptr) {
state.sampled_images[i].variable = state.builder.AllocateId();
}
}
for (uint32_t i = 0; i < state.storage_images.size(); i++) {
const auto view = static_cast<ImageViewKind>(i % ImageViewKindCount);
if (DescriptorBinding(state, StorageBindingKind(i >= ImageViewKindCount, view)) !=
const auto view = static_cast<ImageViewKind>(i % StorageImageViewKindCount);
if (DescriptorBinding(state, StorageBindingKind(i >= StorageImageViewKindCount, view)) !=
nullptr) {
state.storage_images[i].variable = state.builder.AllocateId();
}
@@ -13,16 +13,30 @@ namespace {
constexpr uint32_t MaxPushConstantBytes = 128;
constexpr std::array ImageBindingKinds = {
DescriptorBindingKind::Sampled1D, DescriptorBindingKind::Sampled1DArray,
DescriptorBindingKind::Sampled2D, DescriptorBindingKind::Sampled2DArray,
DescriptorBindingKind::Sampled3D, DescriptorBindingKind::SampledUint1D,
DescriptorBindingKind::SampledUint1DArray, DescriptorBindingKind::SampledUint2D,
DescriptorBindingKind::SampledUint2DArray, DescriptorBindingKind::SampledUint3D,
DescriptorBindingKind::Storage1D, DescriptorBindingKind::Storage1DArray,
DescriptorBindingKind::Storage2D, DescriptorBindingKind::Storage2DArray,
DescriptorBindingKind::Storage3D, DescriptorBindingKind::StorageUint1D,
DescriptorBindingKind::StorageUint1DArray, DescriptorBindingKind::StorageUint2D,
DescriptorBindingKind::StorageUint2DArray, DescriptorBindingKind::StorageUint3D,
DescriptorBindingKind::Sampled1D,
DescriptorBindingKind::Sampled1DArray,
DescriptorBindingKind::Sampled2D,
DescriptorBindingKind::Sampled2DArray,
DescriptorBindingKind::Sampled2DMsaa,
DescriptorBindingKind::Sampled2DMsaaArray,
DescriptorBindingKind::Sampled3D,
DescriptorBindingKind::SampledUint1D,
DescriptorBindingKind::SampledUint1DArray,
DescriptorBindingKind::SampledUint2D,
DescriptorBindingKind::SampledUint2DArray,
DescriptorBindingKind::SampledUint2DMsaa,
DescriptorBindingKind::SampledUint2DMsaaArray,
DescriptorBindingKind::SampledUint3D,
DescriptorBindingKind::Storage1D,
DescriptorBindingKind::Storage1DArray,
DescriptorBindingKind::Storage2D,
DescriptorBindingKind::Storage2DArray,
DescriptorBindingKind::Storage3D,
DescriptorBindingKind::StorageUint1D,
DescriptorBindingKind::StorageUint1DArray,
DescriptorBindingKind::StorageUint2D,
DescriptorBindingKind::StorageUint2DArray,
DescriptorBindingKind::StorageUint3D,
};
bool ImageBinding(const ImageResource& image, DescriptorBindingKind& result) {
@@ -36,6 +50,8 @@ bool ImageBinding(const ImageResource& image, DescriptorBindingKind& result) {
case Dimension::Dim1DArray: result = Kind::Sampled1DArray; return true;
case Dimension::Dim2D: result = Kind::Sampled2D; return true;
case Dimension::Dim2DArray: result = Kind::Sampled2DArray; return true;
case Dimension::Dim2DMsaa: result = Kind::Sampled2DMsaa; return true;
case Dimension::Dim2DMsaaArray: result = Kind::Sampled2DMsaaArray; return true;
case Dimension::Dim3D: result = Kind::Sampled3D; return true;
default: return false;
}
@@ -45,6 +61,8 @@ bool ImageBinding(const ImageResource& image, DescriptorBindingKind& result) {
case Dimension::Dim1DArray: result = Kind::SampledUint1DArray; return true;
case Dimension::Dim2D: result = Kind::SampledUint2D; return true;
case Dimension::Dim2DArray: result = Kind::SampledUint2DArray; return true;
case Dimension::Dim2DMsaa: result = Kind::SampledUint2DMsaa; return true;
case Dimension::Dim2DMsaaArray: result = Kind::SampledUint2DMsaaArray; return true;
case Dimension::Dim3D: result = Kind::SampledUint3D; return true;
default: return false;
}
@@ -71,7 +89,7 @@ bool ImageBinding(const ImageResource& image, DescriptorBindingKind& result) {
}
bool CollectValue(const ScalarProvenance& provenance, uint32_t id, std::vector<uint8_t>& visited,
std::set<uint32_t>& registers) {
std::set<uint32_t>& registers) {
if (id <= ScalarProvenance::Unknown) {
return true;
}
@@ -104,7 +122,7 @@ bool CollectValue(const ScalarProvenance& provenance, uint32_t id, std::vector<u
}
bool CollectSource(const Program& program, uint32_t source, bool allow_unknown,
std::vector<uint8_t>& visited, std::set<uint32_t>& registers) {
std::vector<uint8_t>& visited, std::set<uint32_t>& registers) {
if (allow_unknown && source == ScalarProvenance::Unknown) {
return true;
}
@@ -165,8 +183,7 @@ bool CollectUserData(const Program& program, std::vector<uint32_t>& result) {
return false;
}
for (uint32_t i = 0; i < inst.src_count; i++) {
if (!CollectValue(program.provenance, inst.scalar_sources[i], visited,
registers)) {
if (!CollectValue(program.provenance, inst.scalar_sources[i], visited, registers)) {
return false;
}
}
@@ -199,7 +216,7 @@ bool AllocateBindings(Program& program, const BindingLayoutOptions& options, std
if (!program.shader_info_complete || program.binding_layout_complete) {
if (error != nullptr) {
*error = !program.shader_info_complete ? "shader info is not ready"
: "binding layout already allocated";
: "binding layout already allocated";
}
return false;
}
@@ -0,0 +1,323 @@
#include "graphics/shader/recompiler/ir/ReadLaneElimination.h"
#include "graphics/shader/recompiler/ir/SrtWalker.h"
#include <algorithm>
#include <iterator>
#include <map>
#include <set>
#include <utility>
namespace Libs::Graphics::ShaderRecompiler::IR {
namespace {
constexpr uint32_t FirstTemporaryScalarRegister = 128;
struct LaneKey {
uint32_t reg = 0;
uint32_t lane = 0;
auto operator<=>(const LaneKey&) const = default;
};
using LaneSet = std::set<LaneKey>;
bool PairDwordOpcode(Opcode op) {
switch (op) {
case Opcode::MoveU64:
case Opcode::WqmB64:
case Opcode::SaveexecB64:
case Opcode::BitwiseAndU64:
case Opcode::BitwiseAndNotU64:
case Opcode::BitwiseOrU64:
case Opcode::BitwiseOrNotU64:
case Opcode::BitwiseXorU64:
case Opcode::BitwiseNandU64:
case Opcode::BitwiseNorU64:
case Opcode::BitwiseXnorU64:
case Opcode::BitwiseNotU64:
case Opcode::BitFieldMaskU64:
case Opcode::BitFieldExtractU64:
case Opcode::BitReplicateB64B32:
case Opcode::ShiftLeftLogicalU64:
case Opcode::ShiftRightLogicalU64:
case Opcode::SelectU64: return true;
default: return false;
}
}
bool ResolveLane(const Program& program, const Instruction& inst, uint32_t source_index,
uint32_t& lane) {
if (source_index >= inst.src_count || (program.wave_size != 32 && program.wave_size != 64)) {
return false;
}
const auto& selector = inst.src[source_index];
if (selector.kind == OperandKind::ImmediateU32) {
lane = selector.imm % program.wave_size;
return true;
}
uint32_t folded = 0;
if (!FoldScalarConstant(program.provenance, inst.scalar_sources[source_index], folded)) {
return false;
}
lane = folded % program.wave_size;
return true;
}
bool UniformWriteSource(const Instruction& inst) {
if (inst.src_count == 0) {
return false;
}
const auto& source = inst.src[0];
if (source.kind == OperandKind::ImmediateU32 || source.kind == OperandKind::PcRelativeU32) {
return true;
}
return source.kind == OperandKind::Register &&
(source.reg.file == RegisterFile::Scalar || source.reg.file == RegisterFile::Scc ||
source.reg.file == RegisterFile::M0);
}
bool WriteLaneKey(const Program& program, const Instruction& inst, LaneKey& key) {
if (inst.op != Opcode::WriteLaneU32 || inst.dst.kind != OperandKind::Register ||
inst.dst.reg.file != RegisterFile::Vector || !UniformWriteSource(inst)) {
return false;
}
uint32_t lane = 0;
if (!ResolveLane(program, inst, 1, lane)) {
return false;
}
key = {inst.dst.reg.index, lane};
return true;
}
bool ReadLaneKey(const Program& program, const Instruction& inst, LaneKey& key) {
if (inst.op != Opcode::ReadLaneU32 || inst.src_count < 2 ||
inst.src[0].kind != OperandKind::Register || inst.src[0].reg.file != RegisterFile::Vector) {
return false;
}
uint32_t lane = 0;
if (!ResolveLane(program, inst, 1, lane)) {
return false;
}
key = {inst.src[0].reg.index, lane};
return true;
}
void InvalidateRegister(LaneSet& valid, uint32_t reg) {
const auto first = valid.lower_bound({reg, 0});
const auto last = valid.lower_bound({reg + 1u, 0});
valid.erase(first, last);
}
void ApplyInstruction(const Program& program, const Instruction& inst, LaneSet& valid) {
if (inst.op == Opcode::WriteLaneU32 && inst.dst.kind == OperandKind::Register &&
inst.dst.reg.file == RegisterFile::Vector) {
LaneKey key;
if (WriteLaneKey(program, inst, key)) {
valid.insert(key);
return;
}
uint32_t lane = 0;
if (ResolveLane(program, inst, 1, lane)) {
valid.erase({inst.dst.reg.index, lane});
} else {
InvalidateRegister(valid, inst.dst.reg.index);
}
return;
}
if (inst.op == Opcode::MoveRelDestU32 && inst.dst.kind == OperandKind::Register &&
inst.dst.reg.file == RegisterFile::Vector) {
valid.clear();
return;
}
if (inst.dst.kind == OperandKind::Register && inst.dst.reg.file == RegisterFile::Vector) {
uint32_t dwords = std::max(inst.memory.data_dwords, 1u);
if (PairDwordOpcode(inst.op) || inst.op == Opcode::UMadU64U32) {
dwords = std::max(dwords, 2u);
}
for (uint32_t i = 0; i < dwords && inst.dst.reg.index <= UINT32_MAX - i; i++) {
InvalidateRegister(valid, inst.dst.reg.index + i);
}
}
if (inst.dst2.kind == OperandKind::Register && inst.dst2.reg.file == RegisterFile::Vector) {
InvalidateRegister(valid, inst.dst2.reg.index);
}
}
LaneSet TransferBlock(const Program& program, const BasicBlock& block, LaneSet state) {
for (const auto& inst: block.instructions) {
ApplyInstruction(program, inst, state);
}
return state;
}
LaneSet Intersect(const LaneSet& left, const LaneSet& right) {
LaneSet result;
std::set_intersection(left.begin(), left.end(), right.begin(), right.end(),
std::inserter(result, result.end()));
return result;
}
uint32_t NextTemporaryScalarRegister(const Program& program) {
uint32_t next = FirstTemporaryScalarRegister;
const auto consider = [&next](const Operand& operand) {
if (operand.kind == OperandKind::Register && operand.reg.file == RegisterFile::Scalar &&
operand.reg.index >= next && operand.reg.index != UINT32_MAX) {
next = operand.reg.index + 1u;
}
};
for (const auto& block: program.blocks) {
for (const auto& inst: block.instructions) {
consider(inst.dst);
consider(inst.dst2);
for (uint32_t i = 0; i < inst.src_count; i++) {
consider(inst.src[i]);
}
}
}
return next;
}
Operand ScalarRegisterOperand(uint32_t reg) {
Operand operand;
operand.kind = OperandKind::Register;
operand.reg.file = RegisterFile::Scalar;
operand.reg.index = reg;
return operand;
}
Instruction ShadowWrite(const Instruction& write, uint32_t temporary) {
Instruction shadow;
shadow.pc = write.pc;
shadow.op = Opcode::MoveU32;
shadow.dst = ScalarRegisterOperand(temporary);
shadow.src[0] = write.src[0];
shadow.src_count = 1;
return shadow;
}
Instruction ShadowRead(const Instruction& read, uint32_t temporary) {
Instruction rewritten;
rewritten.pc = read.pc;
rewritten.op = Opcode::MoveU32;
rewritten.dst = read.dst;
rewritten.src[0] = ScalarRegisterOperand(temporary);
rewritten.src_count = 1;
return rewritten;
}
} // namespace
ReadLaneEliminationStats EliminateReadLane(Program& program) {
ReadLaneEliminationStats stats;
if (program.blocks.empty() || (program.wave_size != 32 && program.wave_size != 64)) {
return stats;
}
LaneSet universe;
for (const auto& block: program.blocks) {
for (const auto& inst: block.instructions) {
LaneKey key;
if (WriteLaneKey(program, inst, key)) {
universe.insert(key);
}
}
}
if (universe.empty()) {
return stats;
}
const size_t block_count = program.blocks.size();
std::vector<LaneSet> entry(block_count, universe);
std::vector<LaneSet> exit(block_count, universe);
entry[0].clear();
for (size_t block = 0; block < block_count; block++) {
exit[block] = TransferBlock(program, program.blocks[block], entry[block]);
}
bool changed = true;
while (changed) {
changed = false;
for (size_t block_index = 0; block_index < block_count; block_index++) {
LaneSet next_entry;
const auto& block = program.blocks[block_index];
if (block_index != 0 && !block.predecessors.empty()) {
next_entry = universe;
for (const auto predecessor: block.predecessors) {
if (predecessor >= block_count) {
next_entry.clear();
break;
}
next_entry = Intersect(next_entry, exit[predecessor]);
}
}
auto next_exit = TransferBlock(program, block, next_entry);
if (next_entry != entry[block_index] || next_exit != exit[block_index]) {
entry[block_index] = std::move(next_entry);
exit[block_index] = std::move(next_exit);
changed = true;
}
}
}
LaneSet forwarded;
for (size_t block_index = 0; block_index < block_count; block_index++) {
auto state = entry[block_index];
for (const auto& inst: program.blocks[block_index].instructions) {
LaneKey key;
if (ReadLaneKey(program, inst, key) && state.contains(key)) {
forwarded.insert(key);
}
ApplyInstruction(program, inst, state);
}
}
if (forwarded.empty()) {
return stats;
}
std::map<LaneKey, uint32_t> temporaries;
auto next_temporary = NextTemporaryScalarRegister(program);
for (const auto& key: forwarded) {
if (next_temporary == UINT32_MAX) {
return {};
}
temporaries.emplace(key, next_temporary++);
}
for (size_t block_index = 0; block_index < block_count; block_index++) {
const auto original = std::move(program.blocks[block_index].instructions);
auto& rewritten = program.blocks[block_index].instructions;
rewritten.clear();
rewritten.reserve(original.size() + temporaries.size());
auto state = entry[block_index];
for (const auto& inst: original) {
LaneKey read_key;
if (ReadLaneKey(program, inst, read_key) && state.contains(read_key)) {
const auto temporary = temporaries.find(read_key);
if (temporary != temporaries.end()) {
rewritten.push_back(ShadowRead(inst, temporary->second));
stats.rewritten_reads++;
ApplyInstruction(program, inst, state);
continue;
}
}
rewritten.push_back(inst);
LaneKey write_key;
if (WriteLaneKey(program, inst, write_key)) {
const auto temporary = temporaries.find(write_key);
if (temporary != temporaries.end()) {
rewritten.push_back(ShadowWrite(inst, temporary->second));
stats.shadow_writes++;
}
}
ApplyInstruction(program, inst, state);
}
}
return stats;
}
} // namespace Libs::Graphics::ShaderRecompiler::IR
@@ -0,0 +1,20 @@
#ifndef EMULATOR_INCLUDE_EMULATOR_GRAPHICS_SHADER_RECOMPILER_READLANEELIMINATION_H_
#define EMULATOR_INCLUDE_EMULATOR_GRAPHICS_SHADER_RECOMPILER_READLANEELIMINATION_H_
#include "graphics/shader/recompiler/ir/ShaderIR.h"
namespace Libs::Graphics::ShaderRecompiler::IR {
struct ReadLaneEliminationStats {
uint32_t rewritten_reads = 0;
uint32_t shadow_writes = 0;
};
// Replaces fixed-lane ReadLane operations that are reached by a matching WriteLane on every
// control-flow path. A synthetic scalar register snapshots the value at WriteLane execution time,
// so the rewrite remains valid when the source SGPR is subsequently overwritten.
[[nodiscard]] ReadLaneEliminationStats EliminateReadLane(Program& program);
} // namespace Libs::Graphics::ShaderRecompiler::IR
#endif /* EMULATOR_INCLUDE_EMULATOR_GRAPHICS_SHADER_RECOMPILER_READLANEELIMINATION_H_ */
@@ -12,10 +12,11 @@ namespace {
constexpr uint64_t AddressMask = 0x0000ffffffffffffull;
Decoder::ImageDimension DescriptorDimension(const DescriptorValue& descriptor,
Decoder::ImageDimension DescriptorDimension(const DescriptorValue& descriptor,
Decoder::ImageDimension requested) {
const bool is_array = requested == Decoder::ImageDimension::Dim1DArray ||
requested == Decoder::ImageDimension::Dim2DArray;
requested == Decoder::ImageDimension::Dim2DArray ||
requested == Decoder::ImageDimension::Dim2DMsaaArray;
switch (static_cast<Prospero::ImageType>((descriptor.dwords[3] >> 28u) & 0xfu)) {
case Prospero::ImageType::kColor1D: return Decoder::ImageDimension::Dim1D;
case Prospero::ImageType::kColor1DArray:
@@ -26,13 +27,17 @@ Decoder::ImageDimension DescriptorDimension(const DescriptorValue& descrip
case Prospero::ImageType::kColor3D: return Decoder::ImageDimension::Dim3D;
case Prospero::ImageType::kCube: return Decoder::ImageDimension::Dim2DArray;
case Prospero::ImageType::kColor2DArray:
case Prospero::ImageType::kColor2DMsaaArray:
if (is_array) {
return Decoder::ImageDimension::Dim2DArray;
}
return Decoder::ImageDimension::Dim2D;
case Prospero::ImageType::kColor2D:
case Prospero::ImageType::kColor2DMsaa: return Decoder::ImageDimension::Dim2D;
case Prospero::ImageType::kColor2DMsaaArray:
if (is_array) {
return Decoder::ImageDimension::Dim2DMsaaArray;
}
return Decoder::ImageDimension::Dim2DMsaa;
case Prospero::ImageType::kColor2D: return Decoder::ImageDimension::Dim2D;
case Prospero::ImageType::kColor2DMsaa: return Decoder::ImageDimension::Dim2DMsaa;
default: return Decoder::ImageDimension::Unknown;
}
}
@@ -51,8 +56,7 @@ bool ValidImageDescriptor(const DescriptorValue& descriptor) {
const auto base_level = (descriptor.dwords[3] >> 12u) & 0xfu;
const auto fragments = (descriptor.dwords[3] >> 16u) & 0xfu;
const auto max_mip = (descriptor.dwords[5] >> 4u) & 0xfu;
return base_level == 0 && fragments >= 1 && fragments <= 3 &&
max_mip == fragments;
return base_level == 0 && fragments >= 1 && fragments <= 3 && max_mip == fragments;
}
return true;
}
@@ -61,6 +65,11 @@ uint32_t DescriptorImageSwizzle(const DescriptorValue& descriptor) {
return descriptor.dwords[3] & 0xfffu;
}
bool DescriptorIsCube(const DescriptorValue& descriptor) {
return static_cast<Prospero::ImageType>((descriptor.dwords[3] >> 28u) & 0xfu) ==
Prospero::ImageType::kCube;
}
bool DecodeBufferDescriptor(const DescriptorValue& descriptor, ShaderBufferResource& result) {
if (descriptor.dword_count != std::size(result.fields)) {
return false;
@@ -171,12 +180,13 @@ bool ValidateResourceSpecialization(const Program& program, const ResourceSnapsh
const auto& image = program.info.images[i];
const auto& descriptor = snapshot.images[i];
if (NullImageDescriptor(descriptor)) {
bool canonical_kind = image.kind == ResourceKind::Image ||
image.kind == ResourceKind::StorageImage;
bool canonical_kind =
image.kind == ResourceKind::Image || image.kind == ResourceKind::StorageImage;
if (image.atomic) {
canonical_kind = image.kind == ResourceKind::StorageImageUint;
}
if (image.dimension != Decoder::ImageDimension::Dim2D || !canonical_kind) {
if (image.dimension != Decoder::ImageDimension::Dim2D || image.cube ||
!canonical_kind) {
if (error != nullptr) {
*error = fmt::format(
"image descriptor {} no longer matches canonical null specialization", i);
@@ -186,7 +196,8 @@ bool ValidateResourceSpecialization(const Program& program, const ResourceSnapsh
continue;
}
const auto dimension = DescriptorDimension(descriptor, image.dimension);
if (dimension == Decoder::ImageDimension::Unknown || dimension != image.dimension) {
if (dimension == Decoder::ImageDimension::Unknown || dimension != image.dimension ||
DescriptorIsCube(descriptor) != image.cube) {
if (error != nullptr) {
*error =
fmt::format("image descriptor {} no longer matches specialized dimension", i);
@@ -361,6 +372,7 @@ bool SpecializeResources(Program& program, const ResourceSnapshot& snapshot, std
auto& image = next.images[i];
if (NullImageDescriptor(descriptor)) {
image.dimension = Decoder::ImageDimension::Dim2D;
image.cube = false;
switch (image.kind) {
case ResourceKind::ImageUint: image.kind = ResourceKind::Image; break;
case ResourceKind::StorageImageUint:
@@ -386,6 +398,7 @@ bool SpecializeResources(Program& program, const ResourceSnapshot& snapshot, std
return false;
}
image.dimension = descriptor_dimension;
image.cube = DescriptorIsCube(descriptor);
if (image.kind == ResourceKind::StorageImage ||
image.kind == ResourceKind::StorageImageUint) {
image.storage_swizzle = DescriptorImageSwizzle(descriptor);
@@ -402,6 +415,7 @@ bool SpecializeResources(Program& program, const ResourceSnapshot& snapshot, std
std::reference_wrapper<Instruction> inst;
ResourceKind kind;
Decoder::ImageDimension dimension;
bool cube;
};
std::vector<ImagePatch> patches;
for (auto& block: program.blocks) {
@@ -420,13 +434,14 @@ bool SpecializeResources(Program& program, const ResourceSnapshot& snapshot, std
return false;
}
const auto& image = next.images[inst.memory.resource];
patches.push_back({std::ref(inst), image.kind, image.dimension});
patches.push_back({std::ref(inst), image.kind, image.dimension, image.cube});
}
}
program.info = std::move(next);
for (const auto& patch: patches) {
patch.inst.get().memory.kind = patch.kind;
patch.inst.get().memory.image_dimension = patch.dimension;
patch.inst.get().memory.image_cube = patch.cube;
}
return true;
}
@@ -31,7 +31,7 @@ bool ValidateResourceSpecialization(const Program& program, const ResourceSnapsh
// Resolves the immutable dense resource topology against one runtime user-data/SRT snapshot.
// On failure the destination is unchanged.
bool MaterializeResources(const Program& program, const SrtRuntime& runtime,
ResourceSnapshot& snapshot, std::string* error);
ResourceSnapshot& snapshot, std::string* error);
// Applies runtime descriptor shape/format facts to a copied dense topology before layout and
// emission. On failure the program is unchanged.
@@ -84,7 +84,7 @@ uint32_t ByteExtent(const Instruction& inst) {
}
bool ContainsUnknown(const ScalarProvenance& provenance, uint32_t id, std::vector<uint8_t>& visited,
std::vector<uint32_t>& path) {
std::vector<uint32_t>& path) {
path.push_back(id);
if (id <= ScalarProvenance::Unknown || id >= provenance.values.size()) {
return true;
@@ -115,7 +115,7 @@ bool ContainsUnknown(const ScalarProvenance& provenance, uint32_t id, std::vecto
}
bool IsLoopInvariantValue(const ScalarProvenance& provenance, uint32_t id,
std::vector<uint8_t>& visiting) {
std::vector<uint8_t>& visiting) {
if (id <= ScalarProvenance::Unknown || id >= provenance.values.size()) {
return false;
}
@@ -432,6 +432,7 @@ struct MemoryInfo {
bool typed = false;
bool formatted = false;
bool image_has_mip = false;
bool image_cube = false;
bool glc = false;
bool slc = false;
bool idxen = false;
@@ -607,6 +608,7 @@ struct ImageResource {
bool written = false;
bool atomic = false;
bool depth_compare = false;
bool cube = false;
bool operator==(const ImageResource& other) const = default;
};
@@ -677,11 +679,15 @@ enum class DescriptorBindingKind {
Sampled1DArray,
Sampled2D,
Sampled2DArray,
Sampled2DMsaa,
Sampled2DMsaaArray,
Sampled3D,
SampledUint1D,
SampledUint1DArray,
SampledUint2D,
SampledUint2DArray,
SampledUint2DMsaa,
SampledUint2DMsaaArray,
SampledUint3D,
Storage1D,
Storage1DArray,
+14 -14
View File
@@ -530,8 +530,8 @@ bool BuildSrtPlan(Program& program, std::string* error) {
}
bool EvaluateDescriptorSource(const Program& program, uint32_t source, uint32_t use_pc,
const SrtRuntime& runtime, DescriptorValue& result,
std::string* error) {
const SrtRuntime& runtime, DescriptorValue& result,
std::string* error) {
const DescriptorSourceRequest request {source, use_pc};
std::vector<DescriptorValue> results;
if (!EvaluateDescriptorSources(program, std::span {&request, 1}, runtime, results, error)) {
@@ -542,11 +542,11 @@ bool EvaluateDescriptorSource(const Program& program, uint32_t source, uint32_t
}
static bool EvaluateRuntimeSourcesImpl(const Program& program,
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime,
std::vector<DescriptorValue>& results,
std::vector<uint32_t>& flat, bool evaluate_flat,
std::string* error) {
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime,
std::vector<DescriptorValue>& results,
std::vector<uint32_t>& flat, bool evaluate_flat,
std::string* error) {
if (!program.srt_plan_complete) {
if (error != nullptr) {
*error = Diagnostic(program, 0, "SRT plan is not ready");
@@ -602,22 +602,22 @@ static bool EvaluateRuntimeSourcesImpl(const Program&
}
bool EvaluateDescriptorSources(const Program& program,
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime, std::vector<DescriptorValue>& results,
std::string* error) {
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime, std::vector<DescriptorValue>& results,
std::string* error) {
std::vector<uint32_t> ignored;
return EvaluateRuntimeSourcesImpl(program, requests, runtime, results, ignored, false, error);
}
bool EvaluateRuntimeSources(const Program& program,
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime, std::vector<DescriptorValue>& results,
std::vector<uint32_t>& flat, std::string* error) {
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime, std::vector<DescriptorValue>& results,
std::vector<uint32_t>& flat, std::string* error) {
return EvaluateRuntimeSourcesImpl(program, requests, runtime, results, flat, true, error);
}
bool WalkSrt(const Program& program, const SrtRuntime& runtime, std::vector<uint32_t>& flat,
std::string* error) {
std::string* error) {
std::vector<DescriptorValue> ignored;
return EvaluateRuntimeSources(program, {}, runtime, ignored, flat, error);
}
@@ -30,22 +30,22 @@ bool FoldScalarConstant(const ScalarProvenance& provenance, uint32_t value, uint
bool BuildSrtPlan(Program& program, std::string* error);
bool EvaluateDescriptorSource(const Program& program, uint32_t source, uint32_t use_pc,
const SrtRuntime& runtime, DescriptorValue& result,
const SrtRuntime& runtime, DescriptorValue& result,
std::string* error);
// Evaluates one runtime snapshot transactionally. Scalar values and ReadConst results shared by
// several descriptors are memoized once across the batch.
bool EvaluateDescriptorSources(const Program& program,
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime, std::vector<DescriptorValue>& results,
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime, std::vector<DescriptorValue>& results,
std::string* error);
// Evaluates descriptor sources and the flattened immediate SRT with one memoized scalar walk.
// On failure neither destination is changed.
bool EvaluateRuntimeSources(const Program& program,
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime, std::vector<DescriptorValue>& results,
std::vector<uint32_t>& flat, std::string* error);
std::span<const DescriptorSourceRequest> requests,
const SrtRuntime& runtime, std::vector<DescriptorValue>& results,
std::vector<uint32_t>& flat, std::string* error);
bool WalkSrt(const Program& program, const SrtRuntime& runtime, std::vector<uint32_t>& flat,
std::string* error);
+9 -12
View File
@@ -13,8 +13,8 @@
#include "graphics/guest_gpu/graphicsRun.h"
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/shader/recompiler/decompiler/ShaderDecoder.h"
#include "graphics/shader/recompiler/ShaderRecompiler.h"
#include "graphics/shader/recompiler/decompiler/ShaderDecoder.h"
#include "graphics/shader/shaderVertexMetadata.h"
#include "libs/errno.h"
#include "spirv-tools/libspirv.h"
@@ -828,11 +828,10 @@ static void ShaderGetStaticInputInfoPS(
vs_info.stage.program != nullptr && !vs_info.stage.program->bindings.descriptors.empty()
? 1
: 0;
ps_info.push_constant_offset =
vs_info.stage.program != nullptr
? vs_info.stage.program->bindings.push_constant_offset +
vs_info.stage.program->bindings.push_constant_size
: 0;
ps_info.push_constant_offset = vs_info.stage.program != nullptr
? vs_info.stage.program->bindings.push_constant_offset +
vs_info.stage.program->bindings.push_constant_size
: 0;
for (int i = 0; i < 8; i++) {
ps_info.target_output_mode[i] = sh.target_output_mode[i];
@@ -1294,9 +1293,8 @@ static void DumpShaderRecompilerSpirv(const char* type, uint64_t shader_hash,
static std::atomic_int id = 0;
const auto base_name =
Config::GetShaderLogFolder() /
fmt::format("{:04d}_new_shader_{}_{:016x}", id++, type, shader_hash);
const auto base_name = Config::GetShaderLogFolder() /
fmt::format("{:04d}_new_shader_{}_{:016x}", id++, type, shader_hash);
Common::File::CreateDirectories(base_name.parent_path());
Common::File spv_file;
@@ -1345,9 +1343,8 @@ static void DumpShaderRecompilerOriginal(const char* type, uint64_t shader_hash,
static std::atomic_int id = 0;
const auto base_name =
Config::GetShaderLogFolder() / "original" /
fmt::format("{:04d}_new_shader_{}_{:016x}", id++, type, shader_hash);
const auto base_name = Config::GetShaderLogFolder() / "original" /
fmt::format("{:04d}_new_shader_{}_{:016x}", id++, type, shader_hash);
Common::File::CreateDirectories(base_name.parent_path());
Common::File bin_file;

Some files were not shown because too many files have changed in this diff Show More