How To Make PrintScreen Work In Exclusive Fullscreen

Phillip Trudeau-Tavara
TL;DR: You have to do a "Low Level Keyboard Hook" and then write the screenshot function yourself.

Hey there Happenlancers, this is a quick aside technical post about windowing & fullscreen - more devlogs are forthcoming, promise!

I recently read "Fullscreen Exclusive is a Lie" by Mason Remaley and was happy to discover that Happenlance already offers most of the functionality recommended in the conclusion of the article. It's a good read and I recommend you investigate this on your own project before jumping into this quick blog post.

In the article, Mason dives into the ways you can enable, and test for, "exclusive fullscreen" mode -- the only window display mode that bypasses Windows's compositor, or roughly, "allows screen tearing". As it turns out, most tests for exclusivity are wrong, but one correct way to test is to take a screenshot with the PrintScreen key. If the screenshot is wrong, then your game is successfully bypassing the Windows compositor. Thanks, Microsoft!
More importantly, it turns out that there's no foolproof way to enable exclusive fullscreen at all on the average player's setup (Windows 10, one monitor, NVIDIA graphics card). So, any code to try and enable it will be a best-effort attempt (namely setting the window style to WS_POPUP).

SDL2, the library Happenlance uses for windowing, does indeed perform this best-effort attempt, so we were automatically getting exclusive fullscreen on our test machines. However, as a result, we noticed that PrintScreen yielded wrong screenshots (without knowing why), and so I ended up writing code to "make PrintScreen work", whatever it took.

Coincidentally, in the final paragraph, Mason has to weigh the tradeoff between allowing exclusivity for some players vs. letting those players actually use the PrintScreen key to take proper screenshots. Since I accidentally solved (sort of?) this tradeoff, I thought I'd at least propose the solution and post the code that I use currently to implement that solution.

In short, it installs a "low-level keyboard hook", normally used to disable the Windows key (as Mason points out), and catches the PrintScreen key before it does anything. Then, it just flags the game to take a screenshot once it's done rendering a frame and write it to disk, and opts not to propagate the PrintScreen keypress down the line to other programs -- every other program on your computer effectively didn't know there was any key pressed at all. Powerful!

Of course, this took some elbow grease - low level key hooks are known to cause full-machine stalls when debugging, since the keyboard's hook is halted by its own debugger. Plus if you don't run the hook on a separate thread, then lag in your game will stall that hook until the next event pump, which means your entire computer will get delayed key presses -- unacceptable in a commercial game. Lastly, ignoring key releases (which you need to do sometimes) is risky since hooking mid-press will make other apps think the key is still pressed, so keys become "sticky" in a bad way.

What I did was to spawn a simple thread that installs the hook and pumps messages in a green way (use GetMessage, not PeekMessage!) and also opportunistically unhooks itself and exits thread if it detects that a debugger is running. I also made sure to prevent the "sticky-keys" issue by copying some of the SDL2 source code to address it. Lastly, in order to early-out the hook procedure when the window is not focused, I needed to make the main thread update a global variable every frame (technically should be an atomic on less forgiving platforms than x64).

Yuck!!

It's certainly "fun" (a.k.a gross) having to deal with Windows in this way, but the lack of frame hitching from the PrintScreen capture (which again, gives you wrong screenshots anyway) was a win in my personal opinion. Plus, I took the opportunity to capture and ignore the Windows key as well, because screw the Windows key.

Here's the key hook code, verbatim:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
static bool takeScreenshot = false;
static bool hasInputFocus = false;
#ifdef _WIN32
static u8 preHookKeyState[256]; //the docs explicitly say 256
static bool restartHook = false;
static void snapshotHook() {
    GetKeyboardState(preHookKeyState);
    restartHook = true;
}
static LRESULT CALLBACK LowLevelKeyboardProc(int nCode, WPARAM wParam, LPARAM lParam) {
    //you don't want to be hooking the keyboard while debugging...
    if (IsDebuggerPresent() || nCode < 0 || nCode != HC_ACTION) return CallNextHookEx(nullptr, nCode, wParam, lParam);
    if (!hasInputFocus) { //can't use GetFocus() - that's only for the current thread
        snapshotHook();
        return CallNextHookEx(nullptr, nCode, wParam, lParam);
    }

    bool eat = false;
    auto p = (PKBDLLHOOKSTRUCT) lParam;
    switch (wParam) {
        case WM_KEYDOWN:  
        case WM_SYSKEYDOWN:
        case WM_KEYUP:    
        case WM_SYSKEYUP: {
            if (p->vkCode == VK_SNAPSHOT) {
                eat = true;
                //you have to track when the key went up because this receives key-repeats
                static bool lastWasUp = true;
                if (restartHook) lastWasUp = !preHookKeyState[p->vkCode];
                if (wParam == WM_KEYDOWN || wParam == WM_SYSKEYDOWN) {
                    if (lastWasUp) takeScreenshot = true;
                    lastWasUp = false;
                } else if (wParam == WM_KEYUP || wParam == WM_SYSKEYUP) {
                    lastWasUp = true;
                }
            } else if (p->vkCode == VK_LWIN || p->vkCode == VK_RWIN) {
                eat = true;
            }
        } break;
    }
    restartHook = false;

    if (wParam == WM_KEYUP || wParam == WM_SYSKEYUP) {
        //Pass on key releases from pre-hook state (big brain move from SDL source):
        //"If the key was down prior to our hook being installed, allow the
        // key up message to pass normally the first time. This ensures other
        // windows have a consistent view of the key state, and avoids keys
        // being stuck down in those windows if they are down when the grab
        // happens and raised while grabbed."
        if (p->vkCode <= 0xFF && preHookKeyState[p->vkCode]) {
            // print_log("Passing on snapshotted key!\n");
            preHookKeyState[p->vkCode] = 0;
            eat = false;
        }
    }
    return eat? 1 : CallNextHookEx(nullptr, nCode, wParam, lParam);
}
#endif



Installing the hook at program startup (written in C++; converting to C is an exercise for the reader):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
#ifdef _WIN32
        //Install the low-level keyboard hook
        if (!IsDebuggerPresent()) {
            auto hookThread = [](LPVOID) -> DWORD {
                snapshotHook();
                auto hook = SetWindowsHookEx(WH_KEYBOARD_LL, LowLevelKeyboardProc, GetModuleHandle(nullptr), 0);
                //"the thread that installed the hook must have a message loop"
                //https://docs.microsoft.com/en-us/windows/win32/winmsg/lowlevelkeyboardproc
                MSG m;
                while (GetMessage(&m, NULL, 0, 0) > 0) {
                    if (IsDebuggerPresent()) { UnhookWindowsHookEx(hook); break; }
                    TranslateMessage(&m); DispatchMessage(&m);
                }
                return 0;
            };
            CreateThread(NULL, 0, hookThread, NULL, 0, NULL); //ignore if it fails, oh well.
        }
#endif

... "exclusive fullscreen" mode -- the only window display mode that bypasses Windows's compositor

Exclusive fullscreen is not only mode where Windows bypasses compositor for your window. Starting with Windows 8.1 (if I'm not mistaken with version) it automatically bypasses compositor if you have proper pixel format and resolution matches monitor and there are no other elements on screen (popups, system tray, etc...).

And I'm not sure why WS_POPUP would have to do anything with exclusive fullscreen. It never did. Only way to enable exclusive fullscreen is to use Direct3D api's when creating swap chain. In OpenGL you cannot do that.

Anyways "Exclusive Fullscreen" mode is kind obsolete in Win10 world. New DXGI swap chains have FLIP presentation modes that has all the advantages of old exclusive fullscreen modes without any of its drawbacks (changing resolutions, messing up icons on desktop, etc..). This video goes into more details on this: https://www.youtube.com/watch?v=E3wTajGZOsA

And if you want to use these presentation modes in GL, you can with WGL_NV_DX_interop2 extension - use GL to render then DXGI swap chain to present to window.
mmozeiko
Only way to enable exclusive fullscreen is to use Direct3D [...] "Exclusive Fullscreen" mode is kind obsolete in Win10 world

I see. If I understand correctly, then Mason's article and my post were using the phrase "exclusive fullscreen" more flippantly, right? I think we're both just referring to a catch-all term meaning any fullscreen mode that you activate to try to intentionally and explicitly bypass the compositor. I'll edit the blog post to clarify that, thanks!

Also, you're totally right here:

mmozeiko
Starting with Windows 8.1 it automatically bypasses compositor


Thanks for the reminder! If I understand correctly, those are the fullscreen optimizations Microsoft added, and I'll edit the blog post to mention them. Mason decided not to distinguish between FSO and "exclusive fullscreen" (used loosely again) and gives this reason:

I’m aware that there’s technically a distinction [...] but I’m not particularly concerned about that difference. I ran a number of these tests with FSO disabled and saw no discernible difference.


The empirical tests Mason mentions there imply that WS_POPUP is doing something to make composition-bypass happen, and the SDL2 source code appears to prefer this style for fullscreen too. What does it do, if anything? I'd love to hear your input on what that might be!

Thanks for the resources on new DXGI swapchains! I'll take a look at the video, and I'd certainly recommend the new presentation modes to anyone reading this, if you're writing your renderer in a handmade style. For Happenlance specifically, we're mainly leaning on SDL here, so I wouldn't know the first thing about fitting a custom DXGI swapchain into SDL2's OpenGL backend (does SDL2 even use DX12 ever?) and I haven't had the time to look into it. I can't wait for my next project ;)

By the way, Mārtiņš -- I'd also like your input on the key hook stuff! 😅 I hear you're something of a Windows guru around here, so do you know of any ways to improve the hooking or any pitfalls it might trigger? Thanks in advance!
Phillip
The empirical tests Mason mentions there imply that WS_POPUP is doing something to make composition-bypass happen, and the SDL2 source code appears to prefer this style for fullscreen too. What does it do, if anything? I'd love to hear your input on what that might be!


WS_POPUP style simply makes window to not have title area - the one you can hold & drag window around. That's it. Without this style window will include non-client area - which you don't want when you want to avoid compositor, because you want to cover whole monitor with your client area. There's nothing else special about WS_POPUP.

To do GL rendering to D3D swapchain, you would need to ignore GL stuff in SDL. Create plain SDL window, get its HWND handle and then create your own swap chain and after that it is just regular GL & D3D calls.

As for low level window keyboard hooks - sorry, no idea, I've never used them.