Automated Crash Reporting in Basically One 400-Line Function

Too Long; Tell Me The Downsides First

Sure, here you go!

• This covers Windows only. The principles map on to Linux, but the code sample doesn't!

• Spoilers for the rest of the post, if that's the sort of thing you care about --

Disadvantages to having your crash handler be a side-path of your main program's EXE:

• Missing DLLs. They couldn't possibly be caught by this setup -- the crash handler process can't launch if it's the same process that's missing DLLs!
• Static initializers. If you have a very C++-y program with a bunch of global variables with constructors that can crash, then you can launch the process, but you can't safely enter main()!

• Addressing these disadvantages is covered in the article: build the crash hander as a separate EXE.

Okay, on with the full post.

The unexamined app is not worth shipping

As a Windows app developer, you will eventually want to write a crash reporter, so that you can fix bugs in your program without relying on users to report them. Compared to just crossing your fingers and hoping that your sprawling test suites cover everything, I feel like the idea of just collecting crashes from real runs of the program experienced by real users is a more greasy-wrench-engineering-y way to fix glitches, even if you already do a lot of testing. You want something with maximal coverage of the surface area your program actually takes up on a user's computer, no matter what is causing the crash. How do you do that?

Last year, while working on Happenlance, I didn't know. The impression I got online was that I should do something along these lines:

Call SetUnhandledExceptionFilter at the start of your program, passing in a callback function that calls MiniDumpWriteDump on the running process.

Here are the qualms I had with that:

• In my experience, calling SetUnhandledExceptionFilter at program startup somehow didn’t catch all exceptions. Was this my fault? Was it a thread thing? Was it a DLL thing? I don't know! I'm sure it depends highly on what the game code looked like at the time.
• In the same vein, I was never able to call MiniDumpWriteDump in-process in a way that produced minidumps with parseable stack traces, regardless of whether the program was built using Clang or MSVC. Even WinDBG couldn’t analyze any stack trace(!!!).
• Even if SetUnhandledExceptionFilter does catch all exceptions for you, and MiniDumpWriteDump does work for you when called in-process... the unhandled exception filter apparently doesn't suspend other running threads. I believe this means a minidump produced in this way will not accurately reflect the state of the threads at the time of the unhandled exception. I think you can even get multiple threads minidumping the process at the same time. Or maybe even a deadlock!

The crappy version

Due to what I saw to be the situation there, instead of minidumps I shipped the game with just a basic wrapper block that ran main inside an SEH handler. The handler, which is also what assert() got defined to, would just call RtlStackBackTrace followed by a loop of SymGetAddr64 calls to achieve a crappy text stacktrace (not even with a log file!):

static bool stackdump(const char * name, EXCEPTION_POINTERS * ep) {
    if (!file_exists("Happenlance.pdb")) return false; // YUCK!

    void *stack[512];
    int stack_count = RtlCaptureStackBackTrace(0, 512, stack, 0);

    // ... SymGetAddr64 loop, etc ...

}
void __attribute__((noreturn)) (dumping_assert)(EXCEPTION_POINTERS * ep) {
    MessageBoxA(nullptr,
                "Once you click OK, the game will show you a crash dump file.\n\n"
                "Please post this file to the Discord server and write down what happened before the crash.\n\n",
                "Critical Error", MB_OK | MB_ICONERROR | MB_SYSTEMMODAL);

    // ... etc ...

    if (!stackdump("CrashDump.dmp", ep)) print_log("failed to stack dump\n"); //¯\_(ツ)_/¯
    if (file_exists("CrashDump.dmp")) view_in_system_file_browser("CrashDump.dmp");

    // ... etc ...

    ExitProcess(-1);
    TerminateProcess(GetCurrentProcess(), -1);
}
int WINAPI CALLBACK WinMain(HINSTANCE, HINSTANCE, LPSTR, int) {
    __try {
        return game_main(__argc, __argv);
    } __except(unhandled_handler(GetExceptionInformation())) { // Needed because for some reason SetUnhandledExceptionFilter isn't working???
        return -1;
    }
}
static int game_main(int argc, char ** argv) {
    SetUnhandledExceptionFilter(unhandled_handler); // Seems not to work?
    SymInitialize(GetCurrentProcess(), NULL, TRUE); // Needed??

    // ... etc ...

This meant we had to ship the PDB, which sucks, but more importantly, it was torturously bad to try to pinpoint people's glitches. There was essentially no avenue of attack. But it was the only way I could see any stack trace at all! How can this be improved?

The better version

Well, Microsoft says the ONLY safe place to call MiniDumpWriteDump is outside the process, and I found that calling the function from another process did indeed generate the dumps I was looking for, so we'll need to use another process to get dumps. Multiple processes isn't ideal, but what can I say -- we're finally seeing stack traces with local variables!

I also came across this great article by Mason Remaley, How To Write a Crash Reporter, in which the author sets up their app as a self-calling self-watchdog process. If we need to generate the minidump from outside the main process, then we might as well set up the app as described in this article, since it's cleaner than writing and deploying a separate executable alongside the main one.

One thing, though. In the article, the watchdog waits until the subprocess exits and checks its exit code/error code to determine whether an error occurred. However, if you just wait for an error code, the process has already exited or terminated! I am pretty sure that the most accurate, most informative minidump is going to come from intercepting the exception that caused the crash the moment it happens.

In addition to intercepting the exception, you also need read permission on the process's memory in order to be able to generate the minidump in the first place.

These two requirements led me to the conclusion that, well, we need to call CreateProcess with DEBUG_ONLY_THIS_PROCESS... and we need to write a debugger loop in the crash handler!

Basic outline

With that established, here's how the crash handler could work:

• Create the subprocess and debug it in a loop.
• If the subprocess exited:
  • If the process's exit code was 0, exit quietly.
  • Otherwise, display a Yes/No dialog box asking the user whether they want to restart the app.
• Handle last-chance exceptions by extracting the exception code, building an EXCEPTION_POINTERS out of the thread context and then calling MiniDumpWriteDump.
• By this point, a dump is generated, but if the game is fullscreen then there is still a window covering the screen, and if there is a massive memory leak then none of that is free yet. The crash handler process needs to terminate the subprocess and detach (important!) once the minidump is written to free up this space.
• After that, show a message box explaining that a crash happened and prompting to automatically upload the crash dump.
• If the user clicks "No" to auto-uploading:
  • Select the crash dump file in Windows Explorer.
  • Show another message box asking them to email the file to us, and prompt the user to restart the app.
• If the user clicks "Yes" to auto-uploading:
  • Create a progress bar window.
  • Perform a multipart-form HTTP POST request to the Discord webhook. (As explained in the article linked above.)
  • Prompt the user to retry if the request fails, and try again if they say yes, in an infinite loop.
  • Once the request succeeds or is cancelled, prompt the user to restart the app.
• Remember to handle all error conditions along the way - can’t open file, can’t terminate, can’t write minidump, can’t show messagebox! :)

The nice thing about this algorithm is that there's no reason it can't be implemented in basically just one function. All you have to do to get crash reporting is to call do_crash_handler at the top of your main(). If the crash handler detects that this process is already the subprocess, then it will return without doing anything, and your main app continues running none the wiser. Neat!

Nothing comes with no strings attached

Here's some things to keep in mind:

• The best way for your app to crash is now __debugbreak(). Neither abort() nor exit(1) allow the crash handler to generate a dump, which means no stack information. This might feel like opposite day to you!
• Sadly, any sufficiently complex program is going to behave differently when it is being debugged. There is a Windows function, IsDebuggerPresent(), that facilitates this naughty trend. In Happenlance's crash reporter, we specifically trick the subprocess into thinking that it is not being debugged, so that IsDebuggerPresent() returns false. We only do this for Steam API load times, but you might want this for various reasons. (You might even want to not trick it, because you might prefer consistency between your debugged build and your deployed build!)
• When you debug your program in a debugger, make sure to pass -no-crash-handler in the command line, so you're debugging the actual program instead of debugging the crash handler.
• You probably want to store crash dump files in %APPDATA%/YourAppName/, so set that file path up before calling do_crash_handler.
• In Happenlance, we set the window icon for all the process's windows using a global hook. Do that before entering the crash handler so that the crash handler's dialogs have your window icon.
• Make sure to only enable DPI awareness after entering the crash handler, so it applies to the subprocess but not the watchdog process. If the crash handler says it's DPI-aware, then the progress bar window will appear too small on high-DPI displays. I suppose you could also just do the extra work to render the progress bar with DPI-accurate scaling, but I didn't do that!
• Remember that when you debug crashdumps in Visual Studio, you are going to need the exact PDBs associated with your app, and sometimes extra info on third-party loaded modules - including symbols for your graphics drivers, and any antivirus programs or game overlays hooking into your program!

Disadvantages to having your crash handler be a side-path of your main program's EXE:

• Missing DLLs. They couldn't possibly be caught by this setup -- the crash handler process can't launch!
• Static initializers. If you have a very C++-y program with a bunch of global variables with constructors that can crash, then you can launch the process, but you can't safely enter main()!

Thankfully, those disadvantages can be addressed by just factoring do_crash_handler into a separate program -- presumably the "entry program" from a UX perspective -- but at that point, it's your job to evaluate the pros and cons of that tradeoff in your specific case yourself.

Additional concerns not addressed here:

• App store certification. I have no idea how the Microsoft Store feels about a bunch of Win32 dialogs in your store app. Nor do I know about the Epic Games Store. How does Valve feel about your Steam game popping up native window dialogs that don't support Steam Controller input? Careful!
• Internationalization. How will you get translations of all the error messages? How will you determine what language to display? (GetUserDefaultUILanguage() presumably -- but what if it's a Steam game and Steam is overriding with a different language? Will you be loading the Steam API in your crash handler!? How long will that take? How crash-prone is that?)
• Privacy regulations. What personally identifiable information will your minidumps contain? What does your country ask that you inform the user of before uploading the dump?

Room for improvement that I haven't done here:

• Piping the subprocess's stdout+stderr/print_log() output to files or named pipes accessible and uploaded by the crash handler.
• Piping a profiling trace from the subprocess into a file or named pipe accessible and uploaded by the crash handler.
• Showing the crash dialog windows on the same monitor as the game instead of always on the primary monitor.
• Zipping the crash dumps instead of uploading them raw.
• Take a screenshot at some interval to a shared memory region to show the general scenario of the crash.
• A more generalized feedback form that lets users type what happened, include the screenshot, etc. before uploading the crash (or just upload the feedback if there was no crash!).
• Threaded/async progress bar that doesn't block window message processing while the HTTP request is going through.

Just show me the code, everything you said is explained faster by code!

Here is a direct rip of the crash handler function as used in Happenlance. It's called around the top of main after console attaching, appdata path, and window icons have been set up. To be able to link it, you'll need to pass in WinHTTP.lib, DbgHelp.lib, Comctl32.lib, and maybe a few other libraries. Implementing the handful of functions and variables not defined here is left as an exercise for the reader. (I have been strapped for time lately, so forgive me for not factoring out an entirely independent crash handler library for this post -- I may write a nice C89 library based on this code at some point in the future!)

#define DISCORD_WEBHOOK_ENDPOINT "/api/webhooks/<your_server_id>/<your_webhook_id>"

#define ErrBox(msg, flags) MessageBoxW(nullptr, L"" msg, L"Fatal Error", flags | MB_SYSTEMMODAL | MB_SETFOREGROUND)
#define fatal_init_error(s) ErrBox("The game had a fatal error and must close.\n\"" s "\"\nPlease report this to [email protected] or the Happenlance Discord.", MB_OK | MB_ICONERROR)

// TODO: Internationalization for error messages.
void do_crash_handler() {

    // Get the command line
    int argc = 0;
    wchar_t *cmd = GetCommandLineW();
    if (!cmd || !cmd[0]) {
        return; // Error: just run the app without a crash handler.
    }
    wchar_t **wargv = CommandLineToArgvW(cmd, &argc); // Passing nullptr here crashes!
    if (!wargv || !wargv[0]) {
        return; // Error: just run the app without a crash handler.
    }

    // Parse the command line for -no-crash-handler
    bool crashHandler = true;
    for (int i = 0; i < argc; ++i) {
        if (!wcscmp(wargv[i], L"-no-crash-handler")) {
            crashHandler = false;
        }
    }
    if (!crashHandler) { // We already *are* the subprocess - continue with the main program!
        return;
    }

    // Concatenate -no-crash-handler onto the command line for the subprocess
    int cmdLen = 0;
    while (cmd[cmdLen]) { // could use wcslen() here, but Clang ASan's wcslen() can be bugged sometimes
        cmdLen++;
    }
    const wchar_t * append = L" -no-crash-handler";
    int appendLen = 0;
    while (append[appendLen]) {
        appendLen++;
    }
    wchar_t *cmdNew = (wchar_t *)calloc(cmdLen + appendLen + 1, sizeof(wchar_t)); // @Leak
    if (!cmdNew) {
        return; // Error: just run the app without a crash handler.
    }
    memcpy(cmdNew, cmd, cmdLen * sizeof(wchar_t));
    memcpy(cmdNew + cmdLen, append, appendLen * sizeof(wchar_t));

    // Crash handler loop: run the program until it succeeds or the user chooses not to restart it
    restart:;

    // Parameters for starting the subprocess
    STARTUPINFOW siw = {};
    siw.cb = sizeof(siw);
    siw.dwFlags = STARTF_USESTDHANDLES;
    siw.hStdInput = GetStdHandle(STD_INPUT_HANDLE); // @Leak: CloseHandle()
    siw.hStdOutput = GetStdHandle(STD_ERROR_HANDLE);
    siw.hStdError = GetStdHandle(STD_OUTPUT_HANDLE);
    PROCESS_INFORMATION pi = {}; // @Leak: CloseHandle()

    // Launch suspended, then read-modify-write the PEB (see below), then resume -p 2022-03-04
    if (!CreateProcessW(nullptr, cmdNew, nullptr, nullptr, true,
                        CREATE_SUSPENDED | DEBUG_ONLY_THIS_PROCESS, nullptr, nullptr, &siw, &pi)) {
        // If we couldn't create a subprocess, then just run the program without a crash handler.
        // That's not great, but it's presumably better than stopping the user from running at all!
        return;
    }

    // NOTE: SteamAPI_Init() takes WAY longer On My Machine(tm) when a debugger is present.
    //       (The DLL file steam_api64.dll does indeed call IsDebuggerPresent() sometimes.)
    //       It's clear that Steam does extra niceness for us when debugging, but we DO NOT
    //       want this to destroy our load times; I measure 3.5x slowdown (0.6s -> 2.1s).
    //       The only way I know to trick the child process into thinking it is free of a
    //       debugger is to clear the BeingDebugged byte in the Process Environment Block.
    //       If we are unable to perform this advanced maneuver, we will gracefully step back
    //       and allow Steam to ruin our loading times. -p 2022-03-04
    auto persuade_process_no_debugger_is_present = [] (HANDLE hProcess) {

        // Load NTDLL
        HMODULE ntdll = LoadLibraryA("ntdll.dll");
        if (!ntdll) return;

        // Get NtQueryInformationProcess function
        auto NtQueryInformationProcess =
            (/*__kernel_entry*/ NTSTATUS (*)(HANDLE, PROCESSINFOCLASS, PVOID, ULONG, PULONG))
                GetProcAddress(ntdll, "NtQueryInformationProcess");
        if (!NtQueryInformationProcess) return;

        // Query process information to find the PEB address
        PROCESS_BASIC_INFORMATION pbi = {};
        DWORD queryBytesRead = 0;
        if (NtQueryInformationProcess(hProcess, ProcessBasicInformation, &pbi, sizeof(pbi), &queryBytesRead) != 0
            || queryBytesRead != sizeof(pbi)) return;

        // Read the PEB of the child process
        PEB peb = {};
        SIZE_T processBytesRead = NULL;
        if (!ReadProcessMemory(hProcess, pbi.PebBaseAddress, &peb, sizeof(peb), &processBytesRead)
            || processBytesRead != sizeof(peb)) return;
        print_log("Child process's peb.BeingDebugged is %d, setting to 0...\n", peb.BeingDebugged);

        // Gaslight the child into believing we are not watching
        peb.BeingDebugged = 0;

        // Write back the modified PEB
        SIZE_T processBytesWritten = NULL;
        if (!WriteProcessMemory(hProcess, pbi.PebBaseAddress, &peb, sizeof(peb), &processBytesWritten)
            || processBytesWritten != sizeof(peb)) return;
    };
    persuade_process_no_debugger_is_present(pi.hProcess);

    // Helper function to destroy the subprocess
    auto exit_child = [&] {
        TerminateProcess(pi.hProcess, 1); // Terminate before detaching, so you don't see Windows Error Reporting.
        DebugActiveProcessStop(GetProcessId(pi.hProcess)); // Detach
        WaitForSingleObject(pi.hProcess, 2000); // Wait for child to die, but not forever.
    };

    // Kick off the subprocess
    if (ResumeThread(pi.hThread) != 1) {
        exit_child();
        fatal_init_error("Could not start main game thread");
        ExitProcess(1); // @Note: could potentially "return;" here instead if you wanted.
    }

    // Debugger loop: catch (and ignore) all debug events until the program exits or hits a last-chance exception
    char * filename = nullptr;
    HANDLE file = nullptr;
    for (;;) {

        // Get debug event
        DEBUG_EVENT de = {};
        if (!WaitForDebugEvent(&de, INFINITE)) {
            exit_child();
            fatal_init_error("Waiting for debug event failed");
            ExitProcess(1);
        }

        // If the process exited, nag about failure, or silently exit on success
        if (de.dwDebugEventCode == EXIT_PROCESS_DEBUG_EVENT && de.dwProcessId == pi.dwProcessId) {

            // If the process exited unsuccessfully, prompt to restart it
            // @Todo: in these cases, no dump can be made, so upload just the stdout log and profiling trace
            if (de.u.ExitThread.dwExitCode != 0) {

                // Terminate & detach just to be safe
                exit_child();

                // Prompt to restart
                MessageBeep(MB_ICONINFORMATION); // MB_ICONQUESTION makes no sound
                if (MessageBoxW(nullptr,
                                L"The game had a fatal error and must close.\n"
                                "Unfortunately, a crash report could not be generated. Sorry!\n"
                                "Please report this to [email protected] or the Happenlance Discord.\n"
                                "Restart the game?\n", L"Fatal Error",
                                MB_YESNO | MB_ICONQUESTION | MB_SYSTEMMODAL | MB_SETFOREGROUND) == IDYES) {
                    goto restart;
                }

            }

            // Bubble up the failure code - this is where successful program runs will end up!
            ExitProcess(de.u.ExitThread.dwExitCode);
        }

        // If the process had some other debug stuff, we don't care.
        if (de.dwDebugEventCode != EXCEPTION_DEBUG_EVENT) {
            ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_CONTINUE);
            continue;
        }

        // Skip first-chance exceptions or exceptions for processes we don't care about (shouldn't ever happen).
        if (de.u.Exception.dwFirstChance || de.dwProcessId != GetProcessId(pi.hProcess)) {
            ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_EXCEPTION_NOT_HANDLED);
            continue;
        }

        // By here, we have hit a real, last-chance exception. This is a crash we should generate a dump for.
        #define crash_report_failure(str) \
            ErrBox( \
                "The game had a fatal error and must close.\n" \
                "A crash report could not be produced:\n\"" str "\"\n" \
                "Please report this to [email protected] or the Happenlance Discord.", \
                MB_OK | MB_ICONERROR);

        // Create crash dump directory
        if (!create_dir_if_not_exist(appdataRoamingFolder)) {
            exit_child();
            crash_report_failure("The crash report directory could not be created. Sorry!");
            ExitProcess(1);
        }

        // Create crash dump filename
        filename = mprintf("%sCrashDump_%d.dmp", appdataRoamingFolder, (int) time(NULL)); //@Leak
        if (!filename) {
            exit_child();
            crash_report_failure("The crash report filename could not be built. Sorry!");
            ExitProcess(1);
        }

        // Convert filename to UTF-16
        wchar_t *wfilename = utf8_to_utf16(filename);
        defer { free(wfilename); };
        if (!wfilename) {
            exit_child();
            crash_report_failure("The crash report filename could not be converted. Sorry!");
            ExitProcess(1);
        }

        // Create crash dump file
        file = CreateFileW(wfilename, GENERIC_WRITE | GENERIC_READ, 0, nullptr,
                                      CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, nullptr);
        if (file == INVALID_HANDLE_VALUE) {
            exit_child();
            crash_report_failure("The crash dump file could not be created. Sorry!");
            ExitProcess(1);
        }

        // Generate exception pointers out of excepting thread context
        CONTEXT c = {};
        if (HANDLE thread = OpenThread(THREAD_ALL_ACCESS, true, de.dwThreadId)) {
            c.ContextFlags = CONTEXT_ALL;
            GetThreadContext(thread, &c);
            CloseHandle(thread);
        }
        EXCEPTION_POINTERS ep = {};
        ep.ExceptionRecord = &de.u.Exception.ExceptionRecord;
        ep.ContextRecord = &c;
        MINIDUMP_EXCEPTION_INFORMATION mei = {};
        mei.ThreadId = de.dwThreadId;
        mei.ExceptionPointers = &ep;
        mei.ClientPointers = false;

        // You could add some others here, but these should be good.
        int flags = MiniDumpNormal
                  | MiniDumpWithHandleData
                  | MiniDumpScanMemory
                  | MiniDumpWithUnloadedModules
                  | MiniDumpWithProcessThreadData
                  | MiniDumpWithThreadInfo
                  | MiniDumpIgnoreInaccessibleMemory;

        // Write minidump
        if (!MiniDumpWriteDump(pi.hProcess, GetProcessId(pi.hProcess), file,
                               (MINIDUMP_TYPE)flags, &mei, nullptr, nullptr)) {
            exit_child();
            crash_report_failure("The crash dump could not be written. Sorry!");
            ExitProcess(1);
        }

        // @Todo: ZIP compress the crash dump files here, with graceful fallback to uncompressed dumps.

        // Cleanup: Destroy subprocess now that we have a dump.
        // Note that we want to do this before doing any blocking interface dialogs,
        // because otherwise you would leave an arbitrarily broken program lying around
        // longer than you need to.
        exit_child();
        break;
    }

    // Prompt to upload crash dump
    int res = 0;
    bool uploaded = false;
    if (!(res = ErrBox(
        "The game had a fatal error and must close.\n"
        "Send anonymous crash report?\n"
        "This will go directly to the developers on Discord,\n"
        "and help fix the problem.",
        MB_YESNO | MB_ICONERROR))) ExitProcess(1);

    // Upload crash dump
    if (res == IDYES) {

        // Setup window class for progress window
        WNDCLASSEXW wcex = { sizeof(wcex) };
        wcex.style = CS_HREDRAW | CS_VREDRAW;
        wcex.lpszClassName = L"bar";
        wcex.hbrBackground = (HBRUSH)(COLOR_WINDOW);
        wcex.hCursor = LoadCursor(GetModuleHandleA(nullptr), IDC_ARROW);
        wcex.lpfnWndProc = [] (HWND h, UINT m, WPARAM w, LPARAM l) -> LRESULT {
            return m == WM_QUIT || m == WM_CLOSE || m == WM_DESTROY? 0 : DefWindowProcW(h, m, w, l);
        };
        wcex.hInstance = GetModuleHandleA(nullptr);
        if (!RegisterClassExW(&wcex)) {
            ExitProcess(1);
        }
        HWND hWnd = nullptr;
        HWND ctrl = nullptr;
        
        // Initialize common controls for progress bar
        INITCOMMONCONTROLSEX iccex = { sizeof(iccex) };
        iccex.dwICC = ICC_PROGRESS_CLASS;
        if (InitCommonControlsEx(&iccex)) {
            
            // Create progress window and progress bar child-window
            hWnd = CreateWindowExW(0, wcex.lpszClassName, L"Uploading...",
                WS_SYSMENU | WS_CAPTION | WS_VISIBLE, CW_USEDEFAULT, SW_SHOW,
                320, 80, nullptr, nullptr, GetModuleHandleA(nullptr), nullptr);
            ctrl = CreateWindowExW(0, PROGRESS_CLASSW, L"",
                WS_CHILD | WS_VISIBLE | PBS_SMOOTH, 10, 10,
                280, 20, hWnd, (HMENU)12345, GetModuleHandleA(nullptr), nullptr);

        } else {
            ExitProcess(1);
        }

        // Infinite loop: Attempt to upload the crash dump until the user cancels or it succeeds
        do {

            // Position the progress window to the centre of the screen
            RECT r; GetWindowRect(hWnd, &r);
            int ww = r.right - r.left, wh = r.bottom - r.top;
            int sw = GetSystemMetrics(SM_CXSCREEN), sh = GetSystemMetrics(SM_CYSCREEN);
            SetWindowPos(hWnd, HWND_TOP, (sw - ww) / 2, (sh - wh) / 2, 0, 0, SWP_NOSIZE);

            // Helper function to set the loading bar to a certain position.
            auto update_loading_bar = [&] (float amt) {
                if (hWnd && ctrl) {
                    SendMessageW(ctrl, PBM_SETPOS, (WPARAM)(amt * 100), 0);
                    ShowWindow(hWnd, SW_SHOW);
                    UpdateWindow(hWnd);
                    MSG msg = {};
                    while (PeekMessageW(&msg, nullptr, 0, 0, 1) > 0) {
                        TranslateMessage(&msg);
                        DispatchMessageW(&msg);
                    }
                }
            };


            auto try_upload = [&] () -> bool {
                float x = 0;
                update_loading_bar(x);

                // Build MIME multipart-form payload
                static char body[1 << 23]; //ouch that's a big static buffer!!!
                const char * bodyPrefix =
                    "--19024605111143684786787635207\r\n"
                    "Content-Disposition: form-data; name=\"payload_json\"\r\n\r\n{\"content\":\""
                    "Happenlance "
#if DEMO
                    "Demo "
#endif
                    APP_VERSION_STRING " Anonymous Crash Report"
                    "\"}\r\n--19024605111143684786787635207\r\n"
                    "Content-Disposition: form-data; name=\"files[0]\"; filename=\"";
                const char * bodyInfix = "\"\r\n"
                    "Content-Type: application/octet-stream\r\n"
                    "\r\n";
                const char * bodyPostfix = "\r\n--19024605111143684786787635207--\r\n";

                // Printf the prefix, filename, infix
                int headerLen = snprintf(body, sizeof(body), "%s%s%s", bodyPrefix, filename, bodyInfix);
                if (headerLen != strlen(bodyPrefix) + strlen(filename) + strlen(bodyInfix)) return false;
                update_loading_bar(x += 0.1f);

                // Get crash dump file size
                LARGE_INTEGER fileSizeInt = {};
                GetFileSizeEx(file, &fileSizeInt);
                uint64_t fileSize = fileSizeInt.QuadPart;
                if (fileSize >= 8000000) return false; //discord limit
                int bodyLen = headerLen + fileSize + strlen(bodyPostfix);
                if (bodyLen >= sizeof(body)) return false; //buffer overflow
                update_loading_bar(x += 0.1f);

                // Seek file to start
                if (SetFilePointer(file, 0, nullptr, FILE_BEGIN) != 0) return false;

                // Copy entire file into the space after the body infix
                DWORD bytesRead = 0;
                if (!ReadFile(file, body + headerLen, fileSize, &bytesRead, nullptr)) return false;
                if (bytesRead != fileSize) return false;
                update_loading_bar(x += 0.1f);

                // Print the body postfix after the data file (overflow already checked)
                strcpy(body + headerLen + fileSize, bodyPostfix);
                update_loading_bar(x += 0.1f);

                // Windows HTTPS stuff from here on out...
                HINTERNET hSession = WinHttpOpen(L"Discord Crashdump Webhook",
                    WINHTTP_ACCESS_TYPE_DEFAULT_PROXY, WINHTTP_NO_PROXY_NAME,
                    WINHTTP_NO_PROXY_BYPASS, 0);
                if (!hSession) return false;
                defer { WinHttpCloseHandle(hSession); };
                update_loading_bar(x += 0.1f);

                // Connect to domain
                HINTERNET hConnect = WinHttpConnect(hSession, L"discord.com", INTERNET_DEFAULT_HTTPS_PORT, 0);
                if (!hConnect) return false;
                defer { WinHttpCloseHandle(hConnect); };
                update_loading_bar(x += 0.1f);

                // Begin POST request to the discord webhook endpoint
                HINTERNET hRequest = WinHttpOpenRequest(hConnect, L"POST",
                    L"" DISCORD_WEBHOOK_ENDPOINT,
                    nullptr, WINHTTP_NO_REFERER, WINHTTP_DEFAULT_ACCEPT_TYPES, WINHTTP_FLAG_SECURE);
                if (!hRequest) return false;
                defer { WinHttpCloseHandle(hRequest); };
                update_loading_bar(x += 0.1f);

                // Send request once - don't handle auth challenge, credentials, reauth, redirects
                const wchar_t ContentType[] = L"Content-Type: multipart/form-data; boundary=19024605111143684786787635207";
                if (!WinHttpSendRequest(hRequest, ContentType, ARR_SIZE(ContentType),
                    body, bodyLen, bodyLen, 0)) return false;
                update_loading_bar(x += 0.1f);

                // Wait for response
                if (!WinHttpReceiveResponse(hRequest, nullptr)) return false;
                update_loading_bar(x += 0.1f);

                // Pull headers from response
                DWORD dwStatusCode, dwSize = sizeof(dwStatusCode);
                if (!WinHttpQueryHeaders(hRequest,
                    WINHTTP_QUERY_STATUS_CODE | WINHTTP_QUERY_FLAG_NUMBER,
                    nullptr, &dwStatusCode, &dwSize, nullptr)) return false;
                if (dwStatusCode != 200) return false;
                update_loading_bar(x += 0.1f);

                return true;
            };
            res = 0;
            uploaded = try_upload();
            if (!uploaded) {
                if (!(res = MessageBoxW(hWnd, L"Sending failed. Retry?", L"Fatal Error", MB_RETRYCANCEL | MB_ICONWARNING | MB_SYSTEMMODAL))) ExitProcess(1);
            }
        } while (res == IDRETRY);

        // Cleanup
        if (hWnd) {
            DestroyWindow(hWnd);
        }
        UnregisterClassW(wcex.lpszClassName, GetModuleHandleA(nullptr));
    }

    // Cleanup
    CloseHandle(file);

    // Prompt to restart
    MessageBeep(MB_ICONINFORMATION); // MB_ICONQUESTION makes no sound
    if (MessageBoxW(nullptr, uploaded?
                    L"Thank you for sending the crash report.\n"
                    "You can email more info to [email protected] or\n"
                    "the Happenlance Discord.\n"
                    "Restart the game?\n" :
                    view_file_in_system_file_browser(filename)?
                    L"The crash report folder has been opened.\n"
                    "You can email the file to [email protected] or\n"
                    "send it to the Happenlance Discord.\n"
                    "Restart the game?\n" :
                    L"The crash report can be found in the program installation directory.\n"
                    "You can email the file to [email protected] or\n"
                    "send it to the Happenlance Discord.\n"
                    "Restart the game?\n", L"Fatal Error",
                    MB_YESNO | MB_ICONQUESTION | MB_SYSTEMMODAL | MB_SETFOREGROUND) == IDYES) {
        goto restart;
    }

    // Return 1 because the game crashed, not because the crash report failed
    ExitProcess(1);
}