Modifying Windows System Menus in wxWidgets

System menus are those standard menus that open when you click an application’s icon in the top left of its title bar.

A System Menu opened for the Amazon Music player. It was invoked by left-clicking the title bar icon.

There doesn’t seem to be a standard way to modify and respond to them in wxWidgets, so I’m going to go over what I did in my wxWidgets WindowsOS applications to modify the System Menu.

This solution isn’t going to be portable. So this is only for Windows UIs.

My application using wxWidgets on Windows with a modified System Menu.

Modifying the app’s System Menu

The first thing to do is to add extra System Menu options to your application window. This is done by getting the wxWindow’s HWND and calling Windows API functions on it. This only works for windows derived from wxTopLevelWindow that were created with the wxSYSTEM_MENU flag (which is a part of wxDEFAULT_FRAME_STYLE).

For the menu above, my top-level window is called TopDockWin, which derives from a wxFrame, and I modify the menu in the constructor. The changes are persistent, so this only needs to be done once (per top-level window).

TopDockWin::TopDockWin(const wxPoint& pos, const wxSize& size)
    :   wxFrame(NULL, wxID_ANY, "PrecisionDock", pos, size) // Flags param value not specified, so defaulted to wxDEFAULT_FRAME_STYLE
{
    // Get the System Menu
    HMENU sysMenu = (HMENU)GetSystemMenu(this->GetHWND(), FALSE);
    
    if (sysMenu != NULL)
    {
        // Counter to insert entries at the top. To append to the bottom use:
        // InsertMenu(..., -1, MF_BYPOSITION | MF_POPUP, ...,  TEXT("..."));
        int menuPosIns = 0; 
    
        // Create, populate and add the "Manage All" submenu
        HMENU subMenuManip = CreatePopupMenu();
        if (subMenuManip != NULL)
        {
            // Add the submenu to the System Menu
            InsertMenu(sysMenu, menuPosIns, MF_BYPOSITION | MF_POPUP, (UINT_PTR)subMenuManip, TEXT("Manage All"));
            ++menuPosIns;
            //
            // Add submenu entries
            InsertMenu(subMenuManip, -1, MF_BYPOSITION, (int)CMDID::ReleaseAll,  TEXT("Release All"));
            InsertMenu(subMenuManip, -1, MF_BYPOSITION, (int)CMDID::DetachAll,   TEXT("Detach All All"));
            InsertMenu(subMenuManip, -1, MF_BYPOSITION, (int)CMDID::CloseAll,    TEXT("Close All"));
            InsertMenu(subMenuManip, -1, MF_BYPOSITION, wxID_EXIT,               TEXT("Force Close All"));
        }

        // Create, populate and add the "View" submenu
        HMENU subMenuView = CreatePopupMenu();
        if (subMenuView != NULL)
        {
            InsertMenu(sysMenu, menuPosIns, MF_BYPOSITION | MF_POPUP, (UINT_PTR)subMenuView, TEXT("View"));
            ++menuPosIns;
            
        // ... Add items into subMenuView
        // ... And add another submenu called "Help"
    } // if (sysMenu != NULL)
    
}

The IDs added for the menu options are the same ones you’ll use for normal menu command declarations in the event table.
I like to use class enums, so you’ll see (int) castings for my command IDs. It’s nothing special, though.

wxBEGIN_EVENT_TABLE(TopDockWin, wxFrame)
    EVT_MENU(wxID_EXIT,  TopDockWin::OnExit)
    EVT_MENU((int)CMDID::ToggleStatusbar,       TopDockWin::OnMenu_ToggleStatusbar  )
    EVT_MENU((int)CMDID::ReleaseAll,            TopDockWin::OnMenu_ReleaseAll       )
    EVT_MENU((int)CMDID::DetachAll,             TopDockWin::OnMenu_DetachAll        )
    EVT_MENU((int)CMDID::CloseAll,              TopDockWin::OnMenu_CloseAll         )
wxEND_EVENT_TABLE()

Redirecting the System Menu events

Even though we used the regular EVT_MENU macro, and the System Menu is a menu, this doesn’t automatically work. This is because System Menu events are handled differently. So we need to do some message plumbing work for our top-level window.

For wxWindows on WindowsOS, there’s a virtual function you can override called MSWTranslateMessage(). This allows for a per-class implementation of intercepting Windows messages. Specifically, we’re interested in when a System Menu entry is selected, which sends a WM_SYSCOMMAND message; we need to intercept that.

In the documentation for WM_SYSCOMMAND, we see that certain defined values are reserved (which are prefixed with SC_*). Using these reserved IDs with EVT_MENU is prohibited. The lowest value is 0xF000 so you have ~60,000 values you can enumerate through before reaching those values. The value SCF_ISSECURE is pretty low, but we’re ignoring that one.

So in the .h of my class,

class TopDockWin: public wxFrame
{
    // ... etc
    bool MSWTranslateMessage(WXMSG* msg) override;
    // ... etc
};

And in the .cpp of my class, some plumbing work to turn WM_SYSTEM messages into wxWidgets EVT_MENU events. But only for things we’re allowed to.

bool TopDockWin::MSWTranslateMessage(WXMSG* msg)
{
    if (msg->message == WM_SYSCOMMAND)
    {
        switch(msg->wParam)
        {
            // For every wparam documented for WM_SYSCOMMAND, we leave alone and 
            // let the default implementation do whatever with it.
            // https://learn.microsoft.com/en-us/windows/win32/menurc/wm-syscommand
        case SC_CLOSE:
        case SC_CONTEXTHELP:
        case SC_DEFAULT:
        case SC_HOTKEY:
        case SC_HSCROLL:
        // case SCF_ISSECURE: 
        //  Except this, t'is a troublemaker!
        case SC_KEYMENU:
        case SC_MAXIMIZE:
        case SC_MINIMIZE:
        case SC_MONITORPOWER:
        case SC_MOUSEMENU:
        case SC_MOVE:
        case SC_NEXTWINDOW:
        case SC_PREVWINDOW:
        case SC_RESTORE:
        case SC_SCREENSAVE:
        case SC_SIZE:
        case SC_TASKLIST:
        case SC_VSCROLL:
            break;

        default:
            // Everything else, we're claiming as ID space we're processing ourselves.
            // We convert to a menu command
            wxCommandEvent evt(wxEVT_MENU, msg->wParam);
            this->ProcessEvent(evt);
            return true;
        }
        
    } // if (msg->message == WM_SYSCOMMAND)
    
    // Delegate everything we didn't handle to parent implementation.
    return wxFrame::MSWTranslateMessage(msg);
}

The menu IDs we added for the menus are in the wParam, which is also what was used for our EVT_MENU IDs, so it’s a pretty easy conversion. We have do-nothing case statements to specify what we want to reject handling (that was reserved via the WM_SYSTEM documentation) and delegate everything else that isn’t WM_SYSTEM to our parent (wxFrame) implementation.