fix: controller navigation hardening (ported from PR #17)

CHANGELOG.en-US.md

@@ -4,6 +4,11 @@ English (US) release notes, mirroring CHANGELOG.md (Brazilian Portuguese). Befor
 
 ## [Unreleased]
 ### What's new
+- Desktop interface navigable with the controller: D-pad/stick move, A activates (lists, switches, cards), B closes a list or leaves Settings, Start toggles Settings, Y goes to the console interface; in the tour, A advances and B skips. Based on PR #17 by @nextestudios.
+
+### Fixes
+- The controller went dead for good if a pad disconnected mid-read; the read now just skips that sample. Also from PR #17.
+- Controller navigation: the first press only reveals the focus ring; in scrolled lists focus falls back to tab order when the spatial search finds nothing; D-pad diagonals count as vertical; reads only count while the window is in the foreground (checked per window, not per event).
 - Session menu: while the game is open, hold Select + Y for a menu over the game with volume, resolution, audio output, FPS limit, HDR, "Back to the PC" and "Exit Console Mode". Also via `consolemode://menu`.
 
 ## [1.5.0-beta.5]

CHANGELOG.md

@@ -4,6 +4,11 @@ Notas em português do Brasil; a versão em inglês (Estados Unidos) fica em `CH
 
 ## [Unreleased]
 ### Novidades
+- Interface Desktop navegável pelo controle: D-pad/analógico movem, A ativa (listas, interruptores, cards), B fecha a lista ou volta de Ajustes, Start abre/fecha Ajustes, Y vai para a interface Console; no tutorial, A avança e B pula. Baseado no PR #17 de @nextestudios.
+
+### Correções
+- O controle deixava de responder de vez se um controle desconectasse no meio de uma leitura; agora a leitura só pula a amostra. Também vindo do PR #17.
+- Navegação por controle: o primeiro toque só mostra o anel de foco; em listas roladas o foco cai na ordem de tabulação quando a busca espacial não acha nada; diagonais no D-pad contam como vertical; a leitura só vale com a janela em primeiro plano (verificação por janela, não por evento).
 - Menu da sessão: com o jogo aberto, segure Select + Y para abrir um menu sobre o jogo com volume, resolução, saída de áudio, limite de FPS, HDR, "Voltar ao PC" e "Sair do Console Mode". Também por `consolemode://menu`.
 
 ## [1.5.0-beta.5]

docs/TESTING.md

@@ -80,6 +80,9 @@ Pré-condições: Ajustes → Interface = **Automático** (padrão).
 - [ ] Trocar o idioma em Ajustes e voltar para Console: todos os textos, incluindo os cartões, trocam. Resultado: ______
 
 ### Navegação por controle (Xbox **e** PlayStation; a janela precisa estar em primeiro plano)
+- [ ] **Interface Desktop:** D-pad move o foco entre tiles, o seletor de papel, os cards de Ajustes e os ComboBoxes; A abre um ComboBox, D-pad percorre, A escolhe, B fecha; Start abre e fecha Ajustes; Y vai para a interface Console; no tutorial, A avança e B pula. Resultado: ______
+- [ ] Desconectar o controle no meio do uso e reconectar: a navegação volta a funcionar sem reiniciar o app (log mostra no máximo uma linha `Controles: ...`). Resultado: ______
+- [ ] O primeiro toque após um clique de mouse só mostra o anel de foco; o segundo age. Resultado: ______
 - [ ] Ao abrir, o foco está em **Jogar agora** (anel grosso). Resultado: ______
 - [ ] D-pad e analógico esquerdo movem o foco: Jogar → cartões de ajustes → cartões de telas → "Modo desktop", e de volta. Segurar uma direção repete. Resultado: ______
 - [ ] **A** num cartão de tela abre o painel "O que a tela X faz?" com foco em "Jogar aqui"; A escolhe; **B** fecha sem mudar. Resultado: ______

src/ConsoleMode/MainWindow.xaml.cs

@@ -52,10 +52,51 @@ public MainWindow(MainViewModel viewModel)
         // The console interface fills the screen, like a console; the desktop one gets its size back.
         ViewModel.PropertyChanged += (_, e) =>
         {
-            if (e.PropertyName != nameof(MainViewModel.IsConsoleUi) || appWindow.Presenter is not OverlappedPresenter p) return;
-            if (ViewModel.IsConsoleUi) p.Maximize();
-            else if (p.State == OverlappedPresenterState.Maximized) p.Restore();
+            if (e.PropertyName == nameof(MainViewModel.IsConsoleUi) && appWindow.Presenter is OverlappedPresenter p)
+            {
+                if (ViewModel.IsConsoleUi) p.Maximize();
+                else if (p.State == OverlappedPresenterState.Maximized) p.Restore();
+            }
+            if (e.PropertyName is nameof(MainViewModel.IsConsoleUi) or nameof(MainViewModel.IsConsoleActive))
+                RefreshDesktopNavigator();
         };
+        WatchDesktopPad(hwnd);
+    }
+
+    // The desktop interface, driven by the pad like a console menu (ported from PR #17):
+    // D-pad/stick move, A activates (lists, toggles, cards), B closes a list or leaves Settings,
+    // Start toggles Settings, Y jumps to the console interface, tour: A next / B skip.
+    // The console interface has its own navigator inside ConsoleHomeView.
+    private GamepadNavigator? _desktopPad;
+
+    private void WatchDesktopPad(nint hwnd)
+    {
+        _desktopPad = new GamepadNavigator(DispatcherQueue, (DependencyObject)Content, hwnd)
+        {
+            Intercept = action =>
+            {
+                if (ViewModel.TourStep <= 0) return false;
+                if (action == ControllerAction.Confirm) ViewModel.TourNextCommand.Execute(null);
+                else if (action == ControllerAction.Back) ViewModel.EndTourCommand.Execute(null);
+                return true;
+            }
+        };
+        _desktopPad.MenuRequested += () =>
+        {
+            if (ViewModel.IsSettingsPage) ViewModel.GoHomeCommand.Execute(null);
+            else ViewModel.OpenSettingsCommand.Execute(null);
+        };
+        _desktopPad.BackRequested += () => { if (ViewModel.IsSettingsPage) ViewModel.GoHomeCommand.Execute(null); };
+        _desktopPad.AltRequested += () => ViewModel.SwitchUiCommand.Execute("console");
+        RefreshDesktopNavigator();
+    }
+
+    private void RefreshDesktopNavigator()
+    {
+        if (_desktopPad is null) return;
+        var listen = !ViewModel.IsConsoleUi && !ViewModel.IsConsoleActive;
+        if (listen && !_desktopPad.IsRunning) _desktopPad.Start();
+        else if (!listen && _desktopPad.IsRunning) _desktopPad.Stop();
     }
 
     private static class Win32Dpi

src/ConsoleMode/Resources/Strings.en-US.json

@@ -51,7 +51,7 @@
   "UiModeConsole": "Console",
   "SwitchToDesktop": "Desktop mode",
   "SwitchToConsole": "Console mode",
-  "ControllerHintDesktop": "Controller detected · press A for console mode",
+  "ControllerHintDesktop": "Controller detected · A navigates, Y opens console mode",
   "SessionMenuTitle": "Session menu",
   "BackToGame": "Back to the game",
   "VolumeRow": "Volume",

src/ConsoleMode/Resources/Strings.es-ES.json

@@ -51,7 +51,7 @@
   "UiModeConsole": "Consola",
   "SwitchToDesktop": "Modo escritorio",
   "SwitchToConsole": "Modo consola",
-  "ControllerHintDesktop": "Mando detectado · pulsa A para el modo consola",
+  "ControllerHintDesktop": "Mando detectado · A navega, Y abre el modo consola",
   "SessionMenuTitle": "Menú de la sesión",
   "BackToGame": "Volver al juego",
   "VolumeRow": "Volumen",

src/ConsoleMode/Resources/Strings.pt-BR.json

@@ -51,7 +51,7 @@
   "UiModeConsole": "Console",
   "SwitchToDesktop": "Modo desktop",
   "SwitchToConsole": "Modo console",
-  "ControllerHintDesktop": "Controle detectado · aperte A para o modo console",
+  "ControllerHintDesktop": "Controle detectado · A navega, Y abre o modo console",
   "SessionMenuTitle": "Menu da sessão",
   "BackToGame": "Voltar ao jogo",
   "VolumeRow": "Volume",

src/ConsoleMode/Services/ControllerInput.cs

@@ -122,8 +122,22 @@ public void Start()
 
     public void Dispose() => _timer.Stop();
 
+    /// <summary>
+    /// Only act while this returns true (e.g. our window is in the foreground). Reading resumes
+    /// primed, so a button held while inactive doesn't fire on the way back.
+    /// </summary>
+    public Func<bool>? IsActive { get; set; }
+
+    private bool _loggedError;
+
     private void Poll()
     {
+        if (IsActive is not null && !IsActive())
+        {
+            _primed = false;
+            return;
+        }
+
         var held = new bool[AllActions.Length];
         try
         {
@@ -132,11 +146,18 @@ private void Poll()
         }
         catch (Exception ex)
         {
-            AppLog.Write($"Controles: {ex.Message}");
-            _timer.Stop();
+            // A pad unplugged mid-read throws; skip the sample instead of going deaf for good
+            // (ported from nextestudios' controller PR, #17).
+            if (!_loggedError) AppLog.Write($"Controles: {ex.Message}");
+            _loggedError = true;
+            _primed = false;
             return;
         }
 
+        // A diagonal on the D-pad or stick moves vertically, the common case in these lists.
+        if (held[(int)ControllerAction.Up] || held[(int)ControllerAction.Down])
+            held[(int)ControllerAction.Left] = held[(int)ControllerAction.Right] = false;
+
         // First sample only records state, so a button already held when the prompt opens
         // (e.g. the A that launched console mode) doesn't answer it.
         var now = DateTime.UtcNow;

src/ConsoleMode/Services/GamepadNavigator.cs

@@ -1,17 +1,21 @@
+using ConsoleMode.Native;
 using Microsoft.UI.Dispatching;
 using Microsoft.UI.Xaml;
 using Microsoft.UI.Xaml.Automation.Peers;
 using Microsoft.UI.Xaml.Automation.Provider;
 using Microsoft.UI.Xaml.Controls;
 using Microsoft.UI.Xaml.Controls.Primitives;
 using Microsoft.UI.Xaml.Input;
+using Microsoft.UI.Xaml.Media;
 
 namespace ConsoleMode.Services;
 
 /// <summary>
 /// WinUI 3 desktop apps get keyboard focus navigation for free but nothing from a
 /// controller. This turns controller presses into focus moves and invokes, scoped to one
-/// root element, so the console interface is fully usable from the couch.
+/// root element, so the whole app is usable from the couch. Several rules come from
+/// nextestudios' controller PR (#17): foreground gating by HWND, first press only shows
+/// the focus ring, tab-order fallback when the spatial search finds nothing, ComboBox handling.
 /// </summary>
 public sealed class GamepadNavigator : IDisposable
 {
@@ -24,10 +28,18 @@ public sealed class GamepadNavigator : IDisposable
     public event Action? AltRequested;
     public event Action? OptionRequested;
 
-    public GamepadNavigator(DispatcherQueue dispatcher, DependencyObject root)
+    /// <summary>Return true to consume a direction (e.g. a slider row using Left/Right).</summary>
+    public Func<FocusNavigationDirection, bool>? BeforeMove { get; set; }
+
+    /// <summary>Handles a press before focus logic; return true to swallow it (e.g. tour tips).</summary>
+    public Func<ControllerAction, bool>? Intercept { get; set; }
+
+    /// <param name="hwnd">When given, presses only count while this window is in the foreground.</param>
+    public GamepadNavigator(DispatcherQueue dispatcher, DependencyObject root, nint hwnd = 0)
     {
         _root = root;
         _input = new ControllerInput(dispatcher);
+        if (hwnd != 0) _input.IsActive = () => NativeWindows.GetForegroundWindow() == hwnd;
         _input.Pressed += OnPressed;
     }
 
@@ -45,52 +57,125 @@ public void Dispose()
 
     private void OnPressed(ControllerAction action)
     {
-        switch (action)
+        try
         {
-            case ControllerAction.Up: Move(FocusNavigationDirection.Up); break;
-            case ControllerAction.Down: Move(FocusNavigationDirection.Down); break;
-            case ControllerAction.Left: Move(FocusNavigationDirection.Left); break;
-            case ControllerAction.Right: Move(FocusNavigationDirection.Right); break;
-            case ControllerAction.Confirm: if (GetXamlRoot() is { } root) Invoke(FocusManager.GetFocusedElement(root)); break;
-            case ControllerAction.Back: BackRequested?.Invoke(); break;
-            case ControllerAction.Menu: MenuRequested?.Invoke(); break;
-            case ControllerAction.Alt: AltRequested?.Invoke(); break;
-            case ControllerAction.Option: OptionRequested?.Invoke(); break;
+            if (Intercept?.Invoke(action) == true) return;
+            switch (action)
+            {
+                case ControllerAction.Up: Move(FocusNavigationDirection.Up); break;
+                case ControllerAction.Down: Move(FocusNavigationDirection.Down); break;
+                case ControllerAction.Left: Move(FocusNavigationDirection.Left); break;
+                case ControllerAction.Right: Move(FocusNavigationDirection.Right); break;
+                case ControllerAction.Confirm: Confirm(); break;
+                case ControllerAction.Back: Back(); break;
+                case ControllerAction.Menu: MenuRequested?.Invoke(); break;
+                case ControllerAction.Alt: AltRequested?.Invoke(); break;
+                case ControllerAction.Option: OptionRequested?.Invoke(); break;
+            }
+        }
+        catch (Exception ex)
+        {
+            AppLog.Write($"Controles: navegação: {ex.Message}");
         }
     }
 
     private XamlRoot? GetXamlRoot() => (_root as UIElement)?.XamlRoot;
 
-    /// <summary>Return true to consume a direction (e.g. a slider row using Left/Right).</summary>
-    public Func<FocusNavigationDirection, bool>? BeforeMove { get; set; }
+    private Control? Focused() =>
+        GetXamlRoot() is { } root && FocusManager.GetFocusedElement(root) is Control c && IsShown(c) ? c : null;
+
+    /// <summary>After a mouse click or at startup focus has no ring; the first press only reveals it.</summary>
+    private bool RevealFocus(Control control)
+    {
+        if (control is ComboBoxItem || control.FocusState == FocusState.Keyboard) return false;
+        control.Focus(FocusState.Keyboard);
+        return true;
+    }
+
+    private void FocusFirst() => (FocusManager.FindFirstFocusableElement(_root) as Control)?.Focus(FocusState.Keyboard);
 
     private void Move(FocusNavigationDirection direction)
     {
         if (BeforeMove?.Invoke(direction) == true) return;
+        if (Focused() is not { } control) { FocusFirst(); return; }
+        if (RevealFocus(control)) return;
+
+        // Inside an open ComboBox the arrows walk its items instead of leaving the popup.
+        if (control is ComboBoxItem item && ItemsControl.ItemsControlFromItemContainer(item) is ComboBox combo)
+        {
+            if (direction is not (FocusNavigationDirection.Up or FocusNavigationDirection.Down)) return;
+            var index = combo.IndexFromContainer(item) + (direction == FocusNavigationDirection.Down ? 1 : -1);
+            if (index >= 0 && index < combo.Items.Count && combo.ContainerFromIndex(index) is Control next)
+                next.Focus(FocusState.Keyboard);
+            return;
+        }
+
         var options = new FindNextElementOptions { SearchRoot = _root };
-        if (GetXamlRoot() is not { } root || FocusManager.GetFocusedElement(root) is null)
+        var target = FocusManager.FindNextElement(direction, options) as Control;
+        // Scrolled lists: the spatial search can miss controls outside the viewport, so up/down
+        // fall back to tab order, which does reach them (Focus scrolls them into view).
+        target ??= direction switch
+        {
+            FocusNavigationDirection.Down => FocusManager.FindNextElement(FocusNavigationDirection.Next, options) as Control,
+            FocusNavigationDirection.Up => FocusManager.FindNextElement(FocusNavigationDirection.Previous, options) as Control,
+            _ => null
+        };
+        target?.Focus(FocusState.Keyboard);
+    }
+
+    private void Confirm()
+    {
+        if (Focused() is not { } control) { FocusFirst(); return; }
+        if (RevealFocus(control)) return;
+        Invoke(control);
+    }
+
+    private void Back()
+    {
+        if (Focused() is ComboBoxItem item && ItemsControl.ItemsControlFromItemContainer(item) is ComboBox combo)
         {
-            // Nothing focused yet: land on the first focusable control.
-            (FocusManager.FindFirstFocusableElement(_root) as Control)?.Focus(FocusState.Keyboard);
+            combo.IsDropDownOpen = false;
+            combo.Focus(FocusState.Keyboard);
             return;
         }
-        FocusManager.TryMoveFocus(direction, options);
+        BackRequested?.Invoke();
     }
 
-    /// <summary>A = click the focused control, or flip it when it is a switch.</summary>
+    /// <summary>A = activate whatever has focus: pick a list item, open a list, flip a switch, click a button.</summary>
     public static void Invoke(object? focused)
     {
         switch (focused)
         {
+            case ComboBoxItem item when ItemsControl.ItemsControlFromItemContainer(item) is ComboBox combo:
+                combo.SelectedIndex = combo.IndexFromContainer(item);
+                combo.IsDropDownOpen = false;
+                combo.Focus(FocusState.Keyboard);
+                break;
+            case ComboBox combo:
+                combo.IsDropDownOpen = true;
+                break;
             case ToggleSwitch sw:
                 sw.IsOn = !sw.IsOn;
                 break;
+            case SelectorItem selectorItem:
+                selectorItem.IsSelected = true;
+                break;
             case ButtonBase button:
-                if (FrameworkElementAutomationPeer.CreatePeerForElement(button)?.GetPattern(PatternInterface.Invoke) is IInvokeProvider invoke)
-                    invoke.Invoke();
-                else if (FrameworkElementAutomationPeer.CreatePeerForElement(button)?.GetPattern(PatternInterface.Toggle) is IToggleProvider toggle)
-                    toggle.Toggle();
+                var peer = FrameworkElementAutomationPeer.FromElement(button) ?? FrameworkElementAutomationPeer.CreatePeerForElement(button);
+                if (peer?.GetPattern(PatternInterface.Invoke) is IInvokeProvider invoke) invoke.Invoke();
+                else if (peer?.GetPattern(PatternInterface.Toggle) is IToggleProvider toggle) toggle.Toggle();
+                // e.g. SettingsCard: a ButtonBase whose peer has no Invoke pattern.
+                else if (button.Command?.CanExecute(button.CommandParameter) == true) button.Command.Execute(button.CommandParameter);
                 break;
         }
     }
+
+    private static bool IsShown(DependencyObject element)
+    {
+        for (var current = element; current is not null; current = VisualTreeHelper.GetParent(current))
+        {
+            if (current is UIElement { Visibility: Visibility.Collapsed }) return false;
+        }
+        return true;
+    }
 }

src/ConsoleMode/SessionMenuWindow.xaml.cs

@@ -43,7 +43,7 @@ public SessionMenuWindow(MainViewModel viewModel, ScreenRect? target)
         }
         WindowPlacement.CenterOn(appWindow, target, WidthDip, HeightDip);
 
-        _navigator = new GamepadNavigator(DispatcherQueue, Root)
+        _navigator = new GamepadNavigator(DispatcherQueue, Root, _hwnd)
         {
             // Left/Right on the volume row adjust it instead of moving focus.
             BeforeMove = direction =>

src/ConsoleMode/Views/ConsoleHomeView.xaml.cs

@@ -27,7 +27,8 @@ public ConsoleHomeView()
 
     private void OnLoaded(object sender, RoutedEventArgs e)
     {
-        _navigator ??= new GamepadNavigator(DispatcherQueue, this);
+        _navigator ??= new GamepadNavigator(DispatcherQueue, this,
+            App.MainWindowInstance is { } w ? WinRT.Interop.WindowNative.GetWindowHandle(w) : 0);
         _navigator.BackRequested += GoBack;
         _navigator.MenuRequested += () => { if (ViewModel.CanStart && !ViewModel.IsRolePanelOpen && !ViewModel.IsConsoleSettingsOpen && !ViewModel.IsPickerOpen) ViewModel.StartCommand.Execute(null); };
         _navigator.AltRequested += () =>