Live screen. Real mouse. Real keyboard. On your own Wi-Fi, at 60 fps.

---

Most remote desktop apps treat your phone as a touchscreen and hand you a floating cursor to poke at. Backmac treats it as a computer.

Plug a mouse and keyboard into the phone and it takes pointer capture — Android's cursor disappears, and your mouse drives the Mac's directly in relative deltas. Physical keys go straight through to macOS virtual keycodes, Command and Option included. Touch still works as a trackpad, when that's all you have.

No account, no relay server, no cloud. Two devices, one Wi-Fi network, one TCP port.

Quick start

./build.sh

That builds both halves, installs Backmac.app into /Applications and launches it, then builds the APK and installs it on every attached Android device.

Then give macOS its three things — the script checks all three and tells you which are missing:

	Why	Where
Screen Recording	Without it the phone connects to a black screen	Privacy & Security → Screen & System Audio Recording
Accessibility	Without it the screen arrives but nothing moves	Privacy & Security → Accessibility
Firewall: Allow	Without it the phone finds your Mac, then times out	The prompt on first launch

Add /Applications/Backmac.app to both permission lists and relaunch — macOS only re-reads Screen Recording at process start. Details, failure modes and the full troubleshooting table live in docs/setup.md.

Finally, pair:

1. Mac — menu bar icon → Pair new device…. A six-digit code appears.
2. Phone — your Mac is already listed. Tap it, type the code.
3. The window closes itself the moment the phone gets through.

The code is single use and only live while that window is open. In exchange the phone gets a 32-byte token and reconnects with one tap from then on.

The Mac, offering a code	The phone, having already found it

Using it

Show the controls	Tap the handle at the top edge. The pill fades away on its own.
Capture the mouse	The pointer button in the pill. Android's cursor disappears.
Release it	Double-tap Right Ctrl, or switch away from the app.
Touch	Drag to move, tap to click, two-finger tap to right-click, two-finger drag to scroll, tap-then-drag to drag.
Latency	The pill leads with the echo — keypress to the frame that answers it — with p95 and the raw network ping beside it. The dot grades it: green under 60 ms, amber under 110, red above.

While the mouse is captured there is no chrome on screen at all.

Laptop mode

Half-open a foldable mid-session and the phone becomes a self-contained terminal for the Mac: a full US QWERTY along the bottom edge — the half now lying flat — with the Mac's screen filling everything above it. No external hardware, no reconnect, no decoder restart.

• Real modifiers. Tap Command once to arm it for the next key, twice to lock it. Both states are visible on the key, which is what makes Cmd+Shift+4 reachable one-handed.
• Feedback is local. A press flashes, pops a preview bubble clear of your fingertip and ticks — all before the packet leaves the phone. Waiting for the Mac's answer is a screen-capture round trip, and it's what made keys feel dead.
• The keys are a trackpad too. Land a finger in the middle of the keyboard and sweep, and the keys become a pad under it — same gestures, same sensitivity, two big click buttons where your thumbs already are. It lingers three seconds after you lift so you can re-centre and click again; touch the top row and it's gone. It costs no layout, because it's an overlay: the keys stay full size.
• The screen above is a trackpad as well. Plus l-click and r-click keys held exactly as long as your finger is, so you can hold one and drag above it.
• Drag the bar between them to trade keyboard for screen. Shrinking it doesn't squash the keys flat — the keyboard splits, and the halves walk further apart the smaller it gets, so the keys keep their shape and land under your thumbs on the way down. Tap the same bar for the session controls.
• It knows when to stay away — never in DeX, never against a real keyboard or mouse, never while the pointer is captured.

_{Mid-sweep: the keys are a pad, with the function row left uncovered as the way out.}

It works unfolded too — set On-screen keyboard to Always. An ultrawide Mac letterboxes into a phone-shaped window and leaves bands of dead screen; this spends them on a keyboard.

It knows how you're holding it

A foldable doesn't have a screen size, it has a posture, and it changes four times a minute without the app restarting. Width alone gets two of them wrong — half-opened, the window measures exactly like a tablet — so the hinge and the attached hardware are part of the answer too.

Phone	One column, thumb-sized targets. The icon is the interface.
Bent	Split on the hinge, from FoldingFeature rather than half the window. Identity above the crease, everything you touch below it.
Tablet	The same split, turned ninety degrees.
Desktop	A keyboard and mouse are attached, so nothing is oversized and the soft-keyboard buttons leave the pill entirely.

How it works

Two TCP connections, not one. A 200 KB keyframe and a 9-byte mouse move on one socket means the mouse waits behind the keyframe. Splitting them costs one handshake and removes the coupling entirely.

Latency over throughput, everywhere. The encoder runs realtime with B-frames off; the decoder renders each buffer the moment it appears rather than scheduling by presentation time; both sockets set TCP_NODELAY.

The phone says what it actually got. TCP won't tell you: a send completes when the local stack takes the bytes, so the server's queue reads empty while a megabyte sits in a kernel buffer ahead of the frame answering your last keystroke. So every frame carries a sequence number, the phone acks it on the control channel, and the server stops sending past six unacknowledged frames. Six is measured rather than reasoned — three threw away 14.3 frames a second against full-screen video, and ten bought nothing six didn't.

Repairs are cheap. A dropped P-frame corrupts everything that references it, so the stream has to be re-anchored — classically with an IDR, which on a mostly-static desktop is a six-figure stall. Because the phone confirms which frames it genuinely decoded, the server can mark one as a long-term reference and repair with a small P-frame instead: 251 bytes where an IDR was 166 KB.

Only changed frames are encoded. ScreenCaptureKit reports .idle/.blank when nothing moved and those never reach the encoder, so a static desktop costs almost no bandwidth.

Aspect ratio is the Mac's, not the phone's. The client asks for a maximum box; the server fits the real display inside it. An ultrawide stays ultrawide.

Full wire format: PROTOCOL.md.

Security

The connection is authenticated but not encrypted. Tokens are never replayable — they only ever act as HMAC keys over a server-chosen nonce, and never cross the wire after pairing — but video and keystrokes travel in cleartext.

That's fine behind your own WPA2/WPA3 Wi-Fi and nowhere else. Don't run it on a network you don't control, and don't forward the port. AES-GCM over the key material the handshake already produces is the intended next step.

What you need

Mac	macOS 14 or newer, Xcode command line tools (xcode-select --install)
Phone	Android 12 or newer (API 31)
To build	A JDK between 17 and 21, the Android SDK, and adb on your PATH
Network	Both devices on the same Wi-Fi, with client isolation off

build.sh finds a supported JDK itself, so you don't need one as your system default — just installed. brew install --cask temurin@17 is enough.

./build.sh --mac-only      just the menu-bar app
./build.sh --android-only  just the phone app
./build.sh --no-install    build both, install neither
./build.sh --dest ~/Apps   put Backmac.app somewhere other than /Applications

Known limits

• Main display only. The Mac reports how many displays it has, but always captures the primary.
• No audio, clipboard sync, or file transfer.
• One client at a time — a second connection displaces the first.

Settings — everything optional, in the sheet behind the home-screen icon

• Video — Smooth (1280 px, 10 Mbps), Balanced (1920 px, 20 Mbps), Sharp (2560 px, 40 Mbps), or Custom with raw sliders. All presets run at 60 fps; dropping frame rate hurts the feel of a cursor far more than dropping resolution does.
• PC keyboard layout — makes Alt act as Command and Win act as Option, matching the keys that physically sit where a Mac's would. Leave it off for an Apple keyboard, which already reports Command as Meta.
• Natural scrolling — match whichever direction the Mac already uses.
• Show on-screen input — Auto (only when half-opened, and never against real hardware), Always, or Never.
• Split keyboard — pulls the halves apart so the middle columns land under your thumbs, at any height. 6, Y, H, N and space sit on both halves, so neither thumb crosses the gap. Off, the keyboard is whole until you drag it shorter, which splits it anyway — that is what makes a short keyboard still worth typing on.
• Function row — F1–F12 above the numbers. The other rows get taller without it.
• Forget every paired Mac — drops the stored tokens.

Source layout

build.sh                       build + install both halves
mac/
  Makefile                     build → Backmac.app, signing, run/stop
  Sources/BackmacServer/
    main.swift  AppDelegate    menu bar, permissions, pairing window
    Server.swift               listener, sessions, pipeline, backpressure
    PeerConnection.swift       framed TCP, send-queue accounting
    Protocol.swift             message types, big-endian codec, frame parser
    Auth.swift                 pairing codes, device tokens, HMAC challenge
    ScreenCapturer.swift       ScreenCaptureKit → NV12
    VideoEncoder.swift         VideoToolbox H.264, AVCC → Annex-B
    InputInjector.swift        CGEvent mouse/keyboard/scroll/text
  Tools/ltrprobe.swift         standalone: is a cheap repair real on this Mac?
android/app/src/main/java/com/c0di/backmac/
    MainActivity.kt            discovery, pairing, settings sheet (Compose)
    RemoteActivity.kt          fullscreen session, auto-hiding controls, capture
    RemoteView.kt              video surface + all three input modes
    OnScreenInputView.kt       laptop mode: the band the keyboard lives in
    VirtualKeyboardView.kt     latching modifiers, key repeat, local feedback
    KeyboardTrackpad.kt        the pad the keys turn into, and its click buttons
    Trackpad.kt                touch → mouse, shared by the video pane and the keyboard
    KeyboardLayout.kt          which key sits where, in Android keycodes
    DeviceState.kt             fold posture, and spotting real keyboards and mice
    Stage.kt                   phone / bent / tablet / desktop, and their metrics
    EchoLatency.kt             finger-to-frame measurement, and what it discards
    VideoDecoder.kt            MediaCodec low-latency decode
    BackmacClient.kt           sockets, handshake, reader threads
    Discovery.kt               NSD, with the API-34 callback split
    KeyMap.kt                  Android keycodes → macOS virtual keycodes
    Theme.kt  Protocol.kt  Prefs.kt  Session.kt

Further reading

docs/setup.md	Permissions, the firewall, and why a stale rule lies to every local check
PROTOCOL.md	The wire format, message by message
docs/laptop-mode.md	The original brief — historical, but it's where "custom keyboard, not the system IME" is argued out
docs/keyboard-ux-review.md	The review that followed: why the keyboard felt laggier than the pointer when neither was, and what got cut
docs/typing-latency-investigation.md	A running log of why typing felt excruciating and video didn't — including three conclusions that were confidently wrong before they were measured