Repository navigation
Platform-agnostic ECCE: what a native macOS and Windows port would take #133
Description
Decisions and status (updated 2026-10-07)
Decided
- Native clients. ECCE runs natively on macOS and Windows. Containers, WSL and XQuartz are not the route to either platform.
- The server stays Linux. The data server and broker for a central install keep running on Linux (Debian primary, RHEL secondary). A native client either connects to such a server with
ecce -remote, or runs in local mode (Local storage option: run the client with a data folder instead of a data server #216). Local mode is not the default yet; it becomes the default once the native clients ship. Clients will get the site configuration (machines, queues) from the server (Remote mode: the server provides site configuration (machines, queues), so a client works out of the box #192, designed together with Jobs submitted and monitored by the ECCE server, not the client #208). - Viewer: Coin3D replaces the vendored SGI Open Inventor (Port the viewer to Coin3D (9.0) #166). Coin is the default since 9.0.0-alpha.4. The vendored core will be deleted once nobody needs the fallback.
- Messaging: Mosquitto (MQTT) replaces ActiveMQ and the Java relay (Evaluate replacing ActiveMQ with MQTT (Mosquitto) before #194's broker authentication #213). There is no Java in 9.0.
- No X11/Xt outside the Linux GTK build (Remove the X11/Xt dependencies (native macOS/Windows clients) #232).
- No paid code signing. The apps are signed ad hoc only, and the README says how to open them.
- macOS
- The release's dependencies (wxWidgets 3.3, Coin3D, OpenSSL, libssh, Mosquitto, Xerces-C, FreeType, libjpeg-turbo) are built from source on CI for a fixed minimum macOS: 11.0 on Apple silicon, 10.15 on Intel. Homebrew is not used for them, because its bottles carry the runner's macOS as their minimum.
- Apple silicon ships first; Intel follows when it is ready and never holds a release.
- Windows
- The build uses MSYS2 (MinGW-w64, wxMSW).
- The first client bundles a portable Strawberry Perl, an embeddable Python with wxPython, and a minimal MSYS2
shwith the coreutils the job scripts use. gensub, the importers and the code-registration dialogs then run unchanged; rewriting them is a later option, not a prerequisite. - Processes are started with CreateProcess.
ecce-flockand the Mosquitto auth plugin are not built on Windows.- On Windows, the message link uses Mosquitto's own network loop instead of a pipe.
Done
macOS: on main (merged in c0ead20). CI builds ECCE.app and a .dmg for both architectures.
- Minimum version:
- Every Mach-O in the app (51 files) is checked against the minimum: 11.0 for arm64 and 10.15 for x86_64.
- The job fails if any file is above the minimum or of the wrong architecture.
- Start test: the CI start test opens every application from the dmg on both architectures, with screenshots.
- Catalina: on macOS 10.15.7 (Intel, a private VM), all 34 executables of the Intel app load with every symbol bound at launch (
DYLD_BIND_AT_LAUNCH=1), including the Fortran tools. - Opening the app: the README describes it, including the macOS 15 "Open Anyway" step.
Windows: branch wip/133-windows, not on main yet.
- Builds: the whole tree builds with 0 failures. Test programs that need POSIX are excluded on Windows.
- Starts in local mode: Organizer, Builder, the Calculation Editor, Basis Set Tool, Periodic Table, Machine Browser, Machine Registration and the Launcher.
- Fixed on the way, as Windows-only changes:
- a
C:/drive letter read as a URL host; - helper programs found without their
.exe; - a UTF-8 label;
- static-library link order;
- a numeric locale problem that also affected Linux users with a decimal comma. That fix is on
main.
- a
- Process launch: done with CreateProcessW. A Windows unit test (23 cases) covers exit status, separate stdout and stderr, large output without deadlock, the working directory, the environment, timeouts, detached starts and cancelling a whole process tree.
Also found by the macOS start test and fixed on main: a Builder crash when opening a structure file in the List + detail layout, which affected every platform (#238).
Open
- Windows
- a local job from submit to "completed" without the GUI (in progress);
- Perl and shell bundling into the install tree;
- an installer;
- merging the branch to
main.
- macOS
- a person using the app windows on an older Mac. So far only CI and a load check have been run;
- the Organizer message pane, whose top line is still partly cut off;
- job submission from the bundled app.
- Both: setting the server address without
ecce-remote-setup, which needs root and bash. Native clients need a per-user first-start setting (v9 roadmap: the revamped ECCE (server/client split, new transport, native clients) #186).
The survey below is the original assessment (September 2026). Parts of it have been overtaken by the decisions above (ActiveMQ, the vendored Inventor, "Windows is a rewrite").
What it would take to run ECCE natively on macOS and Windows. Surveyed rather
than estimated; counts below are from the tree as it stands.
The short version
macOS is a port. Windows is a rewrite of the service layer. They should not
be planned as one project, and most of the preparatory work is worth doing on
Linux regardless of whether either ever ships.
Layer by layer
Already portable
- Build -- CMake/CPack. CPack needs generators adding (DragNDrop or
productbuild; NSIS or WiX) but the structure is there. - GUI toolkit -- wxWidgets 3.2. This is the quiet dividend of the 2.8->3.2
port: wxOSX/Cocoa and wxMSW are real targets now, where wx 2.8 would not have
been. - JMS broker -- ActiveMQ and the JMSDispatcher relay are Java.
- Codereg dialogs -- wxPython Phoenix.
- XML -- Xerces-C.
The 3-D viewer -- smaller than feared
src/inv is SGI Open Inventor, vendored (274 sources, Copyright 1996-97
Silicon Graphics), plus moiv (Molecular Inventor) and wxinv -- the
wxWidgets binding written for this modernization, which already replaced SoXt.
Only 12 files touch X11 or GLX directly, and the binding sits on wx and
OpenGL, both portable. The specific ones that need work:
sysfonts.C-- X11 font handling.SoOffscreenRenderer-- GLX pbuffers; wants framebuffer objects instead.SoWxViewer.C/SoWxRenderArea.C-- the binding's own X11 bits.
Decide early: port the vendored Inventor, or move to Coin3D. Coin is an
API-compatible open reimplementation, maintained and already cross-platform,
which would delete most of this problem -- but moiv is SGI's chemistry
extension and would have to build on it. That evaluation gates the viewer work
and should happen before any of it starts.
The scripting layer -- 234 Perl scripts, and a scoping insight
Not all of it has to be portable. Perl runs in two places and only one of
them follows the user to a Mac:
- Client side:
ai.<code>(input generation),wr*GBS.pm/rdStandardGBS.pm
(basis writers),*.expt(import). These must be portable. - Compute-server side:
gensub,eccejobmonitor. These run on the machine
the job runs on, which stays Linux in every realistic deployment.
That roughly halves the exposure. macOS ships Perl; Windows would need
Strawberry Perl bundled.
Eight csh scripts, including scripts/ecce -- the main launcher. csh has
no Windows story and is a deprecated dependency on macOS. This should go
regardless of any port.
Sixteen Fortran sources (NWChem's symmetry library, reached by shelling out
from SymmetryOps::find). Fine with gfortran on macOS; awkward on Windows.
POSIX in the C++
unistd.h 60 files
system() 25 files
kill / waitpid 8 files
fork() 7 files
BSD sockets 5 files
hardcoded /bin/ 13 files
hardcoded /tmp 11 files
All fine on macOS. On Windows, fork has no equivalent and the spawn sites
would need routing through one abstraction -- wxExecute/wxProcess already
covers most of what these do, so this is consolidation rather than
reimplementation.
The dataserver -- the real Windows blocker
A per-user, non-root Apache 2.4 with mod_dav on port 8096, serving the
structure library, basis sets, saved calculations and help.
- macOS: Apache exists, and the per-user non-root model works.
- Windows: no. Bundling httpd is possible; replacing it with a small embedded
WebDAV server would be more portable and would drop a heavyweight
dependency -- but it is a rewrite of a load-bearing component and every save
path goes through it.
Session state keyed by $DISPLAY
ecce-gateway-start keys the dispatcher's port file by $DISPLAY, and several
scripts and 30 C++ files read it. There is no $DISPLAY on macOS or
Windows. This needs replacing with a session identifier that exists
everywhere.
Worth doing anyway: CLAUDE.md already records this keying causing every app to
abort with SIGABRT on a second display (VNC, X-forwarding, a second seat, Xvfb
in CI), and the fix there was to make the pidfile per-display rather than to
question the concept.
Recommended staging
1. Make the tree platform-honest, on Linux, with no behaviour change.
Every item here is independently worth doing and testable today:
- retire csh, starting with
scripts/ecce - route process spawning through one abstraction instead of 25 scattered
system()calls - replace hardcoded
/tmpand/bin/...with portable equivalents - replace
$DISPLAY-keyed session state with a real session id
2. Decide Coin3D versus porting the vendored Inventor. Gates everything
about the viewer.
3. macOS. The realistic target: it is Unix, so Perl, Apache, fork/exec and
gfortran are all present, and the work concentrates in wxOSX behavioural
differences, the viewer's X11 bits, and packaging/notarisation.
Budget a fresh round of toolkit-specific bugs. The GTK3 port produced a
characteristic family of them -- a radio box whose command event never arrived
because of a pushed handler chain, wxGrid::CreateGrid() firing a selection
event synchronously into a half-built panel, layout reentrancy. Cocoa will have
its own set, in the same places, for different reasons.
4. Windows, only after macOS, and only with the dataserver question
answered first.
Worth asking before any of it
What is the actual goal? If it is "colleagues on Macs can use ECCE", then a
container image or a remote/X-forwarded deployment reaches that far sooner and
for a fraction of the cost. Native is the right answer if the goal is
distribution to people who will never touch a terminal -- but that is a
different objective and worth stating out loud, because it changes which of the
four stages above are actually required.
Activity
ohlincha commentedon Sep 28, 2026
Revised plan (Andy, 2026-09-28): split into server and client, then native clients
Strategy: split ECCE into a server package and a client package.
gensub,eccejobmonitor). It serves a lab (the central deployment, which is unconditional) or a single Linux user.This removes this issue's "real Windows blocker" (per-user Apache/mod_dav) from the client. Windows users run the client natively, with the server on a lab machine or under WSL2/Docker.
What the client still needs, in order:
scripts/eccefirst), one spawn abstraction instead of 25system()calls, no hardcoded/tmpor/bin, and a real session id instead of$DISPLAY. Lands on main, in small steps with no behaviour change.Branching: long-lived branch
native/split(local for now), in its own worktree, for the disruptive parts only: the package split, STOMP, the viewer port, and non-Linux packaging. Items 1 and 3 go tomainin small commits, and the branch rebases ontomainregularly so it stays small.Order: the package split on the branch (cheap, and it formalises what
-remotemode already does), with 1 and 3 on main in parallel. Then 2, then 4, then macOS, then Windows.LocalData: the stat timestamp members on macOS (#133)
CI macOS: build with ECCE_USE_COIN against Homebrew's coin3d (#166, #133
ohlincha commentedon Oct 4, 2026
ecce --bugon native clients (2026-10-04). On Linux,--bugis part of the basheccewrapper (packaging/ecce.in) plusecce-diagnose; it stays as it is. It does not carry over: macOS ships bash 3.2 (the script needs bash 4) and has notimeoutor/proc; Windows has no bash and needs a native launcher anyway.Plan: one implementation per platform, sharing only what is common:
ECCE_DEBUG_*switches--bugturns on (read by the C++ programs on every platform) and the list of ECCE's own logs (session, mosquitto, data server if local).log showfor the session's time window, system and GL information (sw_vers,system_profiler SPDisplaysDataType), packed as a zip; started from the launcher (menu item or option).Close-on-exec pipes without pipe2() on macOS (#133)
README: macOS state by reference to #133, not a percentage that goes …
README: roadmap points to #133 for the macOS build state
wxString::c_str() through sprintf printed garbage (MD Prepare orient,…
macOS compile fixes from the first full CI error list (#133)
TreeListCtrl scrolls through wx's own helper on wx 3.3 (#133)
ohlincha commentedon Oct 5, 2026
macOS CI progress on branch
wip/133-macos-build(not merged):dc2878a6transports:consumePendingSignal()(include/util/ConsumeSignal.H) keepssigtimedwaiton Linux and usessigwaiton macOS. It cannot block there, because it is only called aftersigpending()has shown SIGPIPE pending and blocked in this thread.socketpairCloexec()sits next topipeCloexec().319d07baCI: XQuartz and/opt/X11removed; ninja-k 0lists every failing file.45f8a582a real Linux bug:wxString::c_str()passed tosprintfprinted garbage, so MD Prepare's orient command got wrong atom/molecule/segment ids. Apple clang rejects the call.5271950f13 small portability fixes (glibcvalues.h,st_mtim, an arm64 pointer cast, wx 2.8 Carbon code,wxComboBoxnot being awxControlWithItemson wxOSX, BSDsed -i, and others).aac5a5f2TreeListCtrl: Homebrew ships wx 3.3.3, whereHandleOnScroll()is private. On wx 3.3 the event is skipped instead; wx 3.2 is unchanged.Run 37365127429 left only that TreeListCtrl error: everything else compiled. The run after the fix (37367542714) never reached a macOS runner ("not acquired", arm64 capacity), so link status is not known yet.
Linux: full build,
ctest -R transport8/8, tests/launch and tests/queues--suite localpass, and all Linux CI jobs pass.Merge wip/133-macos-build: portable signal/socket helpers, XQuartz dr…
ohlincha commentedon Oct 6, 2026
@mattasplund The native macOS build now compiles and links on GitHub's macOS runner (Homebrew, Cocoa wxWidgets 3.3, Coin3D, no XQuartz): run https://github.com/FriendsofECCE/ECCE/actions/runs/37372957954, main at f9535bf. All applications link, including the Organizer and the Builder.
What changed to get there (all on main): the X11/Xt dependencies are gone on macOS (#232); portable replacements for two Linux-only calls in the remote-shell code; small portability fixes; and a wx 3.3 change in the tree-list control.
What is not known yet: whether it runs. CI only builds, nothing is packaged or started. The next step needs a Mac: start
eccefrom the build tree, log in, open the Organizer and the Builder, and report what happens (withecce --bugwhere something goes wrong). That would be the most useful thing your student could do now; building from main needsbrew installof the packages listed in the macOS job of.github/workflows/build.yml.84 remaining items
ohlincha commentedon Oct 7, 2026
Plan for the fixes: the Retina drawing fix and the Terminal/editor fix are planned for 9.0.0-alpha.8, scheduled for Friday morning (9 October). If either turns out to block your work before then, say so here and we can make an earlier fix build. Anything else you find before Friday will be considered for the same release.
mattasplund commentedon Oct 7, 2026
Those are the only ones that I have seen. I need to confirm that a clean linux 9.0 connect to the remote server to make certain where the bug is. I am still planning to have my students use the 8.0 build so nothing is blocking right now. When I have a proper test I will post.
docs/claude: viewer sizes are framebuffer pixels; macOS terminal and …
Merge wip/mac-fixes: Retina viewer viewport/picking, Terminal.app and…
ohlincha commentedon Oct 7, 2026
@mattasplund thank you for the detailed report. Points 2 and 3 are fixed on
main(merge d8a342f); both are in the next build.3D viewer drawn in the lower-left corner. On a Retina display the OpenGL canvas has twice as many pixels as its size in screen points. The viewer took the size in points as its drawing area, so the molecule was drawn into the lower-left quarter of the canvas. It now uses the pixel size, and mouse positions are converted the same way, so rotating and clicking atoms happen under the cursor. Linux with
GDK_SCALE=2had the same bug and is used as the test: at scale 2 the picture must be 960×960 pixels with the molecule centred, and clicks must select the atom aimed at. Not changed yet: the small status text in the corners ("Mode: Rotate", "6 atoms") is still drawn in pixels, so on a Retina display it is half the normal size.Terminal and editor on macOS. ECCE starts a terminal as
xterm -e command. Terminal.app accepts no command on its command line, so the default terminal on macOS is now a small script included with ECCE. It writes the command to a.commandfile, opens that in Terminal.app, and waits until the command has finished. This is used for Open Shell, shells in a calculation directory and terminal editors such as vi.open -efailed becauseopenreturns immediately. ECCE took that as the end of the edit and deleted the temporary file before TextEdit could read it. ECCE now adds-W -ntoopen, which makes it wait until that copy of TextEdit quits. The default editor on macOS is nowopen -t(your default text editor). Saves appear in ECCE while the editor is open. The edit ends when you quit TextEdit (Cmd-Q); closing its window is not enough. Help and "Open data folder" also useopennow. An editor or terminal set in Edit > Preferences,ECCE_EDITORorECCE_TERMINALstill takes precedence. If you set "open -e" earlier, that setting now works.Please try the next macOS build (the
ECCE-macos-dmg-arm64artifact from the Build workflow onmain, or the next alpha) and tell us whether the viewer fills the window and whether Open Shell and editing an input file work. Point 4 (remote connections) is not affected by these changes. If it still fails, please post the error message.mattasplund commentedon Oct 7, 2026
I have been digging in, and I think that part of the problem that I am having is that some of the scripts are setting ECCE_HOME as /Users/runner/work/ECCE and so some scripts are not running correctly. That may be wrong as I am not certain that I understand all of MacOS's shell quirks.
I have confirmed that a new Ubuntu virtual machine connects to my remote server just find, and that the MacOS computer can connect to the data server from a web browser just fine.
I am trying to debug, but all I get is
ecce-gateway-start: the central ECCE server *** is nto answering on ***:8088 (broker) and ****:8096 (data-server)
ohlincha commentedon Oct 7, 2026
Windows: a per-user MSI is now built next to the zip in the Windows CI job (artifact
ECCE-windows-msi, built with thewix.NET tool v5 directly, not CPack's WiX generator, because the install tree is completed by the bundle scripts aftercmake --install). It installs under %LOCALAPPDATA%\Programs\ECCE without admin, adds a Start-menu entry "ECCE" (wscript +ecce.vbs->ecce.cmd, no console window) and an Apps & Features entry, and upgrades with MajorUpgrade. CI (run 37699090745) installs it withmsiexec /i /qn, runs the local jobs (complete, cancel) and the start test from the installed copy, launches via the shortcut command, uninstalls, and checks the folder is gone and~/ecce-localand~/.ECCEare kept. Size 161.5 MB (596 MB, 12063 files installed). Not yet tried on a machine with a non-admin account.ohlincha commentedon Oct 8, 2026
@mattasplund thanks, both findings are useful.
/Users/runner/work/ECCEin the scripts is a packaging bug on our side: that is the build directory on GitHub's macOS build machine, and it must not end up in the installed app. We are removing it so that the app finds its own location wherever it is installed, and adding a check to the build that fails if the build path appears in a package.We'll post here when there is a build to try.
macOS: wrappers find ECCE_HOME beside themselves (#133)
ecce-gateway-start: portable reachability check that says why (#133)
ohlincha commentedon Oct 8, 2026
@mattasplund thank you, both observations were correct. Both are fixed on
main(0a29510).ECCE_HOME = /Users/runner/work/ECCE/... The 51 command scripts in
ECCE.app/Contents/Resources/bin(ecce,ecce-gateway-start,ecce-organizer, ...) setECCE_HOMEto the CI machine's build directory unless it was already set. Starting ECCE.app from the Finder hid this, because the app's launcher setsECCE_HOMEfirst. Started from a terminal, the scripts looked for ECCE in a directory that exists only on the build machine. The scripts now find the ECCE installation next to themselves, wherever the app has been put, also when they are called through a symbolic link. Every macOS and Linux build now fails if an installed script contains the build directory."not answering on :8088 (broker) and :8096 (data server)". The check that the central server is reachable ran
timeout 5 .... macOS has notimeoutcommand, so the check failed before it tried to connect, and both ports were reported as not answering whether or not the server was up. That is why the Ubuntu VM and your browser could reach the server and alpha.7 on the Mac could not. The check now connects with Perl (included in macOS) and reports the reason for each port, for example:On macOS, a reason of "No route to host" for a server on your local network can also mean that macOS has not given ECCE (or Terminal, if you start it from there) permission to reach the local network. That setting is under System Settings > Privacy & Security > Local Network.
The macOS CI job now runs the packaged app's
ecce-gateway-startin-remotemode against test ports on the runner, without settingECCE_HOME: the server is accepted when the ports answer, and both ports are reported with their reason when they do not.To try: the
ECCE-macos-dmg-arm64artifact of Build run https://github.com/FriendsofECCE/ECCE/actions/runs/37736314864 (or the next alpha). A setup made withecce-remote-setup --user(in~/.ECCE/RemoteServer) is kept; one written into the old app's ownsiteconfigis replaced with the app, so runecce-remote-setupagain in that case. Start it from Terminal withor open ECCE.app as before.
If it still fails, please post:
/Applications/ECCE.app/Contents/Resources/bin/ecce-gateway-status,sw_vers).ohlincha commentedon Oct 8, 2026
Thanks, Matt. Both models make sense to us.
1. All local. This is what 9.0.0-alpha.8 delivers on Linux, macOS and Windows: no data server; the built-in engine and any codes installed on the laptop. On first start ECCE asks "Store data on this computer" or "Connect to a server", and the answer can be changed later.
2. Local broker, remote data server, remote compute. Today this split is not available as such. With
-remote, the client uses the server's broker as well as its data server. A per-user broker on the laptop (as in the local mode) combined with a remote data server looks like a small change, because the broker mainly connects one user's own programs. Two things have to be checked first:The compute side of a cloud machine needs no new mode: it is a registered machine like any cluster.
I've filed this as a design issue: #246.
Documentation: yes, gladly. User help is in
help/src/*.md; build, install and deployment are inGETTING_STARTED.md. A "try it out" walkthrough from a new user's point of view would help most: install on macOS/Windows, the first-start question, registering a machine, a first calculation. Pull requests are welcome; we'll review them.mattasplund commentedon Oct 8, 2026
Latest build from last night/this morning fixed the visual problems. I need to set up a matching data server to test the remote option. I was initially confused about the division of data and messaging, as I see data (and monitoring) as together, but messaging as always local, and this is where we are at.
I have blank Ubuntu and Redhat VMs that I can use for practice. I did a fresh install on Ubuntu, and the apt install ran without issue and put in all necessary support packages. I will test Redhat later today. I also have a blank MacOS users, and when the Windows version is ready, I can test that too. I will put together introduction documents for each platform, with both local and remote computer, and an additional one for remote data.
ohlincha commentedon Oct 20, 2026
Closing: the native Windows (per-user MSI and zip) and macOS (dmg, arm64 and x86_64) clients exist, build in CI and run on real machines (Windows 10/11, macOS 12 Intel, Apple Silicon). Remaining work has its own tickets or work-plan lines: a two-machine test against a central server, the Windows slowness on one managed PC (#386/#387), macOS icons (#354).