Disposable interpreters, which free the memory of what they compiled [skip-ci]

[osschar]

Adds disposable interpreters: cling interpreters that give back all the memory of the code they
compiled when they are destroyed.

Motivation

cling never frees the code it compiles. Every line gInterpreter processes stays in memory until
the process ends, about 32 kB per line for REve's method calls. Code that is needed only once, such
as REve's calls from the browser or jitted expressions, can instead be compiled in an interpreter of
its own, which is destroyed afterwards.

Usage

auto d = gInterpreter->CreateDisposable();
d->Declare("#include \"ROOT/REveBox.hxx\"");
d->ProcessLine("((ROOT::Experimental::REveBox *)0x...)->SetMainColor(3);");
// ... more work ...
d.reset();   // all the memory of the compiled code is returned

A disposable interpreter is created with the same arguments as gInterpreter, so it finds the same
headers and C++ modules. Creating one takes about 60 ms, and loading REve's declarations into it
another 35 ms (Debug build). It does not see what was declared in gInterpreter. Nothing may point
into its code once it is destroyed.

Changes

Three things in cling kept the memory of a destroyed interpreter. The first two commits deal with
them.

1. [cling] Let the ASTReader be freed with its interpreter. The ASTReader leaked with every
   interpreter, gInterpreter included. This bug fix stands on its own.

2. [cling] Add --disposable, for interpreters that free all their memory. cling sets
   DisableFree for every interpreter, so the AST is never deleted, and it never releases JIT
   memory. An interpreter created with the new option does both.

   The JIT release was disabled in TClingCallFunc needs to support unloading root-project/root#10898, because unloading with .U freed code that
   TClingCallFunc wrappers still pointed to. It stays disabled for gInterpreter, and
   TClingCallFunc never uses disposable interpreters.

3. [core] Add TInterpreter::CreateDisposable(). The interface TInterpreter::Disposable, with
   Declare() and ProcessLine(), its implementation in core/metacling, and tests. ROOT's LLVM
   is built without thread support, so a disposable interpreter takes gInterpreterMutex for
   everything it does.

4. [eve7] Optionally execute MIRs in a disposable interpreter. An example of use, which can be
   dropped.

Results

17 different REve setter calls on four elements, 6000 calls in all (Linux, Debug build):

	time per 1000 calls	memory kept at the end
gInterpreter->ProcessLine()	8.7 to 9.6 s	196 MB heap, 271 MB RSS
a new disposable interpreter every 1000 calls	7.4 s	0.24 MB heap, 3 MB RSS

What the disposable interpreters kept stayed from the first one. From the second on, each kept
nothing.

With the first three commits, ROOT's tests and roottest pass, apart from three that fail the same
way without these changes. REve's tests pass with the fourth.

The REve example

REve executes method calls from the browser, such as SetMainColor(3) on element 17, through
TMethodCall. With WebEve.MIRInterpreter: Disposable in .rootrc, it compiles them in a
disposable interpreter instead, replaced every 1000 calls.

TMethodCall is about 40 times faster for the calls it can handle. The disposable interpreter is
for the calls it cannot: a call with a string argument, such as SetTitle("box"), crashes REve's
server through TMethodCall. Compiled, it works, and the memory stays bounded by one live
interpreter:

6000 calls	time per 1000 calls	heap kept
TMethodCall	0.14 s	grows by 0.33 MB per 1000
disposable, replaced every 1000 calls	5.4 s	45 MB, constant
disposable, replaced every 100 calls	6.6 s	22 MB, constant

REve also compiles lambdas for collection filters and table columns. These are kept in
std::functions, which would have to be reassigned before their interpreter is replaced, so they
are not part of this.

Open questions

• Should the ASTReader fix go in on its own first?
• Is TInterpreter::CreateDisposable() the right shape and place for the API?
• Would RDataFrame use it, eg one disposable interpreter per computation graph?
• Is concurrency wanted? In a build with LLVM_ENABLE_THREADS=ON, disposable interpreters ran
  concurrently on 4 threads, and helgrind found two races in cling, both with small fixes.

🤖 Generated with Claude Code