Full tailcall support on Unix

[jkotas]

It is important to have uniform managed tailcall support accross platforms for F#.

theme:tail-call

[jkotas]

The "slow" helper based tailcalls are missing - see dotnet/coreclr#703.

The existing design the "slow" helper based tailcalls has a strong dependency on how varargs work on Windows (it assumes that va_list is castable to void* that does not hold on Unix), and it is super complex (requires ~500 line handwritten assembly thunk emitter). I believe the contract between JIT and VM for slow tailcalls needs to be redesigned to make the Unix implementation feasible.

[dsyme]

Super, thanks Jan.

If possible, could you add some notes about when the slow tailcall helper is used (which AFAIU should correspond to when tailcalls will currently not be taken for CoreCLR on Unix?). It would be useful to have docs on when the slow helper is used on Windows in any case.

[jkotas]

when the slow tailcall helper is used

This is implemented in fgCanFastTailCall method https://github.com/dotnet/coreclr/blob/master/src/jit/morph.cpp#L5715.

The simple rule that should hold across platforms over time is that the fast tailcall will be used if both of the following are true:

• Return value and call target arguments are all either primitive types, reference types, or valuetypes with a single primitive type or reference type fields
• The aligned size of call target arguments is less or equal to aligned size of caller arguments

There are additional platform-specific and implementation-specific situations where the fast tailcall will be used that can be found by consulting fgCanFastTailCall sources.

[DemiMarie]

I had a rather wild thought: what about using a different calling convention (callee-pops) for tail calls?

This would break lots of stuff on Windows, but as I understand it DWARF unwinding is more flexible.

[janvorli]

Managed code unwinding is not using DWARF on Unix. We use the same unwind info format as on Windows. Also, having a completely different calling convention for selected functions would complicate a lot of stuff for runtime (e.g. for reflection) and JIT.

[DemiMarie]

@janvorli What about using explicit frame pointers? I think that LLVM can do this too, even on Windows (source: LLVM supports tail calls on Windows, and I can't see any other way that this could be implemented).

Why does the calling convention impact reflection?

[janvorli]

@DemiMarie the reflection prepares arguments for a function call based on the calling convention - putting register arguments into appropriate slots in a transition block structure and the others on the stack. It then transitively calls assembler helper CallDescrWorkerInternal that loads registers from the transition block, prepares stack, calls the actual method invoked via reflection and after it returns, stores its result into the CallDescrData. See RuntimeMethodHandle::InvokeMethod here:
https://github.com/dotnet/coreclr/blob/master/src/vm/reflectioninvocation.cpp#L1322

Regarding a different calling convention, there is another problem. Any method can be called via tail call in general and the same method can be called from another place using regular call. So having a different calling convention for tail calls would mean you'd have to have two jitted versions of the same function in case a function was called both ways.

As for the explicit frame pointers - can you please be more specific on your idea on taking advantage of these?

[DemiMarie]

@janvorli yes, I know that some methods would be JITed twice. My hypothesis is that this would be an acceptable overhead for the use cases (as an alternative to slow tail call).

Regarding explicit frame pointers, my basic idea is to figure out exactly what LLVM does, and do that. I don't know the details, but according to this source a frame pointer can be used to allow the stack pointer to be manipulated. I know (from the LLVM documentation) that LLVM can support tail calls on Windows/x64, and I strongly suspect that it supports unwinding, as otherwise debugging would be impossible.

Regarding reflection: under my proposal, this would be done using the version of the method with the standard calling convention.

[cartermp]

We're now hitting this more prominently with F# on Unix being far more widespread. We're also going to have a .NET Core-based editing experience in VSCode and eventually VS for Mac.

[BruceForstall]

cc @jashook @jakobbotsch

[AndyAyersMS]

@erozenfeld we can close this now, right?

[erozenfeld]

Yes, this is fixed by #341.