Support for compile-time plugins from external source directories

[ingowald]

This adds support for a plugin mechanism in which external source directories can extend barney by additional geometry and spatial field types.

The way this works:

• at cmake configure time, the user can supply a list of additional build directories via -DBARNEY_PLUGIN_DIRS=...
• barney's build process then includes all these directories in the build, assuming each directory can build itself, and link itself into the final libananri_library_barney[_backend].so (ie, plugins get included into barney itself, they do NOT become additional shared libraries!).
• each such plugin directory can make use of the PluginInfrastructure in barney/anari/BarneyGlobalState.h to define a 'Plugin', to be registered under a plugin-specific name (eg, 'myPlugin'). Each such plugin can then define a set of geometries and/or spatial fields it wants to support (eg 'myNewGeom').
• at runtime, any anariNewGeometry('myNewGeom@myPlugin') will make barney try, on he current backend, to load the specific 'myPlugin', and, assuming that was indeed included in the build process, ask this plugin to create a geometry of type 'myNewGeom'.

The key benefit of this way of handling plugins is that they are not additional shared libraries, and also do not require barney to be built with any whole_archive etc flags, because they can actually link to the respective barney_static_backend while the final libanari_library_barney.so is built. They are also fully included in the backend-specific build path of barney, so will automatically get built for whatever backends barney itself gets built for.

[tarcila]

I feel dlopen and friends are non obvious, error prone and I think unnecessary in our current compile time plugin case.

I'd rather have:

---

A static initialization triggering the plugin initialization, possibly using an immediately invoked lambda. Fully implicit, no symbol search, portable with our supported compilers.
Something like (untested):

class BarneyGlobalState {
  ...
#define BARNEY_REGISTER_PLUGIN(name) \
  namespace { \
    const auto init = []() { \
      registerPlugin_barney_@BARNEY_BACKEND_NAME@_##name(...); return 0; }(); \
  }

Those static lifetime variables are guaranteed to be initialized before barney device is created.
This approach, along with the current one, is having the issue that any mis-registration (forgetting to call the macro), will silently ignore the plugin.

Worth noting, static library linking might be an issue if the plugin itself does:

add_library(myPlugin STATIC ...)
...
target_link_libraries(barney PRIVATE myPlugin)

as only used symbols will be pulled from the archive, which might leave out the initialization variable. A way to solve that it to use CMake OBJECT library instead of STATIC.

---

Another approach, more explicit, is to use the fact that we know all plugins at CMake time. From there we can generate the initialization call list in a function that is called at device init time. Basically completing the foreach (plugin_dir ${BARNEY_PLUGIN_DIRS}) loop in barney/anari/CMakeLists.txt to write out that cpp file.

The plugin then needs to advertise a name, either deduced from its folder name or, possibly using CMake set_properties. Any mis-registration now fails at compile time, which I like a lot.

[iwald-nvidia]

Re "OBJECT" library - yes, that is correct, the plugin registration has to be OBJECT, not static; my sample(s) use that, too. Note this is only required for the one file that has the public symbol, that itself can then link to a static library; but yes, that one registry file has to be object.

Re dlopen: I'm not a huge fan of dlopen, either, BUT: static initialization unfortunately is not guaranteed to happen in the right order, i've run into issues with that in the past: the problem is that the registerPlugin() function itself has to append itself to some kind of global registry, but there is no guarantee that this is already initialized. The initialization of global variables in different compiliation units of the same shared library depends on link order.

The generation of a list of plugins-to-be-loaded in the cmakefile is an interesting alternative that i've also thought about, but haven't found a good way of doing it. A simply string that lists all the names is easy to generate, but isn't enough, because somebody would still have to translate strings to symbols (at which point we'd be back to dlopen/dlsym). The only way to generate true symbol names for compiler and linker would be to have cmake generate actual c++ code, and that is also somewhat daunting.... but possible, if you think that better than dlsym.

[tarcila]

static initialization unfortunately is not guaranteed to happen in the right order

Indeed, the usual trick is using a function local static, runtime allocated once and leaked:

static PluginRegistry &get() {
  static PluginRegistry *instance = new PluginRegistry;
  return *instance;
}

All accesses to the registry then goes through calling PluginRegistry::get(). C++11 guarantees this is initialized once on first access and thread-safety. Maybe std::once could help here too? Probably overkill compared to that simple approach tho...

Another way maybe... C++20 enables constinit construction for vectors (std::map or like are not yet supported I think).
So, it means that we should be able to default constinit the vector so it is always ready by the time the static init functions are called.

// In BarneyGlobalState
struct PluginEntry { std::string_view name; PluginInfrastructure::Plugin *(*create)(); };
static inline constinit std::vector<PluginEntry> plugins{};

The only way to generate true symbol names for compiler and linker would be to have cmake generate actual c++ code

That's actually what I was having in mind. Something along the lines of:

# plugin_names holds the names gathered from BARNEY_PLUGIN_DIRS
set(PLUGIN_DECLS   "")
set(PLUGIN_ENTRIES "")
foreach(name IN LISTS plugin_names)
  string(APPEND PLUGIN_DECLS   "PluginInfrastructure::Plugin *registerPlugin_${name}();\n")
  string(APPEND PLUGIN_ENTRIES "  { \"${name}\", &registerPlugin_${name} },\n")
endforeach()
file(CONFIGURE OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/Plugins.cpp
  CONTENT [[
#include "anari/BarneyGlobalState.h"
namespace BARNEY_NS { namespace anari {
@PLUGIN_DECLS@
const PluginInfrastructure::Entry PluginInfrastructure::plugins[] = {
@PLUGIN_ENTRIES@
  { nullptr, nullptr }
};
}}
]] @ONLY)
target_sources(anari_library_barney_${backend}_static PRIVATE
  ${CMAKE_CURRENT_BINARY_DIR}/Plugins.cpp)

The plugin declare macro needs to be updated to:

#define BARNEY_REGISTER_PLUGIN(name) \
  BARNEY_NS::anari::PluginInfrastructure::Plugin *BARNEY_NS::anari::registerPlugin_##name()

With the plugin code doing something like:

BARNEY_REGISTER_PLUGIN(myPlugin) {
 auto *plugin = new PluginInfrastructure::Plugin;
  plugin->exportGeometry(...);
  return plugin;
}

Static initialization might be the simplest way to deal with that, no CMake infrastructure change or like, even simpler if the C++20 route can work. I like the fact that a wrongly declared plugin will fail building with the CMake based approach, but that more invasive changes.

[iwald-nvidia]

Hm. I'm didn't think that static locals are guaranteed to be initialized before globals initialization has run - I'd have naively assumed to just be implicit globals and be initialized in that same order - but it's certainly worth a try. I'll save current version and rework to try that out... we can always go back :-)

[iwald-nvidia]

Just pushed an update in which i use a local static member to provide a list of plugins that each plugin can then add itself in its globals initialization. This leaves it to linker's globals init order in which order the plugins to register themselves, but - at least if the local static members does indeed initialize on first call even if that call itself happens in globals init - the should always have a valid list to add themselves to.

note i split plugin initialization into two stages: first stage (in globals init) the plugin just registers an init function, but doesn't yet register any geom/spatial types (becuase who knows what is or isn't initialized at this stage, yet). Those initfunctions then get called by BarneyGlobalState (only) when this state gets created, at which time these plugins can export their geom/spatial type(s) into a valid barneyglobalstate.