LSL for Apple Vision Pro?

[AviLPA]

Hi!

I am trying to connect LSL and Labrecorder to an App in AVP.

I am unable to get the streams to appears once downloaded and pushed through Xcode for the Vision Pro....

Any ideas on this?

[cboulay]

It's almost always something to do with enabling permissions for multicasting.

[TienEnChang]

@AviLPA Have you been able to receive LSL stream since? I'm having the same issue on iOS even after enabling multicast networking ...

[StevanD02]

I managed to get it working on iOS.

You need to have an active Apple Developer account. After that, you need to obtain
https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.networking.multicast
via the request form listed on the link.

After that, both inlets and outlets work normally.

[TienEnChang]

I managed to get it working on iOS.

You need to have an active Apple Developer account. After that, you need to obtain https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.networking.multicast via the request form listed on the link.

After that, both inlets and outlets work normally.

Hi Stevan, thanks for your response. However, I believe I've done all the steps you've mentioned above. Can you elaborate more on the implementation details you have?

My previous attempt was as follows: (1) Install the LSL4Unity package (2) Add a modification of the outlet example code in my program that sends markers periodically (3) Build the project as an iOS app first in Unity and then in Xcode (with multicasting permission enabled).

The build was successful but once we launched the app, the Xcode debugger ran into an error saying that iOS does not support the .dylib file type in which the liblsl for macOS was compiled (there's no liblsl dedicated for iOS inside the LSL4Unity package). Can you tell me how did you resolve this issue? Much appreciated!

Edit: By the way, I've tested the exact same program compiled into a macOS app and it worked just fine. So that rules out the possibility of bugs in my modified code.

[cboulay]

You'll likely have to recompile the dylib targeting the correct iOS version.

[StevanD02]

Chadwick is right.

Our project was setup so that it builds the whole lsl lib from scratch (we've imported lib LSL into our project as a C++ target), and then we instructed Xcode on how to build it, and then used that. To bridge it to Swift, we had to write ObjectiveC wrapper around C API.

Right now, with Swift 6, they claim to have full C++ interoperability, so maybe it can be used directly from Swift code. That would be nice to know :)

[TienEnChang]

@StevanD02 Got it, but I don't know how to modify the source code to tell Xcode to compile for the iOS version I currently have. Perhaps can I get the already working Unity project you have and start from there by matching my iOS version to yours?

Also @cboulay can I ask you what kind of issues would my current workaround of sending markers from iPad through TCP to a dedicated marker relay for LSL on desktop have? Do you strongly advise against this method or is the latency kind of tolerable?

[StevanD02]

Unfortunately I cannot help you with Unity project, as our app was pure SwiftUI iOS app, fully created and built with Xcode :/
I also do not remember it explicitly creating .dylib artifacts, I think they were implicitly packaged with the app by Xcode, if they were created as .dylib files at all, as everything was being built together, both C++ and Swift.

[TienEnChang]

@StevanD02 Oh I thought the whole time your project was built with Unity, sorry that I misread your response.

So basically, what you're saying is that the only plausible (and proven to work) way of doing it is to abandon the LSL4Unity package entirely and build from scratch with the original C++ liblsl and write a C# wrapper for Unity?

If that's the case, it would be very helpful if you could kindly share just the ObjectiveC wrapper part of the code and the instructions to build it with Xcode (I assume it's build in C++, which we could also make use of with a C# wrapper)?

As for .dylib, I think you're right that there should be no such file since the correct procedure is to build the whole LSL library together with the iOS app, instead of what is currently done by LSL4Unity which is to use a .dylib file (it stands for dynamic linking).

Edit: If my attempt turns out to be successful, I'll make sure to provide a public tutorial for future researchers who need the same LSL support for their iOS Unity app 🙂

[cboulay]

@TienEnChang , I think @StevanD02 is just saying that was their solution. But given that it's the same hardware and operating system, it's proof that the protocol works.

I think that you can use LSL4Unity if you replace the dylib with one that you build yourself.

[TienEnChang]

@cboulay Sorry it's just imprecise wording, and I've corrected my comment.

It's plausible that I can reuse most of the C# wrapper written inside LSL4Unity, but I still very much need to know how Stevan built liblsl with Xcode as a static library together with the whole program. I'm afraid that it's not as simple as replacing the .dylib with my own build because I've already mentioned that Xcode debugger says iOS does not support .dylib.

[cboulay]

If I had to guess, it's the same as any other open source 3rd party library. There are really 2 options for static:

1. download the liblsl source code and build it as a static library (read the CMakeSettings file for ideas on how to do this), then link to it from your app.
2. add the liblsl source code to your project and build it just as you would any other set of source files that are part of your project.

[TienEnChang]

@cboulay Got it. I know it sounds really simple as the way you described it. But as someone who has little experience with Xcode as an independent compiler, it still very much feels like a headache trying to implement the options you've mentioned without a proper guide (I can only hope that working with ChatGPT is able to get this job done 🥲). Not to mention that I don't really know how much of the C# wrapper in LSL4Unity is reusable when we change from dynamic linking to static linking. So having a working wrapper for static liblsl as a reference could be very beneficial.