Repository navigation
uvloop does not support original unix udp socketpair #304
Description
Activity
This seems to be a low-level behavior:
#include <unistd.h> #include <stdint.h> #include <stdlib.h> #include <stdio.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <sys/errno.h> #include <sys/ioctl.h> void do_send(int fd) { int32_t w_buf = 32; write(fd, &w_buf, sizeof(w_buf)); } void do_recv(int fd) { // copied from uv__udp_recvmsg struct sockaddr_storage peer; struct msghdr h; ssize_t nread; struct iovec buf; int r; int nonblocking = 1; buf.iov_base = malloc((32) * sizeof(char)); buf.iov_len = 32; memset(&h, 0, sizeof(h)); memset(&peer, 0, sizeof(peer)); h.msg_name = &peer; h.msg_namelen = sizeof(peer); h.msg_iov = (void*) &buf; h.msg_iovlen = 1; // copied from uv__nonblock_ioctl do r = ioctl(fd, FIONBIO, &nonblocking); while (r == -1 && errno == EINTR); if (r) { printf("ioctl errno: %d %s\n", r, strerror(errno)); } nread = recvmsg(fd, &h, 0); if (nread == -1 || errno != 0) { printf("%ld %s\n", nread, strerror(errno)); } else { printf("nread=%ld sa_family=%d msg_namelen=%d\n", nread, ((struct sockaddr *)&peer)->sa_family, h.msg_namelen); } } int main() { int socket_vector[2]; // use either SOCK_DGRAM or SOCK_STREAM if(0 != socketpair(AF_UNIX, SOCK_DGRAM, 0, socket_vector)) { perror("cannot create socket pair"); return 2; } do_send(socket_vector[0]); do_recv(socket_vector[1]); }
OS socketpair type nonblocking peer.sa_family msg_namelen Linux SOCK_DGRAM 0 AF_UNSPEC 0 Linux SOCK_DGRAM 1 AF_UNSPEC 0 Linux SOCK_STREAM 0 AF_UNSPEC 0 Linux SOCK_STREAM 1 AF_UNSPEC 0 macOS SOCK_DGRAM 0 AF_UNIX 16 macOS SOCK_DGRAM 1 AF_UNIX 16 macOS SOCK_STREAM 0 AF_UNSPEC 0 macOS SOCK_STREAM 1 AF_UNSPEC 0 And
((struct sockaddr_un *) &peer)->sun_pathis always an empty string.Related code:
- https://github.com/libuv/libuv/blob/0ed9692a80b2f310ad52060a8cb400f2a963d09c/src/unix/udp.c#L285-L301
Line 335 in 2870219
Lines 64 to 68 in 2870219
@ZhuYuJin 's code fails because
__convert_sockaddr_to_pyaddrdoesn't supportAF_UNSPEC. @1st1 shall we patch__convert_sockaddr_to_pyaddrso that it returns an empty string if the family isAF_UNSPEClike this (Solution A)?diff --git a/uvloop/dns.pyx b/uvloop/dns.pyx index a252a24..843b5d5 100644 --- a/uvloop/dns.pyx +++ b/uvloop/dns.pyx @@ -65,6 +67,9 @@ cdef __convert_sockaddr_to_pyaddr(const system.sockaddr* addr): addr_un = <system.sockaddr_un*>addr return system.MakeUnixSockPyAddr(addr_un) + elif addr.sa_family == uv.AF_UNSPEC: + return "" + raise RuntimeError("cannot convert sockaddr into Python object")
Alternatively, we could patch
udp.pyxto handle it like this (Solution B):diff --git a/uvloop/handles/udp.pyx b/uvloop/handles/udp.pyx index 675407d..5a0f225 100644 --- a/uvloop/handles/udp.pyx +++ b/uvloop/handles/udp.pyx @@ -330,6 +330,8 @@ cdef void __uv_udp_on_receive(uv.uv_udp_t* handle, if addr is NULL: pyaddr = None + elif addr.sa_family == uv.AF_UNSPEC: + pyaddr = "" else: try: pyaddr = __convert_sockaddr_to_pyaddr(addr)
Or less efficiently but support potential future changes to
__convert_sockaddr_to_pyaddr(Solution C):diff --git a/uvloop/handles/udp.pyx b/uvloop/handles/udp.pyx index 675407d..16aced6 100644 --- a/uvloop/handles/udp.pyx +++ b/uvloop/handles/udp.pyx @@ -334,8 +334,11 @@ cdef void __uv_udp_on_receive(uv.uv_udp_t* handle, try: pyaddr = __convert_sockaddr_to_pyaddr(addr) except BaseException as exc: - udp._error(exc, False) - return + if addr.sa_family == uv.AF_UNSPEC: + pyaddr = "" + else: + udp._error(exc, False) + return if nread < 0: exc = convert_error(nread)
Yeah, I think it is from Linux kernel - I tried to track it down but it's a lot of code to read, so we might want to work it around in uvloop first.
C or B would work. How does CPython deal with this btw?
That's a really good point - let me check.
Looks like it'll just return
None:static PyObject * makesockaddr(SOCKET_T sockfd, struct sockaddr *addr, size_t addrlen, int proto) { if (addrlen == 0) { /* No address -- may be recvfrom() from known socket */ Py_RETURN_NONE; }
Guess solution D would be to update
__convert_sockaddr_to_pyaddrto include this logic.Guess solution D would be to update __convert_sockaddr_to_pyaddr to include this logic.
Yep!
Bad news: libuv didn't provide the
addrlen(themsg_namelenbelow as a value-result argument) for callbacks (I'll file an issue in libuv):static void uv__udp_recvmsg(uv_udp_t* handle) { struct sockaddr_storage peer; struct msghdr h; ... memset(&h, 0, sizeof(h)); memset(&peer, 0, sizeof(peer)); h.msg_name = &peer; h.msg_namelen = sizeof(peer); ... handle->recv_cb(handle, nread, &buf, (const struct sockaddr*) &peer, flags);
And I don't think it is a reliable way to check if the first byte is
\0as an equivalent to checkingaddrlen == 0in__convert_sockaddr_to_pyaddr, because we don't know for sure whataddr->sa_familyis.In this case, I would go for solution B if no objections.
By the way, @ZhuYuJin 's code doesn't work with vanilla asyncio - it uses
sendtoifpeernameis not set:def sendto(self, data, addr=None): ... if self._extra['peername']: self._sock.send(data) else: self._sock.sendto(data, addr)
But the
peernameis always empty for asocketpair:import socket s1, s2 = socket.socketpair(socket.AF_UNIX, socket.SOCK_DGRAM, 0) print("peername:", repr(s1.getpeername())) s1.sendto(b'xxx', None) s1.sendto(b'xxx', b'')
peername: '' Traceback (most recent call last): File "t304.py", line 5, in <module> s1.sendto(b'xxx', None) TypeError: a bytes-like object is required, not 'NoneType' Traceback (most recent call last): File "t304.py", line 5, in <module> s1.sendto(b'xxx', b'') OSError: [Errno 56] Socket is already connectedShould we consider this a bug in CPython asyncio and fix it there too?
Yeah, let's to B.
Bad news: libuv didn't provide the addrlen (the msg_namelen below as a value-result argument) for callbacks (I'll file an issue in libuv):
I think I also encountered this limitation myself one time, so resolving this at the libuv level would be great. IIRC I also had problems with dealing with abstract unix sockets in libuv, and maybe it was somehow related to this, I don't remember now.
Reacted by Fantix King- added 3 commits that reference this issue
on Apr 27, 2020 - added a commit that references this issue
on Apr 28, 2020
I initialize a unix udp socketpair, and try to use uvloop to manage both sockets.
The code above has the following err:
However, when I bind address to both sockets, the err disappears.
It shows uvloop does not support the original unix udp socketpair.