prov/tcp: Process EPOLLERR properly

[jxiong]

The original code just clears MSG_ERRQUEUE but not handle SO_ERROR, which makes the socket always have something to report from epoll.

[jxiong]

It would be nice if we could fetch remote address here in the log message, something like this:

ssize_t get_remote_addr(char *buf, size_t buflen, int sockfd) {
        struct sockaddr_in remote_addr;
        socklen_t addr_len = sizeof(remote_addr);

        if (getpeername(sockfd, (struct sockaddr *)&remote_addr, &addr_len) == -1) {
    strerror_r(errno, buf, buflen);
                return 0;
  }

        char remote_ip[INET_ADDRSTRLEN];
        if (inet_ntop(AF_INET, &(remote_addr.sin_addr), remote_ip, INET_ADDRSTRLEN) == NULL) {
    strerror_r(errno, buf, buflen);
                return 0;
  }

        int remote_port = ntohs(remote_addr.sin_port); // Network to host short

        return snprintf(buf, buflen, "%s:%d", remote_ip, remote_port);
}

but I don't know how to make more generic in the context of libfabric

@shefty @j-xiong FYI