Viosock : recv stuck in client, while the socket is being SHUTDOWN from host

[54shady]

Describe the bug

on client side, stuck while call the recv, even using the latest driver code

To Reproduce
Run attachment application as below description

server.txt
mv server.txt server.c and compile it to run on linux host
run server.c on the Linux host

client.txt
mv client.txt client.cpp and compile it to run in windows guest

Expected behavior

on client side, recv can return after the host SHUTDOWN the connection

Screenshots

on server , the socket is being close

on client windows, the recv stuck:

Host:
seems no related to the host and qemu version and command line options etc.

• Disto: ubuntu

• Kernel 6.2.0-31-generic

• QEMU emulator version 7.0.50

• QEMU

• qemu-system-x86_64
  ...
  -device vhost-vsock-pci,guest-cid=3
  ...

VM:

• Windows win10 18363.592
• vsock
• the latest vsock driver

the problem i found when i comment out the server.txt line 192, which sleep 10 second after close the socket
what if i comment out the line 193, which do a write on host side, the recv stuck issue on client side is disappear

[MartinDrab]

Hello,

can you provide a kernel memory dump for the WIndows VM?

[54shady]

Hello,

can you provide a kernel memory dump for the WIndows VM?

I config the kernel memory dump in windows, but it seems there is no system crash or a BSOD which generate the dump file?

[MartinDrab]

Hello,
can you provide a kernel memory dump for the WIndows VM?

I config the kernel memory dump in windows, but it seems there is no system crash or a BSOD which generate the dump file?

With a bit more config, you should be able to initiate the system crash (thus, generate a kernel memory dump) manually. I think the Keyboard method should be best for your case.
https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/forcing-a-system-crash

Also, can you point me to the version of the Virtio drivers you are using (especially the viosock.sys)? If you compiled them yourself, I would be glad to get viosock.sys and viosock.pdb (the debug symbols for the driver) if you have these files.

Thanks.

[54shady]

Hello,
can you provide a kernel memory dump for the WIndows VM?

I config the kernel memory dump in windows, but it seems there is no system crash or a BSOD which generate the dump file?

With a bit more config, you should be able to initiate the system crash (thus, generate a kernel memory dump) manually. I think the Keyboard method should be best for your case. https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/forcing-a-system-crash

Also, can you point me to the version of the Virtio drivers you are using (especially the viosock.sys)? If you compiled them yourself, I would be glad to get viosock.sys and viosock.pdb (the debug symbols for the driver) if you have these files.

Thanks.

I test the latest version driver on the github, which seems the bug can be reproduce too
I will try to get a kernel memory dump later

[54shady]

dump00.zip
dump01.zip
dump02.zip
dump03.zip

the dump00-04 contain the windows vm kernel memory dump file, and viosock.sys, viosock.pdb file i use

i reproduce the test flow and trigger the windows to dump the memory

[54shady]

@MartinDrab any progress for the issue now? or is there any info that i need to provide?

BTW i checkout the code in Socket.c as below, which comment out some lines
which i guess if it relate to this kind of issue?

when the guest recv the SHUTDOWN form peer, and by then, guest close the recv end by force?

VIOSockShutdownFromPeer(
    PSOCKET_CONTEXT pSocket,
    ULONG uFlags
)
{
....
//     if (uDrain & VIRTIO_VSOCK_SHUTDOWN_SEND)
//         VIOSockReadDequeueCb(pSocket, WDF_NO_HANDLE);
//     if (uDrain & VIRTIO_VSOCK_SHUTDOWN_RCV)
//     {
//         //TODO: dequeue requests for current socket only
//     }

Then I try to uncomment the line above, and do a WdfRequestComplete(ReadRequest, STATUS_CANCELLED); in function VIOSockReadDequeueCb, but i currently cannot figure out where should i do this. or is it doable like this?

[MartinDrab]

DMP and PDB files should be enough for me. I will look into it tomorrow and let you know.

[MartinDrab]

Hello,

I reproduced the issue also with both server and the client on one (Windows) VM. Looking at the code, the shutdown signal is sent during socket close, however, the recv request seems not to be completed. I will look into it deeper.

[MartinDrab]

You may try #1036, that should fix your issue.