Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INSTREAM is slower then SCAN #1424

Open
marijusGood opened this issue Dec 19, 2024 · 1 comment
Open

INSTREAM is slower then SCAN #1424

marijusGood opened this issue Dec 19, 2024 · 1 comment

Comments

@marijusGood
Copy link

Describe the bug

While using INSTREAM we noticed that the speed to scan a file is the same or slower if you just used SCAN and point to a file. We thought the whole idea of INSTREAM is so that it would scan the file while its being read. Could you please help us diagnose the problem or did we miss understood INSTREAM.

How to reproduce the problem

put your files in a dir and run to get the baseline of SCAN:
sudo chmod 777 *
chgrp clamav *
echo "SCAN /yourDir" | nc localhost 3310

For INSTREAM we used java and we send 1k chunks, here is a snippet:
`public String scan(InputStream inputStream) throws IOException {
try (Socket socket = new Socket(CLAMAV_HOST, CLAMAV_PORT);
OutputStream out = socket.getOutputStream();
InputStream in = socket.getInputStream()) {

        out.write("zINSTREAM\0".getBytes());

        byte[] buffer = new byte[1024];
        int bytesRead;

        while ((bytesRead = inputStream.read(buffer)) != -1) {
            byte[] sizeBytes = intToByteArray(bytesRead);
            out.write(sizeBytes);
            out.write(buffer, 0, bytesRead); // Send chunk
            out.flush();
        }

        out.write(new byte[] {0, 0, 0, 0});
        out.flush();

        ByteArrayOutputStream responseBuffer = new ByteArrayOutputStream();
        byte[] responseBytes = new byte[1024];
        int responseLength;

        while ((responseLength = in.read(responseBytes)) != -1) {
            responseBuffer.write(responseBytes, 0, responseLength);
        }

        return responseBuffer.toString("UTF-8");
    }
}

private byte[] intToByteArray(int value) {
    return new byte[] {
        (byte) ((value >>> 24) & 0xFF),
        (byte) ((value >>> 16) & 0xFF),
        (byte) ((value >>> 8) & 0xFF),
        (byte) (value & 0xFF)
    };
}`
@micahsnyder
Copy link
Contributor

ClamAV need the whole file to do a scan. Many file formats put important structural bits at the end, such as the file entry directory of a ZIP. Some file formats like DMG can't even be identified without seeking to the end of the file.

The INSTREAM feature copies a file over a TCP stream to ClamD . ClamD writes the streamed bytes to a temp file on disk and then scans the temp file. It is going to be slower than scanning a file directly.

The purpose of INSTREAM is make a file available to ClamD that it would not normally have access to. E.g.

  1. if the scan client (your program, clamdscan, clamonacc, etc) is on a different computer than ClamD , or
  2. if the ClamD process does not have read-permissions to the file.

The second case may also be solved on unix / linux systems using the FILEDES (file descriptor passing) feature. In this mode, the scan client would open the file and then transfer the open file descriptor to ClamD over a UNIX socket. The unix or linux kernel translates the file descriptor from one process to the other, giving ClamD the ability to read from the open file even though it would not have permission to open the file itself.

File descriptor passing would be the same speed as having ClamD scan a file directly. But it requires using the UNIX socket (clamd.conf "LocalSocket" option) and not the TCP socket. It is a bit more complicated. You can see how we do it in C, here: https://github.com/Cisco-Talos/clamav/blob/main/common/clamdcom.c#L175-L216
I don't know how to do it in Java.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants