[Bug]: Importing via remote disk s3 failing due to error in ReadChunk

[Ashay84]

Is the bug applicable and reproducable to the latest version of the package and hasn't it been reported before?

• Yes, it's still reproducable

What version of Laravel Excel are you using?

^3.1

What version of Laravel are you using?

^10.0

What version of PHP are you using?

8.2.27

Describe your issue

I am using this package to import file entries to database through remote file stored on s3 using chunk reading and queues.
Our server setup is build on top of docker and using aws cloud container service .
The issue is the imports are working properly when build is run on local but failing when run on this configuration on aws ECS .

1. I have checked s3 object permissions and also updated the excel.php config file as given in docs.
2. I have also checked file permissions and they are set to web server group www-data .
3. Tried setting no_of_procs on supervisor to 1
4. s3 is setup using package league/flysystem-aws-s3-v3 and filesystems.php was checked and is working to upload files .
5. Dev ops team also checked following

• ECS Task Operating system/Architecture : Linux/ARM64
• Also tried upscaling the resources. The file we were trying to process here is just 170 bytes and the server currently has 0 traffic so the resource bottleneck issue gets eliminated already.
• Tried both ECS cluster types : EC2 and Fargate. Having same issue. However pulling and running the same image in local machine (mac arm64) also works fine.

Using import as below

Import class file

Docker file

# Use Ubuntu as the base image
FROM ubuntu:22.04

# Set the working directory
WORKDIR /var/www

# Set environment variables
ENV DEBIAN_FRONTEND=noninteractive TZ=FJT

# Install system dependencies and add PHP PPA
RUN apt-get update && apt-get install -y \
    software-properties-common \
    lsb-release \
    curl \
    unzip \
    zip \
    cron \
    git \
    redis-server \
    supervisor \
    apache2 \
    tzdata \
    && add-apt-repository ppa:ondrej/php -y && apt-get update

# Install PHP 8.2 and extensions
RUN apt-get install -y \
    php8.2 \
    php8.2-cli \
    php8.2-curl \
    php8.2-gd \
    php8.2-mysql \
    php8.2-zip \
    php8.2-mbstring \
    php8.2-bcmath \
    php8.2-opcache \
    php8.2-intl \
    php8.2-readline \
    php8.2-redis \
    php8.2-imagick \
    libapache2-mod-php8.2 \
    php8.2-xml \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

# Enable Apache modules
RUN a2enmod rewrite

# Install Composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

# Copy the existing application directory contents
COPY . /var/www

# Install Composer dependencies
RUN composer install --no-dev --optimize-autoloader --no-interaction

# Set permissions for Laravel directories
RUN chown -R www-data:www-data /var/www \
    && chown -R www-data:www-data /var/www/storage \
    && chown -R www-data:www-data /var/www/bootstrap/cache

# Create necessary log directories and set permissions
RUN mkdir -p /var/log/redis /var/www \
    && touch /var/log/redis/redis-server.log /var/www/worker.log \
    && chown -R www-data:www-data /var/log/redis /var/www

# Copy Apache configuration
COPY ./apache2/000-default.conf /etc/apache2/sites-available/000-default.conf

# Copy Supervisor configuration
COPY ./supervisor/supervisord.conf /etc/supervisor/conf.d/supervisord.conf

# Create a new cron file for Laravel
RUN echo "SHELL=/bin/sh" >> /etc/cron.d/laravel-cron && \
    echo "PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin" >> /etc/cron.d/laravel-cron && \
    echo "* * * * * cd /var/www && php artisan schedule:run >> /var/log/cron.log 2>&1\n" >> /etc/cron.d/laravel-cron

# Give execution rights on the cron job file
RUN chmod 0644 /etc/cron.d/laravel-cron

# Apply the cron job
RUN crontab /etc/cron.d/laravel-cron
RUN touch /var/log/cron.log

# Expose port 80
EXPOSE 80

# Start Apache, Cron, and Supervisor
CMD ["sh", "-c", "service apache2 start && service cron start && /usr/bin/supervisord -n -c /etc/supervisor/conf.d/supervisord.conf"]

excel.php temporary file part

`
'temporary_files' => [

    /*
    |--------------------------------------------------------------------------
    | Local Temporary Path
    |--------------------------------------------------------------------------
    |
    | When exporting and importing files, we use a temporary file, before
    | storing reading or downloading. Here you can customize that path.
    | permissions is an array with the permission flags for the directory (dir)
    | and the create file (file).
    |
    */
    'local_path'          => storage_path('framework/cache/laravel-excel'),

    /*
    |--------------------------------------------------------------------------
    | Local Temporary Path Permissions
    |--------------------------------------------------------------------------
    |
    | Permissions is an array with the permission flags for the directory (dir)
    | and the create file (file).
    | If omitted the default permissions of the filesystem will be used.
    |
    */
    'local_permissions'   => [
        //             'dir'  => 0755,
       //             'file' => 0644,
    ],

    /*
    |--------------------------------------------------------------------------
    | Remote Temporary Disk
    |--------------------------------------------------------------------------
    |
    | When dealing with a multi server setup with queues in which you
    | cannot rely on having a shared local temporary path, you might
    | want to store the temporary file on a shared disk. During the
    | queue executing, we'll retrieve the temporary file from that
    | location instead. When left to null, it will always use
    | the local path. This setting only has effect when using
    | in conjunction with queued imports and exports.
    |
    */
    'remote_disk'         => 's3',
    'remote_prefix'       => 'payment-csv-files/',

    /*
    |--------------------------------------------------------------------------
    | Force Resync
    |--------------------------------------------------------------------------
    |
    | When dealing with a multi server setup as above, it's possible
    | for the clean up that occurs after entire queue has been run to only
    | cleanup the server that the last AfterImportJob runs on. The rest of the server
    | would still have the local temporary file stored on it. In this case your
    | local storage limits can be exceeded and future imports won't be processed.
    | To mitigate this you can set this config value to be true, so that after every
    | queued chunk is processed the local temporary file is deleted on the server that
    | processed it.
    |
    */
    'force_resync_remote' => true,
],`

After debugging found that the resource is not found on this line.

Exception raised in failed_jobs:

{#10197
     +"id": 1352,
     +"uuid": "8b48bb5c-0196-4aa6-9066-0dcaefaee90d",
     +"connection": "database",
     +"queue": "bulk_transfer_queue",
     +"payload": "{"uuid":"8b48bb5c-0196-4aa6-9066-0dcaefaee90d","displayName":"Maatwebsite\\Excel\\Jobs\\ReadChunk","job":"Illuminate\\Queue\\CallQueuedHandler@call","maxTries":null,"maxExceptions":null,"failOnTimeout":false,"backoff":null,"timeout":null,"retryUntil":null,"data":{"commandName":"Maatwebsite\\Excel\\Jobs\\ReadChunk","command":"O:32:\"Maatwebsite\\Excel\\Jobs\\ReadChunk\":23:{s:7:\"timeout\";N;s:5:\"tries\";N;s:13:\"maxExceptions\";N;s:7:\"backoff\";N;s:5:\"queue\";s:19:\"bulk_transfer_queue\";s:10:\"connection\";N;s:40:\"\u0000Maatwebsite\\Excel\\Jobs\\ReadChunk\u0000import\";O:39:\"App\\Imports\\PaymentCsvTransactionImport\":1:{s:17:\"\u0000*\u0000paymentCsvFile\";O:45:\"Illuminate\\Contracts\\Database\\ModelIdentifier\":5:{s:5:\"class\";s:25:\"App\\Models\\PaymentCsvFile\";s:2:\"id\";i:163;s:9:\"relations\";a:0:{}s:10:\"connection\";s:5:\"mysql\";s:15:\"collectionClass\";N;}}s:40:\"\u0000Maatwebsite\\Excel\\Jobs\\ReadChunk\u0000reader\";O:35:\"PhpOffice\\PhpSpreadsheet\\Reader\\Csv\":18:{s:15:\"\u0000*\u0000readDataOnly\";b:1;s:17:\"\u0000*\u0000readEmptyCells\";b:1;s:16:\"\u0000*\u0000includeCharts\";b:0;s:17:\"\u0000*\u0000loadSheetsOnly\";N;s:13:\"\u0000*\u0000readFilter\";O:49:\"PhpOffice\\PhpSpreadsheet\\Reader\\DefaultReadFilter\":0:{}s:13:\"\u0000*\u0000fileHandle\";i:0;s:18:\"\u0000*\u0000securityScanner\";N;s:50:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000inputEncoding\";s:5:\"UTF-8\";s:53:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000fallbackEncoding\";s:6:\"CP1252\";s:46:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000delimiter\";s:1:\",\";s:46:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000enclosure\";s:1:\"\"\";s:47:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000sheetIndex\";i:0;s:47:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000contiguous\";b:0;s:52:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000escapeCharacter\";s:1:\"\\\";s:51:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000testAutodetect\";b:1;s:31:\"\u0000*\u0000castFormattedNumberToNumeric\";b:0;s:28:\"\u0000*\u0000preserveNumericFormatting\";b:0;s:55:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000preserveNullString\";b:0;}s:47:\"\u0000Maatwebsite\\Excel\\Jobs\\ReadChunk\u0000temporaryFile\";O:43:\"Maatwebsite\\Excel\\Files\\RemoteTemporaryFile\":3:{s:49:\"\u0000Maatwebsite\\Excel\\Files\\RemoteTemporaryFile\u0000disk\";s:2:\"s3\";s:53:\"\u0000Maatwebsite\\Excel\\Files\\RemoteTemporaryFile\u0000filename\";s:68:\"payment-csv-files\/laravel-excel-7yo894cbjK0a5nkqzxPbUuWNWJ8KoJeN.csv\";s:63:\"\u0000Maatwebsite\\Excel\\Files\\RemoteTemporaryFile\u0000localTemporaryFile\";O:42:\"Maatwebsite\\Excel\\Files\\LocalTemporaryFile\":1:{s:52:\"\u0000Maatwebsite\\Excel\\Files\\LocalTemporaryFile\u0000filePath\";s:97:\"\/var\/www\/storage\/framework\/cache\/laravel-excel\/laravel-excel-7yo894cbjK0a5nkqzxPbUuWNWJ8KoJeN.csv\";}}s:43:\"\u0000Maatwebsite\\Excel\\Jobs\\ReadChunk\u0000sheetName\";s:9:\"Worksheet\";s:45:\"\u0000Maatwebsite\\Excel\\Jobs\\ReadChunk\u0000sheetImport\";r:8;s:42:\"\u0000Maatwebsite\\Excel\\Jobs\\ReadChunk\u0000startRow\";i:2;s:43:\"\u0000Maatwebsite\\Excel\\Jobs\\ReadChunk\u0000chunkSize\";i:25;s:42:\"\u0000Maatwebsite\\Excel\\Jobs\\ReadChunk\u0000uniqueId\";N;s:15:\"chainConnection\";N;s:10:\"chainQueue\";s:19:\"bulk_transfer_queue\";s:19:\"chainCatchCallbacks\";N;s:5:\"delay\";N;s:11:\"afterCommit\";N;s:10:\"middleware\";a:0:{}s:7:\"chained\";a:1:{i:0;s:2693:\"O:37:\"Maatwebsite\\Excel\\Jobs\\AfterImportJob\":15:{s:45:\"\u0000Maatwebsite\\Excel\\Jobs\\AfterImportJob\u0000import\";O:39:\"App\\Imports\\PaymentCsvTransactionImport\":1:{s:17:\"\u0000*\u0000paymentCsvFile\";O:45:\"Illuminate\\Contracts\\Database\\ModelIdentifier\":5:{s:5:\"class\";s:25:\"App\\Models\\PaymentCsvFile\";s:2:\"id\";i:163;s:9:\"relations\";a:0:{}s:10:\"connection\";s:5:\"mysql\";s:15:\"collectionClass\";N;}}s:45:\"\u0000Maatwebsite\\Excel\\Jobs\\AfterImportJob\u0000reader\";O:24:\"Maatwebsite\\Excel\\Reader\":5:{s:14:\"\u0000*\u0000spreadsheet\";N;s:15:\"\u0000*\u0000sheetImports\";a:0:{}s:14:\"\u0000*\u0000currentFile\";O:43:\"Maatwebsite\\Excel\\Files\\RemoteTemporaryFile\":3:{s:49:\"\u0000Maatwebsite\\Excel\\Files\\RemoteTemporaryFile\u0000disk\";s:2:\"s3\";s:53:\"\u0000Maatwebsite\\Excel\\Files\\RemoteTemporaryFile\u0000filename\";s:68:\"payment-csv-files\/laravel-excel-7yo894cbjK0a5nkqzxPbUuWNWJ8KoJeN.csv\";s:63:\"\u0000Maatwebsite\\Excel\\Files\\RemoteTemporaryFile\u0000localTemporaryFile\";O:42:\"Maatwebsite\\Excel\\Files\\LocalTemporaryFile\":1:{s:52:\"\u0000Maatwebsite\\Excel\\Files\\LocalTemporaryFile\u0000filePath\";s:97:\"\/var\/www\/storage\/framework\/cache\/laravel-excel\/laravel-excel-7yo894cbjK0a5nkqzxPbUuWNWJ8KoJeN.csv\";}}s:23:\"\u0000*\u0000temporaryFileFactory\";O:44:\"Maatwebsite\\Excel\\Files\\TemporaryFileFactory\":2:{s:59:\"\u0000Maatwebsite\\Excel\\Files\\TemporaryFileFactory\u0000temporaryPath\";s:46:\"\/var\/www\/storage\/framework\/cache\/laravel-excel\";s:59:\"\u0000Maatwebsite\\Excel\\Files\\TemporaryFileFactory\u0000temporaryDisk\";s:2:\"s3\";}s:9:\"\u0000*\u0000reader\";O:35:\"PhpOffice\\PhpSpreadsheet\\Reader\\Csv\":18:{s:15:\"\u0000*\u0000readDataOnly\";b:1;s:17:\"\u0000*\u0000readEmptyCells\";b:1;s:16:\"\u0000*\u0000includeCharts\";b:0;s:17:\"\u0000*\u0000loadSheetsOnly\";N;s:13:\"\u0000*\u0000readFilter\";O:49:\"PhpOffice\\PhpSpreadsheet\\Reader\\DefaultReadFilter\":0:{}s:13:\"\u0000*\u0000fileHandle\";i:0;s:18:\"\u0000*\u0000securityScanner\";N;s:50:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000inputEncoding\";s:5:\"UTF-8\";s:53:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000fallbackEncoding\";s:6:\"CP1252\";s:46:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000delimiter\";s:1:\",\";s:46:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000enclosure\";s:1:\"\"\";s:47:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000sheetIndex\";i:0;s:47:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000contiguous\";b:0;s:52:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000escapeCharacter\";s:1:\"\\\";s:51:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000testAutodetect\";b:1;s:31:\"\u0000*\u0000castFormattedNumberToNumeric\";b:0;s:28:\"\u0000*\u0000preserveNumericFormatting\";b:0;s:55:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000preserveNullString\";b:0;}}s:52:\"\u0000Maatwebsite\\Excel\\Jobs\\AfterImportJob\u0000dependencyIds\";a:0:{}s:47:\"\u0000Maatwebsite\\Excel\\Jobs\\AfterImportJob\u0000interval\";i:60;s:9:\"\u0000*\u0000events\";a:0:{}s:3:\"job\";N;s:10:\"connection\";N;s:5:\"queue\";N;s:15:\"chainConnection\";N;s:10:\"chainQueue\";N;s:19:\"chainCatchCallbacks\";N;s:5:\"delay\";N;s:11:\"afterCommit\";N;s:10:\"middleware\";a:0:{}s:7:\"chained\";a:0:{}}\";}s:9:\"\u0000*\u0000events\";a:0:{}s:3:\"job\";N;}"}}",
     +"exception": """
       TypeError: stream_copy_to_stream(): Argument #1 ($from) must be of type resource, null given in /var/www/vendor/maatwebsite/excel/src/Files/Disk.php:73\n
       Stack trace:\n
       #0 /var/www/vendor/maatwebsite/excel/src/Files/Disk.php(73): stream_copy_to_stream()\n
       #1 /var/www/vendor/maatwebsite/excel/src/Files/RemoteTemporaryFile.php(103): Maatwebsite\Excel\Files\Disk->copy()\n
       #2 /var/www/vendor/maatwebsite/excel/src/Jobs/ReadChunk.php(194): Maatwebsite\Excel\Files\RemoteTemporaryFile->sync()\n
       #3 /var/www/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Maatwebsite\Excel\Jobs\ReadChunk->handle()\n
       #4 /var/www/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()\n
       #5 /var/www/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()\n
       #6 /var/www/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod()\n
       #7 /var/www/vendor/laravel/framework/src/Illuminate/Container/Container.php(662): Illuminate\Container\BoundMethod::call()\n
       #8 /var/www/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(128): Illuminate\Container\Container->call()\n
       #9 /var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}()\n
       #10 /var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()\n
       #11 /var/www/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(132): Illuminate\Pipeline\Pipeline->then()\n
       #12 /var/www/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(123): Illuminate\Bus\Dispatcher->dispatchNow()\n
       #13 /var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\Queue\CallQueuedHandler->Illuminate\Queue\{closure}()\n
       #14 /var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()\n
       #15 /var/www/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(122): Illuminate\Pipeline\Pipeline->then()\n
       #16 /var/www/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(70): Illuminate\Queue\CallQueuedHandler->dispatchThroughMiddleware()\n
       #17 /var/www/vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php(102): Illuminate\Queue\CallQueuedHandler->call()\n
       #18 /var/www/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(439): Illuminate\Queue\Jobs\Job->fire()\n
       #19 /var/www/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(389): Illuminate\Queue\Worker->process()\n
       #20 /var/www/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(176): Illuminate\Queue\Worker->runJob()\n
       #21 /var/www/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(137): Illuminate\Queue\Worker->daemon()\n
       #22 /var/www/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(120): Illuminate\Queue\Console\WorkCommand->runWorker()\n
       #23 /var/www/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\Queue\Console\WorkCommand->handle()\n
       #24 /var/www/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()\n
       #25 /var/www/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()\n
       #26 /var/www/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod()\n
       #27 /var/www/vendor/laravel/framework/src/Illuminate/Container/Containers:46:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000delimiter\";s:1:\",\";s
:46:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000enclosure\";s:1:\"\"\";s
:47:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000sheetIndex\";i:0;s:47:\"
\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000contiguous\";b:0;s:52:\"\u0000
PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000escapeCharacter\";s:1:\"\\\";s:51:\"
\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000testAutodetect\";b:1;s:31:\"\u
0000*\u0000castFormattedNumberToNumeric\";b:0;s:28:\"\u0000*\u0000preserveNumeri
cFormatting\";b:0;s:55:\"\u0000PhpOffice\\PhpSpreadsheet\\Reader\\Csv\u0000prese
rveNullString\";b:0;}}s:52:\"\u0000Maatwebsite\\Excel\\Jobs\\AfterImportJob\u000
0dependencyIds\";a:0:{}s:47:\"\u0000Maatwebsite\\Excel\\Jobs\\AfterImportJob\u00
00interval\";i:60;s:9:\"\u0000*\u0000events\";a:0:{}s:3:\"job\";N;s:10:\"connect
ion\";N;s:5:\"queue\";N;s:15:\"chainConnection\";N;s:10:\"chainQueue\";N;s:19:\"
chainCatchCallbacks\";N;s:5:\"delay\";N;s:11:\"afterCommit\";N;s:10:\"middleware
\";a:0:{}s:7:\"chained\";a:0:{}}\";}s:9:\"\u0000*\u0000events\";a:0:{}s:3:\"job\
";N;}"}}",

How can the issue be reproduced?

It can only be reproduced on aws ECS , by running the docker image and deploying it and testing for import via s3 . As its running as expected on local machine (docker desktop on windows 10) not able to figure out the issue when image run on aws ECS .

What should be the expected behaviour?

Expected behavior should stream data from s3 and store on local file and import the rows onto the database .

[Ashay84]

Any update here ?

[Ashay84]

Any help would be appreciated .

[Ashay84]

Did anyone check this ?