[DAV] Download fails for big files: stream_copy_to_stream() throws an error

[derkostka]

Steps to reproduce

1. Try to download a big file on Nextcloud, PHP7.0, 32 Bit arm (> 2 GB)
2. Check downloaded file and log
3. File is empty, log shows the error immediately

Expected behaviour

File with size of 9 GB should be downloaded

Actual behaviour

0 Byte file created, no error message but logged error

Server configuration

Operating system:
Ubuntu 16.04

Web server:
Apache

Database:
MySQL

PHP version:
PHP 7

Nextcloud version: (see Nextcloud admin page)
Nextcloud 11.0.1

Updated from an older Nextcloud/ownCloud or fresh install:
Updated from Owncloud 9 -> Nextcloud 10 -> Nexcloud 11 alpha
Where did you install Nextcloud from:
Git
Signing status:
List of activated apps:
The content of config/config.php:
Are you using external storage, if yes which one: local/smb/sftp/...
local
Are you using encryption: yes/no
no
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no

Client configuration

Browser: Chrome

Operating system: MacOS 10.12

Logs

Nextcloud log (data/nextcloud.log)

Nextcloud log

{"reqId":"He91ouATEMBQ7rkEw65e","remoteAddr":"127.0.0.1","app":"PHP","message":"stream_copy_to_stream() expects parameter 3 to be integer, string given at \/media\/data\/www\/nextcloud\/3rdparty\/sabre\/http\/lib\/Sapi.php#78","level":3,"time":"October 11, 2016 20:58:34","method":"GET","url":"\/remote.php\/webdav\/Media-Fernsehaufnahmen\/arte_HD\/arte_HD_Der_Tag_der_Kr%C3%A4hen20160615_135900.ts","user":"name","version":"9.2.0.4"}
{"reqId":"DKWW6ajns8HKX8pETQS+","remoteAddr":"127.0.0.1","app":"PHP","message":"stream_copy_to_stream() expects parameter 3 to be integer, string given at \/media\/data\/www\/nextcloud\/3rdparty\/sabre\/http\/lib\/Sapi.php#78","level":3,"time":"October 11, 2016 20:58:58","method":"GET","url":"\/remote.php\/webdav\/Media-Fernsehaufnahmen\/arte_HD\/arte_HD_Die_wahre_Macht_des_Vatikan_(2_2)20160207_144900.ts","user":"name","version":"9.2.0.4"}

[derkostka]

Update: According to sabre-io/http#61, the issue is located somewhere else. The file size seems to be invalid (NaN) which leads to that error. However, if i apply that patch, it works - but is not solving the root cause as noted in the comments.

Update2: The file breaks at exact 4 GB (real size is 9,1 GB). So there must be an issue somewhere.

Can anybody please test large file downloads in an environment with 32 bit an php 7 ? thx !

[nickvergessen]

Well you said you use 32 bits, the max int there is 2147483647.
From min to max, the number is 4294967294, which is exactly 4GB.
So I'd say try to switch to 64 bits if you want to have so huge files.

I guess the problem is, that before your cast to int (the intval from the patch), the size is an invalid string. casting to int seems to use the max int, when the value is too high:
See that the following 3 numeric strings all result in the same integer:
https://3v4l.org/OMMWp

var_dump((int) '9223372036854775807');
var_dump((int) '9223372036854775808');
var_dump((int) '92233720368547758081231');

So not sure if there is anything we can do.

[go2sh]

There seems to be a general issue with files bigger than 2GB on 32-bit systems. 🙈

[derkostka]

There has been a discussion about that topic, generally, stating that as an alternative either floats may be used (which are essentially 64 bits, even on the 32bit arm platform) or numeric strings are an option:
https://github.com/fruux/sabre-dav/issues/408

var_dump((int) '9223372036854775807');

var_dump((int) '9223372036854775808');

var_dump((float) '92233720368547758081231');

results in:

int(9223372036854775807) int(9223372036854775807) float(9.2233720368548E+22)

... but as far as i understood, floats are not an option for sabre dav, so it possibly won´t change. Nextcloud shall catch the error and throw a warnings prior downloading instead, i guess.

[nickvergessen]

Can you try, whether #1740 fixes the issue?

[nickvergessen]

And by the way, you should not run any nextcloud 11 alpha version yet. We didn't make any release for that Oo

[derkostka]

No, does not seem to solve it. Download is still 0 Byte. Log:

{"reqId":"cIm0o8yJYIzbld34vhUX","remoteAddr":"127.0.0.1","app":"PHP","message":"stream_copy_to_stream() expects parameter 3 to be integer, string given at /media/data/www/nextcloud/3rdparty/sabre/http/lib/Sapi.php#78","level":3,"time":"October 15, 2016 11:24:15","method":"GET","url":"/remote.php/webdav/Media-Fernsehaufnahmen/arte_HD/arte_HD_Der_Tag_der_Kr%C3%A4hen20160615_135900.ts","user":"me","version":"9.2.0.4"}

-> I will investigate and test further by the end of tomorrow.

[derkostka]

The proposed fix did unfortunately not help. So is there really no chance to download files > 4 GB in an 32 Bit Environment ? I am not into the internals of NC, but i think a download is "streamed" in some kind of chunks, so Filesize should not matter for the download itself, correct ? Is it needed to know the actual filesize (as variable) before downloading ?

[nickvergessen]

Well that seems to exactly be the problem. Before streaming the size is given. But in your case the size is not a number, but a string (because its bigger than the max int) and therefor the error occures.

The problem is, that the log is not really helpful. Let me check if I can find out where this comes from and whether you can add some debug calls for us, that help us finding the real issue.