Read lines until ReadLineAsync returns null in the File polyfills

[SimonCropp]

The File.ReadAllLinesAsync (.NET Framework / netstandard2.0) and File.ReadLinesAsync (below net7) polyfills loop on reader.EndOfStream. When the reader's buffer is empty at that point, EndOfStream refills it with a synchronous Stream.Read, which on a FileStream opened for asynchronous I/O blocks the calling thread on the overlapped read.

Change

Both loops now read until ReadLineAsync returns null:

string? line;
while ((line = await reader.ReadLineAsync().ConfigureAwait(false)) != null)
{
    cancellationToken.ThrowIfCancellationRequested();
    ...
}

The lines returned are the same. The token is still checked once per line, and, as before, never once the end of the file has been reached, so an already cancelled token still throws for a file with content and still returns nothing for an empty file.

Measurements

net48, x64, Release. A 12 MB file of 200,000 lines, read through the old and the new loop with the FileStream replaced by a subclass that counts calls.

	Before	After
blocking Stream.Read calls per file	112	0
Stream.ReadAsync calls per file	11,695	11,807

EndOfStream only has to refill when a line happens to end exactly where the buffer does, so it accounts for about 1% of the reads rather than all of them; the rest already went through ReadLineAsync. The elapsed time is accordingly unchanged within the noise of the machine (best of 15 runs, three rounds):

	Before	After
ReadAllLinesAsync	204 / 202 / 447 ms	217 / 200 / 366 ms
ReadLinesAsync	157 / 164 / 390 ms	160 / 209 / 293 ms

So this removes the remaining synchronous reads from the two asynchronous methods; it is not a throughput change.

Tests

In a unique temp directory that is deleted afterwards:

• ReadAllLinesAsync and ReadLinesAsync against File.ReadAllLines for a set of file contents built around the reader's buffer sizes: empty, only terminators, no final terminator, leading and trailing blank lines, a first line ending 2 bytes before to 2 bytes after the 1024, 2048, 4096 and 8192 byte marks with each of \n, \r\n, \r and no terminator (so terminators are split by a buffer boundary and lines end exactly on one), 5,000 short lines, lines exactly one buffer long, and a 100,000 character line
• ReadAllLinesAsync with UTF-8, UTF-8 with BOM, UTF-16 LE/BE and UTF-32, including a byte order mark overriding the requested encoding
• an already cancelled token throws and leaves the file closed
• ReadLinesAsync cancelled while enumerating throws, having returned only a prefix of the lines, and leaves the file closed; breaking out of the enumeration early leaves the file closed