optimization: rs274ngc: parse numbers with std::from_chars

[alex-pres]

read_real_number built a std::string and a std::stringstream for every number in a program, and read_integer_unsigned handed the digits it had already delimited to sscanf("%d"). std::from_chars needs neither, and is locale-independent where the stream was not. Accepted syntax, parsed values and both error messages are unchanged, except that a magnitude too small for a double, and a line number too large for an int, are now parse errors. Parsing a 13.7M line program is 2.1x faster.

Benchmark on real program [ 400Mb 13.6M lines ]

Interp parse only       : master 12.10s (1.00x),    A + #2 (branch) 5.83s (2.08x)

It was hard to see before graphical preview was moved to C++ (PR #4481) , but now this small optimisation of Interp becomes useful.

Found and reported by @AlexPform

[BsAtHome]

Hm, that is quite an improvement.

What I don't understand is why there is no full and proper lexer/tokenizer reading the file. The ngc format is both lexicographically and grammatically defined, so a direct tokenizer+parser would generally be faster. Even with the state dependencies of the format.

[alex-pres]

Hm, that is quite an improvement.

What I don't understand is why there is no full and proper lexer/tokenizer reading the file. The ngc format is both lexicographically and grammatically defined, so a direct tokenizer+parser would generally be faster. Even with the state dependencies of the format.

I agree completely. My guess: parser code was written decades ago and it was not as easy as now, no such tooling as we have today.

[BsAtHome]

I agree completely. My guess: parser code was written decades ago and it was not as easy as now, no such tooling as we have today.

Hm, used lex and yacc (flex+bison) more than 30 years ago (damn, I'm getting old...) and they were considered "slow" with respect to manual hand-written tokenizers and descent parsers. I guess you need to have some taste for them and know some of the trickery involved.