Repository navigation
Proposal: allow peer type resolution to expand integer sizes #15737
Description
Activity
Question
Would this also mean that
(u|i)N * (u|i)N->(u|i)(2*N)(essentially equivalent tostd.math.mulWide), etc. for every integer operation, or am I misunderstanding?Regarding the original
In C, embedded programmers often have to go out of their way to avoid accidental int promotion. It seems this is not that different (despite the semantics being much saner overall) and could be a foot-gun for some users, especially given the widespread use of type inference.
Just for reference, with the new
_BitIntfeature in C23, the expression evaluates asunsigned _BitInt(N) + _BitInt(N)->unsigned _BitInt(N).Reacted by Zachary RaineriI like the idea. It will immediately signal "Something is wrong, you are doing unsafe operations". But this should be very clearly articulated in language specs.
And we should clearly provide way to make this operations safe and staying within
u8(math with overflow checking?). As I'm working with database blocks, I can't just writeu9to disk, whereu8was expected.Also what to do with performance? As I understand on hardware
u16will be allocated and it will be justu16addition operation? Don't think working withu9is a wise idea.From practical view I think this proposal bans mixin
u8+i8operations. No one will deal withu9in reality. But from security perspective making this explicit is very important, so many hidden bugs 🪲 in such promotions/conversions.Security and safety is much more important than faster integer math, especially considering memory access is much slower.
Would this also mean that
(u|i)N * (u|i)N->(u|i)(2*N)(essentially equivalent tostd.math.mulWide), etc. for every integer operation, or am I misunderstanding?Ah, no - that's about something else. @likern, I think you have the same misunderstanding - that's my bad, let me clarify.
The promotion here is not happening to ensure the addition result is always in range. That's a separate proposal, and something that may well end up being a part of #3806 if it gets in. That would be the thing where
uX * uXgives you au(2*X)for instance; again, it's something which I think would fit nicely with ranged ints, but is certainly a more drastic change.This change is about peer type resolution, which is essentially the question of: "given types
T1..Tn, what type can I use which fits all of them"? Arithmetic operations do (currently) use PTR, but in hindsight they're a misleading example here - instead, consider something like this:var a: u8 = undefined; var b: i8 = undefined; const c = if (foo()) a else b;
What type should
chave? It needs to be a type which can hold au8or ani8. This also comes up a lot with large switches:switch (thing) { .foo => a, .bar => b, .baz => c, else => d, }
The result type of this expression is found through peer type resolution of the types of
a,b,c, andd. Peer type resolution can do things like stripping off sentinels if needed, addingconstto pointers, wrappings things in error unions or optionals, etc. For instance, if you run PTR on the typeserror{Foo}andu32, you geterror{Foo}!u32, while*const [4]u8and[*]u8resolve to[*]const u8. The idea isn't to make operations like arithmetic safe, but instead to find a supertype of all your input types that they can coerce to in order to be "combined" in some way (in this case just by being the result of the same conditional expression).Now we can consider this again in the context of integers. The results of PTR where all operand types are the same are unchanged under this proposal. For instance, arithmetic like
u8 + u8still yields au8;i16 + i16gives ani16; etc. The point here is specifically about the case of peer type resolution between signed and unsigned integers. Again, I think this is much clearer to understand if you consider aniforswitchstatement (see the first code snippet in this comment). Currently, that statement would emit an error; this proposal is to instead make it emit ani9. (Again, the equivalent in #3806 is that you just get a ranged int containing both ranges, which in fact appears to be an actual part of that proposal!)I hope this explanation makes sense? Sorry for the slight ambiguity in my initial overview :-]
Reacted by Yuri and cnThis change is about peer type resolution, which is essentially the question of: "given types T1..Tn, what type can I use which fits all of them"?
What does mean "fits" in this proposal?
As I understand if I
const c = @as(u8, 255) + @as(i8, 127)what would I get? Looks like overflow fori9? What the usefulness in this regard of thati9?
And howu8 + i8fits intoi9?I think intentional compilation error with good explanation is more sane and safe, as in
if elsebranches.Develop will stop here and think, probably double check condition branches (check overflow, convert to
u8, etc.).What I think more useful is to be able to borrow
u16type ofcin this example and check all branches fits intou16. This helpful message to add type toccould be provided by the compiler.const c: u16 = if(cond) @as(u8, 255) else @as(i8, 127)`;
@likern
Peer Type Resolution is about finding a common type that all of the operands can coerce to (meaning, it is an unambiguous and safe transformation); the result type is not really under discussion in this proposal. Sou8+u8PTR is stillu8since both operands trivially coerce tou8.u8+i8could coerce toi9in this proposal becausei9can hold the full range of values of bothi8andu8.Ensuring that an arithmetic result fits into the result type is a bigger change (e.g.
u8*u8would requireu16, which this proposal doesn't address; you'd probably need to introduce new operators like+^or something like that to avoid breaking almost all existing code, but I could see that being useful for writing code that is guaranteed not to overflow regardless of the build mode, without needing explicit@ascasts).@likern Peer Type Resolution is about finding a common type that all of the operands can coerce to (meaning, it is an unambiguous and safe transformation); the result type is not really under discussion in this proposal. So
u8+u8PTR is stillu8since both operands trivially coerce tou8.u8+i8could coerce toi9in this proposal becausei9can hold the full range of values of bothi8andu8.Ensuring that an arithmetic result fits into the result type is a bigger change (e.g.
u8*u8would requireu16, which this proposal doesn't address; you'd probably need to introduce new operators like+^or something like that to avoid breaking almost all existing code, but I could see that being useful for writing code that is guaranteed not to overflow regardless of the build mode, without needing explicit@ascasts).Thanks, I get it. "Unambiguous coercion". But what to do later with this
i9and how it would be represented in hardware?Since Zig is a system programming language, there is not much use of these
i9types. Except, of course, packing as unified struct of these bit types. But I think it's very specific use case and should stay that.People might otherwise start passing these
i9integers as function parameters, etc. to "shut down" compiler.And what should we get with
i9 + u8? If justi9why not just coerce toi16? Zig should represent real-life hardware, since it's system programming language.Thanks, I get it. "Unambiguous coercion". But what to do later with this
i9and how it would be represented in hardware?You can use arbitrary-bit-sized integers like any other integer. I believe it'll be represented as an
i16while doing arithmetic on it, with negligible runtime cost in release builds.Since Zig is a system programming language, there is not much use of these
i9types. Except, of course, packing as unified struct of these bit types. But I think it's very specific use case and should stay that.What makes you say that? Bit-sized integers are a useful feature of Zig for safety and documentation. For instance:
- small
uXintegers are commonly used as enum tag types - the RHS of shifts is a bit-sized integer whose size is log2 of the LHS' size; e.g. you can only shift a
u64by au6 - the standard library represents Unicode codepoints as
u21 - the standard library represents alignments as
u29
People might otherwise start passing these
i9integers as function parameters, etc. to "shut down" compiler.These types are already used as parameters etc, and that's totally fine. What do you mean by "shut down the compiler"?
And what should we get with
i9 + u8? If justi9why not just coerce toi16? Zig should represent real-life hardware, since it's system programming language.You could just as well ask why not coerce to an
i32, or ani64. Informally, the point of peer type resolution is kind of to find the "closest" possible type which all peers coerce to. For instance, PTR between[*]u8and*[5]u8could technically result in*anyopaque, but that's not very useful, because it's unnecessarily different from the peer types (so it instead results in[*]u8). In this context, the type that's closest to our peer int types will be the one that's not any larger than it needs to be; for now, that's a type that's only one bit larger.- small
Thanks for clarification.
What makes you say that? Bit-sized integers are a useful feature of Zig for safety and documentation. For instance:
small uX integers are commonly used as enum tag types
the RHS of shifts is a bit-sized integer whose size is log2 of the LHS' size; e.g. you can only shift a u64 by a u6
the standard library represents Unicode codepoints as u21
the standard library represents alignments as u29👍 Agree! Well, what I can say - lets do it! Thank you spending time for clarification and resolving objections.
Reacted by Victor Malov@mlugg Is there proposal to
borrowdesired type from variablecfor peer type resolution (can't find)?var a: u8 = undefined; var b: i8 = undefined; const c = if (foo()) a else b;
Like to solve this issue with declaring type for
cvar a: u8 = undefined; var b: i8 = undefined; const c: u8 = if (foo()) a else b;
I still think it's more clear. At the same time we can leave
u8 + i8as is, as a compiler error with clear message. Moreover while we haveu21for unicode code points, it has clear semantic.Is there proposal to
borrowdesired type from variablecfor peer type resolution (can't find)?I think this already does what you're expecting. PTR is only used if the expression doesn't have a known "result type"; annotating the type of
cin that snippet forwards it as the result type for both cases of theif, so in your example you'll get a compile error on thebsince it can't coerce to au8, and if you made the annotationi9it would work.Moreover while we have
u21for unicode code points, it has clear semantic.What do you mean by this?
Unicode Standard has encoded characters in the range U+0000..U+10FFFF, which amounts to a 21-bit code space
I think this already does what you're expecting. PTR is only used if the expression doesn't have a known "result type"; annotating the type of c in that snippet forwards it as the result type for both cases of the if, so in your example you'll get a compile error on the b since it can't coerce to a u8, and if you made the annotation i9 it would work.
Yes, thanks. I think problem is already solved in this regard, as I can see it.
When adding u8 to u16, you almost certainly mean for the result to be a u16, so it's ok for the compiler to just go ahead and do that. But when adding
u8toi8, this is not clear. You might want either au8, ani8or an error, depending on the context. Widening to i9 only pushes the problem down the road in most cases, I think, as the error will still resurface when the result is stored into a typed location, only now it may be on a different line.I think this line is the crux of the argument. Is the intent more clear here or not?
For mixing signedness, I lean towards status quo
var a: u8 = undefined; var b: i8 = undefined; var c = a + b; // error: expected type 'u8', found 'i8'vs. proposed
var a: u8 = undefined; var b: i8 = undefined; var c = a + b; // ... const v = some_arr[c]; // error: expected type 'usize', found 'i9'where I (suddenly) have an
i9I didn't expectReacted by Victor Malovstatus quo has said that Zig will never automatically up-cast without the use of something explicit like
@intCast. for that and the others mentioned above I dont think this is a good idea for ZigReacted by Victor Malov, Yoshikage Kira, Matthias and Oto ŠťávaUnder the current semantics, this code triggers a compile error, with peer type resolution incorrectly resolving the type as
u8(if you flip the operands around, it instead resolves it as ani8).After rereading a bit, I'm unsure what this exactly means:
- Is this an internal implementation detail?
- Or are there scenarios in which this peer-type-resolution result can be accessed/queried/used via the language? (Maybe with multi-arg
@TypeOf?)
Since apparently there are compile errors regarding this in some scenarios, I would be inclined to view the cases where there is no compile error as bugs.
From that perspective, it seems clear to me that we shouldn't strive to keep status-quo with those bugs.
But in that case the actual pertinent question is:
"Always allow such peer-type resolution (making it widen by one bit) or always make it a compile error?"While I personally think widening operations (followed by explicit
@intCast) are the saner default overall,
compile errors seem more in-line with the fixed-size operators we have in the language today.
Motivation
Consider the following code snippet:
What is the type of
c? Under the current semantics, this code triggers a compile error, with peer type resolution incorrectly resolving the type asu8(if you flip the operands around, it instead resolves it as ani8). But there's clearly a type which both can coerce to:i9!Proposal
In #15726, I've been fixing some peer type resolution cases, and this was a thing I ended up looking into. Essentially, peer type resolution on conflicting int types should choose a final type by considering the bounds of each type in question and combining them (this is somewhat similar to the recent work on
@min/@max). Practically, this means that when you perform peer type resolution on a set of fixed-width integer types, if their signedness differs, the resulting type may have a bit size one more than the bit size of the input - i.e. ifuXandiXare peers, the result will bei(X+1).This follows very nicely into #3806, which would allow the bounds to be specified even more precisely than the arbitrary-bit-width integers we already have. If that proposal is implemented, this proposal essentially becomes that PTR between
@Int(a, b)and@Int(c, d)should result in@Int(@min(a, c), @max(b, d)). I think that's a stance most people would agree upon if ranged integers are implemented; this proposal is simply to implement the equivalent of this behaviour for status quo integers.Implementation
I looked into this as a part of my initial implementation of #15726, so have the work saved locally. It's relatively simple (I'd argue simpler than the implementation of status quo in fact), plus should pretty much automatically retain the correct behaviour if #3806 is implemented.