builtin function @reify to create a type from a TypeInfo instance

@scurest

for arrays, pointers, error unions, nullables: T.child_type field.
for functions, T.return_type field
for functions, T.is_var_args and
T.arg_types which is of type [N]type
@fieldsOf(T) where T is a struct or enum. Returns anonymous struct { T: type, name: []const u8 }
accessing a field by comptime string name
implement @scurest proposal below
- @typeInfo
- @reify

@AndreaOrru

For functions, would it make sense to also expose the number of arguments and make them individually accessible? (cc @AndreaOrru )

Yes, that would be super useful to implement the IPC in my kernel, for example.

Added a third item for this above

@hasenj

cc @hasenj

<hasenj> for comptime is there a way to perform property access by string or something similar?
<hasenj> for example, say, @field(object, field_name)

@fieldsOf(T) where T is a struct or enum. Returns anonymous struct { T: type, name: []const u8 }

Is the type even needed? I think a list of names as strings should suffice, given that there are other builtin functions that can be used to reflect on the type. For example:

@field(some_struct, name); // expands to some_struct.<value of name>
// e.g.
@field(a, "foo"); // expands to `a.foo`

@typeOf(@field(a, "foo")); // gives type of a.foo

It seems like with just these features we could implement a structural based interface mechanism, similar to go.

Here is a motivating example to implement a print trait of sorts, allowing any struct which implements a print field with the appropriate type to have it called. I don't think anything is too glaringly out of place here, but feel free to correct.

const std = @import("std");
const builtin = @import("builtin");
const TypeId = builtin.TypeId;

fn getField(comptime T: type, x: var, comptime name: []const u8) ->
    (fn (self: T) -> %void)
{
    for (@fieldsOf(x)) |field_name| {
        if (field_name == name) {
            const field = @field(x, name);
            const field_type = @typeOf(field);

            if (@typeId(field_type) != TypeId.Fn) {
                @panic("field is not a function");
            }

            if (field_type.is_var_args) {
                @panic("cannot handle varargs function");
            }

            // Would need to be a bit more in-depth to handle a &T self arg
            const expected_args = []type { T };
            if (!std.mem.eql(type, field_type.arg_types, expected_arg_types)) {
                @panic("prototype does not match");
            }

            if (@typeId(field_type.return_type) != TypeId.Error
                and field_type.return_type.child_type != void)
            {
                @panic("return type does not match");
            }

            return field;
        }
    }

    null
}

// Expects a struct with a field method of type `print(self) -> %void`.
pub fn printTrait(x: var) -> %void {
    const T = @typeOf(x);
    const Id = @typeId(T);

    if (Id != TypeId.Struct) {
        @panic("expected a struct");
    }

    if (getField(T, x, "print")) |func| {
        func(x);
    } else {
        @panic("no print field found!");
    }
}

With this example, if you were going to do printTrait(x) couldn't you instead do x.print() ?

You're right. This example would really only provide slightly more targeted error messages.

A better example would be a printDebug function which could recursively print fields of structs and enums, similar to println!("{:?}", x) in Rust. Another example as well that could be very useful would be for generic serialization code by inspection of field names.

more compile-time type reflection

add compile-time reflection for function arg types

Since zig can store types as regular data, I was wondering if it would be better to, rather than have magic fields that make types seem like structs, expose the reflected data in actual structs. Example:

const NullableType = struct {
    child: type,
};

@reflect(?u32) ==> NullableType { .child = u32 }

const ArrayType = struct {
    child: type,
    len: u64,
};

@reflect([4]u32) ==> ArrayType { .child = u32, .len = 4 }

// Possible example for a struct

const StructType = struct {
    field_names: [][]const u8,
    field_types: []type,
    field_offsets: []u64,
}

const S = struct { x: i32, y: u8 };

@reflect(S) ==> StructType {
    .field_names = [][]const u8 { "x", "y" },
    .field_types = []type { i32, u8 },
    .field_offsets = []u64 { 0, 4 },
}

etc. You get the idea. You could also add a souped-up version of @typeId that returns an enum with variants like Nullable: NullableType, etc.

Some pros: possibly easier documentation (you can now lookup what fields you can access just like for regular structs), fewer special cases in the field-lookup code.

There could also be a @deReflect (deflect? unreflect?) that turns one of these reflected structs into a type, so you could generate eg. a struct programmatically at compile time.

Of course, @reflect could also be implemented as a regular function in userland (I hit #586 when trying it) as long as some reflection mechanism exists.

I like this idea. I ran into an issue which is related which I will type up and link here.

@scurest

I think #588 has to be solved before the idea @scurest outlined here can be implemented.

This issue is now blocking on #2907.

Another random thought on what @reify could achieve: help exchange zig types (think optionals, slices, unions...) accross ABI boundaries. We could create helpers to convert from/to C types representing them automatically and their instances accordingly, so that in the end we can make dynamic zig libraries easier to use. I understand and agree that static linking is to be preferred, but the usecase exists. Makes sense?

Related to AoS to SoA transforms is creating an archetype based ECS. For example, given a user defined archetype with some component fields, I'd like to be able to generate an appropriate concrete "storage" type (here storage is shown by simple slices).

const MyArchetype = struct {
    location: Vec3,
    velocity: Vec3,
};

// I then write some function Storage() that uses the @reify feature
// const MyArchetypeStorage = Storage(MyArchetype);

// I want to generate something like this:
const MyArchetypeStorage  = struct {
    location: []Vec3,
    velocity: []Vec3,
}

Modern game engines tend to jump through lots of hoops (and runtime costs) to pay for this abstraction, because it enables things like batched compile-time dispatch (if I have a bunch of storages I can run some function on each one that contains a particular set of batched field).

Would be great to be able to generate things like this.

I recently came across a problem which @reify would be very useful for. There's a very long C struct I'm working with where large blocks of fields are selectively present for different platforms using macros.

There's no neat way of currently solving this in zig without resorting to code generation, or writing massive amounts of code.

With @reify I could split these blocks into their own structs, and neatly "join" them based on comptime values to form the struct I'm looking for.

This would greatly simplify and reduce the footprint of the code, which would be great for my sanity and for spotting bugs.

I would recommend having a look at Circle (https://www.circle-lang.org/). It's a compile-time machinery on top of C++. It's sometimes complex (not saying it in a bad way - C++ itself is very complex) but it does pass the printf test. (It's possible to create printf without compiler hooks.) And it does much more than that.

I'm running into a desire for full-strength @reify from writing argument parsing machinery; it'd be nice to generate structs & tagged unions which represent the byte slices parsed. https://github.com/MasterQ32/zig-args manages to avoid this currently by restricting its API to a struct with duck-typed, specially-named metadata fields, but this restricts access to other aspects of Zig's rich type system in the API. You can't use a struct to store a few different fields about an option, or a tagged union to clearly show how choices are structured. Trying to avoid unintuitiveness with @reify leads to complexity getting shoved into (comptime) dynamically typed trees that are weakly typed by default.

@alexnask

After some discussion with @alexnask in IRC we came to the conclusion that allowing @Type(.Struct) would be more sane than all the hacks and workarounds based on @TypeOf(.{ … }) and similar techniques.

People are already working around the restrictions that you cannot create struct at comptime creating APIs that are convenient to use, but really hard to understand. Allowing @Type(.Struct) would make the code more straightforward, readable and maintainable.

The question that was not clear is whether to allow creating structs with .decls set to a non-empty set or not. Allowing it would probably open up possibilities for really convenient interface APIs similar to interface.zig, but making code harder to understand, but still easier than the current implementations

For reference, the following is achievable today:

const std = @import("std");

// Create a tuple with one runtime-typed field
fn UniTuple(comptime T: type) type {
    const Hack = struct {
        var forced_runtime: T = undefined;
    };
    return @TypeOf(.{Hack.forced_runtime});
}

/// Types should be an iterable of types
fn Tuple(comptime types: anytype) type {
    const H = struct {
        value: anytype,
    };
    var empty_tuple = .{};

    var container = H{
        .value = empty_tuple,
    };

    for (types) |T| {
        container.value = container.value ++ UniTuple(T){ .@"0" = undefined };
    }
    return @TypeOf(container.value);
}

pub fn StructType2(comptime names: []const []const u8, comptime types: anytype) type {
    std.debug.assert(names.len == types.len);
    const Storage = Tuple(types);

    return struct {
        const field_names = names;
        const field_types = types;
        const Self = @This();

        storage: Storage,

        pub fn create(literal: anytype) Self {
            comptime std.debug.assert(std.meta.fields(@TypeOf(literal)).len == field_names.len);
            comptime std.debug.assert(std.meta.trait.hasFields(@TypeOf(literal), field_names));

            var self: Self = undefined;
            inline for (field_names) |name, idx| {
                self.storage[idx] = @field(literal, name);
            }
            return self;
        }

        fn fieldIndex(comptime name: []const u8) ?comptime_int {
            var i = 0;
            while (i < field_names.len) : (i += 1) {
                if (std.mem.eql(u8, name, field_names[i])) return i;
            }
            return null;
        }

        fn FieldType(comptime name: []const u8) type {
            const idx = fieldIndex(name) orelse @compileError("Field '" ++ name ++ "' not in struct '" ++ @typeName(Self) ++ "'.");
            return field_types[idx];
        }

        pub fn field(self: anytype, comptime name: []const u8) if (@TypeOf(self) == *Self) *FieldType(name) else *const FieldType(name) {
            const idx = fieldIndex(name).?;
            return &self.storage[idx];
        }

        pub fn format(
            self: Self,
            comptime fmt: []const u8,
            options: std.fmt.FormatOptions,
            writer: anytype,
        ) !void {
            try writer.writeAll(@typeName(Self));
            if (std.fmt.default_max_depth == 0) {
                return writer.writeAll("{ ... }");
            }
            try writer.writeAll("{");
            inline for (field_names) |f, i| {
                if (i == 0) {
                    try writer.writeAll(" .");
                } else {
                    try writer.writeAll(", .");
                }
                try writer.writeAll(f);
                try writer.writeAll(" = ");
                try std.fmt.formatType(self.storage[i], fmt, options, writer, std.fmt.default_max_depth - 1);
            }
            try writer.writeAll(" }");
        }
    };
}

pub fn StructType(comptime fields: anytype) type {
    var field_names: [fields.len][]const u8 = undefined;
    var field_types: [fields.len]type = undefined;

    for (fields) |f, idx| {
        field_names[idx] = f[0];
        field_types[idx] = f[1];
    }

    return StructType2(&field_names, &field_types);
}

pub fn main() void {
    const FooStruct = StructType(.{
        .{ "a", usize },
        .{ "b", bool },
    });

    var foo = FooStruct.create(.{ .a = 0, .b = false });
    foo.field("a").* = 1;
    std.debug.print("{}\n", .{foo});
}

This is a simple implementation of a @Type(.Struct)-type construction (which could be improved with default field values and other things).
In my opinion it is clear that @Type(.Struct) would be better solution to this (including .decls although I could see how this is somewhat controversial)

@alexnask

After some discussion with @alexnask in IRC we came to the conclusion that allowing @Type(.Struct) would be more sane than all the hacks and workarounds based on @TypeOf(.{ … }) and similar techniques.

I think this is a strong argument. I'd like to "accept" @Type supporting struct, enum, etc, but keep the decls as an open proposal, with no conclusion yet. Hopefully that feels like progress.

This is now implemented for every type, and can generate everything that syntax can except

Generic function types (these are weird anyway)
Declarations in enums, structs, unions, and opaque types.

I think this issue should be renamed to be about allowing declarations, or closed and replaced with a new one for that.

I think this issue should be renamed to be about allowing declarations, or closed and replaced with a new one for that.

sounds good 👍

Navigation Menu

Repository navigation

builtin function @reify to create a type from a TypeInfo instance #383

37 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

builtin function @reify to create a type from a TypeInfo instance #383

Description

Activity

ranma42 commented on May 30, 2017

AndreaOrru commented on May 30, 2017

andrewrk commented on May 30, 2017

andrewrk commented on Sep 21, 2017

hasenj commented on Sep 21, 2017

tiehuis commented on Sep 22, 2017

andrewrk commented on Sep 22, 2017

tiehuis commented on Sep 28, 2017

scurest commented on Nov 5, 2017

andrewrk commented on Nov 6, 2017

andrewrk commented on Nov 6, 2017

37 remaining items

andrewrk commented on Jul 16, 2019

Sahnvour commented on Nov 30, 2019

cshenton commented on Jan 18, 2020

Sobeston commented on Apr 29, 2020

az5112 commented on Jun 18, 2020

bb010g commented on Jul 7, 2020

ikskuh commented on Aug 3, 2020

alexnask commented on Aug 3, 2020

andrewrk commented on Aug 7, 2020

tadeokondrak commented on Oct 8, 2020

andrewrk commented on Oct 17, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions