[RFC] Relay IR Text Format

[joshpoll]

Keyword Proposal

The final keywords are:

• def for global function definitions
• fn for local function definitions/anonymous functions
• if and else for if-else expressions
• let for let bindings
• True and False for booleans
• data types are tvm-style. e.g. float32x4

Loose Ends

There are some elements of the text format that didn't make it into this PR. See #1935 for details.

---

Please comment on syntax design choices for #1781 here!

Notable Syntax Examples

Functions

Named, Typed

def @foo(%x: int64, %y: int64) -> int64 {
    %x + %y
}

Named, Inferred

def @foo(%x, %y) {
    %x + %y
}

Anonymous, Typed

fn (%x: int64, %y: int64) -> int64 {
    %x + %y
}

Anonymous, Inferred

fn (%x, %y) {
    %x + %y
}

Let Expressions and Mutation

Immutable

let %x = 2;
let %x = 3;
let %y = 4;
%x + %y

Side Effects Only

The following are equivalent

print("hello");
1

let %_ = print("hello");
1

Tuples

()
(0,)
(0, 1)

Types

Base Types

Designed to look like NumPy types.

int32
uint32
float32
bool
int32x4
...

Shape Types

unimplemented

s
(n + 1, 2 * n, 4)

Tensor Annotations

generics unimplemented

def @add(t1: Tensor[s, bt], t2: Tensor[s, bt]) -> Tensor[s,  bt] {
    ...
}

Function Types

fn (int32, int32) -> int32

[tqchen]

Can we move most of the discussion to this issue instead? and post RFC here

[tqchen]

cc @dmlc/tvm-team

[tqchen]

cross posting.
We will need a spec on how nested expressions can be printed,

specifically, instead of

add(%x, add(%y, %z))

We prefer

%1 = add(%y, %z)
%2 = add(%x, %1)

Note that add will become neural network layers such as conv2d etc. When we construct the exprs, it is quite typical that the expression is already CSEed. Example:

%1 = add(%y, %z)
%2 = add(%1, %1)

[MarisaKirisame]

I think we can view it by seeing two kind of let: %1 is the 'explicit let', which map to relay let.
_1 is the 'implicit let', which does not map to relay concept, only to denote whenever _1 is refered, the rhs is used (and shared).

Also to be frank, I do not see a reason to has relay program as graph - we still need let because we have scope and effect(no reference for now, but there is general recursion), we cant see graph as tree (because exponential blowup), and we still need to handle binding as we has let, so we are getting the benefit of neither world.

I had made some facility to automatically turn binder into value (LFunctor), so with it we can operate a tree as graph without actually having one, so I think we should at least sit down and revisit why we want graph right now (we can always add it later).

[joshpoll]

I don't think this concern is within the scope of this issue/PR. IMO the pretty printer should print the AST exactly as it's presented. I think it's too restrictive to force function arguments to be values at parse time, and thus we must allow for nested expressions. We could possibly have a compiler option that forces certain compiler passes like ANF and CSE before printing or even use the pretty printer to reformat code into "graph" style.

[tqchen]

This is an issue because this is how user usually constructs a program via DSL, and we need to make sure when we print, they are print out in a well-defined compact fashion which is readable by the parser(without running any transformation pass). The graph itself is not a graph for the exactly the same reason that it is already CSed as user constructs them.

i.e. if a user constructs the program via DSL(i.e. IRBuilder), she might write

x = some relay_expr
y = relay.op.add(x, 1)
let z = relay.op.add(y, y)

If we simply print out the expression as they are, it will explode as a tree

let %z = add(add(%x, 1), add(%x, 1))

The information that the intermediate value is CSEd is missing here, instead, a better way would be

%1 = add(%x, 1)
let %z = add(%1, %1)

I understand this is normally not considered as good pratice for normal programs, but this makes sense for deep learning workloads, where each operations can be arbitrarily nested.

Most users still think in terms of computational graphs, and it is important for us to support that concept well.

[joshpoll]

Here's a proposal for block syntax per @yuruofeifei's suggestion. It will have the same semantics as ReasonML's for local scope, which is

{
  let %x = 1;
  %x
};
...

desugars to

let _ =
  let %x = 1;
  %x;
...

[tqchen]

cc @junrushao1994 @dmlc/tvm-team @jroesch @MarisaKirisame @yuruofeifei @grwlf please comment on the choices of keywords in the text format. (see beginning of the post)

[tqchen]

I think one general question we want to ask when deciding keyword is consistency, and whether we want to enforce some of these. Note that being consistent with common choices will always reduce the mind burden of users, and we should try to do so when possible

• Consistency with python style
• boolean true/false vs True/False
• tuple: (1) vs (1,)
• Whether we support alternative printing of list
  • e.g. axis=[1,2] or should we always print Array as axis=(1,2)
• Consistency with numpy base type convention(and the printing output of tvm types)
  • e,g. float32, bool vs Float32, Bool
  • I can understand one possible reason to go for Float32 is that it really means Tensor[(), float32] , but do we need to make such distinguishment or not
• Consistent with common choices of existing languages
  • e.g. use of -> vs =>
    • Rust: fn(args) -> return_type
    • Swift func(args)-> return_type
• How to express common constant with type information
  • example proposal:
    • 1 -> const(1, int32)
    • 1.0f -> const(1, float32)
    • float64(1) -> const(1, float64)

[junrushao]

Neat.

BTW, it is necessary to use Int, UInt, Float as we already have Int32, Float32 etc? Will it cause some confusion?

[nhynes]

How to express common constant with type information
Rust has

let float1 = 1f32;
let double = 1.0;
let also_double = 1f64;
// same for {u32, u64, u128, ...}

necessary to use Int, UInt, Float

It will definitely be confusing unless there's a well defined spec for how the compiler chooses to use these annotations. I can see some value of this when compiling one model to different architectures with varied word sizes (e.g., native vs wasm).

tuple

This is already a major point of confusion for novice python devs. Following the convention of tuple = comma will at least keep the confusion consistent.

print Array as axis=(1,2)

Semantically, it's the same thing since the type is fixed at this point. If anything, square brackets might improve readability (e.g., x(y(z, axis=(1, 2)), axis=(3,)) vs x(y(z, axis=[1, 2]), axis=[3]))

[joshpoll]

@junrushao1994 Sorry I should have made this more clear, but Int, UInt, and Float are only used for type constructors, e.g. Float[32, 4] for 32 bit float with 4 lanes.

[junrushao]

@joshpoll I see. Sorry I didn't catch that. That looks cool now.

So why not we make type constructors capitalized like Int, Tensor, and normal types uncased like int64, float32?

[joshpoll]

The idea was to have all terms (e.g. variables, functions, literals) use lowercase (this is currently not enforced) and all types use uppercase (this is currently enforced by the parser). This is similar to Python type annotations and makes it clear whether something is a term or a type.

This may become especially important when we have generic shapes like (n, 1, m), because it should hopefully discourage users from writing things like (float32, 1, m).

Moreover, in Python type annotations are lowercase for builtins like int and str, but (by convention) uppercase for classes. I think this is a meaningless distinction for Relay and leads to more problems than it solves. It also can falsely equate functions and types. E.g. str in python is both a type and a function that returns a str.