[RELAY] Support RelayBuild with Only Constants

[IshwaraK]

The below reduced test-case shows segmentation fault in TVM compiler stack at the optimization level 2 and above. The problem is in call to 'lib.get_source()', as constant folding optimizes away whole graph and lib object becomes NULL.

import tvm
from tvm import relay
from tvm.contrib import graph_runtime
import numpy as np

data_shape = (2, 2)
matrix_x = relay.var('matrix_x', shape=data_shape, dtype='int32')
matrix_y = relay.var('matrix_y', shape=data_shape, dtype='int32')

add = relay.op.add(matrix_x, matrix_y)
fun = relay.Function([matrix_x,matrix_y], add)
relayMod = relay.Module({})
fname = relay.GlobalVar('main')
relayMod.entry_func = fname
relayMod[fname] = fun
print("Relay Module:\n", relayMod)

x_data = np.matrix('1 2; 3 4', dtype='int32')
y_data = np.matrix('5 6; 7 8', dtype='int32')
params = {
    "matrix_x" : x_data,
    "matrix_y" : y_data,
}
with relay.build_config(opt_level=3):
    graph, lib, params = relay.build(relayMod, "llvm", params=params)
print("Model graph:\n", graph)
print("\nModel parameters:")
for item in params:
    print(item, ":\n", params[item])
print("\nOperator library:\n", lib.get_source())
mod = graph_runtime.create(graph, lib, ctx=tvm.cpu(0))
mod.set_input(**params)
mod.run()
res = mod.get_output(0).asnumpy()
print("inference:\n", res)

[tqchen]

Given that this is an clearly actionabble issue, turn this to a task

[tqchen]

cc @zhiics @jroesch

[zhiics]

I saw this before as well. We actually had a warning in the C++ code when the returned module is empty.
https://github.com/apache/incubator-tvm/blob/ee0af843f3c5a3429e888079afb5f30789bd9bee/src/relay/backend/build_module.cc#L479-L480

In order to avoid the erroneously working on the empty module, we can probably assert here:
https://github.com/apache/incubator-tvm/blob/ee0af843f3c5a3429e888079afb5f30789bd9bee/python/tvm/relay/build_module.py#L244

But I think it might be not a good idea to assert as we should probably allow to feed the graph runtime an empty module. It just does nothing.

[kumasento]

Hi @zhiics @tqchen I'm interested in this issue and would be happy to help.

From a user's perspective, we would normally expect that relay.build returns an empty lib. An empty lib can be NULL/None, but it may cause confusion like this issue. As @zhiics mentioned, using assertion may raise unexpected issues for end users as well.

One workaround for this problem, which I'm currently implementing, is inserting a dummy function (with empty content) into the lowered_funcs map before passing to module building. In this way, we won't break other parts of the codebase, and the returned lib still makes sense.

If you prefer this way, I can continue implementing it and create a PR later.

Thanks

[tqchen]

I agree with @zhiics that allowing feeding an empty Module to GraphRT makes more sense here.
In parallel, I am also OK supporting a dummy function to get around with most of the issues we have. There are certain cases where we still want to serialize the module binary blob as so, but there aren't any host functions.

also cc @FrozenGene @yzhliu

[kumasento]

Thanks @tqchen , I've proposed a tentative implementation of this dummy function idea #4847. The script in this issue can be built successively now.

lib.get_source() output:

Operator library:
 ; ModuleID = '__dummy__'
source_filename = "__dummy__"
target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-pc-linux-gnu"

@__tvm_main__ = weak local_unnamed_addr constant [10 x i8] c"__dummy__\00", align 1

; Function Attrs: norecurse nounwind readnone
define dllexport i32 @__dummy__() local_unnamed_addr #0 !dbg !5 {
entry:
  ret i32 0, !dbg !9
}

attributes #0 = { norecurse nounwind readnone }

!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!3, !4}

!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "TVM", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, dwoId: 1)
!1 = !DIFile(filename: "model.tvm", directory: "/tmp/")
!2 = !{}
!3 = !{i32 2, !"tvm_target", !"llvm"}
!4 = !{i32 4, !"Debug Info Version", i32 3}
!5 = distinct !DISubprogram(name: "__dummy__", scope: !1, file: !1, type: !6, isLocal: false, isDefinition: true, flags: DIFlagPrototyped, isOptimized: true, unit: !0, variables: !2)
!6 = !DISubroutineType(types: !7)
!7 = !{!8}
!8 = !DIBasicType(name: "int32", size: 32, encoding: DW_ATE_signed)
!9 = !DILocation(line: 0, scope: !5)

There is one more question though: what the target of the dummy function should be? I currently set it to llvm.

[zhiics]

Not sure if we want to use LLVM in this case as it will be treated as the host module when exporting. I even think it is just okay to leave it as it is because it means we just return a null pointer when returning an empty module, which is a valid return. Considering mod is is essentially a shared pointer, the failure/crash in dereferencing a null pointer sounds good to me.

[FrozenGene]

I even think it is just okay to leave it as it is because it means we just return a null pointer when returning an empty module, which is a valid return. Considering mod is is essentially a shared pointer, the failure/crash in dereferencing a null pointer sounds good to me.

IMO, I agree with @zhiics 's point here.

There is one more question though: what the target of the dummy function should be? I currently set it to llvm.

Should be target_host_. llvm just means x86 cpu.

[tqchen]

I would also like to comment about the future direction here.

In our vision, eventually everything will be stored in an IRModule, and it would be fine to create an empty IRModule and export it. We could also in-theory create an EmptyRuntimeModule, or use CSourceModule(with no content) as an empty module.