Python trained model gives different features set in C++

[MyraBaba]

Hi,

We have model trained in python and want to use it from c++ as for getting 512D float features.

c++ gives different output when we try with image-classification-predict.cpp

We just want to get data (as in code) 512d float vector as in python output ?

model:
https://pan.baidu.com/s/1mj6X7MK
or

https://www.dropbox.com/s/ou8v3c307vyzawc/model-r50-arcface-ms1m-refine-v1.zip?dl=0

Isnt there a clear method for C++ feature extraction ?

Best

[lanking520]

@MyraBaba thanks for your question and thanks for trying out the C++ API. Could you please bring me more context on the difference with the result? How would you build your C++ package and your system info?
@leleamol here since he is the developer with C++.

@mxnet-label-bot please add label [C++, Question]

[MyraBaba]

Hi
@leleamol

System is Mac os X and also ubuntu 18
build mxnet with cpp support

We want to use Mxnet in our all production.

We just stuck in first attemp :)

https://github.com/blob/master/deploy/test.py this is the our first test.

We can extract feature in pyhton.

We tried to use predict-cpp api to use model and extract future with same model above first message .

c++ image-classification-predict-cpp we put same model same picture in the above repo Tom-hanks.

it gives different outpu (512D but different float numbers).

Clearly we are missing something because we are new to mxnet .

Basically ; We want to this in c++ as in python and want to get same features:

img = cv2.imread('Tom_Hanks_54745.png')
img = cv2.resize(img , (112,112))

img = model.get_input(img)
f2 = model.get_feature(img)
dist = np.sum(np.square(f1-f2))
sim = np.dot(f1, f2.T)

Best...

[MyraBaba]

@leleamol
@lanking520

any idea ? what could be the basic steps in C++ for :

img = model.get_input(img)
f2 = model.get_feature(img)
dist = np.sum(np.square(f1-f2))
sim = np.dot(f1, f2.T)

We loved the mxnet framework bu fi we cant use it in our C++ environment unfortunately we have to drop mxnet and back to the old system.

for mxnet experts specially on the C , C++ side it should be trivial example which means a lot for the community.

What we can do to make it works ?

[leleamol]

@MyraBaba
The MXNet supports C++ APIs in cpp-package directory. I am working on an example to demonstrate a workflow that will

1. Load the model,
2. Load the parameters, and
3. Run a forward pass.

I will try out your model with that example. Will update this issue with my observations.
Thanks

[MyraBaba]

We almost exhausted :) It certainly should work but we are very new to mxnet framwork. On the other hand training speed and mem usage fantastic.

If we succesfully get same results in c++ as in our python test.py we will change our AI to the mxnet completely .

Thanks for your help and attention.

Best

[leleamol]

With my test program I was able to get the output of float vector of 512. Without using sklearn.preprocessing.normalize on that output (similar to what was done in the test python script), the maximum accuracy was reported to be 2.7216 at index 84. The rest of the entries in 512D vector were 0.03412. ( I have used the same image as that of test.py)

I did not normalize the image before running forward pass.
If this helps, I can share the code of my example.

[MyraBaba]

I will work on your c++ code and try to make progress. I will run under the cpp-package /example/ feature_extract. Right ? I can finalise this week this we will move money and there will be support requested we can discuss separately Best

…

On 22 Oct 2018, at 20:42, Amol Lele ***@***.***> wrote: With my test program I was able to get the output of float vector of 512. Without using sklearn.preprocessing.normalize on that output (similar to what was done in the test python script), the maximum accuracy was reported to be 2.7216 at index 84. The rest of the entries in 512D vector were 0.03412. ( I have used the same image as that of test.py) I did not normalize the image before running forward pass. If this helps, I can share the code of my example. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12869 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQscn9zC_lg0ZG6AfMOZUy_oGMhnxeygks5ungN2gaJpZM4XvOz7>.

[leleamol]

@MyraBaba
The cpp-package/example/feature_extract is a good start but it is not very optimized and does not follow the recommended approach to run the inference workflow.
For example, the program is invoking "SimpleBind" operation every time before running the Forward() call. Since, SimpleBind() is an expensive operation, it should be invoked only once.

Please let us know if you need help.

[MyraBaba]

Can you share your code that you mentioned above please , I will start from your code.

[leleamol]

@MyraBaba
I have submitted a PR #12915. It will be under review. I have used the same example to load the model you have mentioned in the issue and run the inference workflow with the image Tom_Hanks_54745.png

[MyraBaba]

@leleamol
Thanks I started to working on it.
inception_inference is asking synset file. All we need 512D feature we will use it for our own purposes. we dont need labels. How we can just produce 512 features ?

I used synset.tx (inception) with below command and get error below.:

How did you run exactly for tom_hanks (112x112) ?

`./inception_inference --symbol /Users/alpullu/Projects/inSAF-face/models/model-r50-am-lfw/-symbol.json --params /Users/alpullu/Projects/inSAF-face/models/model-r50-am-lfw/-0000.params  --input_shape 3 112 112 --image /Users/alpullu/Projects/inSAF-face/deploy/Tom_Hanks_54745.png --synset ./sysnset.txt --mean mean_224.nd 
[13:50:28] inception_inference.cpp:121: Loading the model from /Users/alpullu/Projects/inSAF-face/models/model-r50-am-lfw/-symbol.json

[13:50:28] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v0.12.1. Attempting to upgrade...
[13:50:28] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
[13:50:28] inception_inference.cpp:130: Loading the model parameters from /Users/alpullu/Projects/inSAF-face/models/model-r50-am-lfw/-0000.params

[13:50:28] inception_inference.cpp:153: Loading the synset file.
libc++abi.dylib: terminating with uncaught exception of type dmlc::Error: [13:50:28] ../../include/mxnet-cpp/symbol.hpp:211: Check failed: MXSymbolInferShape(GetHandle(), keys.size(), keys.data(), arg_ind_ptr.data(), arg_shape_data.data(), &in_shape_size, &in_shape_ndim, &in_shape_data, &out_shape_size, &out_shape_ndim, &out_shape_data, &aux_shape_size, &aux_shape_ndim, &aux_shape_data, &complete) == 0 (-1 vs. 0) 

Stack trace returned 9 entries:
[bt] (0) 0   inception_inference                 0x0000000107ca386b dmlc::StackTrace() + 299
[bt] (1) 1   inception_inference                 0x0000000107ca35ff dmlc::LogMessageFatal::~LogMessageFatal() + 47
[bt] (2) 2   inception_inference                 0x0000000107ca5ad3 mxnet::cpp::Symbol::InferShape(std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > > > > const&, std::__1::vector<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > > >*, std::__1::vector<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > > >*, std::__1::vector<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > > >*) const + 947
[bt] (3) 3   inception_inference                 0x0000000107ca4a64 mxnet::cpp::Symbol::InferExecutorArrays(mxnet::cpp::Context const&, std::__1::vector<mxnet::cpp::NDArray, std::__1::allocator<mxnet::cpp::NDArray> >*, std::__1::vector<mxnet::cpp::NDArray, std::__1::allocator<mxnet::cpp::NDArray> >*, std::__1::vector<mxnet::cpp::OpReqType, std::__1::allocator<mxnet::cpp::OpReqType> >*, std::__1::vector<mxnet::cpp::NDArray, std::__1::allocator<mxnet::cpp::NDArray> >*, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::OpReqType, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::OpReqType> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&) const + 548
[bt] (4) 4   inception_inference                 0x0000000107c9e6cf mxnet::cpp::Symbol::SimpleBind(mxnet::cpp::Context const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::OpReqType, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::OpReqType> > > const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mxnet::cpp::NDArray, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, mxnet::cpp::NDArray> > > const&) + 127
[bt] (5) 5   inception_inference                 0x0000000107c9d981 Predictor::Predictor(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, mxnet::cpp::Shape const&) + 641
[bt] (6) 6   inception_inference                 0x0000000107ca1d49 main + 1849
[bt] (7) 7   libdyld.dylib                       0x00007fffb9881235 start + 1
[bt] (8) 8   ???                                 0x000000000000000f 0x0 + 15

`

[leleamol]

@MyraBaba The synset file and mean image file are not mandatory. But there is a bug in code that mandates synset file to be specified. I am sending out the fix.
The crash you are seeing is due to incorrect input image shape.
The program assumes the input image shape to be 3 * 224 * 224, if not specified on the command line.
For your model your command line would something like this:

inception_inference --symbol <model_json_file> --params --image --synset <./synset file> --input_shape "3 112 112"

[leleamol]

Here is the PR #12930 after fixing the example.