using entity.wall_id instead of entity.id for wall lookup

[chinmay0301ucsd]

This PR is regarding the issue #79. One of the reasons for performance regression on doors and windows comes from the evaluation script eval.py. I found that in lines 181 :

attach_wall = wall_id_lookup.get(entity.id, None)

and 200

attach_wall = wall_id_lookup.get(entity.id, None)

of the file eval.py, entity.wall_id should be used instead of entity.id, as that would actually fetch the correct wall id of the attached entity (door / window) from the wall id lookup dictionary. This change affects only the door and window evaluation numbers, not the wall numbers. On making this change, I re-ran 3 evaluations

1. Finetuned the SpatiaLM Model SpatialLM1.1-Qwen-0.5B on structured3D train set with the labels and gt provided by the authors, and evaluated it on the structured3D test set -
   | Layouts | F1 @.25 IoU | F1 @.50 IoU |
   |---------|-------------|-------------|
   | wall | 0.9300944007413725 | 0.9158097842802695 |
   | door | 0.9301076299430355 | 0.9245620838810897 |
   | window | 0.892983727373873 | 0.8768681686048276 |

2. SpatialLM model finetuned on Structured3D provided by the authors https://huggingface.co/ysmao/SpatialLM1.1-Qwen-0.5B-Structured3D-SFT evaluated on structured3D test set.
   | Layouts | F1 @.25 IoU | F1 @.50 IoU |
   |---------|-------------|-------------|
   | wall | 0.9418111124360891 | 0.9318676751934882 |
   | door | 0.9503952201855035 | 0.9459975133713943 |
   | window | 0.9055098651958046 | 0.892037246299918 |

3. SpatialLM Model SpatialLM1.1-Qwen-0.5B without any finetuning evaluated on the structured3D test set.
   | Layouts | F1 @.25 IoU | F1 @.50 IoU |
   |---------|-------------|--------------|
   | wall | 0.7516214808780001 | 0.6963917824685191 |
   | door | 0.4925566276008396 | 0.4578708554555917 |
   | window | 0.6665149238502482 | 0.27879356972595465 |

For the all the 3 results above, the performance for doors and windows improves significantly to the previous results obtained in the discussion in #79.

[SamMaoYS]

Thank you for carefully reviewing and catching the issues in the evaluation script for the door and window instances!

[tmzcometolearn]

1. Finetuned the SpatiaLM Model SpatialLM1.1-Qwen-0.5B on structured3D train set with the labels and gt provided by the authors, and evaluated it on the structured3D test set

Hello, you mentioned that you finetuned the SpatiaLM Model SpatialLM1.1-Qwen-0.5B on structured3D train set with the labels and gt provided by the authors, and evaluated it on the structured3D test set. Is this dataset structured3d-spatiallm.
Look forward to your reply. Best wishes to your work and life.
Thanks a lot.

[tmzcometolearn]

This PR is regarding the issue #79. One of the reasons for performance regression on doors and windows comes from the evaluation script eval.py. I found that in lines 181 :

attach_wall = wall_id_lookup.get(entity.id, None)

and 200

attach_wall = wall_id_lookup.get(entity.id, None)

of the file eval.py, entity.wall_id should be used instead of entity.id, as that would actually fetch the correct wall id of the attached entity (door / window) from the wall id lookup dictionary. This change affects only the door and window evaluation numbers, not the wall numbers. On making this change, I re-ran 3 evaluations

1. Finetuned the SpatiaLM Model SpatialLM1.1-Qwen-0.5B on structured3D train set with the labels and gt provided by the authors, and evaluated it on the structured3D test set -

   Layouts	F1 @.25 IoU	F1 @.50 IoU
   wall	0.9300944007413725	0.9158097842802695
   door	0.9301076299430355	0.9245620838810897
   window	0.892983727373873	0.8768681686048276

2. SpatialLM model finetuned on Structured3D provided by the authors https://huggingface.co/ysmao/SpatialLM1.1-Qwen-0.5B-Structured3D-SFT evaluated on structured3D test set.

   Layouts	F1 @.25 IoU	F1 @.50 IoU
   wall	0.9418111124360891	0.9318676751934882
   door	0.9503952201855035	0.9459975133713943
   window	0.9055098651958046	0.892037246299918

3. SpatialLM Model SpatialLM1.1-Qwen-0.5B without any finetuning evaluated on the structured3D test set.

   Layouts	F1 @.25 IoU	F1 @.50 IoU
   wall	0.7516214808780001	0.6963917824685191
   door	0.4925566276008396	0.4578708554555917
   window	0.6665149238502482	0.27879356972595465

For the all the 3 results above, the performance for doors and windows improves significantly to the previous results obtained in the discussion in #79.

And I'd like to know, what specific parameters did you use when you fine-tuned it? Especially num_bins. Because the author provided SpatialLM1.1-Qwen-0.5B-Structured3D-SFTwith num_bins as 640.