attn_base和attn_repalce的选择 #30

我可以更换attn_base和attn_repalce的选择吗，这样选择的理由是什么呢
attn = attn.reshape(self.batch_size, h, *attn.shape[1:])
attn_base, attn_repalce = attn[0], attn[1:]
if not is_cross:
"""
==========================================
========= Self Attention Control =========
=== Details please refer to Section 3.4 ==
==========================================
"""
#调用 replace_self_attention，用 attn_base 替换掉 attn_replace
self.loss += self.criterion(attn[1:], self.replace_self_attention(attn_base, attn_repalce))
attn = attn.reshape(self.batch_size * h, *attn.shape[2:])