Let's also note that PyTorch 1.2 has introduced native transformer modules ... math.sqrt(d_k) if mask is not None: scores = scores.masked_fill(mask == 0, ...
確定! 回上一頁