python - 了解PyTorch einsum

我很熟悉einsum在纽比的运作方式。pytorch也提供了类似的功能：torch.einsum()。在功能和性能方面有什么相同点和不同点？在Py火炬文档中提供的信息相当稀少，没有提供任何关于这方面的见解。 最佳答案 由于EIOSM在火把文档中的描述不太充分，所以我决定写这篇文章来比较、对比A如何与AA>相比。差异：NUMPY允许小写字母和大写字母torch.einsum()为“下标字符串”，而PyTrink只允许小写字母numpy.einsum()。除了 > >之外，多语言支持大量关键字参数（例如[a-zA-Z]），而。以下是PyTorch和NumPy的一些例子的实现：# input tensors to work withIn [16]: vecOut[16]: tensor([0, 1, 2, 3])In [17]: atenOut[17]:tensor([[11, 12, 13, 14], [21, 22, 23, 24], [31, 32, 33, 34], [41, 42, 43, 44]])In [18]: btenOut[18]:tensor([[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]])1）矩阵乘法火把：[a-z]；optimize纽比：nd-arraysIn [19]: torch.einsum('ij, jk -> ik', aten, bten)Out[19]:tensor([[130, 130, 130, 130], [230, 230, 230, 230], [330, 330, 330, 330], [430, 430, 430, 430]])2）沿主对角线提取元素喷灯：torch.matmul(aten, bten)纽比：aten.mm(bten)In [28]: torch.einsum('ii -> i', aten)Out[28]: tensor([11, 22, 33, 44])3）Hadamard积（即两个张量的元素乘积）喷灯：np.einsum("ij, jk -> ik", arr1, arr2)纽比：torch.diag(aten)In [34]: torch.einsum('ij, ij -> ij', aten, bten)Out[34]:tensor([[ 11, 12, 13, 14], [ 42, 44, 46, 48], [ 93, 96, 99, 102], [164, 168, 172, 176]])4）按元素对角化喷灯：np.einsum("ii -> i", arr)纽比：aten * btenIn [37]: torch.einsum('ij, ij -> ij', aten, aten)Out[37]:tensor([[ 121, 144, 169, 196], [ 441, 484, 529, 576], [ 961, 1024, 1089, 1156], [1681, 1764, 1849, 1936]])一般：元素智能np.einsum("ij, ij -> ij", arr1, arr2)幂可以通过重复下标字符串和张量aten ** 2倍来实现。例如，计算元素的第四张量的功率可以使用：# NumPy: np.einsum('ij, ij, ij, ij -> ij', arr, arr, arr, arr)In [38]: torch.einsum('ij, ij, ij, ij -> ij', aten, aten, aten, aten)Out[38]:tensor([[ 14641, 20736, 28561, 38416], [ 194481, 234256, 279841, 331776], [ 923521, 1048576, 1185921, 1336336], [2825761, 3111696, 3418801, 3748096]])5）微量元素（即主要对角线元素之和）喷灯：np.einsum("ij, ij -> ij", arr, arr)纽比岛：nthIn [44]: torch.einsum('ii -> ', aten)Out[44]: tensor(110)6）矩阵转置喷灯：n纽比岛：torch.trace(aten)In [58]: torch.einsum('ij -> ji', aten)Out[58]:tensor([[11, 21, 31, 41], [12, 22, 32, 42], [13, 23, 33, 43], [14, 24, 34, 44]])（向量的）外积喷灯：np.einsum("ii -> ", arr)纽比岛：torch.transpose(aten, 1, 0)In [73]: torch.einsum('i, j -> ij', vec, vec)Out[73]:tensor([[0, 0, 0, 0], [0, 1, 2, 3], [0, 2, 4, 6], [0, 3, 6, 9]])（向量的）内积喷灯：np.einsum("ij -> ji", arr)纽比岛：torch.ger(vec, vec)In [76]: torch.einsum('i, i -> ', vec, vec)Out[76]: tensor(14)9）沿轴0求和喷灯：np.einsum("i, j -> ij", vec, vec)纽比岛：torch.ger(vec1, vec2)In [85]: torch.einsum('ij -> j', aten)Out[85]: tensor([104, 108, 112, 116])10）沿轴1求和喷灯：np.einsum("i, i -> ", vec1, vec2)纽比岛：torch.sum(aten, 0)In [86]: torch.einsum('ij -> i', aten)Out[86]: tensor([ 50, 90, 130, 170])11）批量矩阵乘法喷灯：np.einsum("ij -> j", arr)纽比：torch.sum(aten, 1)# input batch tensors to work withIn [13]: batch_tensor_1 = torch.arange(2 * 4 * 3).reshape(2, 4, 3)In [14]: batch_tensor_2 = torch.arange(2 * 3 * 4).reshape(2, 3, 4)In [15]: torch.bmm(batch_tensor_1, batch_tensor_2)Out[15]:tensor([[[ 20, 23, 26, 29], [ 56, 68, 80, 92], [ 92, 113, 134, 155], [ 128, 158, 188, 218]], [[ 632, 671, 710, 749], [ 776, 824, 872, 920], [ 920, 977, 1034, 1091], [1064, 1130, 1196, 1262]]])# sanity check with the shapesIn [16]: torch.bmm(batch_tensor_1, batch_tensor_2).shapeOut[16]: torch.Size([2, 4, 4])# batch matrix multiply using einsumIn [17]: torch.einsum("bij, bjk -> bik", batch_tensor_1, batch_tensor_2)Out[17]:tensor([[[ 20, 23, 26, 29], [ 56, 68, 80, 92], [ 92, 113, 134, 155], [ 128, 158, 188, 218]], [[ 632, 671, 710, 749], [ 776, 824, 872, 920], [ 920, 977, 1034, 1091], [1064, 1130, 1196, 1262]]])# sanity check with the shapesIn [18]: torch.einsum("bij, bjk -> bik", batch_tensor_1, batch_tensor_2).shape12）沿轴2求和喷灯：np.einsum("ij -> i", arr)纽比岛：torch.bmm(batch_tensor_1, batch_tensor_2)In [99]: torch.einsum("ijk -> ij", batch_ten)Out[99]:tensor([[ 50, 90, 130, 170], [ 4, 8, 12, 16]])13）求一个nd张量中所有元素的和喷灯：np.einsum("bij, bjk -> bik", batch_tensor_1, batch_tensor_2)纽比岛：torch.sum(batch_ten, 2)In [101]: torch.einsum("ijk -> ", batch_ten)Out[101]: tensor(480)14）多轴求和（即边缘化）喷灯：np.einsum("ijk -> ij", arr3D)纽比：torch.sum(batch_ten)# 8D tensorIn [103]: nDten = torch.randn((3,5,4,6,8,2,7,9))In [104]: nDten.shapeOut[104]: torch.Size([3, 5, 4, 6, 8, 2, 7, 9])# marginalize out dimension 5 (i.e. "n" here)In [111]: esum = torch.einsum("ijklmnop -> n", nDten)In [112]: esumOut[112]: tensor([ 98.6921, -206.0575])# marginalize out axis 5 (i.e. sum over rest of the axes)In [113]: tsum = torch.sum(nDten, dim=(0, 1, 2, 3, 4, 6, 7))In [115]: torch.allclose(tsum, esum)Out[115]: True15）双点积（同：火炬：和（Hadamard积）参见3）喷灯：np.einsum("ijk -> ", arr3D)纽比：torch.sum(arr, dim=(dim0, dim1, dim2, dim3, dim4, dim6, dim7))In [120]: torch.einsum("ij, ij -> ", aten, bten)Out[120]: tensor(1300)关于python - 了解PyTorch einsum，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/55894693/