本文介绍了重塑大量数据时提升循环性能的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

这是上一个问题的延伸问题。我正在根据下面的代码重塑数据。但是,当数据量增加时,上一个问题中的答案(预分配)就不能再处理了,MATLAB甚至被冻结了整个笔记本电脑。因此,正如Teddy在最后一个问题中提出的那样,我打开了一个新的问题来提高循环的性能。



有些相似但不相似的问题可以找到。从那里,给出的答案建议修改循环为列式。然而,因为我的循环需要同时访问行和列,似乎答案是不适用于我的情况。



是否仍然可以修改此循环以改进其性能?或者,这个循环可以完成而不冻结整个笔记本电脑?如果可能的话,不涉及GPU。

  in = rand(291081,1920); 
m = 581;
[R,C] = size(in);
R_out = R / m;

out =零(m * C,R_out);对于k = 1,
:从第1行到第n行的m%
对于i = 1:C%重塑第n行的每一列
out(i + C *(k-1), :) = in(k:m:end,i)';
end
end

P / S:在前面的问题中,大小越来越大,循环和 arrayfun 似乎没有大的性能差异。



感谢提前!首先,感谢 @NLindros 答案是提供了答案,这启发了这个答案。看来使用重塑是迄今为止处理大量数据的最有效的方式。因此,为避免使用转置,建议使用 permute reshape ,如下所示。 b
$ c $ out = reshape(permute(resmpe(in,m,R_out,[]),[1 2 3]),[],R_out);

这是迄今为止最快速的重塑方式。



我将其标记为回答如果没有更好的答案。


This is an extended question from previous question here. I'm reshaping the data according to the code below. However when the data size get increased, in=rand(291081,1920);, the answer (preallocation) in the previous question can't handle it anymore, MATLAB even freezed the whole laptop. Thus as suggested by Teddy in the last question, I'm opening a new question to boost the performance of the loop.

Some quite similar but not alike question can be found here. From there, the answer given suggested to modify the loop to be column-wise. However, since my loop required to access row and column simultaneously, it seem the answer is not applicable in my case.

Is this loop can still be modified to improve its performance? Or can this loop be done without freezes the whole laptop? If possible, without getting involve with GPUs.

in=rand(291081,1920);
m=581;
[R,C]=size(in);
R_out=R/m;

out=zeros(m*C,R_out);
for k=1:m %from row 1 to nth row
    for i=1:C %reshape every column of nth row
        out(i+C*(k-1),:) = in(k:m:end,i)';
    end
end

P/S: In previous question, when the data size getting bigger, the loop and arrayfun seem to not have large performance difference.

Thanks in advance!

解决方案

Firstly, thanks @NLindros for providing the answer, which had inspired this answer. It seems using reshape is so far the most efficient way to handle huge data. Thus to avoid the usage of transpose as suggested, permute and reshape is used, as below.

out=reshape(permute(reshape(in,m,R_out,[]),[1 3 2]),[],R_out);

This by far is the fastest way to reshape.

I'll mark this as answer if there's no more better answer.

这篇关于重塑大量数据时提升循环性能的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

08-30 04:47