问题描述
我必须使用线程将彩票产生的所有可能组合写入文件中.示例:
- 0 0 0 0 0 0 0(第一个组合)
- 0 0 0 0 0 1 1(第二组合)
- 0 0 0 0 0 2
. . .
最后的. 38 38 38 38 38 38(最后组合)
在我的主类中，我只使用一个线程，因为我不知道如何使用更多线程来更快地写入文件.
要生成数字，我会在内部使用6个循环，这种方式是这样:
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.util.Scanner;
public class Proceso extends Thread {
File file = new File("lottery.txt");
public Proceso(String msg) {
super(msg);
}
public void run() {
try {
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file, true)));
StringBuffer linea = new StringBuffer();
for (int i = 0; i < 15;i++) {
for (int j = 0; j < 39; j++) {
for (int j2 =0; j2 < 39; j2++) {
for (int k = 0; k < 39; k++) {
for (int k2 = 0 ; k2 < 39; k2++) {
for (int l = 0; l < 39; l++) {
linea.append(i + " " +j + " " +j2 + " " +k + " " +k2 + " " +l + "\n");
bw.write(linea.toString());
}
}
}
}
}
}
bw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
该文件(在执行过程中)的大小也增加了8GB，由于我的PC崩溃，我不得不停止该文件.
如果我不能做到这一点，至少我可以几乎同时在同一文件中使用两个不同的线程进行写操作?如果我能做到那我该怎么做?
您首先要停止.认真地:退一步.
首先，由于tkausl正确地进行了注释，因此当您的瓶颈是IO性能时，使用多个线程可能无济于事.相反，让多个线程写入相同文件可能并非全部有用.此外，要做到这一点很难！
因此，如果有的话，您将必须考虑将此任务分配到几个系统上.然后我们调查您的代码；很明显，您无法使用这样的代码来做到这一点.
这是我的初衷:您的代码极其是低级的.那里几乎没有抽象方法可以将此任务中的不同活动彼此分开.只是在循环内循环.那是可能完成"其工作的代码(或者，如果预期"的输出会更小).但是它不包含将其适应"实际需求所需的任何有用的抽象.
长话短说:如果您真的想尝试多线程的事情；然后将代码转换为 基于流 的解决方案.然后，如果可行，您可以研究 并行 流；并使用它们来并行"运行代码.仅作记录:您的代码中存在错误.
linea.append(i + " " +j + " " +j2 + " " +k + " " +k2 + " " +l + "\n");
bw.write(linea.toString())
这在后面添加了一行新行；然后写入整个缓冲区.但是:这些行停留在您的缓冲区中！
因此，也许在这种情况下，简单"的答案是:在写入缓冲区内容后，清除！像:linea.delete(0, linea.length()).顺便说一句:使用StringBuilder;不是StringBuffer.您可以简单地为每行创建一个新的构建器(而不是一直使用/清除同一构建器).
由于StringBuffer不断增长，您很有可能会遇到异常.直到内存用完为止.
最后一点:退一步仍然是这里要做的事情.如果我没记错的话，您打算创建的文件将有38 * 38 * 38 * 38 * 38 * 38行，占3.010.936.384，因此大约有30亿个条目.假设每行包含10、15个字节.因此，您最终得到的文件大小约为30 GB.但为什么?该文件将包含这些值的简单排列.加上一点数学知识，仅行号就可以用来计算在该行中找到的字符串！因为您可以将1到3.010.936.384之间的任何值映射到相应的置换".
因此，长话短说:不管哪个程序正在使用该30 GB文件-都不需要.您知道有多少种排列方式；并且您可以轻松地计算(也称为预测)排列.需要这些排列的任何程序都可以做到！如果您可以如此轻松地计算数据，则没有必要将30 GB数据写入文件系统！
因此，从本质上讲:无需将这些内容写入文件.你没有你.这只会花费您IO时间来写和读稍后的内容.而且您知道，"IO"仍然是最昂贵的事情.
I have to write in a file all possible combinations resulting from the lottery using threads. Example:
- 0 0 0 0 0 0 (first combination)
- 0 0 0 0 0 1 (Second combination)
- 0 0 0 0 0 2
. . .
Last. 38 38 38 38 38 38 (Last combination)
In my main class i just use one thread because i dont know how i can use more threads for write in the file faster.
For generates the numbers i uses 6 loops one inside the other this way:
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.util.Scanner;
public class Proceso extends Thread {
File file = new File("lottery.txt");
public Proceso(String msg) {
super(msg);
}
public void run() {
try {
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file, true)));
StringBuffer linea = new StringBuffer();
for (int i = 0; i < 15;i++) {
for (int j = 0; j < 39; j++) {
for (int j2 =0; j2 < 39; j2++) {
for (int k = 0; k < 39; k++) {
for (int k2 = 0 ; k2 < 39; k2++) {
for (int l = 0; l < 39; l++) {
linea.append(i + " " +j + " " +j2 + " " +k + " " +k2 + " " +l + "\n");
bw.write(linea.toString());
}
}
}
}
}
}
bw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Also the file (in middle of the execution) rise a size of 8gb and i have to stop it because my pc collapse.
Edit: if i cant do that, at least i can write with two different threads in the same file at almost the same time? If i can do that how i would do it?
You start by stopping. Seriously: step back for a second.
First of all, as tkausl is correctly commenting, using more than one thread probably will not help when your bottleneck is IO performance. To the contrary - having multiple threads writing into the same file might not all be helpful. Besides, it is hard to get that right!
So, if at all, you would have to look into distributing this task onto several systems. And we then look into your code; it becomes obvious that you can't do that with such code.
That is my initial point: your code is extremely low-level. There is almost no abstraction in there that would separate the different activities within this task from each other. Just loop within loop within loop. That is code that probably "does" its job (or maybe would, if the "expected" output would be smaller); but it doesn't contain any of the helpful abstractions that you would need to "adapt" it to your actual needs.
Long story short: if you really want to try the multi-threaded thing; then start by turning your code into a stream based solution. And then, when that works, you can study parallel streams; and use those to run your code "in parallel".
And just for the record: there is a bug in your code.
linea.append(i + " " +j + " " +j2 + " " +k + " " +k2 + " " +l + "\n");
bw.write(linea.toString())
This appends a new line to your buffer; and then writes the whole buffer. But: the lines stay in your buffer!
So, maybe in this case, the "easy" answer is: clear the buffer after you wrote its content! Like: linea.delete(0, linea.length()). And btw: use StringBuilder; not StringBuffer. You could simply create a new builder for each line (instead of using/clearing the same builder all the time).
Most likely you get your exception because your StringBuffer keeps growing and growing; until you run out of memory.
Final note: stepping back is still the thing to do here. If I am not mistaken, the file you intend to create will have 38*38*38*38*38*38 lines,that makes up 3.010.936.384, so roughly 3 billion entries. Assume that each line contains 10, 15 bytes. So you end up with a file around 30 GB or so. But why? This file will contain a simple permutation of those values.And with a bit of math ... alone the line number could be used to compute the string that would be find in that line! Because you can map any value between 1 and 3.010.936.384 to the corresponding "permutation".
So, longer story short: whatever program is working with that 30 GB file - it doesn't need to. You know how many permutations exist; and you can easily compute (aka predict) the permutations. Any program that needs those permutations can do that! There is no point in writing 30 GB of data to the file system if you can compute the data so easily!
Thus, in essence: forget about writing this stuff to a file. You don't have you. That just costs you IO time writing and reading in later on. And you know, "IO" is still the most expensive thing to do.
这篇关于在这种情况下，如何有效使用线程?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！