本文介绍了优化以下CSV阅读器代码的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

大家好,



主要是由于可读性我使用以下代码来a)在驱动器上找到特定的csv文件,b)读取csv文件使用streamReader和c)将其解析为List以便以后能够使用LINQ和/或PLINQ。



但是,这个过程大约需要4-5秒,这简直太长了。关于如何改进以下(或者甚至可以替换它?)的任何建议:



Hi everyone,

mostly due to readability I am using the following code to a) find a specific csv file on a drive , b) read that csv file using a streamReader and c) parse it into a List to be able to use LINQ and/or PLINQ later on.

However, the process takes about 4-5 seconds, which is simply put too long. Any suggestions on how to improve the following (or maybe even replace it?) :

var query = (from x in Directory.GetFiles(_path, "*.csv").AsParallel()
             where x.Contains(dateTime)
             select x).First();


            #region Read CSV File
            List _tempList = new List();

            FileStream stream = new FileStream(query, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
            using (StreamReader reader = new StreamReader(stream, Encoding.Default, true, 1024))
            {
                string currentLine;
                while ((currentLine = reader.ReadLine()) != null)
                {
                    string[] temp = currentLine.Split(new char[] { ',' }, StringSplitOptions.None);
                    _tempList.Add(temp);
                }
            }
            #endregion





我感谢你的帮助。 ;)csv文件中的顺序很重要 - 该文件将包含(2000-7000)x 25个条目。



跟进问题。 csv文件有几个我不感兴趣的字段 - 我不一定在我的List(或string [] [])中需要它们。

我试图使用类似于以下伪代码的LINQ语句过滤它们:



I am thankfull for help. ;) The order inside the csv file is important - the file will contain between (2000-7000) x 25 entries.

Follow up question. The csv file has several fields which I am not interested in - and I don't necessarily need them in my List (or string[][]) .
I have tried to filter them with a LINQ statement similar to the following pseudo code:

var query = from x in MyListOfStringArray
            select new {Col1 = x[1] , Col2 = c[4]}.ToArray();





这种方法导致一种奇怪的查询类型 - 而且这个问题非常缓慢。 (我会有一个大约有9-11个属性的数组)。

关于第二个问题的任何想法?



This approach results in a strange type for query - and for that matter was pretty slow. (I would have an array with about 9-11 properties).
Any ideas on that second issue ?

推荐答案


这篇关于优化以下CSV阅读器代码的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

09-05 12:07
查看更多