优化以下CSV阅读器代码

问题描述

大家好，



主要是由于可读性我使用以下代码来a）在驱动器上找到特定的csv文件，b）读取csv文件使用streamReader和c）将其解析为List以便以后能够使用LINQ和/或PLINQ。



但是，这个过程大约需要4-5秒，这简直太长了。关于如何改进以下（或者甚至可以替换它？）的任何建议：

Hi everyone,

mostly due to readability I am using the following code to a) find a specific csv file on a drive , b) read that csv file using a streamReader and c) parse it into a List to be able to use LINQ and/or PLINQ later on.

However, the process takes about 4-5 seconds, which is simply put too long. Any suggestions on how to improve the following (or maybe even replace it?) :

var query = (from x in Directory.GetFiles(_path, "*.csv").AsParallel()
             where x.Contains(dateTime)
             select x).First();


            #region Read CSV File
            List _tempList = new List();

            FileStream stream = new FileStream(query, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
            using (StreamReader reader = new StreamReader(stream, Encoding.Default, true, 1024))
            {
                string currentLine;
                while ((currentLine = reader.ReadLine()) != null)
                {
                    string[] temp = currentLine.Split(new char[] { ',' }, StringSplitOptions.None);
                    _tempList.Add(temp);
                }
            }
            #endregion

我感谢你的帮助。 ;）csv文件中的顺序很重要 - 该文件将包含（2000-7000）x 25个条目。



跟进问题。 csv文件有几个我不感兴趣的字段 - 我不一定在我的List（或string [] []）中需要它们。

我试图使用类似于以下伪代码的LINQ语句过滤它们：

I am thankfull for help. ;) The order inside the csv file is important - the file will contain between (2000-7000) x 25 entries.

Follow up question. The csv file has several fields which I am not interested in - and I don't necessarily need them in my List (or string[][]) .
I have tried to filter them with a LINQ statement similar to the following pseudo code:

var query = from x in MyListOfStringArray
            select new {Col1 = x[1] , Col2 = c[4]}.ToArray();

这种方法导致一种奇怪的查询类型 - 而且这个问题非常缓慢。 （我会有一个大约有9-11个属性的数组）。

关于第二个问题的任何想法？

This approach results in a strange type for query - and for that matter was pretty slow. (I would have an array with about 9-11 properties).
Any ideas on that second issue ?

推荐答案

这篇关于优化以下CSV阅读器代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！