This CSV reader will provide you with uncompromised performance. Thanks to the power of MEX compiler, for the first time in history you can load _HUGE_ CSV files into MATLAB in seconds. Enjoy!
Usage:
quote = '"'; sep = ';'; escape = ''; [numbers, text] = swallow_csv(filename, quote, sep, escape);
The attached archive contains compiled MEXW64 version. For other OS/architecture you need to:
mex swallow_csv.cpp
DOWNLOAD: swallow_csv.zip
Thanks for this.
It is truly fantastic 🙂
This is truly awesome, the original csvimport function i used took 2 hours to load a single csv and this is like 10 seconds
Hi Cecil,
May I ask how to use the csv reader? I am an absolute beginner for matlab and I would like to import a 1.3G .csv file data in matlab. All the data in the file is one header line and the other are five columns of eight decimal numbers.
Thanks
This is light years faster than anything that I have used. Do you have a version that reads a certain range in the csv file ? In the original csvread in MATLAB you can specify a range from which to read from (row and column to row and column). The tool is already hugely useful but this addition will make it immensely hugely useful 🙂
Thanks for this, it’s great !
Thanks a lot. Tried in vain to get textscan to read a file with datestrings,'”‘-escaped data and some missing values, this solved all problems + it is quick. Good work!
There is an error in the cpp file; the escape character is not in the if-else chain of characters that may be escaped, preventing literal escape characters from being included in fields. I recommend inserting:
} else if (buf[i] == escape) {
cell += escape;
below line 111.
Otherwise, this library worked charmingly.
Hi,
I was wondering if you have a 32bit compiled MEX.I have a problem with my version of Matlab I use and a free C++ compiler to compile mex.Any help would be greatly appreciated.
Many Thanks
vivek
Try this one: https://www.youtube.com/watch?v=au-3DV20QDY
That helped me.
Hi,
is there a way to skip the header in a csv fle?
Thanks!!
I got an Out of memory error on importing a 1.9Gb csv file, so I’m not too impressed with this program. I’ll keep searching for an alternative method.
For me the program didn’t work because of “Encountered unknown escape sequence”. Tried to fix it but no solution yet..
Here, I have seen many discussions with respect to .CSV to .MAT file converter, but still I am little bit confused on this. Can any one help me to write a Matlab code which converts .csv file to .mat file dynamically.
Here is the first 4 lines of csv file. Sorry I am not allowed to post an csv file here, or I would have.
[Name] M1 Speed M1 Shaft Torque M2 Speed M2 Shaft Torque [Unit] min-1 N.m min-1 N.m 1 0 0.16805 0 -0.01404 2 0 0.16301 0 -0.01042
FYI: this is just a sample, My CSV file has 40 columns and 48076 rows. First 2 rows and 2 columns has alphabets, then down remaining all rows n columns has only numbers.
thanks – ksc
It doesnt work on big data that contains tab delimited CSV files
I have tried this function. It is awesome. However, it can only load the first 1048575 rows of the files… Is it possible to load all data from the file.
By the way, what is the meaning of ‘quote’, ‘sep’, and ‘escape’. Since there is no introduction of these variable, it is hard to apply it in different cases.
Thanks a lot!
Ashely
Hello,
I have an excel sheet which has 400000 rows and 22 columns. I have used xlsread to read file, but its taking lots of time. Please help to read it faster.
Wow, how can this be faster than matlab’s CSV reader? You are just reading the file in chunks. Can matlab’s reader be that bad!?!
Lol – yes, it can 😉
As others have said, this would be a lot more useful if it just had a little bit of explanation as to how to use it.
IT DOESNT WORK WITH COMMA SEPERATED VALUES, what is the meaning is not explained, whoever says it is useful please confront it with me
it works. what’s your problem?
thanks a lot!!! amazing!!!
Can anyone send me an example of how this works? I attempted the following code.
quote = ‘”‘;
sep = ‘;’;
escape =”;
[numbers,text] = swallow_csv(Navitas_v141_000000_CAN1_000.csv, quote, sep, escape);
I attempted this Navitas_v141_000000_CAN1_000 without csv extension and had no luck.
This sounds very appealing, but I can’t get it to work.
[numbers, text] = swallow_csv(filename, [quote, [sep, [escape]]]);
Regardless of the values I save for quote, sep, and escape, I get a single column of combined text and a single column of NaN.
txt output: ‘-0.041098142,1.26102481,1.04647969,2.47291482,2.47184668,0,0’
numbers ouput: NaN
I would really appreciate if someone could share how to use the software correctly.
I discovered the problem.
In case anyone else would like to know. I needed to change the values wtihin the .cpp file directly (in my case, change “sep” from ‘;’ to ‘,’) and re-run “mex swallow_csv.cpp”, rather than inputting different values within “[numbers, text] = swallow_csv(filename, [quote, [sep, [escape]]])”
Guys, have been using your code on Win 10 x32 and receive the following error message: “Unexpected Standard exception from MEX file.
What() is:bad allocation”
Seems like not enough memory can be used. Any ideas how to fix that?
The ‘user manual’ is a little terse…
That is, non existant… what do you do with this?
For my csv files, the MEX function worked with the function call given below:
[numbers, text] = swallow_csv(filename);
Compared to MatIab, the MEX function is incredibly fast: It imports a 250 MB csv file (1.8e6 x 18 values) in just 14 s.
Thanks for providing this extremely useful function 🙂
I am running some MATLAB code which uses swallow_csv; I am running the code on Windows 10 and I keep getting a usage error no matter what I do…
quote = ‘”‘;
sep = ‘,’;
escape = ‘\’;
[raw, text] = swallow_csv(filename, quote, sep, escape);
Error using swallow_csv
Usage: [numbers, text] = swallow_csv(filename, [quote, [sep, [escape]]])