Every now and then you have to deal with tabular data files that are larger than average. I, for one, occasionally have to deal with CSV files as large as 500MB – 1GB each (e.g. 10000 rows, 30000 columns). If you already had the doubtful pleasure of using Excel to handle such files you know that it can fail even to display them (16384 columns limit), not to mention any editing. Recently I became so desperate about Excel’s incompetence that I wrote my own tool specifically for browsing large CSV files. It’s called FastTable and features data display in tabular form with constant width/height of the cells. It has very low memory overhead, essentially the only thing that is loaded and kept in memory is the CSV file itself, plus very little cached information about positions of different cells in the file. Everything else required for the display is efficiently computed on the fly. You can open CSV files either by dropping them on the main window or using Open File dialog. At the moment only semicolon-separated files are supported and no semicolons within cells. Give it a try. Thanks to FastTable I was finally able to instantly get an overview of all those big CSV files without Excel’s shiny interface squirming in agony of processing 1GB of data.
DOWNLOAD (Win32 binary): fasttable.zip
SOURCE CODE (BSD-old licensed): fasttable_src.zip
Very useful. Two possible improvements:
(i) Add a button (or better still a keyboard shortcut) for quick reloading of the current document. The postcript/pdf viewers gv and xpdf have this functionality, and it makes them valuable as previewers for document that are being worked on. As applied to fasttable, this would allow the user to tweak a data-processing pipeline in a command-prompt window and then immediately see the result by switching to a fasttable window.
(ii) Add an option to the build proces for using separators other than semicolons. It’s easy enough to edit QFastTableView.cpp by hand, replacing ‘;’ with, for instance ‘t’, but some automation would be nice for those of use who want a fasttable for viewing tsv data rather than semicolon-separated data.
Thanks for the praise. I will add these features and more to my upcoming “CSV Studio” application.