Table Import and Export

We can save a table object to disk in binary mode with the function saveTable, or save to a text file with the function saveText . Compared to saveTable, saveText requires much more space and is more time consuming. However, it is convenient if users need to take a quick look at the data. It is recommended for saving small data sets.

To save a table to a DFS database, please first use function createPartitionedTable to create a DFS table, and then use function append! or tableInsert.

The example below shows how efficiently save saves a table with 20 million records to disk, compared with saveText.

n=20000000
syms=`IBM`C`MS`MSFT`JPM`ORCL`BIDU`SOHU`GE`EBAY`GOOG`FORD
timestamp=09:30:00+rand(18000,n)
sym=rand(syms,n)
qty=100*(1+rand(100,n))
price=5.0+rand(100.0,n)
t1=table(timestamp,sym,qty,price);

timer saveTable("C:/DolphinDB/Data", t1, `trades);
// output
Time elapsed: 208.963 ms

saveText(t1, "C:/DolphinDB/Data/trades.csv");
// output
Time elapsed: 3231.914 ms

We can use the function loadTable to load previously saved data, or use the function loadText to import a text file. loadText is more time consuming than loadTable (binary mode).

The example below shows how efficiently loadTable loads a table with 20 million records, compared with loadText.

n=20000000
syms=`IBM`C`MS`MSFT`JPM`ORCL`BIDU`SOHU`GE`EBAY`GOOG`FORD
timestamp=09:30:00+rand(18000,n)
sym=rand(syms,n)
qty=100*(1+rand(100,n))
price=5.0+rand(100.0,n)
t1=table(timestamp,sym,qty,price)
saveTable("C:/DolphinDB/Data", t1, `trades);

timer tt1 = loadTable("C:/DolphinDB/Data",`trades,,true);
// output
Time elapsed: 179.423 ms

saveText(t1, "C:/DolphinDB/Data/trades.txt");
timer tt2=loadText("C:/DolphinDB/Data/trades.txt");
// output
Time elapsed: 7609.7 ms