blaze
allows Python users a familiar interface to query data living in diverse data storage systems. Cf. http://blaze.pydata.org/.
import blaze as bz
The first example constructs a blaze.Data
object from native Python objects.
t = bz.Table([('Henry', 'boy', 8),
('Lilli', 'girl', 14)],
columns=['name', 'gender', 'age'])
t
t[t.age > 10]
Let us read data from an in-memory NumPy ndarray
object.
import numpy as np
t1 = bz.Table(np.random.standard_normal((1000, 5)),
columns=['f0', 'f1', 'f2', 'f3', 'f4'])
t1
t1.data
We generate first a CSV file using random data.
import pandas as pd
df = pd.DataFrame(np.random.standard_normal((1000, 5)),
columns=['f0', 'f1', 'f2', 'f3', 'f4'])
df.to_csv('data.csv', index=False)
Let us read the data with blaze
.
c = bz.CSV('data.csv')
t2 = bz.Table(c)
t2.count()
t2
t2.data
We now generate a SQLite3 table with dummy data.
import sqlite3 as sq3
con = sq3.connect('data.sql')
try:
con.execute('DROP TABLE numbers')
except:
pass
con.execute('CREATE TABLE numbers (f0 real, f1 real, f2 real, f3 real, f4 real)')
con.executemany('INSERT INTO numbers VALUES (?, ?, ?, ?, ?)',
np.random.standard_normal((1000, 5)))
con.commit()
con.close()
Now reading the data with blaze
.
t3 = bz.Table('sqlite:///data.sql::numbers')
t3
t3.data
!ls data.* -n
# cleaning up
!rm data.*
Python Quant Platform | http://quant-platform.com
Python for Finance | Python for Finance @ O'Reilly
Derivatives Analytics with Python | Derivatives Analytics @ Wiley Finance