Help with Data Handling in Python

heyitsabi
heyitsabi edited August 2018 in Python client
Hey guys,
1. I am very new to python and algo trading. I have recently subscribed to the API and I am having issue with handling the data.

Example:
When I use KiteConnect - kite.quote to fetch data - I am getting a set of dictionaries and I have used a loop to bifurcate the data, which is working fine but is inefficient because of how dictionaries work in python, sometimes the data received has different location for different keys.

t = kite.quote("NIFTY50","12111106","NSE:HDIL","NSE:HINDALCO","NSE:ITI","NSE:SETCO","SDBL","NSE:MTNL","NSE:DAAWAT","NSE:KOHINOOR")
columns = ['symbol']
temp = list()

for i in t.keys():
values = []
counter = 1
if i == "12111106":
values.append("NIFTY18AUGFUT")
else:
values.append(i)
for j in t[i].keys():
#print(j,t[i][j])
if j == 'ohlc':
for k in t[i][j].keys():
if k not in columns:
columns.append(k)
values.append(t[i][j][k])
elif j == 'depth':
for k in t[i][j].keys():
#print(k)
if counter >= 5:
counter = 1
for items in t[i][j][k]: #this portion handles the buy/sell depth
#for some reason it is a dictionary inside a list
for l in items.keys():
if (k+l+str(counter)) not in columns:
columns.append(k+l+str(counter))
values.append(items[l])

counter += 1
else:
if j not in columns:
columns.append(j)
values.append(t[i][j])
temp.append(values)

tdf = pd.DataFrame(temp,columns=columns)
The above code does the job but is suffering with the fundamental flaw that I am assuming the data received will always have the same location within the dictionary. Also while using the instrument_token of an instrument the quote doesn't return the name of the instrument, I had to manually do that. And when I stream data the format changes and I had to write another set of loops to handle that, which failed badly because of how dictionary works.

Please help me with this - I want to format the data into pandas for easier computation. If there is some other modules that help with the same please let me know about that, I will try my best to learn that module before asking here again.

2. I want to subscribe for the historical API as well, does that api provide data in nested list/dictionary format? I am having trouble bifurcating these format.
Sign In or Register to comment.