Historical minute data- receiving wrong minute data

akulkanojia2414

If i fetch this query:
minuteData=pd.DataFrame(kite.historical_data(1086465, from_date="2021-05-26", to_date="2021-05-26", interval='minute'))

multiple times, I am receiving 2 different copies for the same symbol.
This is happening for multiple symbols.





Attaching screenshots, please help.

rakeshr

There seems to be difference in volume data as well for multiple candles. Can you re-check, if you are making the same request param? Also, can you paste here the complete request param for the same?

akulkanojia2414

Thanks for replying @rakeshr.

I am just running

minuteData=pd.DataFrame(kite.historical_data(1086465, from_date="2021-05-26", to_date="2021-05-26", interval='minute'))

in python, multiple times.

akulkanojia2414

Attaching my unit test logs for you to better understand which include multiple dates and symbols where I am facing this issue.

akulkanojia2414

Adding for reference.

If I run this,

while(1):
minuteData=pd.DataFrame(kite.historical_data(1086465, from_date="2021-05-26", to_date="2021-05-26", interval='minute'))

print(minuteData.loc[minuteData["date"].dt.time==datetime.time(9,33)]["close"].iloc[0])
time.sleep(1)


Output:
2822.0
2822.0
2822.0
2822.0
2822.9
2822.9
2822.9
2822.9
2822.9
2822.9
2822.9
2822.0
2822.0
2822.9
2822.9
2822.0
2822.0

SRIJAN

Actually, this is a problem in some instruments. I saw this post too few months before.
https://kite.trade/forum/discussion/9842/volume-data-inconsistent/p1
Not sure,but I guess there was some error at zerodha on 26 may 2021 which caused this problem in some instruments. This is not an api problem,but problem from zerodha side. The charts on Kite app change too for 26 may 2021 when queried a number of times. But I don't understand how this happens?? I mean, incorrect data is understandable but how can the api/Kite return different data some times?? That means there must be more than 1 database for some stocks accidentally. And the api/Kite app returns data randomly from one of those databases thus different data some times.

PS- After I saw your log,it looks like that this error is in other dates' data too .

akulkanojia2414

Thanks for adding the thread @SRIJAN, it was helpful. Seems like it is a known unresolved issue.

I figured they maybe loadbalancing and stuff. Most probably some db design issue for db updates.

This doesn't seem to be a one-off case from my unit tests.
This is really concerning and critical for the backtests.

akulkanojia2414

Can someone help address the issue?

Some workaround at least to get consistent data even if data has inaccuracies.

sujith

You will have to fetch data once and dump it at your end and use that as source.