This is in addition to a similar thread pointing out the inconsistencies in five-minute data that I had posted two months back https://kite.trade/forum/discussion/comment/17863/#Comment_17863. Although I have found (painful) workarounds to deal with inconsistencies in the data downloaded from zerodha, I sometimes wonder why even simple problems which can be easily fixed (and for which now I've routinely scan for in my code), are not looked at.
One such simple problem is that duplicate entries in the 1 day data. One of the main dates that seem to happen in several instruments is 2015-12-31. Its there in ITC, RELIANCE (and I list about 20 more stocks at least). I hope at some point of time you can just consider adding a few lines of code scan for duplicates. There are more such in 1 day data 2009-04-09.
Similarly, in five-minute data, there are hundreds of instances when the timestamp at the beginning of day is 09:15:00 or 09:15:01 and so on.. Also, the volume is 0 in several instances in the first five minutes. (And this includes data after 2017 also because in the earlier case, I was told that in 2015, the data may be inconsistent).
not here to make zerodha look bad. I'm just grateful that at least there are some steps a brokerage has taken to help the retail traders. And I hope that such messages help to improve the service for all of us.
Thank you for your response. I hope some action does get taken because its been several months since the last report and looks like there was no change.
Here is another quite a large problem. KALPATPOWR has 5 minute data missing from 2018-04-19 until 2019-02-03 Here are the details 2018-04-18 15:25:00,85.2,85.4,85.0,85.25,116402 2019-02-04 09:15:00,349.8,370.0,300.2,370.0,10043
If you wish, I can provide you a list of problems in all the data that I have looked at so far and that will hopefully reduce your workload. I am guessing you already have my email id from my account details and you can reach out through that. Always happy to assist
@rvsw We checked for KALPATPOWR-NSE token-464385 , interval="5minute", intraday candle is available for period 2018-04-18 to 2019-02-04. eg:{'date': 2018-4-19 9:15), 'open': 482.9, 'high': 485, 'low': 479.85, 'close': 482.75, 'volume': 2671} Can you recheck the same?
There have been many enhancements done on the historical data in recent months. We will continue to do the same. Unfortunately, everything can't be fixed overnight since it is a huge live database table with billions of rows which is used by more than a million users.
The intra-day data missing has to be populated by getting it from some other source. We have informed data team to take a look at this.
Here is another quite a large problem.
KALPATPOWR has 5 minute data missing from 2018-04-19 until 2019-02-03
Here are the details
2018-04-18 15:25:00,85.2,85.4,85.0,85.25,116402
2019-02-04 09:15:00,349.8,370.0,300.2,370.0,10043
If you wish, I can provide you a list of problems in all the data that I have looked at so far and that will hopefully reduce your workload. I am guessing you already have my email id from my account details and you can reach out through that. Always happy to assist
We checked for KALPATPOWR-NSE token-464385 , interval="5minute", intraday candle is available for period 2018-04-18 to 2019-02-04.
eg:{'date': 2018-4-19 9:15), 'open': 482.9, 'high': 485, 'low': 479.85, 'close': 482.75, 'volume': 2671}
Can you recheck the same?
The intra-day data missing has to be populated by getting it from some other source. We have informed data team to take a look at this.