Differences in Historic data when taken on different days

nikkite15
1. I took Historic data of 5 min Nifty 50 csv format thru Python client on 8 Feb & 9 Feb.
2. Both should be same but i see in some date-time differences in the range of Rs 0.05 to 1.4
3. e.g.
8 feb 2020 downloaded data has - 6/19/2019 15:05 11668.1 11692.1 11664.25 11690.05
9 feb 2020 downloaded data has - 6/19/2019 15:05 11668.1 11692.1 11664.25 11691.45
Low has difference of 1.4

8 feb 2020 downloaded data has - 4/10/2019 9:15 11645.6 11656.85 11639.45 11656.05
9 feb 2020 downloaded data has - 4/10/2019 9:15 11646.85 11656.85 11639.45 11656.05
open has difference of 1.25

8 feb 2020 downloaded data has 6/20/2019 10:20 11716.95 11719.9 11711.7 11715.4
9 feb 2020 downloaded data has 6/20/2019 10:20 11716.95 11720 11711.7 11715.4
high has difference of 0.1

Apart from these there are 18 to 20 5min entries where difference is present. I expect that NO DIFFERENCE as same data source is used and same params used.whats the remedy /fix for this as My Trading system requires accurate data thats why I purchased this Historic Data module but if data is changing everyday then its not accurate one.

My python code for download csv:
records=kite.historical_data(instrument_token=256265, from_date="2015-01-01 09:15:00", to_date="2015-03-31 15:30:00", interval='5minute',oi=0)
df = pd.DataFrame(records)
df=df.drop(columns=["volume"])
df['date']=df['date'].dt.tz_localize(None)
df.sort_values(by=['date'], inplace=True, ascending=False)
df.to_csv(r"E:\Tools\Kite Connect\Data\NIFTY\5min\2015_1.csv",index=False)
  • rakeshr
    @nikkite15
    We have forwarded it to data team.Our team is investigating this.
  • sujith
    sujith edited February 2020
    Though we have forwarded your query to the data team. It is important to keep some bullets while working with historical data.
    There is nothing called a accurate data. Each platform will create candles using the data that is captured using the live market data. Generally, the idea is to capture the trend.

    No two platform data will match for the same timeframe since tick data is received via the internet and there can be drops.
    You can refer to this thread to know more.

    It is not possible to achieve 100% accuracy unless you have a co-location server which will record each and every tick at the exchange. The setup costs you around 18 lakhs per annum. If you want this setup, you can write to kiteconnect(at)zerodha.com someone from our team will connect with you for assistance.

    If the dataset provided by us is not feasible for your use-case then I would suggest consulting some exchange registered data vendor.
  • sujith
    sujith edited February 2020
    For live strategies, we recommend generating candles at your end using the live market data provided via Kite Ticker.
  • nikkite15
    1) Sujith- Thanks for reply. I already know about different Data sources/vendors will have difference in prices. Also know that data I am getting Level 2 data so its not 100% accurate. Thats why for my system I dont mix data from different vendors.
    2) But I have not raised ticket for that.My concern is- I downloaded data from your source only so no question of different data providers. All parameters are same (same data source, historic data, 5min time frame, same format ) but on different days of download prices are different , which should not be case. I hope your Data Team will correct this and revert.
  • nikkite15
    Any updates on this?
  • nikkite15
    its been 15 days since i have raised this issue. Need answer on this! Can Data team answer on this??
  • nikkite15
    whats the helpline/call center number to address this issue?
  • nikkite15
    since my subscription will be ended by 9 March 2020 and if I dont get this issue resolved by then I would expect to get subscription extended with no further payment. Pls read and respond or give me call on my registered mobile number.
  • rvarade
    @sujith At least historical_data hourly candles should match the chart data on kite web app? This is creating huge difference in strategies that i am implementing
  • sujith
    @rvarade,
    We would like to repeat our stance on this.
    For live strategies, we recommend generating candles at your end using the live market data provided via Kite Ticker.
  • rvarade
    @sujith i get your point. My only point is why data through API and data on kite web platform is not same. As a consumer, i would expect the same data if I am paying for the service.
  • sujith
    At what time did you fetch data?
  • sujith
    As I mentioned in this thread. It seems fine.
  • ganeshv02
    @sujith - If the data is not 100% accurate and you know about it then it is better to put that as a disclaimer in your main documentation page. Because we are paying 2000 rupees for historical data API. Not saying that it has to be 100% accurate but this must be mentioned somewhere rather than us finding and raising this as a concern. Let me know your thoughts.

    Btw - Some vendors are also giving historical expired weekly option data which is very useful for back testing options. Zerodha does not give that as well yet the cost is the same with other vendors. Who should we raise this to?
  • sujith
    @ganeshv02,
    I will pass your feedback about adding the disclaimer to the documentation to the concerned team.

    We don't intend to provide expired options data as of now. You can check out a relevant thread.

  • ganeshv02
    Thanks @sujith. Appreciated.

    I have just written to your data team to look at providing historical weekly option data at least for few months so it is useful for back testing option strategies. FYERS (tied up with Truedata) provides this and it is quite useful. Although I am more inclined to use Zerodha in longer run for algo trading, i have placed the request with your team. This will be helpful for everyone here I think.
Sign In or Register to comment.