Hi, Today morning (around 8:50am) the instruments list downloaded from https://api.kite.trade/instruments was truncated and had very few symbols. This list is used as a cross-check point for all trades in our system. Since this list was truncated, most of the algo trades got rejected internally.
This data truncation has happened after several months of clean runs. Since it was working fine, we didn't setup a check/reload function and so today's failure affected the system badly. Yes, it is a rare failure, but a very critical one.
It would be good if you can check on this and ensure that the instruments download URL is working fine all the time, especially during the morning hours when the list gets updated.
More than trades getting rejected, the issue is that on a daily basis, I download this file and use it to get instrument_tokens for all desired symbols for websocket data streaming. Because of the issue, I was not able to stream data for all symbols.
As a workaround, internally in my system, I made a change. Now I am downloading two instruments file, one for exchange= "NSE" and other for exchange= "NFO" and appending them. However, like in this case, there is no guarantee that this is future-proof.
Hi @skk, @ramatius, How are you guys downloading files? Are you using a browser or curl? Instrument file is generated at around 7:30 AM and it is a static file which is not updated during the day.
The partial list file is attached.
Usually it has 60,000+ symbols, but today it had only 5,000+ symbols.
More than trades getting rejected, the issue is that on a daily basis, I download this file and use it to get instrument_tokens for all desired symbols for websocket data streaming. Because of the issue, I was not able to stream data for all symbols.
As a workaround, internally in my system, I made a change. Now I am downloading two instruments file, one for exchange= "NSE" and other for exchange= "NFO" and appending them. However, like in this case, there is no guarantee that this is future-proof.
How are you guys downloading files?
Are you using a browser or curl?
Instrument file is generated at around 7:30 AM and it is a static file which is not updated during the day.
I think it is better to enable gzip that should resolve half file download issue wherein client can't decode the partial file.