Let me clarify before I start .... I am aware it is impossible for NSE/Zerodha to capture and send every movement of a scrip. I also know Zerodha say they don't modify data before disseminating further. I also know how much does colo costs and who to contact. I am fully aware, and prepared, that there will always be data missed or delayed or both in the streamed ~1Hz tick.
However what I observe is bad data similar to race-condition effects.
For example, Tick1: 10:30:01: 20 shares traded at 100.50. TotalVol=100 Tick2: 10:30:02: 10 shares traded at 100.60. TotalVol=100 Tick3: 10:30:03: 30 shares traded at 100.40. TotalVol=100 Tick4: 10:30:03: 30 shares traded at 100.40. TotalVol=135
Obviously 20, 10 and 30 trades of Tick[1,2,3] are not coming from thin air ... they are reported with distinct timestamps, so must have happened. Now the last total-volume update (35 trades) in Tick4 can not be correct (because we know 20+10+30 trades did occur). This discrepancy can not be by missing or delayed data.... it is messed up data. We can not rely on even coarse-grained 1second ticks??!!
To be frank, either NSE or ZERODHA is at fault. Zerodha is a matured entity, but then NSE is far more so.
1) Can someone confirm if others too have noticed this anomaly? 2) If yes, are we sure it's nothing due to Zerodha ? (NSE messing data stream is unthinkable to me)
Yeah I have also noticed this several times, where depth,LTP and LTQ changes, that means trade occured since last tick, but volume remain the same. This is conterintutive in some sense.
I guess imo, this is happening due to hard limit set by Zerodha or by NSE, where they only show change in volume if cumulative amount of shares are traded above a limit between several ticks.
Or race condition happening, but it would be very hard to believe.
I can confirm, this is not only the problem in python, but also in golang and rust.
The solution I used to solve this problem, is to use data from several ticks and make algo according to that. But that means making an algo even with 1 sec latency is out of the window for NSE.
This data is created by using go client, and saved it into mongoDB, from there to csv(for display). I have marked the data with black line, where Volume discrepency is occuring(changes in LTP and LTQ occuring but not in Volume). The time is saved in UTC. This is data of NSE Reliance.
I have also checked it for BSE, where everything is fine and without any discrepency.
I guess imo, this is happening due to hard limit set by Zerodha or by NSE, where they only show change in volume if cumulative amount of shares are traded above a limit between several ticks.
Or race condition happening, but it would be very hard to believe.
I can confirm, this is not only the problem in python, but also in golang and rust.
The solution I used to solve this problem, is to use data from several ticks and make algo according to that. But that means making an algo even with 1 sec latency is out of the window for NSE.
This data is created by using go client, and saved it into mongoDB, from there to csv(for display). I have marked the data with black line, where Volume discrepency is occuring(changes in LTP and LTQ occuring but not in Volume). The time is saved in UTC.
This is data of NSE Reliance.
I have also checked it for BSE, where everything is fine and without any discrepency.