Is Historic Data Accurate?

Ganesh
Has anyone cross checked the accuracy of Historic data? I just happened to come across differences across all intervals. I checked for 1minutes,5minutes,15minutes and even eod data when I try to match it with my existing data provider there are quite a few difference. My current data provider is proven and I am using it for more than 4 years now so just want to cross check with people here if you are also seeing the same discrepancies?

The live streaming of API is accurate and matches my data provider and terminal however when it comes to historic data it fails. The difference is good enough to screw indicators and averages to generate false signals.

I experienced this for CRUDEOIL future. Anyone any thoughts?

Thanks,
Ganesh
Tagged:
  • nithin
    nithin edited November 2016
    Ganesh,

    I had answered a similar question on tradingqna - why does historical charts seem different across platforms? Charting platforms essentially use historical data, hence the same answer applies.
    Tick Data: Market data that shows price and volume at every print, it also includes information about every change to the best bid and ask.

    In a single trading day, there are millions of such ticks recorded (remember this also includes change in best bid and ask prices) and it is technically not possible for any trading platform to show all ticks, especially live and on a normal broadband internet connection.

    If you really want to see all the ticks, NSE does have Tick by Tick data feed which is a lot more expensive than the normal data feed, and will have to be subscribed to througn any of the authorized NSE data vendors (Not all of them give you tick by tick data). Also to be able to make best use of this tick by tick data, you should ideally be on the NSE collocation which would mean that your systems are inside NSE itself, so that accepting all the millions of ticks will be quite efficient.

    Tick by Tick data is usually what professional High Frequency and Arbitrage trading firms use, don't think there is anything much a normal retail trader can do with it.

    Coming back to the question, all charting platforms are driven by data which is basically a 1 second snapshot data (unless you have spent a few lakhs to get that TBT data). So if I am looking at a 5 min chart, there could have been thousands of ticks in that 5 mins, but our charting platform would be able to recognize only a few hundred. And using those few hundred ticks, 1 Open, High, low and close points are chosen to plot the candle. So these few hundred points recognized, and the 4 points (OHLC) could very easily be different from one platform to another.

    So none of the above charts are wrong, but the chances of any two charting platforms to exactly match each other, is very slim. But like you can see from the above charts, the difference is quite small, and the general trend of the market is captured by all the 3 charts. I am sure you can add charts from a few more platforms and they will all have some kind of variance.

    So if you are a trader, don't break sweat on this. It is not possible to match two charts and find out which one is more correct. They are all most likely right.
  • sameer
    sameer edited November 2016
    U mean NSE only releases tick data and every vendor creates x-minute data from tick data ?
    Doesn't NSE computes 1-minute candle and officially release it in real-time ?

    I guess NSE should be providing official 1-minute data real time stream as
    i) Constructing minute candle from tick data can have errors
    ii) But constructing N-minute candle from 1-minute data is error-free
  • Kailash
    @sameer No, exchanges do not release minute (or any other compounded interval) data in realtime, but raw ticks.
  • Ganesh
    Thanks @nithin ,

    This explains the variation.
This discussion has been closed.