Hello, could you please guide what should be the preferred way of retrieving one minute candle data for multiple stocks? 1) Save most recent tick received for all stocks. At the end of each minute as per local time, check last tick received for a stock and consider the same for latest candle. (Some systems have slightly different time in seconds, which is causing difference in actual candles and data retrieved) 2) Use internet time instead of system time within script to check end of each minute and process the data based on last tick received for each stock 3) Use the time from streaming ticks itself to check open/close of the minute candle. In such case how to handle scenario when a tick is not received at the end of exact minute? For example, for 09:15 to 09:16 candle, what if last tick received is not in 09:16:00 but on either 09:15:55 or 09:16:05? How to cater this?
++ @rakeshr@sujith Would appreciate if you could please guide on priority.
First of all - your systems should never run on local time. They should use NTP to sync their time to reliable NTP servers. Second, if you want minute level candles for live data, the only reliable way is to use the kite ticker to subscribe to the ticks for those instruments, process the ticks and then generate the candle from those ticks.
Third - when creating candle from ticks your local system time does not matter. What you need to reference is the time field within the tick. And you need to retrieve all ticks from the beginning of the minute to the end of the minute. Say you are creating candle for 10:15 am. You will get all ticks from 10:15:00 to 10:15:59. The first tick will be open, last will be close and then you will need to find he max and min to get the high and low.
It does not matter if the first tick is not exactly at 10:15:00 and the last tick is not exactly at 10:15:59. The only way one can get and process all ticks is to have a colo setup at VSNL mumbai which costs 18Lakhs for the rackspace, additional costs for the high end servers and networking starting at 25L and going into crores. Additionally it will also need some advanced IT knowledge to code and maintain those systems and keep them running.
I will repeat again, your systems local time is immaterial. If you use that, all your computations will be wrong. There is a reason each tick comes with a datetime field.
EDIT: Also let me add here that it depends on what instruments you are subscribed to. For liquid instruments you will normally have ticks at the first and last second. Its only for illiquid instruments that you will have ticks intermittently with a large gap between subsequent ticks.
I implemented using the way you suggested. I've a finding and I would like to know your advice on the same.
Total volume for the day as checked on the chart and received through websocket streaming are different. Volume received through websocket streaming is higher than what we see on the chart day candle.
Does volume received in websocket streaming also considers pre-open-market volume? Do we need to subtract that from volume received throughout the day to get accurate minute candle data? Or are there any other points to consider while calculating volume for each minute candle?
I went through the thread but couldn't get answer to the question. My requirement is to get volume for each minute data and not every tick data. I believe volume received for stocks through websocket streaming per tick is a day volume, is that correct? If tick is received at 59th second of any minute I'm considering it for volume calculation, else a nearest tick received before that.
When I checked volume for AARTIIND at 09:15:59 it was 16412 from websocket streaming but 10078 on chart for first one minute candle. I'm trying to understand why this difference.
Please suggest a way to check volume data for every minute.
I'm checking again latest completed candle on the chart. I saved the data at 09:15:59 and checked the values in tick against first candle on the chart, from 09:15:00 -09:15:59
@sujith "Sum of volumes for the whole day should ideally match." this is mentioned in the answer of the link you shared however I see difference between volume on the chart day candle and the last tick received through websocket streaming. Could you please help me to understand why volumes at any minute or even at the EOD isn't same as what we see on the chart?
@rakeshr@sujith But I see the difference in volumes even on 1 minute candles comparing to data received through ticks. What could be the reason or is there any recommended technical approach to get minute volume data? Please guide.
@zerodhaVyankatesh The OHLC information will never match exactly as we get a subset of ticks from the exchange. As long as they are close, you shouldn't worry. Right now I don't do comparisons. But when I started out, I used to do so and they were almost always a few pips away. Sometimes H,L,C would match but O would be off by a few pips. Sometimes two out of four would match and the other two would be off by a few pips. And thats ok. In fact each charting provider will give you slightly different values. This is a part of market dynamics.
Thanks again, @MAG Do we expect some difference in 1 minute volumes as well? I thought total volume calculated at the end of each minute (hh:mm:59) should match on what we see on the chart.
@zerodhaVyankatesh There are N number of ticks per second depending on the liquidity of the instruments. I don't know whats the average but I wouldn't be surprised if a very liquid instrument has hundreds of ticks per second at the exchange level.
Lets call this number N. We don't know the value of N. But 1. It's definitely more than one (unless its an illiquid instrument). 2. And when we say more than one we don't know if its 2 or 200.
It's plain commonsense logic. If you are getting only one or two ticks out of N ticks per second, how can anyone expect the volume to match exactly? Irrespective of whether one is looking at a one minute interval, five minute interval or any other interval.
Second, if you want minute level candles for live data, the only reliable way is to use the kite ticker to subscribe to the ticks for those instruments, process the ticks and then generate the candle from those ticks.
Third - when creating candle from ticks your local system time does not matter. What you need to reference is the time field within the tick. And you need to retrieve all ticks from the beginning of the minute to the end of the minute. Say you are creating candle for 10:15 am. You will get all ticks from 10:15:00 to 10:15:59. The first tick will be open, last will be close and then you will need to find he max and min to get the high and low.
It does not matter if the first tick is not exactly at 10:15:00 and the last tick is not exactly at 10:15:59. The only way one can get and process all ticks is to have a colo setup at VSNL mumbai which costs 18Lakhs for the rackspace, additional costs for the high end servers and networking starting at 25L and going into crores. Additionally it will also need some advanced IT knowledge to code and maintain those systems and keep them running.
I will repeat again, your systems local time is immaterial. If you use that, all your computations will be wrong. There is a reason each tick comes with a datetime field.
EDIT: Also let me add here that it depends on what instruments you are subscribed to. For liquid instruments you will normally have ticks at the first and last second. Its only for illiquid instruments that you will have ticks intermittently with a large gap between subsequent ticks.
I implemented using the way you suggested. I've a finding and I would like to know your advice on the same.
Total volume for the day as checked on the chart and received through websocket streaming are different. Volume received through websocket streaming is higher than what we see on the chart day candle.
Does volume received in websocket streaming also considers pre-open-market volume? Do we need to subtract that from volume received throughout the day to get accurate minute candle data? Or are there any other points to consider while calculating volume for each minute candle?
++ @rakeshr @sujith Maybe you can also advise here?
You need to also note that in the post market session, there will be few settlements happening at the last LTP.
When I checked volume for AARTIIND at 09:15:59 it was 16412 from websocket streaming but 10078 on chart for first one minute candle. I'm trying to understand why this difference.
Please suggest a way to check volume data for every minute.
@sujith @rakeshr @MAG
*** Even if that's not correct I kindly request to suggest a way/steps to perform to check volume data for every minute.
@sujith @rakeshr @MAG
Lets call this number N. We don't know the value of N. But
1. It's definitely more than one (unless its an illiquid instrument).
2. And when we say more than one we don't know if its 2 or 200.
It's plain commonsense logic. If you are getting only one or two ticks out of N ticks per second, how can anyone expect the volume to match exactly? Irrespective of whether one is looking at a one minute interval, five minute interval or any other interval.