I would have continued with the earlier thread..but found it closed therefore posting here.. Apparently 5 seconds delay for re connection is not a good solution. I am storing the ltp and 5 seconds gap would create lot of inconvenience in the data...why is there not a better implementation available?
Hi @pinkpanther, It is still a better solution than calling connect inside disconnect. In your solution, you are relying on disconnect. The thing is disconnect is not at all triggered for around 30 seconds or more than that when there is abrupt network drop and that is the common behavior of any websocket.
We have a mechanism wherein we keep checking if tick arrived every second. If there are no ticks for 5 seconds, we attempt reconnection. If you decrease that time interval you may get python's stack overflow exception. You can reduce that number to 1 or 2 but there is a chance of race condition which might result in opening multiple connections. One more scenario you need to think is that you don't run out of maximum reconnection count soon. If it goes beyond maximum number for reconnection attempt, it might never reconnect. You will have to manually re-run your script.
@sujith first of all i know my mechanism to connect after disconnect is shit solution. I only came up as I didn't want to miss any second data... To understand what you said..disconnection is a natural occurrence and there is not way it can be stopped from your server?
@pinkpanther, Websocket disconnection can be happening because of many reasons like abrupt changes in network or drop in internet connection. The same websocket is being used by all our platforms which serve 3 lakh users and we haven't come across this issue. Ideally, websocket connection must be run on a single dedicated thread. I would suggest using a separate thread for all calculations and IO operations like writing to the database.
@sujith I am using java kiteconnect library..but yes the implementation is same with python...and i am using threads to handle...BlockingQueue to be in particular..for the calculations..so no issues there. The only problem being it disconnects multiple time and with 5 seconds of reconnection logic..the data gets inconvinient
@pinkpanther, Just make sure you are not blocking the thread on which ticks are coming. I am sorry we can't modify javakiteconnect. It is an open source project you can clone and make changes and use it.
@sujith@Kailash . Why I doubt the re-connection implementation is because of the following use-case. Suppose I have a 1 core machine, and I am doing a CPU intensive work,lets say I am saving file worth of data every minute and the data might be big enough to take more than 5-6 seconds. And this might altogether be a separate process than the one on which kite is running. I such a scenario the kite websocket process is blocked for 5-6 seconds which is faulty. This might look like a edge case scenario but if I am storing ltp of say 200-300 scrips,then this becomes a valid scenario.
@pinkpanther, Reconnection was implemented because many people are using it to store ticks and whenever there is an abrupt change in network or disconnection, callbacks are not invoked which resulted in losing tick data for a considerable time interval(may be more than 30 seconds or 1 minute+). The basic assumption behind this is users won't block the thread which receives ticks.
You can create worker threads and do your job on that which will prevent blocking main thread which receives tick data. It was a generic solution. If it doesn't suit your requirement, you can clone and change it.
It is still a better solution than calling connect inside disconnect. In your solution, you are relying on disconnect. The thing is disconnect is not at all triggered for around 30 seconds or more than that when there is abrupt network drop and that is the common behavior of any websocket.
We have a mechanism wherein we keep checking if tick arrived every second. If there are no ticks for 5 seconds, we attempt reconnection. If you decrease that time interval you may get python's stack overflow exception.
You can reduce that number to 1 or 2 but there is a chance of race condition which might result in opening multiple connections.
One more scenario you need to think is that you don't run out of maximum reconnection count soon.
If it goes beyond maximum number for reconnection attempt, it might never reconnect. You will have to manually re-run your script.
Websocket disconnection can be happening because of many reasons like abrupt changes in network or drop in internet connection.
The same websocket is being used by all our platforms which serve 3 lakh users and we haven't come across this issue.
Ideally, websocket connection must be run on a single dedicated thread. I would suggest using a separate thread for all calculations and IO operations like writing to the database.
Just make sure you are not blocking the thread on which ticks are coming.
I am sorry we can't modify javakiteconnect. It is an open source project you can clone and make changes and use it.
.
Why I doubt the re-connection implementation is because of the following use-case.
Suppose I have a 1 core machine, and I am doing a CPU intensive work,lets say I am saving file worth of data every minute and the data might be big enough to take more than 5-6 seconds. And this might altogether be a separate process than the one on which kite is running. I such a scenario the kite websocket process is blocked for 5-6 seconds which is faulty. This might look like a edge case scenario but if I am storing ltp of say 200-300 scrips,then this becomes a valid scenario.
Reconnection was implemented because many people are using it to store ticks and whenever there is an abrupt change in network or disconnection, callbacks are not invoked which resulted in losing tick data for a considerable time interval(may be more than 30 seconds or 1 minute+).
The basic assumption behind this is users won't block the thread which receives ticks.
You can create worker threads and do your job on that which will prevent blocking main thread which receives tick data.
It was a generic solution. If it doesn't suit your requirement, you can clone and change it.
https://github.com/rainmattertech/pykiteconnect
Appreciated.