Kite 3 migration issues : connection was closed uncleanly (None)

joe

Hi,
I am migrating my existing code to kite3.

I am trying to fetch the quotes in an interval of 5 seconds. Providing the code snippet below

from kiteconnect import KiteTicker
import logging
import pprint
import datetime
import sys
import time
import traceback

api_key = API_KEY
access_token = ACCESS_TOKEN

def on_ticks(kws,ticks ) : 
    print(ticks)
    print('waiting for 5 sec')
    time.sleep(5)

def on_connect( kws , response) :
    print("inside on connect")
    subscriptionlist = [907777]

    print( subscriptionlist )
    kws.subscribe( subscriptionlist )
    kws.set_mode(kws.MODE_FULL, subscriptionlist)
    

def main():
    try:
        print('calling main function')

        kws = KiteTicker( api_key , access_token )
        print('Established websocket connection')


        kws.on_ticks = on_ticks
        kws.on_connect = on_connect

        print('calling KWS.connect')
        response = kws.connect( threaded=False, disable_ssl_verification=True, proxy=None)
        print('finished kws connect process')
        
    except:
        exc_type, exc_value, exc_traceback = sys.exc_info()
        lines = traceback.format_exception(exc_type, exc_value, exc_traceback)
        msg = ''.join('!! ' + line for line in lines)
        print (msg)
    finally:
        print('closing connections')
        kws.close()

if __name__ == "__main__":
    main()

Output:

calling main function
Established websocket connection
calling KWS.connect
inside on connect
[907777]
[{'oi_day_low': 0, 'last_price': 6432.6, 'depth': {'buy': [{'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}], 'sell': [{'quantity': 115, 'price': 6432.6, 'orders': 6}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}]}, 'last_quantity': 2, 'oi': 0, 'oi_day_high': 0, 'change': -0.8401288711442709, 'sell_quantity': 115, 'volume': 4082, 'ohlc': {'open': 6480.0, 'close': 6487.1, 'high': 6549.95, 'low': 6395.05}, 'timestamp': datetime.datetime(2018, 4, 17, 15, 47, 44), 'mode': 'full', 'tradable': True, 'buy_quantity': 0, 'instrument_token': 907777, 'last_trade_time': datetime.datetime(2018, 4, 17, 15, 29, 53), 'average_price': 6464.58}]
waiting for 5 sec
Connection error: 1006 - connection was closed uncleanly (None)
Connection closed: 1006 - connection was closed uncleanly (None)
inside on connect
[907777]

I am getting the quotes, but after each 5 seconds I am getting the same warning. Is it because I am not stopping the web socket connection correctly?

Also when I stop the script manually. I need to restart the kernel each time otherwise I am getting the below issue.

twisted.internet.error.ReactorNotRestartable

ERROR:kiteconnect.ticker:Connection error: 1006 - connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
ERROR:kiteconnect.ticker:Connection closed: 1006 - connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)

Can anyone help me to solve this?

Regards
Joe

Vivek

I wouldn't suggest you to wrap the logic with try except block because once you do kws.connect( threaded=False, disable_ssl_verification=True, proxy=None) current thread is blocked and there won't be any exception raised even if it's an error in your callback implementations. Instead it will be sent in on_error with the message.

Also when I stop the script manually. I need to restart the kernel each time otherwise I am getting the below issue.
twisted.internet.error.ReactorNotRestartable

You need to use

ws.stop()

before restarting.

sujith

You need to call ws.stop() inside on_close callback to actually close the thread. You can check out the example here.

joe

Hi Vivek/Sujith,

Thank you for your comments.
I have modified the code with ws.stop and removed the try - except block.

from kiteconnect import KiteTicker
import logging
import pprint
import datetime
import sys
import time
import traceback

api_key = "API KEY"
access_token = "ACCESS TOKEN"

def on_ticks(ws,ticks ) : 
    print(ticks)
    print('waiting for 5 sec')
    time.sleep(5)

def on_connect( ws , response) :
    print("inside on connect")
    subscriptionlist = [907777]

    print( subscriptionlist )
    ws.subscribe( subscriptionlist )
    ws.set_mode(ws.MODE_FULL, subscriptionlist)

def on_close(ws, code, reason):
    # On connection close stop the main loop
    print('inside the on close')
    print(reason)
    ws.stop()

def on_error(ws, code, reason):
    print('inside the error block')
    print(reason)

def main():
        print('calling main function')

        kws = KiteTicker( api_key , access_token )
        print('Established websocket connection')


        kws.on_ticks = on_ticks
        kws.on_connect = on_connect
        kws.on_close = on_close
        kws.on_error = on_error

        print('calling KWS.connect')
        kws.connect( threaded=False, disable_ssl_verification=True, proxy=None)
        print('finished kws connect process')


if __name__ == "__main__":
    main()

But I am still getting the same issues. Now i am getting the response only once and the connections is getting called with the same error message.

calling main function
Established websocket connection
calling KWS.connect
inside on connect
[907777]
[{'ohlc': {'low': 6417.65, 'high': 6595.0, 'open': 6417.65, 'close': 6432.6}, 'buy_quantity': 12865, 'oi_day_low': 0, 'last_quantity': 1, 'mode': 'full', 'last_price': 6501.6, 'oi': 0, 'sell_quantity': 6093, 'tradable': True, 'instrument_token': 907777, 'last_trade_time': datetime.datetime(2018, 4, 18, 14, 45, 38), 'average_price': 6547.96, 'volume': 6881, 'depth': {'sell': [{'orders': 2, 'price': 6501.6, 'quantity': 2}, {'orders': 1, 'price': 6506.6, 'quantity': 1}, {'orders': 1, 'price': 6515.95, 'quantity': 1}, {'orders': 1, 'price': 6516.0, 'quantity': 2}, {'orders': 1, 'price': 6520.0, 'quantity': 5}], 'buy': [{'orders': 1, 'price': 6501.55, 'quantity': 2}, {'orders': 1, 'price': 6500.05, 'quantity': 1}, {'orders': 1, 'price': 6500.0, 'quantity': 3}, {'orders': 1, 'price': 6495.05, 'quantity': 4}, {'orders': 2, 'price': 6495.0, 'quantity': 11}]}, 'timestamp': datetime.datetime(2018, 4, 18, 14, 45, 41), 'oi_day_high': 0, 'change': 1.0726611323570563}]
waiting for 5 sec

"Connection error: 1006 - connection was closed uncleanly (None)
Connection closed: 1006 - connection was closed uncleanly (None)"

inside the error block
connection was closed uncleanly (None)
inside the on close
connection was closed uncleanly (None)
finished kws connect process

And while restarting getting the error:

ReactorNotRestartable:

zartimus

@joe
There is a ping loop going on here.
And you are disrupting it with sleeping 5 seconds inside on_ticks which is wrong. Remove sleep and it will work. I m seeing many forum threads which opposes the way websocket client is supposed to work. If you enable the debug mode on kiteticker, you can pin point what is going wrong and where.

joe

Hi zartimus,
Then how can I add an interval of 5 seconds ?.
I am looking for a controlled stream in a custom interval

zartimus

@joe Well for starters, you can initialise a start_time variable in the first tick and end_time for every ticks, calculate the difference in the end_time and start_time is greater than 5 and proceed with processing the ticks. Using signals inside event loop would be the desirable way.

Note that `ws.stop()` should only be used inside `on_close` callback when you are sure to quit the ticker instead of the default ticker reconnection mechanism.

joe

I removed the sleep part as you suggested.
output:
calling main function
Established websocket connection
calling KWS.connect
inside on connect
[907777]
[{'volume': 7697, 'last_quantity': 1, 'oi_day_low': 0, 'depth': {'buy': [{'quantity': 1, 'price': 6505.05, 'orders': 1}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}], 'sell': [{'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}]}, 'tradable': True, 'oi': 0, 'instrument_token': 907777, 'ohlc': {'low': 6417.65, 'open': 6417.65, 'high': 6595.0, 'close': 6432.6}, 'sell_quantity': 0, 'oi_day_high': 0, 'buy_quantity': 1, 'average_price': 6543.31, 'mode': 'full', 'change': 1.126294188974906, 'timestamp': datetime.datetime(2018, 4, 18, 16, 15, 25), 'last_price': 6505.05, 'last_trade_time': datetime.datetime(2018, 4, 18, 15, 29, 58)}]
waiting for 5 sec

Now Nothing is coming after this. The script is still running but nothing is getting printed.

And about processing the ticks with time difference, once we start the streaming there will be a continuous output. In that case do we have to stop and reconnect each time?

zartimus

@joe Please post your code

Here is a sample

import time

start_time = None
first_tick_flag = True

def on_ticks(ws, ticks):
    global first_tick_flag , start_time
    if first_tick_flag:
        first_tick_flag = False
        start_time = time.time()
    end_time = time.time()
    # Somehow i m only interested in 5 second intervals ticks
    if int(end_time - start_time) >= 5:
        start_time = end_time
        print "Ticks: %s" % ticks
        # Process ticks from 5 seconds interval

joe

Here is the code

from kiteconnect import KiteTicker
import logging
import pprint
import datetime
import sys
import time
import traceback

api_key = API_KEY
access_token = ACCESS_TOKEN

def on_ticks(ws,ticks ) :
print('inside on ticks')
print(ticks)
print('waiting for 5 sec')

def on_connect( ws , response) :
print("inside on connect")
subscriptionlist = [907777]

print( subscriptionlist )
ws.subscribe( subscriptionlist )
ws.set_mode(ws.MODE_FULL, subscriptionlist)

def on_close(ws, code, reason):
# On connection close stop the main loop
print('inside the on close')
print(reason)
ws.stop()

def on_error(ws, code, reason):
print('inside the error block')
print(reason)

def main():
print('calling main function')

kws = KiteTicker( api_key , access_token, reconnect=True, debug = True )
print('Established websocket connection')


kws.on_ticks = on_ticks
kws.on_connect = on_connect
kws.on_close = on_close
kws.on_error = on_error

print('calling KWS.connect')
kws.connect( threaded=False, disable_ssl_verification=True, proxy=None)
print('finished kws connect process')


if __name__ == "__main__":
main()

output:
calling main function
Established websocket connection
calling KWS.connect
2018-04-18 20:33:07+0530 [-] Log opened.
2018-04-18 20:33:07+0530 [-] inside on connect
2018-04-18 20:33:07+0530 [-] [907777]
2018-04-18 20:33:08+0530 [-] inside on ticks
2018-04-18 20:33:08+0530 [-] [{'buy_quantity': 1, 'mode': 'full', 'volume': 7697, 'change': 1.126294188974906, 'oi_day_high': 0, 'last_price': 6505.05, 'instrument_token': 907777, 'depth': {'buy': [{'quantity': 1, 'price': 6505.05, 'orders': 1}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}], 'sell': [{'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}, {'quantity': 0, 'price': 0.0, 'orders': 0}]}, 'timestamp': datetime.datetime(2018, 4, 18, 16, 15, 25), 'oi_day_low': 0, 'average_price': 6543.31, 'oi': 0, 'sell_quantity': 0, 'last_quantity': 1, 'tradable': True, 'last_trade_time': datetime.datetime(2018, 4, 18, 15, 29, 58), 'ohlc': {'low': 6417.65, 'close': 6432.6, 'high': 6595.0, 'open': 6417.65}}]
2018-04-18 20:33:08+0530 [-] waiting for 5 sec

nothing is coming after this

zartimus

@joe 907777 -> TTKPRESTIG
Can you see any tick on kite app? Please check with instruments that gives you ticks Try 53556743

Dileep

hi..@zartimus while upgrade to kite 3.0 i am getting the error as below
how to resolve it?

zartimus

@Dileep This is fine, we are ignoring bdist_wheel. Check the last line. It says successfully installed kiteconnect.

joe

Hi Zartimus,
Yes you are right. I changed the token and now I am getting the continuous feed.

But whenever I am stopping the script and trying to restart , then I am getting the following error.

"ReactorNotRestartable"

I am running the code from jupyter. If I restart the kernel, then it's working.

I have used, ws.stop in on_close and on_error as per the previous comments. But still the same issue.

joe

Also getting the below error whenever I try to stop the script by adding ws.stop() or kws.close() inside on_tick

Connection error: 1006 - connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
Connection closed: 1006 - connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)

what is the difference between ws.stop and kws.close()

zartimus

@joe

1. Please double check this
   
2. ReactorNotRestartable is self explanatory. You simply cannot restart the ticker in the same process.
   
3. Explanation to `stop` and `close` meths are defined in the docstrings here