Missing ticks in websockets. Mismatch with live charts in kite.

abhishek_
Hey,
So I've been running the python client's websockts with 100 around stock ids. I am printing the ticks in logger and saving it in mysql batching it so it runs 1 insert every second. I found that the high is mismatching with a lot of stocks along with open with the graph candles in kite. I am not getting the data in my log file or mysql ticks. Now my question is, Is it because of my code taking too long to miss the next tick (which is unlikely since as I've noticed I was getting a connection closed uncleanly when I was running a lot of operations.) I am running it now with 10 commodities to provide better understanding of the issue I'm facing will post updates. I personally do not think it's the kite api not working, although I would appreciate any help regarding how I should write code so that I don't miss data of 100+ stocks.
  • sujith
    Are you using multithreading? You need to make sure you haven't blocked the main thread.
  • abhishek_
    hey

    @sujith thanks for replying

    async await can be used instead of threads? Also I timed my entire on_ticks function. The output screenshot is pasted above. It's taking only 8-10 ms time for the method. Here's the code. I don't know if writing from scratch is going to help. I was only running 6 commodities still missed data in 2 laptop same code at the same time which I don't think should be our code's fault. Also my laptop's a macbook pro 15".

    import traceback
    import time

    from kiteconnect import KiteTicker
    from process_ticks_csv import ProcessTicksCSV

    STOCKS_ID_LIST = []
    ptc = None
    out = []
    prevSecs = 0
    log = None


    class KiteSocket:
    def __init__(self, credentials, st_list, store_ticks, data, log_p):
    try:
    global STOCKS_ID_LIST
    global ptc
    global log
    log = log_p
    self.kws = KiteTicker(credentials['api_key'], credentials['access_token']['token'])
    STOCKS_ID_LIST = st_list
    ptc = ProcessTicksCSV(store_ticks, data['ohlc'], data['loggers'], data['csv_file_names'],
    data['algorithms'], data['sql'], log)
    log.info('Success to initialize store_ticks_csv:')
    except Exception as ex:
    traceback.print_exc()
    log.error('Failed to initialize store_ticks_csv: %s', str(ex))
    log.exception(ex)

    def connect(self):
    """
    Connect to KiteTicker
    """
    try:
    self.kws.on_ticks = on_ticks
    self.kws.on_connect = on_connect
    self.kws.on_close = on_close

    print("connecting...")
    self.kws.connect()
    print("connection on!!")
    except Exception as ex:
    traceback.print_exc()
    log.exception(ex)


    def on_ticks(web_soc, ticks):
    try:
    global log
    global out
    global prevSecs
    global ptc
    millis = int(round(time.time() * 1000))
    secs = millis // 1000
    for tick in ticks:
    out.append(tick)

    if prevSecs != secs:
    # print(prevSecs, secs, len(out))
    prevSecs = secs
    ptc.process(out)
    millis2 = int(round(time.time() * 1000))
    log.info(f'time to process everything {millis2 - millis} {len(out)}')
    out = []

    except Exception as ex:
    log.exception(ex)
    traceback.print_exc()
    log.error('Failed in on_ticks callback zerodha_web_socket: '+ str(ex))


    def on_connect(web_soc, response):
    try:
    global STOCKS_ID_LIST
    web_soc.subscribe(STOCKS_ID_LIST)
    web_soc.set_mode(web_soc.MODE_FULL, STOCKS_ID_LIST)
    except Exception as ex:
    log.exception(ex)
    traceback.print_exc()
    log.error('Failed in on_connect callback zerodha_web_socket: '+ str(ex))


    def on_close(web_soc, code, reason):
    try:
    if web_soc.is_connected():
    web_soc.stop()
    except Exception as ex:
    log.exception(ex)
    traceback.print_exc()
    log.error('Failed in on_close callback zerodha_web_socket: '+ str(ex))
  • sujith
    It is not a coding fault. When the thread is busy doing computation or storing to the database then it is quite possible that you may miss few ticks.
    And also take a look at this thread to know why charts of two platforms don't match.
  • sujith
    The values will never match but it will be near.
Sign In or Register to comment.