socket.timeout: The read operation timed out

aditya_chauhan
I have a python script which reads the websocket and stores the data. Once in a month, i get following error and it doesn't point to a specific line in the script. I belive it has something to do with the websocket read time out. I am running the script on google cloud. Error log is as follows:

Traceback (most recent call last):
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "", line 3, in raise_from
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.7/http/client.py", line 1352, in getresponse
    response.begin()
  File "/usr/lib/python3.7/http/client.py", line 310, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.7/http/client.py", line 271, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.7/ssl.py", line 1052, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.7/ssl.py", line 911, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 337, in _raise_timeout
    self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.kite.trade', port=443): Read timed out. (read timeout=7)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "data_collection.py", line 520, in
    main()
  File "data_collection.py", line 485, in main
    print(INITIAL_BANKNIFTY_LTP())
  File "data_collection.py", line 105, in INITIAL_BANKNIFTY_LTP
    NIFTY_BANK_LTP = u.ltp(get_instrument_by_symbol('NSE', 'NIFTY BANK'))[str(get_instrument_by_symbol('NSE', 'NIFTY BANK'))]['last_price']
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/kiteconnect/connect.py", line 600, in ltp
    return self._get("market.quote.ltp", params={"i": ins})
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/kiteconnect/connect.py", line 836, in _get
    return self._request(route, "GET", url_args=url_args, params=params, is_json=is_json)
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/kiteconnect/connect.py", line 891, in _request
    raise e
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/kiteconnect/connect.py", line 888, in _request
    proxies=self.proxies)
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.kite.trade', port=443): Read timed out. (read timeout=7)

Please let me know what is the issue? Do i need to update any python module?
Tagged:
  • rakeshr
    I believe it has something to do with the WebSocket read time out.
    requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.kite.trade', port=443): Read timed out. (read timeout=7)
    Read time out is not from the WebSocket but from the request module(which is used in API calls i.e for here Quote API call).
    You can handle these random time out at your end. Something like:
    from requests.exceptions import ReadTimeout
    except ReadTimeout:
    # Try the same request again
  • aditya_chauhan
    Thanks for the help. I will try and let you know if issue persists.

    Another issue I am facing is while loading the instrument list. I am getting the following error.


    Unknown Content-Type (text/html) with response: (b'\r\n502 Bad Gateway\r\n\r\n502 Bad Gateway\r\n\r\n\r\n')


    I am using the following python routine to fetch the instrument lists and store it in a local file.

    def get_master_contract():
    global s, u, profile, initial_master_contract,global_instrument_var
    try:
    global_instrument_var = u.instruments('NFO')
    except Exception as caught_exception:
    print1("While loading master contract (NFO),occured exception is:",caught_exception)
    try:
    global_instrument_var+=u.instruments('NSE')
    except Exception as caught_exception1:
    print1("While loading master contract (NSE),occured exception is:", caught_exception1)

    csv_file = "master_contract.csv"
    try:
    with open(csv_file, 'w') as csvfile:
    writer = csv.DictWriter(csvfile,fieldnames=['name', 'tick_size', 'instrument_token', 'instrument_type', 'tradingsymbol', 'last_price', 'exchange_token', 'strike', 'segment', 'exchange', 'lot_size', 'expiry'])
    writer.writeheader()
    for data in global_instrument_var:
    writer.writerow(data)
    except IOError:
    print("I/O error while writing master_contract")
    return global_instrument_var

    Please guide me how to fix this. This doesn't happen everyday but frequently.
  • aditya_chauhan
    Above python routine with proper indentation:

  • aditya_chauhan
    I have put 2 sec delay between two u.instruments() API calls. I am not seeing the issue any longer. Please suggest the permanent fix for this issue.
  • sujith
    @aditya_chauhan,
    The instruments dump is a huge file. You need to fetch it only once in a day and save it at your end. We wouldn't suggest pulling it for every run of the script also.
  • rakeshr
    I have put 2 sec delay between two u.instruments() API calls. I am not seeing the issue any longer.
    These instrument dump calls are very heavy, so you should call them only once at your end and store the data at your end, for use throughout the day.
    You should avoid calling it multiple times a day.
  • pvsidharthan
    If anyone facing "socket.timeout: The read operation timed out" issue, you may try giving a time value in seconds while creating the kite object like :
    kite = KiteConnect(api_key=ZERODHA_API_KEY, timeout=3600) #Default value is 7 seconds
  • rakeshr
    kite = KiteConnect(api_key=ZERODHA_API_KEY, timeout=3600)
    Yes, you can increase the configurable timeout parameter as per your use-case.
This discussion has been closed.