I have a python script which reads the websocket and stores the data. Once in a month, i get following error and it doesn't point to a specific line in the script. I belive it has something to do with the websocket read time out. I am running the script on google cloud. Error log is as follows:
Traceback (most recent call last): File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.7/http/client.py", line 1352, in getresponse response.begin() File "/usr/lib/python3.7/http/client.py", line 310, in begin version, status, reason = self._read_status() File "/usr/lib/python3.7/http/client.py", line 271, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.7/socket.py", line 589, in readinto return self._sock.recv_into(b) File "/usr/lib/python3.7/ssl.py", line 1052, in recv_into return self.read(nbytes, buffer) File "/usr/lib/python3.7/ssl.py", line 911, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 756, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/packages/six.py", line 770, in reraise raise value File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 706, in urlopen chunked=chunked, File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 447, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 337, in _raise_timeout self, url, "Read timed out. (read timeout=%s)" % timeout_value urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.kite.trade', port=443): Read timed out. (read timeout=7)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "data_collection.py", line 520, in main() File "data_collection.py", line 485, in main print(INITIAL_BANKNIFTY_LTP()) File "data_collection.py", line 105, in INITIAL_BANKNIFTY_LTP NIFTY_BANK_LTP = u.ltp(get_instrument_by_symbol('NSE', 'NIFTY BANK'))[str(get_instrument_by_symbol('NSE', 'NIFTY BANK'))]['last_price'] File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/kiteconnect/connect.py", line 600, in ltp return self._get("market.quote.ltp", params={"i": ins}) File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/kiteconnect/connect.py", line 836, in _get return self._request(route, "GET", url_args=url_args, params=params, is_json=is_json) File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/kiteconnect/connect.py", line 891, in _request raise e File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/kiteconnect/connect.py", line 888, in _request proxies=self.proxies) File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/home/mukesh_algo_buxar/.local/lib/python3.7/site-packages/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.kite.trade', port=443): Read timed out. (read timeout=7)
Please let me know what is the issue? Do i need to update any python module?
Read time out is not from the WebSocket but from the request module(which is used in API calls i.e for here Quote API call). You can handle these random time out at your end. Something like:
from requests.exceptions import ReadTimeout except ReadTimeout: # Try the same request again
@aditya_chauhan, The instruments dump is a huge file. You need to fetch it only once in a day and save it at your end. We wouldn't suggest pulling it for every run of the script also.
I have put 2 sec delay between two u.instruments() API calls. I am not seeing the issue any longer.
These instrument dump calls are very heavy, so you should call them only once at your end and store the data at your end, for use throughout the day. You should avoid calling it multiple times a day.
If anyone facing "socket.timeout: The read operation timed out" issue, you may try giving a time value in seconds while creating the kite object like : kite = KiteConnect(api_key=ZERODHA_API_KEY, timeout=3600) #Default value is 7 seconds
You can handle these random time out at your end. Something like:
Another issue I am facing is while loading the instrument list. I am getting the following error.
Unknown Content-Type (text/html) with response: (b'\r\n502 Bad Gateway\r\n\r\n502 Bad Gateway\r\n\r\n\r\n')
I am using the following python routine to fetch the instrument lists and store it in a local file.
def get_master_contract():
global s, u, profile, initial_master_contract,global_instrument_var
try:
global_instrument_var = u.instruments('NFO')
except Exception as caught_exception:
print1("While loading master contract (NFO),occured exception is:",caught_exception)
try:
global_instrument_var+=u.instruments('NSE')
except Exception as caught_exception1:
print1("While loading master contract (NSE),occured exception is:", caught_exception1)
csv_file = "master_contract.csv"
try:
with open(csv_file, 'w') as csvfile:
writer = csv.DictWriter(csvfile,fieldnames=['name', 'tick_size', 'instrument_token', 'instrument_type', 'tradingsymbol', 'last_price', 'exchange_token', 'strike', 'segment', 'exchange', 'lot_size', 'expiry'])
writer.writeheader()
for data in global_instrument_var:
writer.writerow(data)
except IOError:
print("I/O error while writing master_contract")
return global_instrument_var
Please guide me how to fix this. This doesn't happen everyday but frequently.
The instruments dump is a huge file. You need to fetch it only once in a day and save it at your end. We wouldn't suggest pulling it for every run of the script also.
You should avoid calling it multiple times a day.
kite = KiteConnect(api_key=ZERODHA_API_KEY, timeout=3600) #Default value is 7 seconds