I have been testing this algo for few days now with same API Key. Today morning also I was able to login and start receiving ticks on WebSocket. But when the algo tried to place an order I suddenly got the following error. However the WebSocket kept receiving new ticks, so I am assuming my seesion was still valid. Any idea what was the issue?
On a restart with same API Key it worked the second time without any problem.
Following is the log. As you can notice WebSocket is still connected and it kept receiving new ticks for NIFTY and Bank Nifty while 3 attempts were made to execute order. Sorry I haven't got detailed traceback as my logging was not set in debug, but any insight into what was the issue would be appreciated as even though a restart worked this time I am not sure why the problem occurred and if I need to change anything to make sure it doesn't happen in the main run for my algo.
root - INFO - 2019-11-29 09:34:59 - NIFTY: 12111.95. Not in Trade. root - INFO - 2019-11-29 09:35:00 - BANKNIFTY: 32022.00. Not in Trade. root - INFO - Trade Entry for BANKNIFTY with 1 lots root - INFO - Entering Position root - INFO - Order placement failed: Incorrect `api_key` or `access_token`. root - ERROR - Failed to enter delta position for BANKNIFTY root - INFO - 2019-11-29 09:35:00 - NIFTY: 12111.70. Not in Trade. root - INFO - 2019-11-29 09:35:01 - BANKNIFTY: 32020.30. Not in Trade. root - INFO - Trade Entry for BANKNIFTY with 1 lots root - INFO - Entering Position root - INFO - Order placement failed: Incorrect `api_key` or `access_token`. root - ERROR - Failed to enter delta position for BANKNIFTY root - INFO - 2019-11-29 09:35:01 - NIFTY: 12110.60. Not in Trade. root - INFO - 2019-11-29 09:35:02 - BANKNIFTY: 32014.25. Not in Trade. root - INFO - Trade Entry for BANKNIFTY with 1 lots root - INFO - Entering Position root - INFO - Order placement failed: Incorrect `api_key` or `access_token`. root - ERROR - Failed to enter delta position for BANKNIFTY
Are you sure you are sending the right auth header? I would suggest running in debug mode to get the complete stack trace with the response code including order placement param.
Yeah Sujith, I'd try to recreate the issue with debug mode. However is it possible that WebSocket remains connected and keeps getting new ticks if API Key or access token is incorrect?
I ran the code with debug and this is what I get. No Error in code was thrown. Following is the execution flow: 1) on_tick called couple of functions when certain conditions were meet. 2) These 2 function calls kept the control for about 5-6 seconds while they did some calculations and executed some orders etc. 3) After that the flow came back to on_tick 4) Now as opposed to expectation where I should have got the next tick I simply got the connection error
As you can see after all the process and order execution and just before returning the control back to on_tick it did a normal orders fetch from kite and those orders were fetched with correct values. So it was connected at that point of time
And then the next moment when the control is back to on_tick for receiving the next tick it throws connection error.
Is it possible that only Kite Ticker got disconnected? Is the gap of 5-6 secs in consecutive calls between on_tick due to my code holding the control for processing in some way causing the issue?
Another thing I'd like to add to that. I have been running this code for few weeks now with just one instrument and it worked without any problem. In that case it used to be one function call from on_tick and the execution used to give the control back to on_tick in 2-3 sec max..... Now am trying to test it with multiple instrument and the moment I add a second instrument (Bank Nifty in this case) it causes this issue. Only difference I can think between the two runs is the longer processing time due to second function call for second instrument and the related longer delay in control coming back to on_tick
Only difference I can think between the two runs is the longer processing time due to second function call for second instrument and the related longer delay in control coming back to on_tick
Yeah, you are correct. Longer processing time inside on_tick is blocking on_tick callback, which is resulting in WebSocket dis-connection. You need to pass on the tick to another thread from on_tick method using multi-threading.Have a look to multi-threading example and explanation here.
@rakeshr Thanks for that info! However one doubt. In non-Threaded implementation kws.connect() used to be the last statement that kept the flow in infinite loop till I called ws.stop() at the end of the day.
However now my kws.connect() with threaded true goes into a separate thread so how do I keep the infinite loop running instead of the whole script ending right after kws.connect?
ok I did that with a While True: at the end. That shouldn't cause any problems right? Given that I don't mind missing a few ticks data while my user function flow ends and it comes back to on_tick.
Another option I was considering, what if instead of having the ticker in threaded I call my functions in separate threads? After that I can either wait for those threads to join in which case my execution of multiple instruments would be parallel instead of sequential. But that'd still be 2-3 sec like my run with single instrument. Or the second option would be to simply not wait back for function threads to join back and instead let on_tick just flow without any wait. Do you think this second approach would be a good choice?
Or the second option would be to simply not wait back for function threads to join back and instead let on_tick just flow without any wait.
Yep, the main motto of separate thread is to, pass on the computation to another thread from on_tick callback. So, you should let on_tick method callback flow without any wait.
In the end I decided to go for executing the function calls from on_tick in separate threads without waiting for these threads to join back. To make sure that I maintain sequence of flow by not calling the function while previous call to the function is not finished I put in a in_process flag inside my instance of class whose function I am calling.
So in conclusion I am running my ticker without threaded and each time calling functions in thread from inside on_tick of ticker.
with concurrent.futures.ThreadPoolExecutor(max_workers=len(delta_runners)) as executor: for runner in delta_runners: if not runner.in_process: executor.submit(runner.on_ticks, ws, ticks)
Though I had an off-market run which showed my separate threads placing orders in parallel instead of sequential flow I'd appreciate if you can give your feedback in case you see any potential problem with this implementation as I still need to test this in live market with multiple ticks received while processing takes place.
executor = concurrent.futures.ThreadPoolExecutor(max_workers=len(delta_runners)) for runner in delta_runners: if not runner.in_process: executor.submit(runner.on_ticks, ws, ticks)
this is what finally worked. Have removed the 'with' statement for executors as even the wait to cleanup the threadpool was causing connection to break. Instead am just leaving the threads to end when their function call is done. The main flow just creates the threadpool calls function based on in_process flag and moves ahead to the next tick.
I am trying to run an algo based on ticker data after converting it into OHLC and making kws.connect(threaded=True) ; I have followed all the instructions mentioned in this thread by @rakeshr , all the computations are working correctly but the main problem that I'm facing is the order execution.
Hi, It seems that I made one minuscule error in the login flow, In my earlier code it was not accepting access token as I placed it in a function (probably) Thank you so much for your support.
I would suggest running in debug mode to get the complete stack trace with the response code including order placement param.
However is it possible that WebSocket remains connected and keeps getting new ticks if API Key or access token is incorrect?
Following is the execution flow:
1) on_tick called couple of functions when certain conditions were meet.
2) These 2 function calls kept the control for about 5-6 seconds while they did some calculations and executed some orders etc.
3) After that the flow came back to on_tick
4) Now as opposed to expectation where I should have got the next tick I simply got the connection error
root - INFO - OrderBook: Reload - Fetching orders from kite
urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.kite.trade:443
urllib3.connectionpool - DEBUG - https://api.kite.trade:443 "GET /orders HTTP/1.1" 200 None
root - DEBUG - : {'id': '191202000523978', 'timestamp': datetime.datetime(2019, 12, 2, 9, 35), 'transaction_type': 'SELL', 'product': 'MIS', 'instrument': 'NIFTY19D0512100CE', 'quantity': 75, 'avg_price': 49.5, 'status': 'COMPLETE', 'status_message': None}
root - DEBUG - : {'id': '191202000524442', 'timestamp': datetime.datetime(2019, 12, 2, 9, 35, 2), 'transaction_type': 'BUY', 'product': 'MIS', 'instrument': 'NIFTY19D0512100CE', 'quantity': 75, 'avg_price': 0, 'status': 'TRIGGER PENDING', 'status_message': None}
root - DEBUG - : {'id': '191202000524737', 'timestamp': datetime.datetime(2019, 12, 2, 9, 35, 3), 'transaction_type': 'SELL', 'product': 'MIS', 'instrument': 'BANKNIFTY19D0531900CE', 'quantity': 20, 'avg_price': 210.3, 'status': 'COMPLETE', 'status_message': None}
root - DEBUG - : {'id': '191202000525183', 'timestamp': datetime.datetime(2019, 12, 2, 9, 35, 5), 'transaction_type': 'BUY', 'product': 'MIS', 'instrument': 'BANKNIFTY19D0531900CE', 'quantity': 20, 'avg_price': 0, 'status': 'TRIGGER PENDING', 'status_message': None}
kiteconnect.ticker - ERROR - Connection error: 1006 - connection was closed uncleanly (None)
kiteconnect.ticker - ERROR - Connection closed: 1006 - connection was closed uncleanly (None)
As you can see after all the process and order execution and just before returning the control back to on_tick it did a normal orders fetch from kite and those orders were fetched with correct values. So it was connected at that point of time
And then the next moment when the control is back to on_tick for receiving the next tick it throws connection error.
Is it possible that only Kite Ticker got disconnected?
Is the gap of 5-6 secs in consecutive calls between on_tick due to my code holding the control for processing in some way causing the issue?
I have been running this code for few weeks now with just one instrument and it worked without any problem. In that case it used to be one function call from on_tick and the execution used to give the control back to on_tick in 2-3 sec max..... Now am trying to test it with multiple instrument and the moment I add a second instrument (Bank Nifty in this case) it causes this issue.
Only difference I can think between the two runs is the longer processing time due to second function call for second instrument and the related longer delay in control coming back to on_tick
You need to pass on the tick to another thread from on_tick method using multi-threading.Have a look to multi-threading example and explanation here.
However now my kws.connect() with threaded true goes into a separate thread so how do I keep the infinite loop running instead of the whole script ending right after kws.connect?
Another option I was considering, what if instead of having the ticker in threaded I call my functions in separate threads? After that I can either wait for those threads to join in which case my execution of multiple instruments would be parallel instead of sequential. But that'd still be 2-3 sec like my run with single instrument. Or the second option would be to simply not wait back for function threads to join back and instead let on_tick just flow without any wait. Do you think this second approach would be a good choice?
So in conclusion I am running my ticker without threaded and each time calling functions in thread from inside on_tick of ticker.
with concurrent.futures.ThreadPoolExecutor(max_workers=len(delta_runners)) as executor:
for runner in delta_runners:
if not runner.in_process:
executor.submit(runner.on_ticks, ws, ticks)
Though I had an off-market run which showed my separate threads placing orders in parallel instead of sequential flow I'd appreciate if you can give your feedback in case you see any potential problem with this implementation as I still need to test this in live market with multiple ticks received while processing takes place.
for runner in delta_runners:
if not runner.in_process:
executor.submit(runner.on_ticks, ws, ticks)
this is what finally worked. Have removed the 'with' statement for executors as even the wait to cleanup the threadpool was causing connection to break. Instead am just leaving the threads to end when their function call is done. The main flow just creates the threadpool calls function based on in_process flag and moves ahead to the next tick.
; I have followed all the instructions mentioned in this thread by @rakeshr , all the computations are working correctly but the main problem that I'm facing is the order execution.
I am getting this error:
DEBUG:urllib3.connectionpool:https://api.kite.trade:443 "POST /orders/regular HTTP/1.1" 400 109
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.kite.trade:443
DEBUG:urllib3.connectionpool:https://api.kite.trade:443 "POST /orders/regular HTTP/1.1" 400 109
exception occured:Invalid `api_key` or `access_token`.
Go through Orders FAQs here.
Attaching the code below for reference-
order_id = kite.place_order(variety=kite.VARIETY_REGULAR,
exchange=kite.EXCHANGE_NSE, tradingsymbol="IRCTC",
transaction_type=kite.TRANSACTION_TYPE_BUY,
quantity=1, validity=kite.VALIDITY_DAY,
product=kite.PRODUCT_MIS,
order_type=kite.ORDER_TYPE_MARKET)
Your order params look fine. Are you able to make other API calls?
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.kite.trade
DEBUG:urllib3.connectionpool:https://api.kite.trade:443 "POST /session/token HTTP/1.1" 200 None
[Access token printed]
No, I'm not able to make other API calls, I tried to get orders, instruments, holdings, but it is returned with -
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.kite.trade:443
DEBUG:urllib3.connectionpool:https://api.kite.trade:443 "GET /orders HTTP/1.1" 400 109
kiteconnect.exceptions.InputException: Invalid `api_key` or `access_token`.
And I'm able to get tick by tick data.
It seems that I made one minuscule error in the login flow, In my earlier code it was not accepting access token as I placed it in a function (probably)
Thank you so much for your support.