Hi I see weird behavior on order placements on start of currency market. Orders placed using kite.order_place at 9 am or slightly after don't return an order_id. I get an exception stating "Gateway timeout" However I see that the very same order shows up in order book in maybe 1-2 minutes and gets executed if the price is reached. This is a big problem for me. So is this a known bug? Or is this a system limitation? In either case do you have any workarounds?
AMO order doesn't do as I take make my strategies based on current market rates. Most of currency action gets done in the first 5 mins and not having ability to track orders at this stage put paid to any strategy used. Hope to get a speedy solution to this. Please keep this ticket open till that happens. Thanks .
Hi, We tried placing orders from 9:00:00 AM to 9:05:00 AM for three to four days. We observed this during the first 30 seconds of 9:00:00 AM. We place tens of thousands of AMOs at 9:00:00 AM and RMS will be busy in handling those ordes which will result in this. Now, OMS team is working on prevalidating orders at 8:48:00 AM which will resolve the issue.
@Sujith Thanks for simulating this at your end. Yes this happens during the first 30 seconds on low volume days and extends beyond 9:01 on high volume days. Look forward to your team pre-validating the orders earlier. This will solve a major (for me) glitch.
Adding on.. Sometimes I have also seen this error around same time. This happens after the order has been placed and order id retrieved and post that I query using kite.orders for that order_id. Couldnt find that `order_id`
Happens for orders placed at 09:00 am. Happened today as well. 180216000015312 .
Situation is like this - I place order using order_place, get order_id and then when I try to retrieve the order to see if it has been executed I get this error - it cant find the order id!
@razcads, The OMS team is working on it. They are testing it out. Maybe in a couple of weeks, we will be prevalidating AMOs at 8:48 AM and that should resolve this issue.
@sujith , if you don't mind can you please add more details of root cause of why a order which was responded with a order id , later gets "can't find the order id" error .
It seems like a bottleneck at core OMS. We have tens of thousands of AMOs being sent at the market opening. But the exchange line has some limit on the number of messages that can be sent wherein each order is a message.
We are thinking of speaking to the exchange about the architecture and infrastructure stuff.
Sorry , that doesn't explain the issue right? The issue - Your system issues a order-id and subsequently when queried says order-id not found. How any external dependency come in to play unless you are doing something systemically wrong.
My understanding of general simplified sequence as i would see will be : 1) Your system generates a order-id 2) Your system records the order with its order-id in your backend database. From this point , even if your system crashes, exchanges has issues , the order-id should not be lost (database is commited) 3) you respond to user with order-id (this should be the guaranteed indication to user that my order request got recorded in your system ) 4) you place order to exchange
Now , how is exchange bottleneck stops you from retreiving the record corresponding to the order-id from your own database ? Let me know if i am getting it wrong.
But , in case you are responding with a order-id before commiting it to your database , OR you respond with order id , place order to exchange and then commit , you are doing something systemically wrong and it is a very dangerous situation w.r.t the integrity of the data . user gets the order-id and assumes order is placed , what if in between of you commiting the record to your database , exchange line crashes or your system crashes ....
Thanks , So receiving a order_id at our end doesn't gurantee that the order got recorded at your system ? BTW, OMS is not currently your own implementation ?
Ideally getting order_id means there is an entry in the orderbook. But at market opening, it is taking time to update orderbook. This is an exception which is being looked into. We use OMS provided by the Thomson Reuters.
Hi Sujith, Thanks , this unfolds a lot more scenerio that i need to ring fence in my system - first of all , though you are seeing it at market opening , this can essentially happen whenever OMS is over-loaded . So from my system design i should expect that it can happen anytime . As you say , the order-id is generated by OMS and not at your end , then one scenerio that is the most important for me to know is following - say , 1) i place a order 2) you request the OMS . 3) OMS places a order and generates a order-id 4) But you dont receive this order-id due to a blip in the network (blip i mean short duration issue like a packet drop)
In this case what would be your response ? How will i eventually get to know this order indeed got placed at the OMS ? Will this order-id be orphaned in the OMS ? How do you suggest should i implement by system to ring fence this scenerio ?
Thanks for your time and patience with my long questions.
One of the scenario wherein order is placed and Kite Trade doesn't get the response from OMS then it will result in request timeout. In another scenario wherein you got order_id but the order doesn't exist in orderbook then you will see the following error
Thanks. Then later when we request orderbook , the orderbook is fetched directly from OMS or your local recorded copy ? If former is the case then i must be assured that the order will show up eventually in the orderbook . Can you please confirm.
okey , so do you guarantee that in the former case that you specified above (order is placed and Kite Trade doesn't get the response from OMS then it will result in request timeout) , the order would "eventually" show up in the orderbook (please do consider the situation where the Kite could miss a PUSH message because of its internal issue) .
Yes, even when there is a request timeout while placing an order. We would recommend double-checking orderbook. The order will show up there. This is the behavior I have seen so far.
@sudhirshettyk I can confirm the behavior that @sujith alludes to. As I see it this error wont go away anytime soon as its to do with a different vendor who most likely wont change his OMS behavior for a border case considering there are only a minor fraction (a small subset of retail algo traders on Zerodha) who complain about it... so not much return on investment for them.
Thanks @sujith@razcads .. i am fine with errors (any async system would) as long as we have a clear understanding of it , i can try and ring-fence it. So in this case as long as it is guranteed that a order doesn't get orphaned in OMS and eventually shows in orderbook i am fine . Coming back to this case , we have a issue if Kite recieves the PUSH notification and uses this to build a local orderbook which is what is queried when we request the orderbook . @Sujith , can you please confirm that this is not the case and you always fetch the orderbook from OMS and dont build your local orderbook based on PUSH notification you recieve .
Yes, it is a PUSH from OMS to Kite Trade but these are interprocess communications that happen internally. It shouldn't matter to end user. The whole flow is very complex to put it in one or two paragraph. Obviously, we won't take one year to do Kite Connect 3 if it is so simple right
To answer your query whether there will be an orphan order in the OMS after you get order id. This is not possible. We are talking to OMS and the exchange to come up with a feasible solution to overcome the line capacity issue. In fact there some major backend things to be done which might require us to get the exchange approvals as well.
We will update this thread once a fix is deployed.
I am facing similar Problem in cash market. I placed a cover order which is giving me an order id. Now, to find out the child order id, I have below python code: def get_cover_order_id( parent_order_id): all_orders = self.kite.orders() df = pd.DataFrame(all_orders) df = df[['order_id','parent_order_id']] orders_dict = df.set_index('parent_order_id')['order_id'].to_dict() o_id = orders_dict[parent_order_id] return o_id
This code is not working for my early few trades (usually in the very first minute 9.15), but it works fine later on throughout the day. It returns "Couldn't find that `order_id`"
Is it similar problem which has been discussed in this thread? If so can you suggest an alternative way of doing this.
Can you share more details? Like for which instrument this is happening and at what time exactly you place the order? We had done some changes to avoid this but it seems like this is not resolved still. I will forward details to the concerned team and get this checked.
Sure, PFA the screenshot of the error from last week.
I placed the order successfully (cover order). Got order id. You can check the timings as well. Passed the order id to the code (pasted above already) to retrieve child order id. It could not provide that. Then I tried to modify that child order then it says "Couldn't find that order id".
@sujith - I am also facing two exceptions (mentioned below) time to time at random time in the MIDDLE of log due to kiteConnect.getOrderHistory(order.orderId) My code handles these exception and continue trading. But I would like to know why these come occasionally at random time ? (1)Couldn't find that `order_id` (2)Invalid `api_key` or `access_token` Details are: Example: Time is mentioned in the below log: 2019-05-22 10:57:09 ERROR BuyStocks:1357 - kite exception in placeSellOrder in Buystocks class -->Couldn't find that `order_id`. 2019-05-22 10:57:10 ERROR BuyStocks:1357 - kite exception in placeSellOrder in Buystocks class -->Couldn't find that `order_id`.
2019-05-22 10:57:07 ERROR BuyStocks:1357 - kite exception in placeSellOrder in Buystocks class -->Invalid `api_key` or `access_token`.
Any updates on this issue? Continues to happen every day.
Our OMS processes tens of thousands of pre-market orders at this time. We are in talks with OMS vendors to come up with a feasible solution.
We tried placing orders from 9:00:00 AM to 9:05:00 AM for three to four days. We observed this during the first 30 seconds of 9:00:00 AM.
We place tens of thousands of AMOs at 9:00:00 AM and RMS will be busy in handling those ordes which will result in this. Now, OMS team is working on prevalidating orders at 8:48:00 AM which will resolve the issue.
Sometimes I have also seen this error around same time. This happens after the order has been placed and order id retrieved and post that I query using kite.orders for that order_id.
Couldnt find that `order_id`Situation is like this - I place order using order_place, get order_id and then when I try to retrieve the order to see if it has been executed I get this error - it cant find the order id!
The OMS team is working on it. They are testing it out. Maybe in a couple of weeks, we will be prevalidating AMOs at 8:48 AM and that should resolve this issue.
Any update on this?
Hopefully, if everything goes well we will take it live in a couple of weeks.
We are thinking of speaking to the exchange about the architecture and infrastructure stuff.
Sorry , that doesn't explain the issue right?
The issue - Your system issues a order-id and subsequently when queried says order-id not found. How any external dependency come in to play unless you are doing something systemically wrong.
My understanding of general simplified sequence as i would see will be :
1) Your system generates a order-id
2) Your system records the order with its order-id in your backend database.
From this point , even if your system crashes, exchanges has issues , the order-id should not be lost (database is commited)
3) you respond to user with order-id (this should be the guaranteed indication to user that my order request got recorded in your system )
4) you place order to exchange
Now , how is exchange bottleneck stops you from retreiving the record corresponding to the order-id from your own database ? Let me know if i am getting it wrong.
But , in case you are responding with a order-id before commiting it to your database , OR you respond with order id , place order to exchange and then commit , you are doing something systemically wrong and it is a very dangerous situation w.r.t the integrity of the data . user gets the order-id and assumes order is placed , what if in between of you commiting the record to your database , exchange line crashes or your system crashes ....
Please help address my concerns.
Regards,
Sudhir Shetty
We send a place order request on behalf of the client at OMS and we get order_id from the OMS.
Even though the order is created, OMS is not returning any data for the first 30 seconds of the market opening.
Thanks , So receiving a order_id at our end doesn't gurantee that the order got recorded at your system ?
BTW, OMS is not currently your own implementation ?
We use OMS provided by the Thomson Reuters.
Thanks , this unfolds a lot more scenerio that i need to ring fence in my system -
first of all , though you are seeing it at market opening , this can essentially happen whenever OMS is over-loaded . So from my system design i should expect that it can happen anytime .
As you say , the order-id is generated by OMS and not at your end , then one scenerio that is the most important for me to know is following -
say ,
1) i place a order
2) you request the OMS .
3) OMS places a order and generates a order-id
4) But you dont receive this order-id due to a blip in the network (blip i mean short duration issue like a packet drop)
In this case what would be your response ? How will i eventually get to know this order indeed got placed at the OMS ? Will this order-id be orphaned in the OMS ? How do you suggest should i implement by system to ring fence this scenerio ?
Thanks for your time and patience with my long questions.
Regards,
Sudhir
In another scenario wherein you got order_id but the order doesn't exist in orderbook then you will see the following error
the order would "eventually" show up in the orderbook (please do consider the situation where the Kite could miss a PUSH message because of its internal issue) .
Coming back to this case , we have a issue if Kite recieves the PUSH notification and uses this to build a local orderbook which is what is queried when we request the orderbook . @Sujith , can you please confirm that this is not the case and you always fetch the orderbook from OMS and dont build your local orderbook based on PUSH notification you recieve .
Best Regards,
Sudhir Shetty
To answer your query whether there will be an orphan order in the OMS after you get order id. This is not possible.
We are talking to OMS and the exchange to come up with a feasible solution to overcome the line capacity issue. In fact there some major backend things to be done which might require us to get the exchange approvals as well.
We will update this thread once a fix is deployed.
I am facing similar Problem in cash market. I placed a cover order which is giving me an order id.
Now, to find out the child order id, I have below python code:
def get_cover_order_id( parent_order_id):
all_orders = self.kite.orders()
df = pd.DataFrame(all_orders)
df = df[['order_id','parent_order_id']]
orders_dict = df.set_index('parent_order_id')['order_id'].to_dict()
o_id = orders_dict[parent_order_id]
return o_id
This code is not working for my early few trades (usually in the very first minute 9.15), but it works fine later on throughout the day. It returns "Couldn't find that `order_id`"
Is it similar problem which has been discussed in this thread?
If so can you suggest an alternative way of doing this.
Thanks
We had done some changes to avoid this but it seems like this is not resolved still. I will forward details to the concerned team and get this checked.
Sure, PFA the screenshot of the error from last week.
I placed the order successfully (cover order). Got order id. You can check the timings as well.
Passed the order id to the code (pasted above already) to retrieve child order id. It could not provide that.
Then I tried to modify that child order then it says "Couldn't find that order id".
Let me know your comments.
Thanks
My code handles these exception and continue trading. But I would like to know why these come occasionally at random time ?
(1)Couldn't find that `order_id` (2)Invalid `api_key` or `access_token`
Details are:
Example: Time is mentioned in the below log:
2019-05-22 10:57:09 ERROR BuyStocks:1357 - kite exception in placeSellOrder in Buystocks class -->Couldn't find that `order_id`.
2019-05-22 10:57:10 ERROR BuyStocks:1357 - kite exception in placeSellOrder in Buystocks class -->Couldn't find that `order_id`.
2019-05-22 10:57:07 ERROR BuyStocks:1357 - kite exception in placeSellOrder in Buystocks class -->Invalid `api_key` or `access_token`.