Worst API performance today

vvijay
vvijay edited May 2018 in General
How do you guys think we can run an algo system with such bad infra?
  • Shaha
    Shaha edited May 2018
    Guys please take the stability issue seriously.
  • phantomdrake
    So very true. I had such a big open Position and the API is stuck. Who is going to be liable for the losses. Feels like i am paying for shit.
  • ManishGupta
    @vvijay @Shaha @phantomdrake I am planning to subscribe Zerodha's API for autmatic execution of OHL strategy. How is your experience guys with Zerodha API. Is it not reliable? What if I host it on AWS? Will it help? Your feedback will be highly appreciated.
  • Shaha
    Shaha edited May 2018
    @ManishGupta See the email received from founder of Zerodha today.....

    I have just highlited few statements... If u are smart enough , you will make out the meaning of it.

    "Traders,

    Our clients had severe difficulties logging into Kite web & mobile post 9.03 AM today. It took almost two hours for the issues to be resolved. Clients who were logged in from before and those using our desktop applications were not significantly affected. But considering that most of our clients use Kite and login around 9 AM, today was one of the toughest days we have had as a business.

    The issue was primarily due to leased line connectivity between our data centres in Mumbai. We suspect Metro digging work reason for multiple of our primary and backup lines to "flap" (as this had affected other services in the past several weeks). Flapping is when leased lines don't completely fail and hence don't automatically switch to the backup lines.

    Every day over 2 million trades are executed through us. Even the largest retail brokerages in the US execute just around half of this. At the scale we are at, these infrastructure issues we face in India can have a snowball effect on the trading platform like it did today. Unfortunately, since the lines flapped during market opening, over 3.5 lakh clients constantly tried to log into Kite multiple times causing the login system to build a massive queue of login requests over the already flapping leased lines. The scale of this issue was unprecedented, as you now know.

    Having multiple primary and backup modes of connectivity still did not help today. The only permanent solution for issues related to login is to completely remove such over-the-network dependencies between data centres. To this end, we're rebuilding how the Kite login system works completely. This will also have the added benefit of doing away with the current 2FA security questions in favour of a simpler numeric PIN-based system.

    This should prevent the particular network failure scenario that happened today in the future, in addition to improving your login experience.

    I am aware that a "sorry" won't help, even so, I am extremely sorry for the downtime today. We as a team are constantly working to ensure that we can offer the best trading platforms to you.

    Again, we deeply regret the pain caused.


    Nithin Kamath"


    -------------------------

    And if u combine all these 3 highlited words.... it can be reframed as ....

    "Zerodha has lot of infrastructure issues... and everytime new(as well as old) issue comes up.... And fonder is now fedup of saying Sorry... and know that sorry wont help..."


    @Nithin,
    If you guys have issues with executing over 2 million trades daily, dont worry.. Next year things will ease up... as no users will be left to take trades with u.


    Anyways as the reason of all the zerodha issues is either the OMS, or exchange,or lease lines, and this time a new one the Metro work.... or any other third party... Zerodha is expert to push the cause of the issues on other parties... By the way, why should we be surprised by this pushing of issues over other parties, we all know that, all their work is done by other parties only... zerodha is here only to earn brokerages by doing publicity.

    Have never seen so many consistant issues with any other broker, including discount brokers.
  • Shaha
    It was our mistake that we developed our system using zerodha's API and always hoped that this one is the last issue that happened to zerodha... Thanks for proving us wrong.
  • deep9686
    deep9686 edited May 2018
    Guys, even ISRO/NASA projects fail. Zerodha has maintained a gold standard SLA throughout. Let's not barge their team when they are quite open about root causes. Zerodha has definitely helped us to grow and in the path of growth, such incidents are indeed forgivable if not permissible. Let us give them their time to settle and come out with a foolproof solution for the future.
    @nithin, appreciate your communication. Keep us posted in the future as well.
This discussion has been closed.