Kite Connect 3.0 -- coming soon

Kailash
Kailash edited July 2017 in General
We have been hard at work rebuilding every bit of Kite, Kite Connect, and the underlying low level APIs over the last many months. At the same time, several internal components have already been rebuilt. We're very close to finishing and announcing Kite Connect 3.0 that'll feature several enhancements. Detailed changelog and transition guides will be published as we start rolling the updates out over the next several weeks.

Upcoming features
  • Reduced latency on all HTTP requests
  • Dedicated, low latency, WebSocket channel
  • Order and fill / trade updates via WebSockets
  • More live data fields on WebSockets
  • Reduced rate limits
  • Availability of over 2000 mutual funds via Kite Connect
  • Continuous futures historical data
  • Margin calculation APIs (phase 2)
  • Backoffice reports (phase 2 -- trade and PNL history, funds and ledgers etc. from the new back office that's being built)
  • Realtime sandbox for testing (phase 2)
Thank you for your support.
  • botany02
    @Kailash

    Welcome Back to Kite Forum...Almost After 4 months (Last Comment 28 Feb).
    Its Great...We are eagerly waiting for the update.
  • Kailash
    @botany02 Been hard at work :) Once Kite Connect 3.0's out, I'll be around frequently.
  • anilv13
    how do we get free access? i'm trying to create an app but obviously i need to create a prototype and not looking for a paid version!
  • Rishant
    @Kailash you guys have done a great work. Keep it up !
  • anupshinde
    Amazing.

    @Kailash Can the sandbox be used for paper-trading too?

    Also, is there a major divergence from the current api endpoints?
  • anupshinde
    Are you also planning to give some Kite API status apis? For example: api.status.kite....com. Preferably independent of Kite itself.
    That would help engineer apps better without the workaround. Some data like:

    is_online: boolean
    next_maintenance_window: { start_datetime, end_datetime }
    previous_maintenance_window: { start_datetime, end_datetime }
    can_login: boolean
    load_rating: range 1-10. 10 is highest load (or 1-100...whatever is better).
    //// Basically something like a 8/10 or 9/10 would mean extreme trading conditions where Kite (and probably most other broker platforms) typically break down due to traffic spikes.
    And may be nice to haves:

    is_market_open
    next_market_open_at
    market_close_at
    market_window // may be redundant info from above

    api_stability_rating: range 1-10. // 10 is best.
  • Gopal
    Hi,
    @Kailash Congratulation! for great job.

    Maine abhi kuch din pahle hi Zarodha Kite se trading karni shuru ki hai. Ab main robotic system se trading karna chahta hun. please mujhe bataye ki Zerodha ki PI Bridge, Kite Connect ya NEST plus main se kon se fully automated hai. aur kaise ese apply kiya jata hai. kya cost aati hai. Ami broker ka software free milta hai ya chargeable hota hai. Please help me.
    Kya ye possible hai ki koi mere system main demo de de full setup ka jo bhi jyada achha ho.

    Gopal Thakur
    Delhi
    9810817312
    [email protected]
  • Kailash
    @anilv13 @anupshinde Yep, the idea is to have a full simulated trading environment. The sandbox will be available free of cost.

    @anupshinde That's a good idea, programmatic access to ecosystem health data. There are no changes to endpoints, but there are a few minor (very minor!) breaking changes (renaming certain fields).
  • sujith
    @Gopal,
    Kite Connect is purely an execution platform. If you want to integrate with Ami broker or any other platforms then you will have to do on your own.
    Check out documentation here.
  • ketanshah
    @Kailash Thanks for the update. Quick question. Will the sandbox data contain live/valid data or just dummy data? I am trying to build a market simulator.

    Kind Regards
  • sachet_singh
    When will sandbox be available pls. We are keen to develop an app, and tried the API in paid version, but obviously cannot do full dev w/o sandbox env.
    Could you give some rough date and would it be possible to notify on email pls ?
    most ppl will be happy to pay for api on live environments, but not for developing their app itself.
    great work.
    thank you
  • pybull
    @Kailash, will you be running the two systems in parallel for some time, until our programs can adapt to the new changes (maybe a week or so)? Or would you be providing the sandbox environment of the new version, before stopping the older kite connect service? It would be helpful if we get access to the sandbox with the new API docs atleast a couple of weeks before it goes live, so that the transition is smoother for us.
  • Kailash
    @ketanshah There are regulations around publishing live data outside of trading environments. So the data will follow markets, but won't be completely random.

    @sachet_singh The sandbox is something we've been trying to get from our OMS vendor (TR) for over a year unfortunately, but now we finally have it. We plan to get it going after the 3.0 release. Can't give an exact timeline, but a few weeks, hopefully. Yes, we'll send out an announcement when it's ready.

    @pybull There will be a 30 day window where both versions will run in parallel. Please see my comment above about the sandbox. While we're ready with the Kite Connect layer, the delay has always been the underlying engine from the vendor. But soon, we hope.
  • sabyasm
    @Kailash
    Reduced rate limits
    Does it mean api will allow less number of orders per second than now?
  • iTank
    "Continuous futures historical data"

    @Kailash - What can we expect in this? Would you give data for expired series and how far back will this be supported?
  • Kailash
    @sabyasm Reduced limits of course, so you'd be able to send more requests than now.

    @iTank -- I'm getting the exact numbers from the data team.
  • Kailash
    @iTank Day data from 2011 for NFO instruments.
  • iTank
    Awesome! :)
    Will this be available for MCX instruments too?
  • krisys
    @Kailash , when can we expect this?
  • sujith
    @iTank,
    We don't have MCX data, we have plans to back-populate that data if available at the source.

    @krisys,
    In next couple of weeks, we will start the migration. We are testing new APIs parallelly right now. We will announce it on the forum.
  • sameer
    sameer edited July 2017
    @Kailash
    "Continuous futures historical data" ...

    Would data from 2011 be available for ***Options*** also ?
    If not then for how many years Options data would be available ?
  • Kailash
    @sameer It's only for futures for now.
  • pivottrading.net
    Very nice.

    Superb. Keep it up.
    Bring BO & CO in publisher api as well.
  • sameer
    sameer edited July 2017
    Can you pls add total mkt cap and free float market cap to your API ?

    Reason:
    1. Comparing traded volume and delivery volume wrt historical volume is one way.

    2. Other way is:
    a. Comparing traded volume and delivery volume wrt "Total Mkt cap" is very misleading.
    b. It makes sense to compare traded volume and delivery volume with "Free Float Mkt Cap"
  • gautam_s60
    Hi,
    @sujith
    can you please add master table like instrument table for master field list with field type and length
    i.e.
    NAME TYPE
    average_price N(10,3)
    buy_m2m N(10,3)
    buy_price N(10,3)
    tradingsymbole C(60)
  • sujith
    Hi @gautam_s60,
    You can check out data type of each field here
  • ishwarm
    Hi @Kailash , @sujith ,

    Regarding
    Continuous futures historical data
    I assume this feature will be available for all time intervals including 1-minute candles too, right? And will this be available for futures in CDS segment also?
  • ping2balaji
    ping2balaji edited July 2017
    Great work guys....Waiting for kiteconnect 3.0 :smile:

    One quick question, will the POSTBACK_URL continue to exist even with kiteconnect 3.0?
    Or can the server post event through kiteconnect 3.0 dynamically to client(our python/jave code etc) when an orderstatus change?

    I am asking this to just to see whether the webhook implementation can be avoided with this upgrade as it needs all personal domain/public ip etc which might be too much for retail traders.
  • sujith
    @ping2balaji,
    We are planning to provide order updates via websocket in next version of websockets.
    Users might not need to setup servers for postbacks after that. Since it will take some time for the next version of websockets, you will have to setup a server for postback for now. You don't need a domain to setup postback.
  • joy
    I append the version header with the http requests

    request.Headers.Add("X-Kite-version", "1.3");

    The above used to work fine, now its not. Removing the same however fixed the issue. Hope migration will be smooth.

    @sujith
    Since it will take some time for the next version of websockets, you will have to setup a server for postback for now.
    any ETA. I suppose currently the ball is in TR's court and thus beyond your control.
  • abhizerodha
    Hi Kite team,

    Any idea when kiteconnect (Python) version 3.0 would be available? I am looking for these main features:

    - Order and fill / trade updates via WebSockets
    - Margin calculation APIs (phase 2)

    Also is there an easy way, given a symbol (500570), to obtain reference info such as Name (Tata Motors), Margin Requirements or corresponding active futures for the stock?

    And how can I query current holdings of the portfolio?

    Thanks again for your help.
    Abhi

  • abhizerodha
    abhizerodha edited July 2017
    PS: How can I send sell short orders for Futures?
  • sujith
    @joy, @abhizerodha,
    I am afraid we can't give any exact timeline as of now.
    Maybe in a couple of weeks.
    We are rolling out updates internally and testing it.
    We will announce change log on the forum.

    @abhizerodha,
    Please create a new thread for new queries, it might help others who are looking for the same.
  • abhizerodha
    Thanks Sujith. Looking forward to the new version. Sand boxing will also help to test better execution strategies.
  • sidsverma
    Does that mean that with Connet 3.0, we will be able to trade in Mutual Funds via APIs?
  • sujith
    @sidsverma,
    Yes, you can expect Mutual funds API soon. We will announce it on the forum.
  • arsh_makker
    I was wondering, if it is possible ot find out the most active securities/contracts during the day via KITE api's?

    So basically I want the kind of data in the here from nse
  • sujith
    @ishwarm,
    It will take some time for continuous charts for currency segment. Currently, our data team is working on MCX data.
  • arkochhar
    @sujith , could you give us some indication as to tentatively when would the API 3.0 and sandbox released? Weeks/Months? Thanks
  • Shaj
    @sujith
    ====>More live data fields on WebSockets

    I assume timestamp will be available ? Also is there any plan to give the OI(open interest) info for F&O scripts ?
  • mld
    @Kailash, @sujith
    Not all of us require Live Market Data or Historic Data.

    Is it possible for Zerodha to sub-divide Kite API access in the following way:
    1. Basic Package
    2. Live Market Data Access
    3. Historic Data Access

    In the Basic Package
    1.1 Order and fill / trade updates via WebSockets
    1.2 Margin calculation APIs
    1.3 Backoffice reports (trade and PNL history, funds and ledgers etc.)
    1.4 Opening price for subscribed instruments
    1.5 End of Day Closing price (after market close) and EOD LTP
    1.6 After Market Order / status of AMO
    1.7 Pre-market / Market / After Market is open / is closed
    1.8 Exchange holidays
    1.9 Zerodha emmergency messages (For example, NSE glitch in trading system)
    1.10 Expiry dates for subscribed instruments
    1.11 Price Alert

    I think pi-bridge (costs only 500/-) provides some of the features. But Kite connect with these basic features will be great as
    1. it will be platform independent
    2. later up-gradation to Live Market Data or Historic Data with Kite will be smoother
    3. It will reduce end user cost for semi-automation
    4. End users will be capable of implementing Good Till Date / Good Till Canceled Odrers
  • pinkpanther
    @mld donot generalize your requirement with others
    'Not all of us require Live Market Data or Historic Data' is quite an assumption.
  • arsh_makker
    I was wondering, if it is possible ot find out the most active securities/contracts during the day via KITE api's?

    So basically I want the kind of data in the here from nse
  • sujith
    @arsh_makker,
    We will add it to our list. We will provide this in future.
  • ravi_ranjan
    @Kailash @sujith

    First resolve my issue which is same as the following
    https://kite.trade/forum/discussion/1933/historical-data-unavailability#latest

    Instead of releasing new version first try to make your infrastructure proper scalable. I already paid money for your data services and you are having fun with that. No service till now
  • jvshk78trade
    Hi, Just getting started on Kite connect, which web hosting service is best suited/recommend for using Kite Connect , does it matter if server is located in India or abroad, any specific services required from web hosting service provider...etc, please guide. BR, -Hari Krishna
  • atulsingh7890
    Hello all,

    Before Paying for Historical Data API, Do Look for Forums and other Issues which other are facing with Historical API,

    Support is worse, I seriously comment If you cannot provide an email support or call support why the hell you people are charging it, Provide it for free and let the users discuss what could have gone wrong through Kite Connect Forum.


    One of the issues which I am facing :
    https://kite.trade/forum/discussion/1933/historical-data-unavailability#latest

    There other posts also complaining about support policy, So beware before paying up.


    @jvshk78trade , @arsh_makker , @pinkpanther , @Shaj , @abhizerodha , @iTank , @anupshinde , @ping2balaji , @ishwarm , @joy , @botany02 , @Gopal , @ketanshah
    Need your help, Are you facing problems fetching data with Historical API as specified in the post above. Or am I the only one here stuck?


    If I am offending someone for posting here, Its because this is my last option to get some support from fellow members.
  • Shaha
    I very much like Zerodha and the way it has always been ahead of competitors, and introduced new things for us. But now your competitors have copied your business model and advancing fast. They are providing much more features and lower pricing for interactive APIs. Zerodha's frequent performance glitches although reduced, but still exists. Its high time, Kite connect and historical API, monthly pricing should be dropped. You will otherwise endup losing customer.

    Please take a serious thought on it. As your pace looks to have reduced. And i am afraid your competitors are running like anything.

    Its understood, that you have so many customers and so you need to be more robust, and think twice before introducing new things, so that it doesnt hamper the old stable platform. But... at the end customer demands is the paramount.
  • Kailash
    Kailash edited August 2017
    @Shaha Pricing of the APIs are a business consideration and I'm unable to comment on them right now. I'm aware that our competitors have even copied our documentation word for word. Shows where the true innovation's happening :)

    But, our pace hasn't reduced. Our tech team has doubled in strength and over the last one year, the entire team's been working on revamping every bit of our infra (including Kite) for scale. Our trading activity has grown exponentially and it takes great amount of effort to keep scaling the systems behind the scenes to keep up. Now, like I said, most of it is behind the scenes, which is why you feel we've slowed down, where as we've upped our throughput manifold :)

    The culmination of one year's worth of non-stop work is going to start hitting the markets soon!
  • Prem
    Hi @Kailash,

    You said that "The culmination of one year's worth of non-stop work is going to start hitting the markets soon!"

    Really I am waiting hungrily. But don't know when it will be available. :(

  • jvshk78trade
    Hi Zerodha Guys,
    Make some tutorials for people like myself to start off. Please start tutorials section ASAP, before the expiry of my first paid app I want to atleast once see my trades successfully automated.
  • Kailash
    @ jvshk78trade All the official client libraries (on Github) come with simple, sample how-to code and sample projects.
  • ramprabhu77444
    @Kailash Any chance of including level 3 market depth ? I mean 20 Bid and 20 Ask Prices ?
  • sujith
    Hi @ramprabhu77444,
    Exchanges are very clear that 20 bid and ask market depth is only for displaying on trading platforms. I am afraid we can't provide this feature in Kite Connect.
  • pinkpanther
    @sujith 'displaying on trading platforms' , if that is the case do we expect it to see in Kite web application? or pi?
  • sujith
    Hi @pinkpanther,
    It will be available on Kite soon. It might be included on Pi in future.
  • pinkpanther
    @sujith no disrespect but anything 'soon' coming from Zerodha is like
    'we know you guys(traders) want it,but there are some reason(our beloved exchanges dnt give or wont give or some reason) why we cant have it now or anytime in future,but from business perspective lets not say we cant give it to you,therefore just use an indefinite term "soon" which can be inferred to as- in the next millisecond to the next 1000 years ' .
    Few other examples - feature coming soon as 'the sandbox' .Last year when you guys came up with Kite Connect you said the sandbox feature would be soon..if I ask you now again you would say its going to come soon.

    Point being it doesn't help saying 'soon' . I fully understand that you guys are actually putting a lot of effort to make things happen and I appreciate it, but telling things as it is Helps to get a better direction.
  • Kailash
    @pinkpanther I completely understand your sentiment. We'd planned to release the sandbox last year. But, Thomson Reuters, our OMS vendor, were unable to provide us the backbone for it on time. Coincidentally, just this morning, we were able to procure the license to a test version of the sandbox environment (sandbox of sandbox!).

    Unfortunately, our "soon"s are bounded by so many external variables that we find it extremely hard to commit exact dates. Significant parts of Kite 3.0 has been running in production for months, but we're still waiting on the 5% external variables before we can make a full release.

    Just like you guys are developers waiting on us, internally we're also developers waiting on others, slowed down by bureaucracy :) Personally, I'm positive the 3.0 release and the new features will make up for all the waiting.
  • thepanoramix
    Understand delays in going live, but why don't you publish the API documentation and allied literature at least, so that we can start development at our end as well. If legal issues are stopping you from having sandbox may be you can replay days activity in sandbox after market hours, that way there would be no legal hassle (I think :wink: ) since you are just replaying past
  • soumyadeep
    @Kailash I hope we get more data in websockets. That will be awesome and save us from calling a lot of requests from time to time. BTW in case you guys change the syntax of api (I am using pykiteconnect), please release the documentation too. That will help a lot.
  • Kailash
    Kailash edited August 2017
    @thepanoramix We plan on publishing the changelog next week. And the week after, the preview version of 3.0 (which you'll be able to access by sending an additional header to the API endpoints). There are no legal issues with the sandbox, but the bureaucracy of our vendor-overlords :)

    @soumyadeep Yes, this is definitely happening. Dedicated ticker instances for the developers. Updated protocol with more fields (this'll take some more time). Order postbacks and other updates via WebSockets. Also, the quotes call will take multiple tokens and return OHLCv for all of them at once. This'll simplify polling to a great extend.
  • thepanoramix
    @Kailash Thanks for the update. And excited to see things the next week.
  • arsh_makker
    Not sure, but just wanted to check, if KiteConnect 3.0 could have the functionality from your already existing unexposed API https://zerodha.com/margin-calculator/SPAN/ for specific options?
  • sujith
    Hi @arsh_makker,
    We will add it to our list.
  • arsh_makker
    so today is next to next week... any idea about the changelog?
  • Vivek
    Vivek edited September 2017
    @arsh_makker Tomorrow we will be making some major announcements. Stay tuned.
This discussion has been closed.