Kite Connect 3.0 - release #2 (Sep 11)

Kailash
Kailash edited November 2017 in General
Dear developers,
We are happy to roll out phase 1 of the Kite Connect 3.0 release. You can consider this a pre-release before we publish the official migration guide and flag off the migration deadlines.

Release 2

Query historical API with smaller intervals (full timestamps and not just dates)

Until now, the historical API could only be queried for candles between two days (yyyy-mm-dd), but now, it can be queried between two timestamps (yyyy-mm-dd hh:mm:ss). So, if you want to get the candles between a smaller interval, you can query just that instead of fetching a whole day's worth and filtering.

In this example, minute candles between 9 AM and 10 AM are fetched. Do note that the space in the date string (between the date and the time) has been urlencoded with a `+` here.
https://api.kite.trade/instruments/historical/5633/minute?api_key=xxx&access_token=yyy&from=2017-09-08+09:00:00&to=2017-09-08+10:00:00

Release 1

New bulk market quote APIs.

Bulk fetch OHLC + LTP quotes for up to 200 instruments.
https://api.kite.trade/quote/ohlc?api_key=xxx&access_token=yyy&i=NSE:INFY&i=BSE:SENSEX&i=NSE:NIFTY+50
You can pass the &i={exchange:tradingsymbol} up to 200 times in a single call.

Response:
{
"status": "success",
"data": {
"BSE:SENSEX": {
"instrument_token": 265,
"last_price": 31606.48,
"ohlc": {
"open": 31713.5,
"high": 31713.5,
"low": 31586.53,
"close": 31809.55
}
},
"NSE:INFY": {
"instrument_token": 408065,
"last_price": 890.9,
"ohlc": {
"open": 900,
"high": 900.3,
"low": 890,
"close": 901.9
}
},
"NSE:NIFTY 50": {
"instrument_token": 256265,
"last_price": 9893.4,
"ohlc": {
"open": 9899.25,
"high": 9911.9,
"low": 9882.55,
"close": 9952.2
}
}
}
}
Bulk fetch LTP quotes for up to 200 instruments.
https://api.kite.trade/quote/ltp?api_key=xxx&access_token=yyy&i=NSE:INFY&i=BSE:SENSEX
You can pass the &i={exchange:tradingsymbol} up to 200 times in a single call.

The API for fetching a full market quote (several additional fields + full market depth) will be released shortly.

Connecting to the new 3.0 backend

By passing the header X-Kite-Version: 3 to the HTTP requests, you can connect to the new 3.0 backend. The requests should be processed much faster than the prior version. Changes described in the following sections will only work if this header is passed.

ETags

All GET requests now support Etags, which is an HTTP caching mechanism. Every HTTP response you get will now have an E-Tag, which you can send along with the next request in the If-None-Match header. If the data hasn't changed, instead of returning the full response body, the API will return a 304 header. This should create significant bandwidth savings for you.

What's remaining?

These will be rolled out over the coming days.

- Live order and fill updates via WebSockets
- Long standing read tokens
- New auth schemes
- Updates to official client libraries incorporating the changes and new features
  • arsh_makker
    arsh_makker edited September 2017
    this post is such a disappointment,
    1. only one functionality has been updated after so much of build up from the post https://kite.trade/forum/discussion/comment/7619#Comment_7619
    2. The link to release #2 just goes to search for #2 in the whole of the kite forums
    3. no roadmap of all the features promised? in the post https://kite.trade/forum/discussion/comment/7619#Comment_7619

    Just saying I expected a lot more from Zerodha
  • botany02
    botany02 edited September 2017
    @Kailash
    I didn't get you on Market Quote API.
    What is the targeted use?
    We already getting market quotes via web-sockets

    Below is my understand
    Market API can be used where you occasionally require the latest price for your analysis and don't need streaming prices.Say running some calculation at at interval of 1 hour, so you can avoid using web-socket streaming to reduce resource usage.

    Web-sockets can be used where you need streaming price and doing analysis in real time.

    I hope Market API is resource hungry at user as well as kite server level and should used only it is absolutely necessary otherwise should stick to web-sockets.

    Market API usage should be clear,otherwise this will mislead, and people may start pulling the data via Market API instead of web-sockets.

    What will happen if everyone pulls data via Market API..?
    Do you recommend Market API over web-sockets..?
    Could you pls throw some light on this..?
  • Kailash
    @botany02 Several of our developers do not want to run and maintain a streaming WebSocket client. They just want to query market data at ad-hoc intervals. Maybe once every 15 mins, or even once every minute.

    - For continuous realtime quotes, WebSockets is the way to go.
    - For ad-hoc / batch queries, for simpler use cases, the new quotes APIs are easier to develop with. No need to run a stateful WebSocket server or database.
  • botany02
    Thanks for the clarification @Kailash
  • bk18
    Hi,

    If i use Market API or REST API to get live data - is that a problem? I mean for every second, open a connection, make the API call, get the response and close the connection. I understand this may not be the desired way, but can we do it that way?

    --Thanks.
  • Kailash
    @bk18 If you poll the GET APIs too much, you'll get rate limited. Polling every second is higly inefficient and if you want live updates, you should definitely use the WebSocket library.
  • varaprasady
    Hi,
    is it possible to get only ltp without ohlc just to save data
    Thanks
  • bk18
    Thanks @Kailash. I shall use websocket streaming provided the structure has tick data second/milliseconds. I do not find any date/time stamp in the packet structure. Can you throw some light on how to map every streamed tick data to a second with the current packet structure?

    Also, i understand current limit is 2 calls per second and may be increased to 3. If at all i poll, i will stick to that limits.

    --Thanks.
  • sujith
    Hi @bk18,
    We have this on our list.
    For now, once you get tick may be you can append a timestamp to that and store it.
  • Kailash
    @bk18 Sending the timestamp in the tick packet is going to be a part of the 3.0 release series.
  • bk18
    Thanks @sujith, @Kailash for the updates. Looking forward for more 3.0 release updates.

    Also, for the historical data API, the documentation says "from" and "to" period. I assume "to" can be yesterday. But what is the initial "from" - i mean the very first date of data availability for each instrument/index? Is there a way to find this out?

    --Thanks.
  • sujith
    Hi @bk18,
    Please post this question on the new thread. It might help others.
  • Encrypted
    Encrypted edited September 2017
    Hi @Sujith when trying to execute https://api.kite.trade/quote/ltp?api_key=xxx&access_token=yyy&i=NSE:INFY&i=BSE:SENSEX getting error remote server returned an error 405:method not allowed.I am calling sessionhook.I am using (HttpWebRequest)WebRequest/response.
  • pranksterguru
    when will javakiteconnect be updated to support this?
  • sujith
    Hi @Encrypted,
    It seems to be working fine. You need to make get request.
  • James
    I do not see the demo account/trading function, is that released?
  • Encrypted
    hi @sujith though it is working but often I am getting below error message for 100 scripts
    {"status": "error", "message": "Gateway is busy", "error_type": "NetworkException"}.today is sunday but any working day I don't know how much load will be on server.
    please respond how to overcome the issue.
  • Encrypted
    Encrypted edited September 2017
    Hi @Kailash , as you mentioned "They just want to query market data at ad-hoc intervals. Maybe once every 15 mins, or even once every minute".so how to pass time frame as parameter https://api.kite.trade/quote/ohlc?api_key=xxx&access_token=yyy&i=NSE:INFY&i=BSE:SENSEX&i=NSE:NIFTY+50. is it still in development phase?
    if we want ohlc for only 5 mins for multiple scripts together how to get that?
  • puneetbijnor4020
    I think it is ohlc of the day
  • Kailash
    Kailash edited September 2017
    @Encrypted The API you've quoted is live. It gives you realtime quote and OHLC would be for the day. If you want OHLC of combined timeframes (5 mins, 15 mins etc.), you need to use the historical data API. We're just releasing an update to the historical data API that'll let you query smaller intervals rather than the full day.
  • Encrypted
    Hi @Kailash .thanks for replying.does historical data supports 200 scripts ohlc data in one query?I have read somewhere it allows 3 calls per second.
  • arunrambo2000
    I'm still using the old APIs. Today my order status' are showing as PUT ORDER REQ RECEIVED even for the ones that are marked as complete in kite.zerodha.com . These use to work perfectly till now. Is this somehow related to your pre-release?
  • thepanoramix
    the historical data api with just date is also broken
  • arunrambo2000
    I realised the problem that I'm facing - https://kite.trade/docs/connect/v1/#retrieving-an-individual-order . This documentation says that the multiple stages of the order are recorded as a list with the latest being the first. This was working properly till friday. Suddenly from today, the order of multiple stages of the order is changed and the latest is being put in the last.

    Is this a permanent change? or is it a bug and you'd fix it?
  • Vivek
    @arunrambo2000 We can confirm the issue, it will resolved soon (after market today) and will update you on this.
  • thepanoramix
    In bulk fetch API can you also add following fields which we get currently in quite response ?

    "change_percent": 0.0,
    "change": 0.0,
    "volume": 0,
    "buy_quantity": 0,
    "sell_quantity": 0,
    "open_interest": 0,
    "last_quantity": 5,
    "last_time": "2015-12-15 10:16:36",
  • arsh_makker
    I tried the bulk api, and am not able to get the tradingsymbol symbol for Options 'Nifty september 2017, 10000CE' in the earlier API it was NIFTY17SEP10000CE
    1. Does the bulk api support FnO calls?
    2. If yes to the above, did the tradingsymbol changed in the new api?
    3. if yes to q3, what do i need to get the new trading symbol?

    Regards
    ARsh
  • sujith
    Hi @thepanoramix,
    We will include it in bulk quote call which will be provided in future.

    @arsh_makker,
    It seems to be working fine.
    You need to send params like this,
    i=NFO:NIFTY17SEP10000CE
    Please post these questions on a new thread so that it will help others.
  • mahesh.james
    Hi,

    - Live order and fill updates via WebSockets
    When can we expect release of the above item.

    thanks
    Mahesh
  • Encrypted
    Hi @Kailash is your new historical api support 100 scripts ohlc for every min interval.suppose I want to get ohlc data from 9:15 to 3:30 for 100 scripts in every min interval.Please reply.
  • sujith
    Hi @Encrypted,
    As mentioned in the post you can fetch OHLC for 200 instruments at once.
  • Encrypted
    @sujith :please understand my question.my question is can I get 200 instrument ohlc data every one min interval.do you have any API that support above functionality?
  • sujith
    @Encrypted,
    You can use this
    https://api.kite.trade/quote/ohlc?api_key=xxx&access_token=yyy&i=NSE:INFY&i=BSE:SENSEX&i=NSE:NIFTY+50
  • Encrypted
    As per answer given here by kailash it will give days ohlc data.Not for 1min.@sujith please check.
  • sujith
    Hi @Encrypted,
    This is quote call which will give you a market snapshot. It has nothing to do with Historical data.
    For 200 instruments, you will have to create your own candles using Websockets API.
  • sudhirshettyk
    @Sujith , w.r.t release 2 of historical data access -
    Any update regarding consistent way of determining if the data is adjusted for corporate actions . Eagerly looking forward for this . Unless there is clear way of handling corporate actions on the historical data , i am unable to use it reliably . It would be nice if you could prioritize this . Thanks.
  • sauravkedia
    @sudhirshettyk. I am facing the same issue where I need corporate actions adjusted data on a daily basis. (I also don't want to adjust for actions myself). For this currently, I am exploring another paid subscription http://www.viratechsoftware.com/stocksup-eod.html . They adjust for actions at End of the Day. They will charge around 10K per year for this. Data will come in metastock format which can be easily read in popular charting packages or programming languages.
  • sauravkedia
    @sudhirshettyk , In case you come across other any other solution/fix to issue, do enlighten us.
  • sudhirshettyk
    @skk , currently i do it myself . I use eod data from nse but it is always guranteed that data will never be adjusted . Hence i can handle it consistently . NSE also provides historical and forthcoming corporate actions details . I maintain these in my database and adjust all my stored historical data , technical analysis data etc.
    The problem with Zerodha historical data is it is not consistent whether a data is adjusted or not . As a result , say if you have a multi-day position , and overnight there is a split/bonus , the historical data will end up generating wrong signals and would wrongly trigger . Also , your portfolio profit/loss details go for a toss. I have been telling Zerodha couple of times in the past , but i have been unable to convince them of the importance of consistency of the historical data . The point is if Zerodha cannot provide the upto date adjusted data then atleast dont do it partially . This is my another attempt to convince guys at Zerodha :-)
  • malay
    Hi,
    Are the recent releases incorporated into the official kiteconnect python client?
  • sujith
    Hi @malay,
    We haven't updated documentation. We will update it soon.
  • gautam_s60
    gautam_s60 edited September 2017
    is it done? or any further update is awaited?
  • haribabu
    When will kite 3.0 web released?
  • Raj-Tech
    What is the release date for 3.0?
  • sujith
    Hi,
    Most of the APIs are already migrated. There are a couple of APIs for which response format will change(minor changes), we will announce changelog soon.
  • haribabu
    If everything already done what has changed between December and Oct ?
  • sujith
    Hi @haribabu,
    We have rewritten the backend from scratch at our level and upgraded numerous things at the OMS level for increased speed and stability. We've released some new features last month and will continue releasing more this month.
  • sujith
    Please be assured that we're working very hard to make the underlying systems better.
  • haribabu
    @sujith You are working hard. I appreciate, but OMS is still slow in executing orders, we are not getting good quotes just one or two per second, doing automation with insufficient data is of no use.

    If we get data from external sources still execution is not guaranteed. It is more than a year. Scalability is missing, you increase team size there is no restrictions, to meet the demand

    What I understood about the development being done is only moving from one vendor to other.

    I don't see any useful things which can aid or improve trading is happening.

    As I told earlier delay in adapting technology is inviting trouble of losing customers ( Nokia and Blackberry )

    When there is volatility still we face issues of orders not going through.

    What kind of improvements you are doing we don't know.
  • Shaha
    Shaha edited October 2017
    Is the continuous futures data available for download in new version? If so, what is the syntax? And which all symbols allowed?
  • sujith
    Hi,
    Continuous future data is available for future & options contracts.
    We only provide day's data for expired contracts.
    You can check out documentation here https://kite.trade/docs/connect/v1/#historical-data
  • pivottrading.net
    I was away from the forums for 2 month and i missed this release.

    I dont have words to thank for this.
    New bulk market quote APIs.

    I will try to get this test and integrate as soon as possible. This is what i was exactly looking for.
    Thanks a ton Kailash and your Team :)

    Thanks,
    Sourabh.
  • Prem
    Hi @Kailash,

    Its nearly2 months a go for your kite connect 3.0 release but its major part is still pending like "Order Fill/Update Via Websockets" and "Sandbox Environment"

    Kindly update your progress on this. I am still waiting for this. Please Implement all these major features in Java kiteConnect SDK also.

    Kindly reply ASAP.

  • Vivek
    @prem We will start providing order fill updates (updates for statuses other than COMPLETE, REJECTED and CANCELLED) for postbacks in a day or two. We are working on Websockets updates, since it needs a major changes to our existing system it might take some more time to release.
  • sauravkedia
    @Vivek
    I am missing a lot of trades as well as getting lots of errors on your platform lately. It is damn irritating, not to mention the costs involved. If connection to the 3.0 back-end will solve some of the problems, I suggest you share how to use it.

    How do I connect to 3.0 back-end. On official python library, I tried adding this header to the requests be amending this line:

    r = self.reqsession.request(method,
    url,
    data=params if method in ["POST", "PUT"] else None,
    params=params if method in ["GET", "DELETE"] else None,
    verify=False,
    allow_redirects=True,
    timeout=self._timeout,
    proxies=self.proxies, headers={"X-Kite-Version": 3})

    However, this fails at authentication stage itself and I get the following error:

    user = kite.request_access_token(request_token="xxx",
    secret="yyy")

    Traceback (most recent call last):

    File "", line 2, in
    secret="yyy")

    File "", line 237, in request_access_token
    "checksum": checksum

    File "", line 469, in _post
    return self._request(route, "POST", params)

    File "", line 528, in _request
    raise ex.ClientNetworkException(e.message, code=500)

    AttributeError: 'InvalidHeader' object has no attribute 'message'

    Next I tried it directly on a browser. On browser, when I pass this header X-Kite-Version: 3, I sometimes get Error 404 Not found, and at other times Error 503. On both occasions, if you remove the header, it works.

    Can you tell us how to use it?



  • sujith
    Hi @sauravkedia,
    Can you let us know the exact URL for which you are getting 404 and 503?
    Can you also explain what exactly your proxy is doing?
    Are you using the same proxy on browser as well?
  • sauravkedia
    Hi @sujith
    1. The official urls as mentioned in your official documentation on kite.trade.
    When I tried to add header to official library. I get the InvalidHeader Error.

    So, I tried browser. I used developer mode in Firefox and go to network tab which allows me to amend the headers and resend the request. When I send a request without header, It works. When I add it I get errors.
    2. Too technical for me.
    3. Too technical for me.

    Instead, Why don't you just push an update so that users can start using it without getting into the nitty-gritties? If pushing update is not high-priority, then at-least share, which lines users need to amend in official libraries to benefit.



  • shortwire
    @Kailash For 3.0 access, we understand the header needs to be changed for HTTP requests. However, how about web sockets? How do we subscribe to an instrument over websockets for version 3.0?
  • tonystark
    Hi @shortwire,

    New WebSocket API is not live yet. It is undergoing final phase of development and testing. We will update the docs and client libraries once it is completed.
  • tahseen
    @Kailash Time based historical fetching - This is dream come true. Excellent !!!
  • coolkoti
    Bulk market API is quite unpredictable. I get quotes for certain symbols or when I use instrument tokens, but it doesn't return anything for many. especially, for options.
    The following gives response:
    https://api.kite.trade/quote/ltp?api_key=xxx&access_token=yyy&i=NFO:NIFTY17DEC10000CE

    But, the below doesn't give any data, though the response is success.
    https://api.kite.trade/quote/ltp?api_key=xxx&access_token=yyy&i=NFO:SAIL17DEC80CE
    In this specific case, using instrument token seem to work:
    https://api.kite.trade/quote/ltp?api_key=xxx&access_token=yyy&i=9719810 -> This gives data

    But even this technique doesn't work for many others. For example, INFY 1000 CE doesn't work either with symbol or instrument token.
    https://api.kite.trade/quote/ltp?api_key=xxx&access_token=yyy&i=NFO:INFY17DEC1000CE
    https://api.kite.trade/quote/ltp?api_key=xxx&access_token=yyy&i=21363714
    Both return empty data -> {"status":"success","data":{}}

    Can I rely on this API for getting option quotes?
  • sujith
    Hi @coolkoti,
    Thank you for reporting. We will look into this.
  • coolkoti
    Thanks much Sujit. Any idea on timeframes bhy when this issue gets resolved?
  • sujith
    sujith edited December 2017
    @coolkoti,
    It is fixed. Can you check and update?
  • coolkoti
    @sujith
    This is working now. Thanks a lot sujit for quick response. Saved a lot of trouble :-)
  • coolkoti
    is there any problem with market data api? This is returning the same LTP from 9:58 AM today. example:
    https://api.kite.trade/quote/ltp?api_key=xxx&access_token=yyy&i=NSE:APOLLOTYRE.

    This is always returning the same LTP to from 9:58
    {"status":"success","data":{"NSE:APOLLOTYRE":{"instrument_token":41729,"last_price":277.55}}}
    Thus is happening with all NFO symbols
  • cisk
    Hi Team,
    Is there any official documentation(url) that states the release features for Version v1, v2, v3 or the future roadmaps? Additionally, I'd like to know what is the status of current implementations:
    1. Timestamp for tick prices in realtime web socket streaming
    2. Is order lifecycle status such as "completed", "rejected" etc. of placed orders are available in websockets streaming handler?
    3. When can we expect a sandbox environment to test our code in off market hours?
    4. What is the best method/architecture you suggest to pick up ticker prices from WebSocket object i.e "ws", and make it available to my main program(where strategy runs)?

    Thank you for any help!
  • sujith
    @cisk,
    We are working on documentation for Kite Connect 3. Which Kite Connect client are you using?

    Timestamp for ticks is already available in new Kite Ticker API. For now, we have kept it in another endpoint till everyone upgrades to the new version. We will announce final major release soon.

    Yes, you can receive order updates via webscokets in new Kite Ticker.

    We are still waiting for APIs from our OMS vendor for Sandbox environment.

    You can check out this thread.
  • Shaha
    @sujith how would i know whether i am using the new Kite Ticker API or have been still using the older one? I would prefer to still continue the older one for my production environment... till i test the new one with penny testing.
  • sujith
    @Shaha,
    We have Kite Ticker 3 API as a parallel setup till everyone migrates. You need to connect to the different endpoint to use it as of now. It is already included in new version of client libraries. You can take a look at it in kite3 branch of the repository.
    Please note that we are still making some final changes on these branches.
Sign In or Register to comment.