Websockets API throwing 429

sbdavra

we are not able connect by web socket
we restarted our application lot of times
but it gives error of "too many connections"

please resolve it or give some advice please

its very urgent
thanks

sbdavra

it gives error "to many request" when we connecting our socket to zerodha

sbdavra

anything is changed from your end ?

sbdavra

this message gives in specific connections only,
we are using 3 sockets connections and out of that 1 working find other 2 are not working

sujith

You can check out this thread.

sujith

You may need to log and check if you spawning multiple connections.

sunny

@sujith : I think you should check at your side. My code was working till yesterday and now connection is dropping with same issue. Nothing changed from our side, something must have changed at your side.

sbdavra

@sujith i 100% agreed with sunny , we did not changed any thing but still we are not able to connect with your servers , it gives same error as below

**[Fri Oct 09 10:39:09 IST 2020] setOnErrorListener.Exception = com.neovisionaries.ws.client.OpeningHandshakeException:
The status code of the opening handshake response is not '101 Switching Protocols'.
The status line is: HTTP/1.1 429 Too Many Requests

sbdavra

i already restarted our servers lots of times but still above message message is appearing from your end....

rahulbhayana

same issue here, unable to get live tickers, websocket handshake failed., it is working perfectly fine till yesterday but from today morning no ticker is being fetched.

sbdavra

@sujith kindly reply on this sujith , please.

still i am not able to connect with your servers ,
sames message i am getting

setOnErrorListener.Exception = com.neovisionaries.ws.client.OpeningHandshakeException:
The status code of the opening handshake response is not '101 Switching Protocols'.
The status line is: HTTP/1.1 429 Too Many Requests

rakeshr

@sbdavra
We just re-checked all scenario at our end, 3 connections with up to 4000 instruments are working perfectly fine. You get 429 - TooManyRequests, only if you have spawned more than 3 connection.
You need to check the same at your end, that you are not exceeding the limit of max 3 simultaneous connection.

sbdavra

i already checked all this scenario ,
again i am repeating that i did not changed anything since last 6 months ,
and i already know this 3 connections limit sir and i not crossed this limit...
after closing all my connection i just started one connection only , and on that connection also i am getting same message.

and why you are not accepting your fault . you can check this thread all are said that we did not changed anything , it was as it is since long time....

so requesting you to check and give me some genuine answer, please

thanks

rakeshr

@sbdavra
Which client are you using? Can you paste your WebSocket code here, we will take look?

rahulbhayana

@rakeshr can you explain exceeding the limit of max 3 simultaneous connection.? or if possible can you connect me on my mobile number 9654182422

sujith

@rahulbhayana,
A Kite Connect app can open a maximum of 3 WebSocket connections at any point in time. If you exceed that then you will see 429 error.

rahulbhayana

@rakeshr ticker.js:263 WebSocket connection to 'wss://ws.kite.trade/?api_key=7vk95n09wix8psc8&access_token=***&uid=1602223797217' failed: Error during WebSocket handshake: Unexpected response code: 429
can you check and help me out?

rahulbhayana

@sujith if possible can you call me on my number(9654182422)? i need small clarification..

sbdavra

@rakeshr
we using below client link
https://github.com/zerodhatech/javakiteconnect
The Kite Connect API Java client

and we also upgraded you latest jar and again repeating till yesterday same setup was working fine since last long time..

sunny

absolutely ridiculous.

@sujith @rakeshr : Just think this:

- How come everyone is suddenly spawning more than 3 connections?
- if all of users who are complaining did not change anything in their code base? what changed?

How hard it is to think that there is only one common thing (Zerodha server) among people who are complaining

Regards,
Troubleshooting 101

rahulbhayana

@sujith @rakeshr i am waiting for your reply,because of this issue, i have lost huge amount of money.

sujith

@sbdavra, @sunny,
The change mentioned here was taken live yesterday.
We had given the deadline way before 8th October 2020. We have always told people to keep the maximum simultaneous connections up to 3.

@rahulbhayana,
We don't provide one-to-one support. You can write here to know more details.

sbdavra

@sujith
dear sujith ,

i dont know that either you are not understanding my issue or you don't wants to understand.

i clearly mentioned that i know this limit of 3 simultaneous web socket connections.
and i am not exceeding it.
i clearly mentioned that i stopped all my servers and connections today . i am trying to connect with one server with one web socket connection, still not able to access it.

so can you explain me what exactly you changed on yesterday live. which affect our running system.

for your knowledge again i not exceed your 3 connection limit.

so i wants to know that is there any other technical changed done by you for subscription and passing arguments or anything else ??

i already read your thread you linked here , but it does not affect any matter to me personally.
so please clarify in detail.

hope now you will give some more information and help to us to overcome this issue raised today suddenly.

thanks
Regards
shailesh

rahulbhayana

@sujith can you tell me what is the maximum connection limit prior to today?

sujith

@sbdavra,
We don't see any issue from our end. Even I tested it multiple times. Anyways we have asked the team to revert it for now. We will take it live again soon.
@all
Please make sure you don't exceed a maximum of 3 websocket connections per api_key.
@rahulbhayana,
We don't have any hard restriction for the maximum number of websocket connections per api_key for now because of legacy reasons. But we will enforce this restriction again so make sure you don't exceed the same.

Bhavninder

@sujith What's the point of an API, if you cannot access data on more then 3 webpages? So, in future we will have to attach different API_key to different webpages(as each webpage is being counted as a websocket). It just makes the entire process cumbersome!!!

With your updations, please simplify our lives, dont complicate it.

sujith

@Bhavninder,
I am afraid you can't use our Websocket data to display live market data on a third party website. You may need to check with compliance first. You can write to kiteconnect(at)zerodha.com to reach out to the compliance team.

Bhavninder

We pay Rs4000/- per month for websocket & historical data. These sudden changes disrupts our entire system. As three webpages(websockets) are not sufficient to deploy any algos.

Bhavninder

@sujith we are not displaying any data for third party, but for internal trading.

sujith

@Bhavninder, You need to use one websocket connection unless you are subscribing for more than 4k instruments.

Edit: Current limit is 3k, the new limit is 4k

prakash.jj

Hmm. If we are running multiple algos how are we supposed to use more than 3 algos with limited websockets. If we try to store the ticks to a database and then use from multiple algos then it will unnecessarily add the delay which defeats the whole purpose. Can you suggest a better way to manage without introducing delay. @sujith

sbdavra

One of the important question i have

how can i know my number of web-socket connection running to zerodha for particular key,
if we know this than i think this may help us to overcome this issue

so requesting you to answer on this please

thanks

sujith

@prakash.jj we have always recommended users to use a maximum of 3 websocket connections per api_key.
We didn't have a hard limit till now for legacy reasons, we are going the put the hard limit now. If you need to open more websocket connections then you will have to subscribe to another Kite Connect app.

sbdavra

@sujith
please reply on my question also

how can i know my number of web-socket connection running to zerodha for particular api_key,

i know that i can know it from my servers , but as zerodha discarding connection it means there is something your internal logic to count no of web socket connection.

so is there any api or process to know number of web socket connections running as of time for particular api_key?

please answer on this ..

thanks

sujith

@sbdavra,
We don't have an API to check the number of connections open at the Kite Ticker.
I will forward your query to the concerned team and check if they can do something about it.

prakash.jj

@sujith what is the new date for enforcing these restrictions now? I will have to find some other solution.

sujith

@prakash.jj
We may go live this weekend again.

prakash.jj

Ok. thats too early to design new solution.

sujith

We had announced it here.
It was done at the end of August.

sbdavra

@sujith
but before that kindly provide anything to us for checking number of web socket connections for particular api_key running with zerodha on particular time.

so that we can check it from our end also.

thanks

sujith

We have informed the team to take a look at this.

prakash.jj

@sujith please send communication through emails for any such big changes. It will be helpful. We may not always see the forum.

sujith

We did send an email at the end of August.

alpha_mohu

Websocket connections are disconnected every now and then. Unable to get streaming data. What is causing this issue? What should we do? Almost half a day is wasted.
Just for your info, I am just using 1 websocket connection.

sbdavra

@sujith
i am 100% agreed with alpha_mohu

since 2 days we facing this issue and team is not replying
don't know what to do
why zerodha team is not responding on this point ?..

alpha_mohu

Did anyone got any solution for frequent disconnections in websockets?
@sbdavra were you able to get any solution.

alpha_mohu

@sujith is there any updates on this forum. Am I just knocking a wall instead of door?

rakeshr

Should be fine since yesterday. We did few fixes yesterday around early trading hours as we got to know about the issue. And we logged WebSocket data for the whole of yesterday post that. Haven't encountered any Server-side error post that.
What's the exact error? Can you paste the complete log here? Also, what's the Websocket client are you using?

sbdavra

@rakeshr
@alpha_mohu
yes rakesh , today i did not faced this onDisconnected issue which i faced on monday and tuesday

swas99

Still facing the issue:

Error: The status code of the opening handshake response is not '101 Switching Protocols'. The status line is: HTTP/1.1 429 Too Many Requests
com.neovisionaries.ws.client.OpeningHandshakeException: The status code of the opening handshake response is not '101 Switching Protocols'. The status line is: HTTP/1.1 429 Too Many Requests
at com.neovisionaries.ws.client.HandshakeReader.validateStatusLine(HandshakeReader.java:232)
at com.neovisionaries.ws.client.HandshakeReader.readHandshake(HandshakeReader.java:54)
at com.neovisionaries.ws.client.WebSocket.readHandshake(WebSocket.java:3244)
at com.neovisionaries.ws.client.WebSocket.shakeHands(WebSocket.java:3123)
at com.neovisionaries.ws.client.WebSocket.connect(WebSocket.java:2155)
at com.zerodhatech.ticker.KiteTicker.connect(KiteTicker.java:214)
at com.zerodhatech.ticker.KiteTicker.reconnect(KiteTicker.java:646)
at com.zerodhatech.ticker.KiteTicker.doReconnect(KiteTicker.java:128)
at com.zerodhatech.ticker.KiteTicker$1.run(KiteTicker.java:105)
at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
at java.base/java.util.TimerThread.run(Timer.java:506)

Why don't you just rollback to a stable state & release to prod when the issue is actually fixed.

Note: I have 2 API-keys. The WebSocket on one was relatively stable.

sujith

@swas99,
It means you have opened too many connections. You need to close some of them.

swas99

@sujith: I am following :

"You can subscribe up to 3000 instruments/shares in a single WebSocket connection, and for one user you can have 3 WebSocket connection, so you can stream maximum of 9000 shares for one user. because all WebSockets will be independent of each other, there will be no cross-connection."

From : https://kite.trade/forum/discussion/4732/frequently-asked-questions-faqs#latest

Has this changed?
It was working till last week

UPDATE: The websocket throwing this was subscribed to just 8 instruments, across 3 web-sockets

UPDATE 2: Happened again right now within 2-3 mins of initialising the web-socket at 2:14PM IST.
Also, the problem seems to be more persistent for the API key I bought last weekend.

sujith

Yes, we allow 4000 instruments now per connection.

swas99

Amazing how you make breaking changes without informing the customers. I thought you were saying max limit is 4k.

Irrespective, the issue still persists with 8 instruments across 3 sockets.

Food for thought: It would be so much better if customers get a stable system with a lesser capacity vs a large capacity that does not work.

sujith

@swas99,
You can check out the announcement here.
It was done at the end of August.

sujith

@swas99,
You may have to stop all the scripts and try again. Maybe your previous connections are not closed properly.

swas99

@sujith : I have been doing that since Yesterday. Results are the same:

com.neovisionaries.ws.client.OpeningHandshakeException: The status code of the opening handshake response is not '101 Switching Protocols'. The status line is: HTTP/1.1 429 Too Many Requests
at com.neovisionaries.ws.client.HandshakeReader.validateStatusLine(HandshakeReader.java:232)
at com.neovisionaries.ws.client.HandshakeReader.readHandshake(HandshakeReader.java:54)
at com.neovisionaries.ws.client.WebSocket.readHandshake(WebSocket.java:3244)
at com.neovisionaries.ws.client.WebSocket.shakeHands(WebSocket.java:3123)
at com.neovisionaries.ws.client.WebSocket.connect(WebSocket.java:2155)
at com.zerodhatech.ticker.KiteTicker.connect(KiteTicker.java:214)
at auto_trade.ticker_websocket.TickerWS.init(TickerWS.java:274)
at data_collector.DataCollector.initScripFilter(DataCollector.java:179)
at data_collector.DataCollector.lambda$initDataCollector$0(DataCollector.java:77)
at java.base/java.lang.Thread.run(Thread.java:832)
[WS_1]: Error: The status code of the opening handshake response is not '101 Switching Protocols'. The status line is: HTTP/1.1 429 Too Many Requests

I have 2 API-keys & interestingly one seems to be lot more stable today. (but even that was down for few hours which defeats the purpose of using them)

swas99

And the issue continues for straight 3 days.

Looks like there is no hope.

rakeshr

As we have informed above, Websocket Streaming is working fine now within a prescribed limit of max 3 connection/ each having max 4k instruments. We too are investigating very closely on this and haven't encountered any issue post last fix, same being said here. Others members have confirmed it working too.
It mostly seems to be your unknown spawn/active Websocket connection exceeding max allowed connection i.e 3. You need to debug and track down on this. Below pointers might help:
1. Close all the active/running WebSocket connection
2. Paste your stack trace here if any previous connection closing was not clean
2. Initiate a single connection at a time and check if it's still throwing 429
3. Keep spawning/increasing your connection uptil desired within the prescribed limit and have a check

swas99

As I said earlier: There is nothing wrong with the code I am using.
Same code was working fine till last week & is behaving differently with different API keys now.

It is possible that the ones who don't face the issue anymore are the fortunate ones with the working API key group.

There are many who continue to face the issue:
https://kite.trade/forum/discussion/8692/websocket-api-issue#latesthttps://kite.trade/forum/discussion/8692/websocket-api-issue#latest
https://kite.trade/forum/discussion/8677/disconnection-message-got-from-your-end-today-lot-of-times#latest
https://kite.trade/forum/discussion/8688/why-we-are-getting-disconnection-from-your-end#latest
https://kite.trade/forum/discussion/8680/not-able-stream-quotes-using-websocket#latest
https://kite.trade/forum/discussion/8685/websocket-streaming-is-breaking-multiple-times#latest

XS8910

2020-10-16 09:51:58,766 - ERROR - zerodha_util.py:get_positions():258- Too many requests
2020-10-16 09:51:58,766 - INFO - zerodha_util.py:get_positions():259-

rakeshr

Same code was working fine till last week & is behaving differently with different API keys now.

It was working before because, we didn't have a hard limit of max 3 connections per API key, but now we have imposed a hard limit of max 3 connections.

It is possible that the ones who don't face the issue anymore are the fortunate ones with the working API key group.

We don't have anything like this i.e priority API channel,etc

https://kite.trade/forum/discussion/8692/websocket-api-issue#latesthttps://kite.trade/forum/discussion/8692/websocket-api-issue#latest
https://kite.trade/forum/discussion/8677/disconnection-message-got-from-your-end-today-lot-of-times#latest
https://kite.trade/forum/discussion/8688/why-we-are-getting-disconnection-from-your-end#latest
https://kite.trade/forum/discussion/8680/not-able-stream-quotes-using-websocket#latest
https://kite.trade/forum/discussion/8685/websocket-streaming-is-breaking-multiple-times#latest

Most of these queries are old i.e before the fix. You too can ask respective members on the same thread.

You need to debug and track down on this. Below pointers might help

Give this once a try? Might be useful.

swas99

I take back my words. Issue was indeed with my code.

while(true)
{
         if(!dataCollectorActive && isTradingHours())
                initScripFilter();
         else if(dataCollectorActive) //instead of else if(dataCollectorActive && !isTradingHours())
                disConnectScripFilter();

         Thread.sleep(ONE_MIN);
}

Even with the bug only 3 sockets were active at any time but more than 3 ticker-objects were initiated.

I deployed this change on Monday & you guys would have enforced the hard-limit after that hence it was working on Monday.

Anyway, thanks for looking into it.

swas99

` 429 Too Many Requests ` seems to be resolved for me with the above fix but the other issue of "Websocket Not Receiving Updates After a while" still persists.

This is also something which didn't happen too often till this week.

alpha_mohu

Am not sure whether zerodha guys are listening to this conversation or not. I have paid subscription and just using only one websocket connection. Still it abruptly stops in middle saying too many connections open. I am just fetching data for just 60 instruments and no other APIs am using now so far.
This started from last week onwards and in mail, they told me to post message in this forum, where no-one responds.

sujith

@alpha_mohu,
Can you private message your api_key? We will check and get back to you.

diwakerkla

@sujith @rakeshr : i am calling your getProfile and tickerUsage from test class(Example, java), success came for getProfile but for tickerUsage it came up with below message, whats worng here, can you check please

for ticker : tokens.add(Long.parseLong("256265"));

2020-11-07 07:27:26.135 INFO 14624 --- [nio-8082-exec-1] okhttp3.OkHttpClient :
2020-11-07 07:27:26.135 INFO 14624 --- [nio-8082-exec-1] okhttp3.OkHttpClient : {"status":"success","data":true}
2020-11-07 07:27:26.135 INFO 14624 --- [nio-8082-exec-1] okhttp3.OkHttpClient : <-- END HTTP (32-byte body)
{"data":true,"status":"success"}
com.neovisionaries.ws.client.OpeningHandshakeException: The status code of the opening handshake response is not '101 Switching Protocols'. The status line is: HTTP/1.1 403 Forbidden
at com.neovisionaries.ws.client.HandshakeReader.validateStatusLine(HandshakeReader.java:232)
at com.neovisionaries.ws.client.HandshakeReader.readHandshake(HandshakeReader.java:54)
at com.neovisionaries.ws.client.WebSocket.readHandshake(WebSocket.java:3244)
at com.neovisionaries.ws.client.WebSocket.shakeHands(WebSocket.java:3123)
at com.neovisionaries.ws.client.WebSocket.connect(WebSocket.java:2155)

diwakerkla

@sujith @rakeshr : connected, and above error is not coming, but on calling tickerUsage, flow did not go inside setOnTickerArrivalListener, so its not priting anything from that method, can you tell why it didnot go inside setOnTickerArrivalListener.

diwakerkla

@sujith @rakeshr
why ticks size 0, i have added for ticker : tokens.add(Long.parseLong("256265"));

diwakerkla

@sujith @rakeshr only first time it showed data after that ticker size always 0, can you check please why this is so

ticks size 1
last price 12263.55
open interest 0
day high OI 0
day low OI 0
change 143.25
tick timestamp Fri Nov 06 18:29:21 IST 2020
tick timestamp date Fri Nov 06 18:29:21 IST 2020
last traded time null
ticks size 0
ticks size 0
ticks size 0
ticks size 0
ticks size 0
ticks size 0

rakeshr

only first time it showed data after that ticker size always 0, can you check please why this is so

Today is a non-trading day. So, you get only one tick(while connection establishment) i.e last tick.