Can we get full subscribes update list in one shot in tick?

nameisbk

Hi,

I'm facing yet another issue with ticker tick where I subscribe for 200+ tokens and get a much-delayed response as tick will give me single token details when data is updated but is it possible to get it the full response of all tokens in the list rather than single token entry?

My test:
subscribes ------- Response time
1-100 ------------ Gives fast and match with Zerodha portal
1-200 ------------ delayed for 2-6 sec as compare to Zerodha portal
>200 ---------- 10-20 sec delayed then Zerodha portal

I did notice that in the Zerodha web portal WebSocket response is containing all the tokens updated details which I have a subscribed in portal whereas in ticker I'll get it single so it will put much more load to the system when we subscribe more than 200 tokens at the same time.

your comment would be appreciated.

Thanks

trade_then

could you post a few lines of your code here please.
what are you doing in your OnTick

what is you computer hardware :-

• RAM
  
• Processor
  
• Disk capacity
  

if you are on Window 10 right-click WindowButton then click System.

apart from what one might conclude, after one can figure out
what is going on at you end. there are a few considerations to keep in mind

as you keep on increasing your subscribed tokens. number of less liquid Instruments
are also increasing in your list. which means Instruments that are hyper-active on which
trades are happening very fast or frequently like ICICIBANK or NIFTY etc etc. are decreasing.
not all instruments are active all the time.

Which can cause delay in ticker data. for those instruments.

The two points of connection

1. where you are recieving data. ( OnTick ).
   
2. where you are consuming
   • NamedPipe
     
   • Memory Mapped Files
     
   • Websocket
     
   • TCP
     

are these isolated into threads/Tasks



Thanks
Regards

nameisbk

Hi @trade_then

Below are the code picture with InitializeTicker and OnTick method in which I use a separate thread to sent data to UI without blocking the current thread. I also post another pic with system configuration with SSD hard drive of 256GB. Using 50Mbps internet speed plan.

Is it fine or do I need to make any change in terms of code to get it faster?

trade_then

So it is a laptop.
we will come to the software part in a bit.

First let us adjust hardware:

• have you enabled Turboboost / hyperthreading on it. i5-8250u has that
  
• and set power settings of your laptop to that of High performance.
  
• ensure using electric/charging wire plugged in and not battery while crunching Ticks.
  

Thanks
Regards

nameisbk

Hi @trade_then

I adjusted all the hardware points that you suggested. What's next?

trade_then

Next comes an apology.
But first, was it a laptop or a 2-in-1 desktop!?.

because on daily bases i dont subscribe to more than 50 ticks. it has been long since i last
subscribed to 200 ticks. back then i never noticed any such delay.

In your above pictures you have hidden you computers local_name but not your

• Product-ID
  
• Device-ID
  

kindly avoid its disclosure too.

Now then, let us investigate!.

Did you notice any difference at your end after adjusting performance settings?

See in your above or 1st posts you try to compare with KiteWeb.
stating that 180+ tickers are upadated properly on that platform but not on your platform.
This could be misleading Kiteweb never keeps that many subscriptions open all the time.

• it keeps subscribing and unsubscribing as and when required.
  
• it is very conservative actually.
  

Only scrips in visible Marketwatch are subscribed and that too in LTP mode.

when you open up 5-depth or 20-depth widget, does it switch to FULL or Full20 mode and as soon as you close the Depth widget it changes the subscription mode back to either LTP or Quote depending on situation.

So FULL Mode only happens in KiteWeb when you are actually viewing the 5-Depth widget.
otherwise it is LTP or some times Quote mode. all through websocket.

Just like OnData in your Code. KiteWeb is parsing data in the background as it recieves.
in parseBinary(e) at Line 378.

Now let us move forward to what might be causing the delay at your end.

• perphaps something might have gone wrong at zerodhas end. ( we cant do anything there ).
  
• internet speed. ( you say it is 100mbps ) .
  
• how your hardware handles processing . ( you have tuned it to high performance ).
  
• software ( let us see ).
  

Althou it is said that avoid blocking OnTick so that websocket
is not blocked to recieve new ticks. This is a bit misleading actually. because there is something
else which is always coming in the way of Websocket delivery even before OnTick and that is OnData.

It is hidden from the usual user. the data that you recieve in OnTick is not coming directly from Kite tick server or Zerodha as it might appear to be. but actually it is coming to you from inside of your own KiteConnect.dll. after incomming websocket message has been parsed by OnData. Actually Full, LTP, Quote Mode. values are also inserted second hand after comprehending number of bytes in a message. KiteServer is not declaring its intent while sending its data, intent is understood as a compromise from the count of bytes that that was the intention.

It is for this reason sometimes you see that when you have subscribed to FULL mode you are actually recieving LTP mode ticks specially in the morning. Because KiteServer is throwing data at you depending upon its own logic about the available data.

I will continue my useless rant in next post because formatting current post is becoming difficult on account of its length.

Thanks
Regards

nameisbk

Hi @trade_then ,

Thanks for detailed explanation.

Yes I'm using laptop only. I did notice slite difference in receiving tick after making Hardware config changes.

That you explained how OnDate reached is good but I can't do anything with that right as every time I also need plain data rather than binary data as I'm also showing that in my portal.

I'm a distributor for trading/live market depth to all my users which is almost 500/1000 now and increasing so let's assume 1 user wants to see 50 tokens live data where all other also have their preference so at the end it becomes more than 900+ unique token I get in the server where I have to process them and ask Kit to give all updates on each of this tokens in tick. so at this point, it becomes slow and users seeing data is not updating rapidly as the market showing them as in the backend server gets ticks late to sent it overall connected clients.

I'm using latest .net core and latest SignalR (WebSocket) but still it seems I'm not getting data as much as quick I want.

As you said Kit portal only subscribe with full mode whom I have a select market depth which is fine but I want that my client see Buy, Sell, LastUpdated... everything which I got in full mode but as I said it becomes slow and not updating data as per marked live trad.

Let me know your thoughts

trade_then

well don`t loose hope just yet. I have not finished my rant just paused for a while.

Thanks
Regards

nameisbk

@trade_then I don't but I'm struggling with this since last two weeks and users are chasing me very badly then.

trade_then

So let us first determine that delay is because of OnData as the subscription grows above 100. or
because of some thing else.

for that add Stopwatch here like.

Stopwatch ondataTiming = new Stopwatch();

then here
like

 ondataTiming?.Restart() ;

and here
like

 ondataTiming?.Stop() ;
TimeSpan ts = stopWatch.Elapsed;
string elapsedTime = String.Format("{0:00}:{1:00}:{2:00}.{3:00}",  ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10) ;
Console.WriteLine($"OnData took { elapsedTime } time" ) ;

try it with different number of subscriptions.
and see how much difference does occur from less to more tick subscriptions.

Thanks
Regards

trade_then

well by the time again with my rant.

Apart from various things that can be done about it.
when you are saying that your primary goal is to distribute the incoming tick to different terminals. That means you are trying to be just another server-node. then why do you want to parse
data on your computer.

it would be best that parsing of data happens at the terminal where data is being sent.
you just push the data at the respective terminal by intercepting OnData before it converts binary to Tick. just like Zerodha is sending it to you, you pass it forward to the respective terminal.

On those terminal use can pass the same library for parsing the data. that is doing it for you.
ofcourse library you are posting on those terminals can be much smaller because it only needs to capture incomming tick via websocket and then parse. I think you can do it by refactoring the Kiteconnect.dll at your end.

and being a server node by being a laptop is not a good idea. because if your system was strong.
parsing of incoming data could have been performed by employing multithreading. or if you had
multiple computers at your place they could have been employed to parse certain amount of data.

ofcourse for that Kiteconnect.dll would have to be readjusted.
which is actually very easy to do.

I would have a lot more to say about intercepting and parsing later.
depending on what you want to do.

Thanks
Regards

sujith

@nameisbk,
I am afraid you can't use our data to show on your website or app. It will be a violation of the regulations. You can write to compliance at kiteconnect(at)zerodha.com.