Getting System.Collections.Generic.KeyNotFoundException while calling kite.GetHistorical method.

Som

While calling kite.GetHistorical() method, sometimes it throws System.Collections.Generic.KeyNotFoundException exception. It will give the results sometimes and sometimes it will throw the exception for the exact same parameter. Also, I am pausing between the method calls so that I won't call the method more than twice in one minute. Stacktrace below-

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at CallSite.Target(Closure , CallSite , Object , String )
at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)
at KiteConnect.Kite.GetHistorical(String InstrumentToken, String FromDate, String ToDate, String Interval, Boolean Continuous)

tonystark

Hi @Som,

Can you give me more details like the parameters you gave, the key which you are trying to access from the historical data etc. so that we can try to reproduce it on our end?

Som

Please find the code-

_kite = new Kite({ApiKey}, {AccessToken});
_kite.GetHistorical("JPASSOCIAT", "2015-01-01", "2015-01-21", "minute");

The above code sometimes works perfectly and sometime it will throw the above-mentioned exception. For example, if I run the _kite.GetHistorical() 10 times, maybe 5 times it will return me the results and another 5 times throw me KeyNotFoundException.

sujith

Hi @Som,
You need to pass instrument token to get historical data. You can check out documentation for more information.

Som

Yeah Sujith, sorry my bad, copied wrong code. Please find the correct code below-

_kite = new Kite({ApiKey}, {AccessToken});
_kite.GetHistorical("2933761", "2015-01-01", "2015-01-21", "minute");

The above code sometimes works perfectly and sometime it will throw the above-mentioned exception. For example, if I run the _kite.GetHistorical() 10 times, maybe 5 times it will return me the results and another 5 times throw me KeyNotFoundException.

tonystark

Hi @Som,

We are getting that error only if we send trading symbol instead of instrument token. When we send proper data we are getting data correctly every time. Can you try enabling the debug mode like:

Kite kite = new Kite(MyAPIKey, Debug: true);

This will log all the requests which will help to debug.

Som

Thanks for the reply but this is not happening in my case. I am getting the error with IntrumentToken also. Please keep this thread open, I will post more data.

As of now, I can figure out that, there is some caching issue from your side. For some of the symbols, for the first requests I am getting the above KeyNotFundException but if I request again after 30-60 seconds of the initial call with the exact same parameters, I am able t get back the data. So I think, for the first request your server-side code doesn't find the data and KeyNotFundException is thrown but the data is being pulled in the meantime to cache. Hence the second request passes. Please have a look.

abhidutta86

I have paid 2000 bucks for Historical data and it never works on my .NET API client. I even tried running the Kite Connect Sample and got the same issue.

Som

Correct, again it is not only 2000, it is 2000+2000=4000. Bad to know that you are also facing the same issue. But at least now they can not say that it is happening only with me and it is working fine for them.

abhidutta86

I have seen this issue come up on a daily basis during Market Hours.It's absolutely useless if it works only offMarket hours. I wanted the last 15 Min Candle during the market hours, but API never returns it and i get an error. Also the Continuous=True overload does nothing as well. ( Yet to see it work without errors)

The only alternative i see to ping the Ticker on 15 Min timer and get the data but Today it seems that even the Ticker is not working is throwing a generic ERROR with message :
Error while connecting. Message: One or more errors occurred.

tonystark

As I mentioned earlier you have to give more information to debug the issue. We tried to reproduce the issue on our end and we couldn't find any.

Kindly try initializing your kite instance like this which will print more information to console on what is happening.

Kite kite = new Kite(MyAPIKey, Debug: true);

Som

HI, I did run the app with "Debug: true" and also did some research from my end. Finally able to figure out how your server end works. From my understanding, it is like below-

1. For some of the popular/high demand symbols, data is pre-cached. So the call to kite.GetHistorical will return immediately with data.
2. But for other not so popular/less demand symbols, data is not cached. So the first call will almost always fail with KeyNotFoundException, if the second call is made after 30/40 seconds then it will pass and able to get the data back.
3. My best guess is, in the first call the data is getting populated in the cache but the exception is thrown because of some server timeout. If the second call is made after 30/40 seconds, it finds the data in cache and return succes with data.

Now some code to prove this hypothesis -

        static void Main(string[] args)
        {
            Kite kite = new Kite("{ApiKey}", "{AccessToken}", Debug: true);
            Thread.Sleep(1000);
            try
            {
                var data = kite.GetHistorical("2933761", "2015-01-01", "2015-01-21", "minute");
                Console.WriteLine("Success:{0}", data.Count);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
                Console.WriteLine("Waiting for 1 minute.");
                Thread.Sleep(1000 * 60);
                try
                {
                    var data = kite.GetHistorical("2933761", "2015-01-01", "2015-01-21", "minute");
                    Console.WriteLine("Success:{0}", data.Count);
                }
                catch (Exception ex1)
                {
                    Console.WriteLine(ex1.ToString());
                }
            }
        }

Console output of the above code-

DEBUG: GET https://api.kite.trade/instruments/historical/2933761/minute?access_token={AccessToken}&from=2015-01-01&to=2015-01-21&api_key={ApiKey}&continuous=0

System.Net.WebException: The operation has timed out
at KiteConnect.Kite.Request(String Route, String Method, Dictionary`2 Params)
at KiteConnect.Kite.GetHistorical(String InstrumentToken, String FromDate, String ToDate, String Interval, Boolean Continuous)
at KiteTest.Program.Main(String[] args) in E:\GitRepo\Working\KiteTest\KiteTest\Program.cs:line 19
Waiting for 1 minute.
DEBUG: GET https://api.kite.trade/instruments/historical/2933761/minute?access_token={AccessToken}&from=2015-01-01&to=2015-01-21&api_key={ApiKey}&continuous=0

DEBUG: 200 {"status": "success", "data": {"candles":
[["2015-01-01T09:15:00+0530",25.05,25.2,24.9,25,134370],
["2015-01-01T09:16:00+0530",25,25.05,24.95,25,93293],
["2015-01-01T09:17:00+0530",25.05,25.05,24.95,25,55037],
["2015-01-01T09:18:00+0530",25,25.15,25,25.15,73008],
["2015-01-01T09:19:00+0530",25.15,25.15,25.05,25.1,89134],
["2015-01-01T09:20:00+0530",25.1,25.1,25,25,69415]
.
.{Removed the lines to keep the post small}
.
["2015-01-21T15:28:00+0530",25.6,25.6,25.5,25.55,492429],
["2015-01-21T15:29:00+0530",25.55,25.6,25.5,25.55,149984]]}}

Success:5611

Now you can see from the above code execution that the first request failed and the second request which was made after 1 minute with the exact same parameters, passed.

To handle the timeout issue, I have set the "Timeout:" parameter of Kite contructor like below-

Kite kite = new Kite("{ApiKey}", "{AccessToken}", Debug: true, Timeout:100000);

With the above code change, I am getting the Console output like below-

DEBUG: GET https://api.kite.trade/instruments/historical/2933761/minute?access_token={AccessToken}&from=2015-01-01&to=2015-01-21&api_key={ApiKey}&continuous=0

DEBUG: 504 {"status": "error", "message": "Gateway timed out", "error_type": "NetworkException"}

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at CallSite.Target(Closure , CallSite , Object , String )
at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)
at KiteConnect.Kite.GetHistorical(String InstrumentToken, String FromDate, String ToDate, String Interval, Boolean Continuous)
at KiteTest.Program.Main(String[] args) in E:\GitRepo\Working\KiteTest\KiteTest\Program.cs:line 19
Waiting for 1 minute.
DEBUG: GET https://api.kite.trade/instruments/historical/2933761/minute?access_token={AccessToken}&from=2015-01-01&to=2015-01-21&api_key={ApiKey}&continuous=0

DEBUG: 200 {"status": "success", "data": {"candles":
[["2015-01-01T09:15:00+0530",25.05,25.2,24.9,25,134370],
["2015-01-01T09:16:00+0530",25,25.05,24.95,25,93293],
["2015-01-01T09:17:00+0530",25.05,25.05,24.95,25,55037],
["2015-01-01T09:18:00+0530",25,25.15,25,25.15,73008],
["2015-01-01T09:19:00+0530",25.15,25.15,25.05,25.1,89134],
.
. {Removed the lines to keep the post small}
.
["2015-01-01T09:33:00+0530",25.15,25.15,25.1,25.15,20481],
["2015-01-01T09:34:00+0530",25.15,25.15,25.1,25.15,29176],
["2015-01-21T15:28:00+0530",25.6,25.6,25.5,25.55,492429],
["2015-01-21T15:29:00+0530",25.55,25.6,25.5,25.55,149984]]}}

Success:5611

Here also the first request failed and the second request which is made after 1 minute with the exact same parameters, passed.
Note: I made this second test after 6/7 hours of the first test, hence the cache was removed for this symbol and I am able to regenerate the KeyNotFoundException.

With this knowledge, I made my app in such a way that, it will retry the same data after 1 minute of the initial failed request and it is somehow working. But this is just a workaround of the main bug, which needs to be fixed from your end. Please have a look.

sujith

Hi,
We have informed data team, they are looking into this.

Som

Any update? It is already 4 days.

sujith

I think requests are timing out before data is fetched from the database. For now, you can reduce the gap between from and to dates.
They are working on it.

Som

Thanks for the response Sujith, that is exactly the same thing what I already mentioned, the request is timed out before the data gets fetched from the DB to cache and it is not honoring the timeout parameter.

Definitely not expected from such a costly product.

praveenBabu

Even for single day i am getting error below error. Due to this i am unable to back test any of my strategy's.

My code:
Kite kite = new Kite(MyAPIKey, MyAccessToken, Debug: true);
List SHistory = kite.GetHistorical("738561", "2017-12-01", "2017-12-01", "15minute");

Error:- System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.

Pending this issue more than a month .