☰
›

GATEWAY TIMEDOUT ERROR WHILE COLLECTING HISTORICAL DATA

sahilsingh
sahilsingh edited September 2017 in Python client 
from datetime import timedelta,date
import time
a = date(2015,1,1)
s1 = []
for i in range(len(w)):
 
 if(i==1):
        break
 while(a<=date(2017,8,31)):
     s = pd.DataFrame(kite.historical(w.instrument_token[i],str(a),str(a+timedelta(29)),'5minute'))
     a = a+timedelta(30)
     s1.append(s)
     time.sleep(7) 
 temp = pd.concat(s1)
 j = str(w.tradingsymbol[i])+'test1.csv'
 temp.to_csv(j)
This was the following test code i wrote to snoop data of the first instrument in the instrument_token file.I also added time.sleep(7) to give an interval of 7 second between each iteration or each call(if many requests are sent within a short period of time then also the error would come) thinking i would get rid of the gateway timedout error,but no even then it is showing the error after sometime.However it works fine when i do it for a period of just 1 year.So please tell me what rules do you have regarding how many queries can be sent in a given amount of time so i can accordingly design my code.
Tagged:
  • sujith
    sujith
    Hi @sahilsingh,
    I think the request is timing out before data is being fetched from the database. Can you try with less time interval?
  • sahilsingh
    sahilsingh
    I tried with 2 seconds and 3.Still the same result.
  • sahilsingh
    sahilsingh edited September 2017
    The funny thing is that in the window of 2015 to 2016 end its working fine.But with 2017 I get the error after 10-12 seconds that's just after extracting 5 months of data.Its like it 'Knows' I wanna extract 2.5 years data.
    UPDATE:
    Seems to be working with an interval of 4 seconds atleast for now!!!
  • sahilsingh
    sahilsingh
    Now i am getting this new error even though previously it was working
    InputException: No candles found based on token and time and candleType.
  • sujith
    sujith
    Hi @sahilsingh,
    Can you give us params to reproduce this? We will check and let you know.
  • sahilsingh
    sahilsingh
    I just simply executed the code which i had already posted.I took the token names from downloading the instrument list by zerodha.
    w = pd.DataFrame(kite.instruments())
    token name was produced by w.instrument_token[i]//iterate through the first 50 instruments in the list
  • sujith
    sujith
    Hi @sahilsingh,
    It is not an error. It means candles doesn't exist for the instrument user has requested for.
    Maybe the instrument was not traded on the day you had requested for.
  • sahilsingh
    sahilsingh
    Not possible.A few day ago the same code had no problem in retrieving data.So did the data suddenly disappear?.Also btw this is happening for many other instrument lists too.
  • sujith
    sujith edited September 2017
    Hi @sahilsingh,
    There many instruments which are of T2T segments in that dump. Normally instruments are traded on EQ segment. So, if you are requesting data for T2T segments then it will not have data. There can be F&O instruments also if you are requesting data for NIFTY17SEPFUT with date 2015 then it will give you the same error.
  • sahilsingh
    sahilsingh
    Yes i know that and thats why i changed the file to contain only equities.The following problem is being experienced with equities.
  • sujith
    sujith
    Hi,
    T2T segment instruments also belong to Equity segment.
    We use the same APIs on all our platforms, it is working fine.
  • sujith
    sujith
    If the error message says no candles then it means it was not traded on the day you are requesting for.
  • sahilsingh
    sahilsingh
    Then tell me how to eliminate the T2T instruments from the file or tell me how do you identify such instruments
  • sujith
    sujith
    Hi @sahilsingh,
    Check out this thread for more information about T2T segments. Usually, tradingsymbol will have the series name like BE, BL appended to it.
    For EQ segment instruments, series is not appended.
  • strikemoney
    strikemoney
    Discover the Chart Patterns for Effective Intraday and enhance your trading strategy today
Sign In or Register to comment.