I am getting following Error while featching historical data in loop, after few iterations.
instruments_list = kite.instruments(exchange=kite.EXCHANGE_NSE) File "/usr/local/lib/python3.6/site-packages/kiteconnect/connect.py", line 550, in instruments return self._parse_instruments(self._get("market.instruments", params)) File "/usr/local/lib/python3.6/site-packages/kiteconnect/connect.py", line 817, in _get return self._request(route, "GET", params) File "/usr/local/lib/python3.6/site-packages/kiteconnect/connect.py", line 894, in _request content=r.content)) kiteconnect.exceptions.DataException: Unknown Content-Type (text/html) with response: (b'\r\n502 Bad Gateway\r\n\r\n
Hi, I am facing probably the same issue. Let me explain.
I am trying to fetch '3minute' data with for loops (100 rows at a time) Here is what I am doing. ------------------------------------------------------------ for stock in stocks: fromDate = d1 toDate = d2 # such that, d2-d1 = 100 days
# trying to fetch data for this particular stock('3minute'data, 20 rows at a time), 200 times # one may put any number here depending upon how back do you want to go. for row in range(0,200): data = kite.historical_data(token, fromDate, toDate, interval='3minute') df = #append "data" to a dataframe del data toDate = fromDate fromDate = toDate-100days
# now df has all the historical (available data) for given stock. # save df to a file. ------------------------------------------------------------
after fetching around 8000 rows (in total) it throws following error DataException: Unknown Content-Type (text/html) with response: 504 Gateway Time-out ....
This corresponds to some date in 10/2018.
I tried to run for this date range separately and found out that it runs successfully a few times and trows error the other times. It seems random to me. Just to clarify, I have tried this over several days and during different times of a day. The behaviour is the same.
Kindly help me out. Please let me know in case of any ambiguity.
I tried to run for this date range separately and found out that it runs successfully a few times and throws an error the other times. It seems random to me.
Is this throwing 504 error, for this contract specific for the mentioned period or for any contract? Do let us know, the instrument token and to and from period?
for most of the scripts (nifty-50 e.g. tokens = 136427780 , 128209924, 136247044), it throws this error for date ranges around from 08-2018 to 11-2018 (it's random though).
One more weird thing happened when I was trying less number of days (looped one). Here it is. token=136316932 days=20 (limit is 100, but more the days, earlier it breaks) interval='3minutes'
I did 2 such iterations
in the first iteration, I stopped getting any data (no error though) for dates prior to 2018-02-20. (Though it later threw an error for dates prior to 2017-11-12)
In the second iteration, I got no data(no error too) from 2017-12-07 to 2018-01-16 but I got data for dates before and after those. eventually it threw the same error for dates prior to 2017-09-18 05.
Please let me know if you need such information in a specific form or something. I'll be glad to be explain it in a way you'd like.
We just checked on this, it might happen for few BSE scrip older minute data. You need to handle this Time-out at your end and make the request again. Something like: from kiteconnect import exceptions
try: # Fetch historical data except exceptions.DataException: # Handle Dataexception and make request again with some delay/sleep
But I should point out that I am looking at only NSE scripts.
for most of the scripts (nifty-50 e.g. tokens = 136427780 , 128209924, 136247044),
Above tokens, which you have mentioned is of BSE exchange. You can refer the same under complete instrument list.
136427780,532921,ADANIPORTS,"ADANI PORTS AND SPECIAL ECONOM",0,,0,0.05,1,EQ,BSE,BSE 128209924,500820,ASIANPAINT,"ASIAN PAINTS.",0,,0,0.05,1,EQ,BSE,BSE 136247044,532215,AXISBANK,"AXIS BANK.",0,,0,0.05,1,EQ,BSE,BSE
Should this still be a problem?
You can use any exchange as per your requirement, shouldn't be a problem.
I am facing probably the same issue. Let me explain.
I am trying to fetch '3minute' data with for loops (100 rows at a time)
Here is what I am doing.
------------------------------------------------------------
for stock in stocks:
fromDate = d1
toDate = d2 # such that, d2-d1 = 100 days
# trying to fetch data for this particular stock('3minute'data, 20 rows at a time), 200 times
# one may put any number here depending upon how back do you want to go.
for row in range(0,200):
data = kite.historical_data(token, fromDate, toDate, interval='3minute')
df = #append "data" to a dataframe
del data
toDate = fromDate
fromDate = toDate-100days
# now df has all the historical (available data) for given stock.
# save df to a file.
------------------------------------------------------------
after fetching around 8000 rows (in total) it throws following error
DataException: Unknown Content-Type (text/html) with response: 504 Gateway Time-out ....
This corresponds to some date in 10/2018.
I tried to run for this date range separately and found out that it runs successfully a few times and trows error the other times. It seems random to me.
Just to clarify, I have tried this over several days and during different times of a day. The behaviour is the same.
Kindly help me out. Please let me know in case of any ambiguity.
Do let us know, the instrument token and to and from period?
DataException: Unknown Content-Type (text/html) with response: (b'<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body bgcolor="white">\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n')for most of the scripts (nifty-50 e.g. tokens = 136427780 , 128209924, 136247044), it throws this error for date ranges around from 08-2018 to 11-2018 (it's random though).
One more weird thing happened when I was trying less number of days (looped one). Here it is.
token=136316932
days=20 (limit is 100, but more the days, earlier it breaks)
interval='3minutes'
I did 2 such iterations
in the first iteration, I stopped getting any data (no error though) for dates prior to 2018-02-20. (Though it later threw an error for dates prior to 2017-11-12)
In the second iteration,
I got no data(no error too) from 2017-12-07 to 2018-01-16 but I got data for dates before and after those.
eventually it threw the same error for dates prior to 2017-09-18 05.
Please let me know if you need such information in a specific form or something. I'll be glad to be explain it in a way you'd like.
from kiteconnect import exceptions
I'll try this.
But I should point out that I am looking at only NSE scripts.
Should this still be a problem ?