unable to download data - timeout all the time

AD6499

hi

i am using kite.historical to download 15 min candles using python, but i am unable to do so.
i keep continuously getting timeout errors. why is reliability so poor for paying customers? what is the way around this?

iteconnect.exceptions.DataException: Unknown Content-Type (text/html) with response: (b'\r\n504 Gateway Time-out\r\n\r\n

504 Gateway Time-out

\r\n

---

nginx

\r\n\r\n\r\n')

rakeshr

Are you trying to fetch continuous data for the intraday candle period?
Can you paste your complete request param for historical data fetch?

AD6499

I am. I am looking to get 15minute candle data going back to 2016 in 3 month periods.
This is essentially the command I am running in a loop

kite.historical_data(128028676, '2016-1-1', '2016-3-31', '15minute, 0, 1)

What am i doing wrong here?

AD6499

@rakeshr appreciate your help

rakeshr

kite.historical_data(128028676, '2016-1-1', '2016-3-31', '15minute, 0, 1)

You are requesting oi=1 for SBIN-EQ symbol. OI value is available only for the F&O contract. You can check all historical data API params here.

AD6499

thanks @rakeshr
it still doesn't work even if i change oi = 0
How far back can I go for EQ BSE and NSE tickers? Is the date that I am trying to download from (2016-1-1) the problem?

rakeshr

I just tried the same request and it's working fine. Actually, we have BSE intra-day data from 2016, 3, 18 onwards. But, still it should show data from 2016, 3, 18 onward inside of time-out error.
kite.historical_data(128028676, '2016-01-01', '2016-03-31', "15minute")
[{'date': datetime.datetime(2016, 3, 18, 9, 15, tzinfo=tzoffset(None, 19800)), 'open': 186.5,
'high': 187.4, 'low': 186.5, 'close': 186.9, 'volume': 116703}, ....

How far back can I go for EQ BSE and NSE tickers? Is the date that I am trying to download from (2016-1-1) the problem?

You can go through historical data FAQs here.

it still doesn't work even if i change oi = 0

Can you paste the complete debug log here for the above request?

AD6499

it works sporadically for me
e.g. kite.historical_data(136453636,'2016-01-01', '2016-03-31', "15minute")

This is the exception
Traceback (most recent call last):
File "/kite_all_code.py", line 112, in kite_download_minute_data_loop
cash_data1 = pd.DataFrame(kite.historical_data(token, start_date, end_date, time_period, 0, 0))
File "/python3.8/site-packages/kiteconnect/connect.py", line 621, in historical_data
data = self._get("market.historical",
File "/python3.8/site-packages/kiteconnect/connect.py", line 836, in _get
return self._request(route, "GET", url_args=url_args, params=params, is_json=is_json)
File "/python3.8/site-packages/kiteconnect/connect.py", line 918, in _request
raise ex.DataException("Unknown Content-Type ({content_type}) with response: ({content})".format(
kiteconnect.exceptions.DataException: Unknown Content-Type (text/html) with response: (b'\r\n504 Gateway Time-out\r\n\r\n

504 Gateway Time-out

\r\n

---

nginx

\r\n\r\n\r\n')

AD6499

" raise ex.DataException("Unknown Content-Type ({content_type}) with response: ({content})".format(
kiteconnect.exceptions.DataException: Unknown Content-Type (text/html) with response: (b'\r\n504 Gateway Time-out\r\n\r\n

504 Gateway Time-out

\r\n

---

nginx

\r\n\r\n\r\n') "

AD6499

i also feel like I am being throttled. It times out after roughly every 10th request.
There is no way I am making more than 3 request/second. Am I hitting some other limit?

rakeshr

This thread explains more on the above BSE scrip related time-out for historical data.

AD6499

this makes no sense. i have random delays between 10 and 30 seconds after every exception. i still can't get this to work reliably for more than few minutes.
is this the kind of service you offer paying customers?

AD6499

no response? @rakeshr @sujith

sujith

@AD6499,
You need to fetch candles and dump it at your end and use that as source for backtesting.
For live market strategies, we suggest you to use generate candles at your end using the live market data provided on the Websockets API. You can get started here.

rakeshr

Also, you can try out other authorized data vendor, as per your requirement for historical data.

AD6499

@sujith that is what i am trying to do, but just not able to download data consistently
@rakeshr don't worry, i will be taking my business elsewhere. i was told that zerodha is a world-class product. how i wish that was even remotely true