Invalid Token for accessing the data for more than one instrument

Hsadikot
Hsadikot edited November 2020 in Python client
Hi,

I was trying to connect to the zerodha Api and get the values for date', 'open', 'high', 'low', 'close', 'volume' for a no of stocks. The stock list is generated daily and is fed into the mysql database. From the database the values are stored in a pandas dataframe which is used as a list of symbols to find the values specified above. PFA the complete code.
from kiteconnect import KiteConnect
import os
import datetime as dt
import pandas as pd
import mysql.connector
from pandas import DataFrame
cwd = os.chdir("C:\\Users\\Administrator\\Desktop\\Zerodha")

#generate trading session
access_token = open("access_token.txt",'r').read()
key_secret = open("credentials.txt",'r').read().split()
kite = KiteConnect(api_key=key_secret[0])
kite.set_access_token(access_token)


#get dump of all NSE instruments
instrument_dump = kite.instruments("NSE")
instrument_df = pd.DataFrame(instrument_dump)
#instrument_df.to_csv("NSE_Instruments_31122019.csv",index=False)
# Connect to the top 100 stocks which are stored in the Database and fetch a list of them
db = mysql.connector.connect(host="localhost", user="abc", passwd="abcd")
pointer = db.cursor()
pointer.execute("use stock")
pointer.execute("select * FROM equity")
ticker = pointer.fetchall()
ticker = pd.DataFrame(ticker, columns =['SYMBOL', 'NAME OF COMPANY', 'SERIES', 'Listing_date', ' PAID UP VALUE', ' MARKET LOT', ' ISIN NUMBER', ' FACE VALUE'])
tickers = ticker["SYMBOL"]
#print (tickers)

def instrumentLookup(instrument_df,symbol):
"""Looks up instrument token for a given script from instrument dump"""
try:
return instrument_df[instrument_df.tradingsymbol==symbol].instrument_token.values[0]
except:
return -1


def fetchOHLCExtended(ticker,inception_date, interval):
"""extracts historical data and outputs in the form of dataframe
inception date string format - dd-mm-yyyy"""
instrument = instrumentLookup(instrument_df,ticker)
from_date = dt.datetime.strptime(inception_date, '%d-%m-%Y')
to_date = dt.date.today()
data = pd.DataFrame(columns=['date', 'open', 'high', 'low', 'close', 'volume'])
while True:
if from_date.date() >= (dt.date.today() - dt.timedelta(100)):
data = data.append(pd.DataFrame(kite.historical_data(instrument,from_date,dt.date.today(),interval)),ignore_index=True)
break
else:
to_date = from_date + dt.timedelta(100)
data = data.append(pd.DataFrame(kite.historical_data(instrument,from_date,to_date,interval)),ignore_index=True)
from_date = to_date
data.set_index("date",inplace=True)
return data


ohlc = fetchOHLCExtended(tickers,"20-08-2019","5minute")

Now when I try to run the last line of code to fetch the ohlc df and use the tickers option it gives me an error as shown in the screenshot below.





The above code works perfectly fine if I replace the tickers variable with any one variable such as INFY OR YESBANK or some other stock.


I have referred the following link with a view to solve this issue:


kite.trade/forum/discussion/6868/getting-invalid-token-error-only-fir-historical-data

I also have checked and have valid subscriptions for both the api and also to get live historical data





I also generate the request and authentication tokens on a daily basis.


Is there any way wherein the open,high,close values can be found for more than one socks as required by me.


Thanks,


Huzefa


  • sujith
    sujith edited November 2020
    Kite Connect doesn't provide bulk fetch API for historical data. You need to dump it at your end and use that as the source for your script.
  • Hsadikot
    Can you direct me for any particular documentation for that
  • sujith
    You can refer to the API documentation here.
  • sujith
    The error says it is an invalid instrument token and not the auth token.
  • Hsadikot
    so how to download instruments token I read the documentation here:
    https://kite.trade/docs/connect/v3/market-quotes/#retrieving-the-full-instrument-list

    But the file which is downloaded contains forex data I need it for BSE or for NSE

  • sujith
    This has all the instruments that are traded for the day.
    Did you download the whole instruments list or instruments list for exchange CDS?
  • Hsadikot
    I went on the following URL:

    "https://api.kite.trade/instruments"


    After this the file automatically got downloaded
  • sujith
    That is a full instruments master, it contains all the instruments that are traded for the day.
  • Hsadikot
    Is there any url or documentation wherein I can get the instrument tokens against their companies names:

    1. Reliance:738561
    2. ACC:5633



    The document which is accessed from the above URL contains only the instrument tokens not the companies they represent
  • sujith
    The instrument master has tradingsymbol, company name, instrument token, exchange token, exchange all the details. We don't have any other ad hoc APIs.
  • dinezh
    dinezh edited November 2020
    @Hsadikot
    Just run this code and you'll get the exact idea what to do next
    This code will return you name:token of the all the symbols in NSE
    just like you asked
    RELIANCE
    Happy trading!

    for instrument in kite.instruments("NSE"):
    ----print(instruement['name']:instrument['instrument_token'])
  • dinezh
    Okay I think I understood wrong, What you want to do is extract 5 minute data for lists of tickers, Right
  • dinezh
    dinezh edited November 2020
    @Hsadikot


    # Once you fetch the SYMBOLS from DataBase
    # Convert the dataframe to a list
    tickers = ticker["SYMBOL"].to_list()
    # you'll have a list of symbols that you want to fetch Historical data


    # Now at end of the code
    for instrument in kite.insturments("NSE"):
    if instrument['tradingsymbol'] in tickers:
    ohlc = fetchOHLCExtended(instrument['instrument_token'], "20-08-2019", "5minute")

  • dinezh
    dinezh edited November 2020
    this will work fine for now

    it's better to save the insturments Name, TradingSymbol, Token etc to your Database so you can fetch it easily.
  • Hsadikot
    Hsadikot edited November 2020
    @dinezh

    I Ran the code as you asked but it is giving me an error as shown:




  • dinezh
    @Hsadikot Oh for all the tokens available
  • Hsadikot
    Hsadikot edited November 2020
    I did not get you

    I just ran the two lines you gave me so as to debug but it is giving me an error as shown
  • Hsadikot
    Hsadikot edited November 2020
    @dinezh

    Thanks.

    But why is mine not working?

    Here is my code:

    from kiteconnect import KiteConnect
    import pandas as pd
    import os

    cwd = os.chdir("C:\\Users\\Administrator\\Desktop\\Zerodha")

    #generate trading session
    access_token = open("access_token.txt",'r').read()
    key_secret = open("credentials.txt",'r').read().split()
    kite = KiteConnect(api_key=key_secret[0])
    kite.set_access_token(access_token)
    for instrument in kite.instruments("NSE"):
    print(instruement['name']:instrument['instrument_token'])

  • dinezh
    @ Hsadikot



    from kiteconnect import KiteConnect
    import pandas as pd

    cwd = os.chdir("C:\\Users\\Administrator\\Desktop\\Zerodha")
    #generate trading session
    access_token = open("access_token.txt",'r').read()
    key_secret = open("credentials.txt",'r').read().split()
    kite = KiteConnect(api_key=key_secret[0])
    kite.set_access_token(access_token)

    exchange = "NSE"
    from_date = "2020-08-20"
    to_date = "2020-11-20"
    interval = "5minute"

    for instrument in kite.instruments(exchange):
    print(f"Fetching {instrument['name']} data")
    try:
    df = pd.DataFrame(kite.historical_data(instrument['instrument_token'], from_date, to_date, interval))
    except Exception as e:
    print("Unable to fetch the data {}".format(instrument['name']))
    print(e)

    df.set_index("date", inplace=True)
    df.to_csv(f"{instrument['tradingsymbol']}.csv", index=False)


  • dinezh
    @ Hsadikot

    The above code will fetch data of all the symbols in NSE
    and it will store the data of each symbol in individual csv file
    You can store it to your SQL DataBase if you want
    I made it simple so everybody can understand.
  • dinezh
    @Hsadikot

    Not a problem!

    and you got a error for this code because I forgot to add the ","
    paste the below code and it'll run just fine


    for instrument in kite.instruments("NSE"):
    print(instrument['name'],':', instrument['instrument_token'])

  • Hsadikot
    Hsadikot edited November 2020
    Hi @dinezh ,

    I tried using your solution but there were too many files downloaded. So I created an excel file for all the NSE Stocks with their corresponding instrument ID. I try to import all the instrument in a pandas DF and try to download the ohlcv data using the kite.historical_data function but am getting an error as shown:




    PFA my code:


    df = pd.read_excel("C:\\Users\Administrator\\Desktop\\Zerodha\\DOCS\\NSE_Instruments_31122019.xlsx")
    tickers = (df['instrument_token'])

    for instrument_token in tickers:
    to_date = datetime.now()
    from_date=to_date-timedelta(days=5)
    interval = "60 Minutes"
    records = kite.historical_data(tickers, from_date, to_date, interval)
    data =pd.DataFrame(records)



  • deepakksharma00079
    Anybody please solve this problem. The below mentioned code gives this error- "IndexError: index 0 is out of bounds for axis 0 with size 0" . I am using spyder latest version. Please tell me solution.

    # -*- coding: utf-8 -*-
    """
    Zerodha Kite Connect - Storing tick level data in db

    """

    from kiteconnect import KiteTicker, KiteConnect
    import datetime
    import sys
    import pandas as pd
    import os
    import sqlite3

    cwd = os.chdir("F:\\Deepakfirst")

    #generate trading session
    access_token = open("access_token.txt",'r').read()
    key_secret = open("api_key.txt",'r').read().split()
    kite = KiteConnect(api_key=key_secret[0])
    kite.set_access_token(access_token)

    db = sqlite3.connect("F://Streaming_data/ticks.db")

    def create_tables(tokens):
    c=db.cursor()
    for i in tokens:
    c.execute("CREATE TABLE IF NOT EXISTS TOKEN{} (ts datetime primary key,price real(15,5), volume integer)".format(i))
    try:
    db.commit()
    except:
    db.rollback()

    def insert_ticks(ticks):
    c=db.cursor()
    for tick in ticks:
    try:
    tok = "TOKEN"+str(tick['instrument_token'])
    vals = [tick['timestamp'],tick['last_price'], tick['volume']]
    query = "INSERT INTO {}(ts,price,volume) VALUES (?,?,?)".format(tok)
    c.execute(query,vals)
    except:
    pass
    try:
    db.commit()
    except:
    db.rollback()


    #get dump of all NSE instruments
    instrument_dump = kite.instruments("NSE")
    instrument_df = pd.DataFrame(instrument_dump)


    def tokenLookup(instrument_df,symbol_list):
    """Looks up instrument token for a given script from instrument dump"""
    token_list = []
    for symbol in symbol_list:
    token_list.append(int(instrument_df[instrument_df.tradingsymbol==symbol].instrument_token.values[0]))
    return token_list

    #####################update ticker list######################################
    tickers = ["ZEEL","WIPRO","VEDL","ULTRACEMCO","UPL","TITAN","TECHM","TATASTEEL",
    "TATAMOTORS","TCS","SUNPHARMA","SBIN","SHREECEM","RELIANCE","POWERGRID",
    "ONGC","NESTLEIND","NTPC","MARUTI","M&M","LT","KOTAKBANK","JSWSTEEL","INFY",
    "INDUSINDBK","IOC","ITC","ICICIBANK","HDFC","HINDUNILVR","HINDALCO",
    "HEROMOTOCO","HDFCBANK","HCLTECH","GRASIM","GAIL","EICHERMOT","DRREDDY",
    "COALINDIA","CIPLA","BRITANNIA","INFRATEL","BHARTIARTL","BPCL","BAJAJFINSV",
    "BAJFINANCE","BAJAJ-AUTO","AXISBANK","ASIANPAINT","ADANIPORTS"]
    #############################################################################

    #create KiteTicker object
    kws = KiteTicker(key_secret[0],kite.access_token)
    tokens = tokenLookup(instrument_df,tickers)

    #create table
    create_tables(tokens)


    def on_ticks(ws,ticks):
    insert_tick=insert_ticks(ticks)
    print(ticks)

    def on_connect(ws,response):
    ws.subscribe(tokens)
    ws.set_mode(ws.MODE_FULL,tokens)


    while True:
    now = datetime.datetime.now()
    if (now.hour >= 9 and now.minute >= 15 ):
    kws.on_ticks=on_ticks
    kws.on_connect=on_connect
    kws.connect()
    if (now.hour >= 15 and now.minute >= 30):
    sys.exit()

    db.close()

    """
    c.execute('SELECT name from sqlite_master where type= "table"')
    c.fetchall()

    c.execute('''PRAGMA table_info(TOKEN975873)''')
    c.fetchall()

    for m in c.execute('''SELECT * FROM TOKEN975873'''):
    print(m)
    """
  • deepakksharma00079
    Please give reply to my above comment ????????
Sign In or Register to comment.