How to efficiently save and retrive tick data

pratiks
pratiks edited June 2021 in Python client
I am using redis and rq to process tick data

def process_data(tick):
opens csv file as dataframe
insert ltp data
alter some columns
save csv file

def on_ticks(ws, ticks):
q.enqueue(process_data,ticks)

this process usually takes longer time than processing in on_ticks directly.
I am accessing ticks of nearly 150 instruments (all stock futures)

processing this data takes long time, and i get delayed signals

so I want to know how other people store/use tick data?

@rakeshr
how does calling helper method this way is diferent than calling it inside on_ticks method directly?
i am unable to understand how this works
please explain

while True:
#Perform required data operation using tick data
def on_ticks(ws, ticks):
..........
helper_method(ticks)
.........

def helper_method(ticks):
.........
Perform computation here
........
#Assign callback
kws.on_ticks=on_ticks
  • rakeshr
    how does calling helper method this way is diferent than calling it inside on_ticks method directly?
    As the thread explains, you are creating another ticker thread using threaded=True and let it handle tick data.
  • pratiks
    you are creating another ticker thread using threaded=True
    ^
    how this another thread gets created
  • pratiks
    @rakeshr i didnt understand exactly when the new ticker thread gets created
Sign In or Register to comment.