Welcome to our Community
Wanting to join the rest of our members? Feel free to sign up today.
Sign up

How to do df.apply parallelly ?

barackobama

Active Member
Jul 25, 2020
61
3
2,300
I have a code segment

Python:
data_preprocess_segment_5 = df.apply(lambda x: ' '.join(segment(x)))

which takes a lot of time maybe more than 24hrs to process.
I was thinking to do
Python:
df[0:1000].apply(lambda x: ' '.join(segment(x)))
df[1000:2000].apply(lambda x: ' '.join(segment(x)))
df[2000:3000].apply(lambda x: ' '.join(segment(x)))

parallelly and later merge those df


 

Sponsored

Topics You Missed

Latest Posts