@azimulh wrote:

I have 2 pandas dataframes. I want to do a find and replace between 2 dataframes. In the df_find dataframe, in the current_title column, i want to search in every row for any occurrence of values from 'keywrod' column in the df_replace dataframe and if found replace it with corresponding value from 'keywordlength' column.

I have been able to get rid of the loop for df_find dataframe since i need to iterate over every row in this dataframe by using str.replace which is a vectorized form of replace function.

Performance matters in my case, as both the dataframes run into GB's. So, i want to get rid of the loop for df_replace here and use any other efficient way of iterating through all rows of df_replace dataframe.

import pandas as pd
df_find = pd.read_csv("input_find.csv")
df_replace = pd.read_csv("input_replace.csv")

replace

for i,j in zip(df_replace.keyword,df_replace.keywordLength):
df_find.current_title=df_find.current_title.str.replace(i,j,case=False)
df_replace This dataframe has the data we need for find and replace

keyword keywordLength
IT Manager ##10##
Sales Manager ##13##
IT Analyst ##12##
Store Manager ##13##

df_find is where we need to do the transformation.

Before executing find and replace code:

current_title
I have been working here as a store manager since after I passed from college
I am sales manager and primarily work in the ASEAN region. My primary rolw is to bring new customers.
I initially joined as a IT analyst and because of my sheer drive and dedication, I was promoted to IT manager position within 3 years

After executing find and replace through above code

current_title
I have been working here as a ##13## since after I passed from college
I am ##13## and primarily work in the ASEAN region. My primary rolw is to bring new customers.
I initially joined as a ##12## and because of my sheer drive and dedication, I was promoted to ##10## position within 3 years

Posts: 1

Participants: 1

Read full topic