Quantcast
Channel: Data Science, Analytics and Big Data discussions - Latest topics
Viewing all articles
Browse latest Browse all 4448

Help : Finding patterns of identical values across fields in large data

$
0
0

Hi

I’m looking for either a visualisation technique, or preferably an algorithm or approach ( and preferably implementable in R) for the following pattern searching problem.

I have a dataset of 1000s of customers where the customer has supplied numerical data. I want to find automatically groups of customers who have provided identical numbers in multiple fields.
I’m not looking to find where every field between customers has been filled in the same, only 2 or more of the fields.

So on the dummy data below:

Customer. f1. f2. f3 … f17. f18. f19. f20
A. 2. 5. 7. … 3. 11. 4. 8
B. 2. 6. 7 … 1. 11. 7. 5

C. 1. 1. 2 … 1. 11. 7. 9

Z. 6. 5. 8 … 3. 9. 6. 8

In the above data: customers A and B share fields f1, f3 and f18
Customers B and C share fields f17, f18 and f19
Customers A and Z share fields f2, f17 and f20

I want to automatically find / highlight these 3 scenarios .

I can obviously find these patterns by grouping by and counting, and then look for groups with a count greater than 1…but I would have to do that for all combinations of two fields, then all combinations of three fields, then four, etc

And a standard k means clustering approach doesn’t really do it.

Is there an algorithm, approach that anyone could recommend?

Regards

Andy

1 post - 1 participant

Read full topic


Viewing all articles
Browse latest Browse all 4448

Trending Articles