Deduplication Memes

Posts tagged with Deduplication

Deduping For Faster Justice
Algorithms Programming Databases Backend
11 days ago 374.1K views 0 shares
Deduping For Faster Justice
Someone finally decided to apply software engineering best practices to a criminal investigation. Converting a list to a set for O(1) lookup time? Chef's kiss. Nothing says "we're serious about justice" quite like eliminating duplicate entries with a simple data structure swap. I can just imagine the meeting: "Detective, we need to search through thousands of names!" "Have you tried... deduplication?" "Brilliant! Promote this person immediately!" The real question is whether they're using a HashSet or a TreeSet. Performance matters when you're fighting crime, people. Also, did nobody think to normalize the data before storing it? Guess they didn't have a DBA on the investigative team.

This Will Surely Eliminate The Fraud
Databases Security Programming
10 months ago 194.4K views 0 shares
This Will Surely Eliminate The Fraud
The code snippet shows a hilariously naive approach to database deduplication that would make any DBA have a heart attack. It's basically iterating through social security numbers in a government database and if there are duplicates, it just deletes ALL matches with that SSN! The comment even includes the infamous Unix rm -rf command, implying this is the digital equivalent of taking a flamethrower to your records. Instead of properly identifying which record is legitimate or merging data, this nuclear option would just obliterate everyone's records if there happened to be a duplicate SSN. Congratulations, you've fixed identity fraud by erasing everyone's identity! Task failed successfully!