Removing duplicates without losing the rows you need
Exact duplicates, duplicates by key, keeping the newest record, and the near-duplicates that whitespace and capital letters hide.
“Remove duplicates” sounds like one operation, but it is really a decision: which columns decide that two rows are the same, and which of them to keep. Getting either wrong silently deletes real data.
Exact duplicates
The safe default: a row is removed only if every cell matches an earlier row. This catches the classic export accident where the same block of rows was pasted twice. It will never remove two different orders from the same customer.
Duplicates by key
Often a row is the “same” record if one or two columns match, e.g. the email address in a contact list, or the order ID in an orders export. Choose those columns in Compare by. Be careful with keys that are not truly unique: deduplicating an orders file by customer email keeps only one order per customer.
Keep first or keep last?
- Keep first when the file is in the order things happened and the first record is the original (sign-ups, first contact).
- Keep last when later rows are updates of earlier ones, e.g. a CRM export appended to over time, so the last row has the newest phone number.
The kept row stays where it was in the file; rows are not re-sorted.
Near-duplicates
NOAH.SMITH@example.com and noah.smith@example.com (with a trailing space) are the same person but not equal text. Turn on Ignore upper/lower case and Ignore extra spaces for the comparison. The kept row keeps its original text; run Trim whitespace first if you also want the output tidied.
Spelling variants (“Jon” vs “John”) are not duplicates by any rule a tool can apply safely. Treat those by hand.
Always look before you download
GridRinse lists every removed row in the Removed rows tab with its original row number and the step that removed it. Glance through it before downloading; if something there should stay, change the key columns or switch the step off.