Remove Duplicate Lines

Drop repeated lines and keep the first time each line appears. The original order of the kept lines does not change.

Use this tool

Result

The result appears in this panel after you run the tool.

About this tool

Lines are split after Windows and old Mac line endings are turned into a line feed. Comparison is exact string equality. Apple and apple are different lines, Turkish i and ı stay distinct, and a trailing space keeps a line distinct. The text locale does not change which lines match. The first copy stays where it was. Later copies are removed. An empty line is kept the first time it appears, including the empty line created by a trailing newline.

The output uses line feeds only. It does not re-sort the remaining lines. Open Sort Lines afterward if you want the unique rows in order. The chain can carry this result to that page without an upload. A refresh can discard the handoff.

How to use it

  1. Paste text that uses one row per line.
  2. Remove duplicates. A line matches only when the whole row is the same, including spaces.
  3. Download the result or continue with another text tool in this browser.

Limitations

First row wins, spaces count

The match is the entire row

A later line is dropped only when it is exactly the same sequence of characters as an earlier line. “Apple” and “apple” both stay. “Apple” and “Apple “ both stay, because the trailing space is part of the row. The first copy keeps its position. Later copies are removed, so the surviving order is the order of first occurrence, not a sorted order. If you wanted alphabetical output, sort after this, or before, knowing that a sort will group the copies and this page will then keep whichever copy ended up first.

Blank lines and punctuation

An empty line matches other empty lines. The first empty line stays and the rest of the empty lines go. That is useful for collapsing a pile of blanks and destructive if a blank line was a paragraph separator you meant to keep. Punctuation is not stripped before the comparison. “end.” and “end” are different rows. This page does not stem, case-fold, or trim. Pair it with trim or case conversion first if you want those differences ignored, and expect the result to follow that earlier edit.

What this is not

It is not a spreadsheet dedupe on one column. Two CSV rows that share an email and differ in the name both stay, because the rows differ. It is not a fuzzy match. A typo is a different line. The comparison is case-sensitive and locale-blind on purpose, so a Turkish capital I does not collapse with a lowercase i unless you case-fold first with a locale that does that mapping.

Order is information this page keeps

Because the first copy stays where it was, a log that repeats an error keeps the earliest line and drops later identical lines. If the later line had a different timestamp, it was not identical and it stays. That is the test. If you needed the latest copy instead, reverse the lines, remove duplicates, and reverse again, and know that you did that on purpose.

Privacy

Your text is processed in your browser and is not uploaded to UtilChain. The lines are not included in analytics.