Text Line Operations: Sort, Deduplicate, Reverse, and Filter Lines

The Core Line Operations

Text processing at the line level covers a small set of fundamental operations:

• Sort: alphabetical, reverse-alphabetical, by line length, numerically
• Deduplicate: remove duplicate lines (case-sensitive or insensitive)
• Reverse: flip the line order
• Shuffle: randomize line order
• Filter: keep or remove lines matching a pattern
• Number: prepend line numbers
• Trim: remove leading/trailing whitespace from each line
• Remove empty: filter out blank lines

Unix/Shell Commands

# Sort alphabetically
sort file.txt

# Sort and remove duplicates
sort -u file.txt

# Sort numerically (for files like "10.txt" that sort wrong alphabetically)
sort -n numbers.txt

# Sort in reverse
sort -r file.txt

# Remove duplicate lines (preserves order, unlike sort -u — needs sorted input)
uniq file.txt

# Remove duplicate lines without sorting (awk trick)
awk '!seen[$0]++' file.txt

# Reverse line order
tac file.txt           # GNU/Linux
tail -r file.txt       # macOS

# Count occurrences of each unique line
sort file.txt | uniq -c | sort -rn

# Filter lines containing "error"
grep "error" logfile.txt
grep -v "error" logfile.txt   # -v = invert, lines NOT matching

# Number lines
cat -n file.txt
nl file.txt

JavaScript Line Processing

const text = `banana
apple
cherry
apple
banana`;

const lines = text.split("\n");

// Sort
const sorted = [...lines].sort();
// ["apple", "apple", "banana", "banana", "cherry"]

// Sort case-insensitive
const sortedCI = [...lines].sort((a, b) => a.toLowerCase().localeCompare(b.toLowerCase()));

// Deduplicate (preserve first occurrence)
const unique = [...new Set(lines)];
// ["banana", "apple", "cherry"]

// Deduplicate case-insensitive
const uniqueCI = lines.filter((line, i, arr) =>
  arr.findIndex(l => l.toLowerCase() === line.toLowerCase()) === i
);

// Reverse
const reversed = [...lines].reverse();

// Remove empty lines
const nonEmpty = lines.filter(l => l.trim() !== "");

// Trim each line
const trimmed = lines.map(l => l.trim());

// Number lines
const numbered = lines.map((l, i) => `${i + 1}. ${l}`);

// Shuffle (Fisher-Yates)
function shuffle(arr) {
  const a = [...arr];
  for (let i = a.length - 1; i > 0; i--) {
    const j = Math.floor(Math.random() * (i + 1));
    [a[i], a[j]] = [a[j], a[i]];
  }
  return a;
}

Common Use Cases

• Deduplicating email lists — copy CSV column, paste as lines, deduplicate
• Sorting import statements — many style guides require sorted imports
• Generating wordlists — for testing, security research, or autocomplete
• Processing log files — filter, deduplicate, and count error lines
• Cleaning scraped data — remove empty lines, trim whitespace, deduplicate

Process Lines Instantly

Use ToolsVito's Line Tools to sort, deduplicate, reverse, shuffle, and filter lines in your browser — no command-line needed.