awk '!seen[$0]++' foo.txt
Quelle: Unix & Linux Stack Echange: How To Remove Duplicate Lines Inside A Text File?, 2012