In this article, we will discuss how to remove duplicate element/objects from ArrayList along with various examples
Removing duplicates from ArrayList :
- Using Java 8 Stream method distinct()
- Using Set approach
- Using java.util.Comparator interface
- Overriding equals() & hashCode() methods
Let’s discuss one-by-one in detail with example/explanation
1. Stream distinct() method
- Stream’s distinct() method returns a stream consisting of the distinct elements (according to
Object.equals(Object)
) of this stream - Below example removes duplicate String elements and maintains original insertion order
- Terminal operation :- Another Stream method collect() is used to collect String elements into new list
- Similarly, sorted() method of Stream helps to sort String elements in alphabetical order once after removing duplicates
RemoveDuplicatesFromArrayList.java
package net.bench.resources.java8;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
public class RemoveDuplicatesFromArrayList {
public static void main(String[] args) {
// create test data
List<String> students = new ArrayList<String>();
// add values to list
students.add("Roger");
students.add("Rafael");
students.add("Djokovic");
students.add("Roger");
students.add("Murray");
students.add("Rafael");
// pretty print
System.out.println("1. Original list with duplicate values :\n");
students.forEach(student -> System.out.println(student));
// Java 8 - distinct() method
List<String> uniqueList = students
.stream() // get stream for original list
.distinct() // distinct method removes duplicates
.collect(Collectors.toList()); // distinct elements stored to new list
// pretty print
System.out.println("\n2. New list with unique values"
+ " maintaining original insertion order:\n");
uniqueList.forEach(uniqueStudent -> System.out.println(uniqueStudent));
// Java 8 - sorting
List<String> sortedList = students
.stream() // get stream for original list
.distinct() // distinct method removes duplicates
.sorted() // uses natural-ordering to sort
.collect(Collectors.toList()); // distinct elements stored to new list
// pretty print
System.out.println("\n3. New list with unique values"
+ " in natural sorting order :\n");
sortedList.forEach(sortedStudent -> System.out.println(sortedStudent));
}
}
Output:
1. Original list with duplicate values :
Roger
Rafael
Djokovic
Roger
Murray
Rafael
2. New list with unique values maintaining original insertion order:
Roger
Rafael
Djokovic
Murray
3. New list with unique values in natural sorting order :
Djokovic
Murray
Rafael
Roger
2. Using Set approach
- Original list contains duplicate String elements
- Using set approach, we can get stream for original list and then we can collect it into set using Collectors.toSet() method
- Collecting into Set helps to remove duplicate elements but at the same time loses insertion order of original list
- This is where Stream‘s distinct() method wins big when comparing with Set approach
- Finally, we are sorting elements in reverse order using another Stream method called sorted() by passing Comparator.reverseOrder(); comparator
UsingSetApproach.java
package net.bench.resources.java8;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
public class UsingSetApproach {
public static void main(String[] args) {
List<String> students = new ArrayList<String>();
// add values to list
students.add("Roger");
students.add("Rafael");
students.add("Djokovic");
students.add("Roger");
students.add("Murray");
students.add("Rafael");
// pretty print
System.out.println("1. Original list with duplicate values :\n");
students.forEach(student -> System.out.println(student));
// Java 8 - Collector.toSet()
Set<String> uniqueSet = students
.stream() // get stream for original list
.collect(Collectors.toSet()); // distinct elements stored to new SET
// pretty print
System.out.println("\n2. New SET with unique values"
+ " which doesn't maintains order:\n");
uniqueSet.forEach(uniqueStudent -> System.out.println(uniqueStudent));
// Java 8 - sorting in reverse order
List<String> sortedList = uniqueSet
.stream() // get stream for unique SET
.sorted(Comparator.reverseOrder()) // reverse-ordering
.collect(Collectors.toList()); // elements are stored to new list
// pretty print
System.out.println("\n3. New list with unique values"
+ " in REVERSE sorting order :\n");
sortedList.forEach(sortedStudent -> System.out.println(sortedStudent));
}
}
Output:
1. Original list with duplicate values :
Roger
Rafael
Djokovic
Roger
Murray
Rafael
2. New SET with unique values which doesn't maintains order:
Roger
Rafael
Djokovic
Murray
3. New list with unique values in REVERSE sorting order :
Roger
Rafael
Murray
Djokovic
3. Custom object – remove duplicates using Comparator
- Student class defined with 4 attributes namely id, name, percentage, rank and constructor, getter/setters & toString() method
- In the Student list, there are 5 Student objects and 1 duplicate Student with Id=2
- To remove duplicate Student, we are converting original list into set which doesn’t allow duplicate by comparing Student’s Id attribute
- Then we are printing again by comparing Student’s rank attribute in ascending order
Student.Java
package net.bench.resources.java8;
// Student class
class Student {
// private member variables
private int id;
private String name;
private double percentage;
private int rank;
// public 4-arg constructor
// getters & setters
// toString() method
}
TestCustomObject.java
package net.bench.resources.java8;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Set;
import java.util.TreeSet;
import java.util.stream.Collectors;
public class TestCustomObject {
public static void main(String[] args) {
// create a list of Student objects
List<Student> studentList = new ArrayList<>();
// add student object to List
studentList.add(new Student(1,"Arun", 67.36, 2));
studentList.add(new Student(2,"Sethu", 88.58, 1));
studentList.add(new Student(3,"Ajith", 55.74, 4));
studentList.add(new Student(4,"Vikcy", 61.32, 3));
studentList.add(new Student(1,"Arun", 67.36, 2)); // duplicate Arun
// pretty print
System.out.println("1. Original Student list with duplicates :\n");
studentList.forEach(student -> System.out.println(student));
// Java 8 - Collector.toCollection()
Set<Student> uniqueStudentSet = studentList
.stream() // get stream for original list
.collect(Collectors.toCollection(//distinct elements stored into new SET
() -> new TreeSet<>(Comparator.comparing(Student::getId)))
); //Id comparison
// pretty print
System.out.println("\n2. New SET with unique Student objects"
+ " in natural order of Id :\n");
uniqueStudentSet.forEach(uniqueStudent -> System.out.println(uniqueStudent));
// Java 8 - sorting in ascending order of Student's Rank
List<Student> sortedList = uniqueStudentSet
.stream() // get stream for unique SET
.sorted(Comparator.comparing(Student::getRank)) // rank comparing
.collect(Collectors.toList()); // elements stored to new list
// pretty print
System.out.println("\n3. Sorted according to ascending order"
+ " of Student's Rank :\n");
sortedList.forEach(sortedStudent -> System.out.println(sortedStudent));
}
}
Output:
1. Original Student list with duplicates :
Student [id=1, name=Arun, percentage=67.36, rank=2]
Student [id=2, name=Sethu, percentage=88.58, rank=1]
Student [id=3, name=Ajith, percentage=55.74, rank=4]
Student [id=4, name=Vikcy, percentage=61.32, rank=3]
Student [id=1, name=Arun, percentage=67.36, rank=2]
2. New SET with unique Student objects in natural order of Id :
Student [id=1, name=Arun, percentage=67.36, rank=2]
Student [id=2, name=Sethu, percentage=88.58, rank=1]
Student [id=3, name=Ajith, percentage=55.74, rank=4]
Student [id=4, name=Vikcy, percentage=61.32, rank=3]
3. Sorted according to ascending order of Student's Rank :
Student [id=2, name=Sethu, percentage=88.58, rank=1]
Student [id=1, name=Arun, percentage=67.36, rank=2]
Student [id=4, name=Vikcy, percentage=61.32, rank=3]
Student [id=3, name=Ajith, percentage=55.74, rank=4]
4. Custom object – remove duplicate by overriding equals/hashCode
- Student class defined with 4 attributes namely id, name, percentage, rank and constructor, getter/setters & toString(), equals(), hashCode() method
- Here, we are overriding equals() & hashCode() methods based on Student’s Id attribute so as to prevent storing duplicate Student object in the Set
- In the Student list, there are 6 Student objects and 2 duplicate Student with Id=2 & Id=3
- To remove duplicate Student, we are using Stream‘s distinct() method and then collecting into Set which stores Student object in natural sorting order of Student’s Id attribute
Student.java
package net.bench.resources.java8;
class Student {
// private member variables
private int id;
private String name;
private double percentage;
private int rank;
// public 4-arg constructor
// getters & setters
// toString() method
// hashCode() method
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + id;
return result;
}
// equals() method
@Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
Student other = (Student) obj;
if (id != other.id)
return false;
return true;
}
}
TestStudentObject.java
package net.bench.resources.java8;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
public class TestStudentObject {
public static void main(String[] args) {
// create a list of Student objects
List<Student> studentList = new ArrayList<>();
// add student object to List
studentList.add(new Student(1,"Arun", 67.36, 2));
studentList.add(new Student(2,"Sethu", 88.58, 1));
studentList.add(new Student(3,"Ajith", 55.74, 4));
studentList.add(new Student(4,"Vikcy", 61.32, 3));
studentList.add(new Student(1,"Arun", 67.36, 2)); // duplicate Arun
studentList.add(new Student(4,"Vikcy", 61.32, 3)); // duplicate Vicky
// pretty print
System.out.println("1. Original Student list with duplicates :\n");
studentList.forEach(student -> System.out.println(student));
// Java 8 - Collector.toCollection()
Set<Student> uniqueStudentSet = studentList
.stream() // get stream for original list
.distinct() // removes duplicate
.collect(Collectors.toSet());
// pretty print
System.out.println("\n2. New SET with unique Student objects"
+ " in natural order of Id :\n");
uniqueStudentSet.forEach(uniqueStudent -> System.out.println(uniqueStudent));
}
}
Output:
1. Original Student list with duplicates :
Student [id=1, name=Arun, percentage=67.36, rank=2]
Student [id=2, name=Sethu, percentage=88.58, rank=1]
Student [id=3, name=Ajith, percentage=55.74, rank=4]
Student [id=4, name=Vikcy, percentage=61.32, rank=3]
Student [id=1, name=Arun, percentage=67.36, rank=2]
Student [id=4, name=Vikcy, percentage=61.32, rank=3]
2. New SET with unique Student objects in natural order of Id :
Student [id=1, name=Arun, percentage=67.36, rank=2]
Student [id=2, name=Sethu, percentage=88.58, rank=1]
Student [id=3, name=Ajith, percentage=55.74, rank=4]
Student [id=4, name=Vikcy, percentage=61.32, rank=3]
Related Articles:
- Java 8 – How to find duplicate from Stream ?
- Java 8 – How to remove duplicates from ArrayList
- Java 8 – How to remove duplicates from LinkedList
- Java 8 – How to find duplicate from Arrays ?
- Java 8 – How to remove duplicate from Arrays ?
- Java 8 – Various ways to remove duplicate elements from Arrays
- Java 8 – How to remove an entry from HashMap by comparing keys
- Java 8 – How to remove an entry from HashMap by comparing values
References:
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html
- https://docs.oracle.com/javase/8/docs/api/java/lang/Iterable.html
- https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html
Happy Coding !!
Happy Learning !!