In this article, we will discuss how to find and count duplicates in an Arrays in different ways
Find and count duplicates in an Arrays :
- Using Stream.distinct() method
- Using Stream.filter() and Collections.frequency() methods
- Using Stream.filter() and Set.add() methods
- Using Collectors.toMap() method and Method Reference Math::addExact for summation of duplicates
- Using Collectors.groupingBy() and Collectors.counting() methods
- Using Map.getOrDefault() and Collection.forEach() methods
- Using Map.merge() and Collection.forEach() methods and lambda Expression for counting duplicates
Let us discuss each one with example and description
1. Using Stream.distinct() method
- Stream.distinct() method eliminates/removes duplicate from Original Arrays and store into new Arrays using toArray(String[]::new) method which results into unique elements in an Arrays
- For finding duplicates,
- Create a new List using original Arrays
- Iterate through new List and remove elements by comparing elements in unique Arrays
- Elements remaining in the new List will contain only duplicates
FindDuplicatesUsingStreamDistinctMethod.java
package in.bench.resources.java.stream;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class FindDuplicatesUsingStreamDistinctMethod {
public static void main(String[] args) {
// 1. String[] array
String[] companies = new String[]{
"Meta",
"Apple",
"Amazon",
"Netflix",
"Meta", // duplicate
"Google",
"Apple" // duplicate
};
// 1.1 print String[] array to console
System.out.println("1. Original String[] Array with duplicates : \n");
Arrays.stream(companies).forEach(System.out::println);
// 2. get unique elements after removing duplicates
String[] distinctCompanies = Arrays
.stream(companies)
.distinct()
.toArray(String[]::new);
// 2.1 print unique elements
System.out.println("\n2. Unique elements in String[] array : \n");
Arrays.stream(distinctCompanies).forEach(System.out::println);
// 2.2 Unique element count
System.out.println("\nNumber of Unique elements = "
+ distinctCompanies.length);
// 3. create List<String> with original String[] array elements
List<String> originalCompanyList = new ArrayList<String>(
Arrays.asList(companies));
// 3. get duplicate elements
for (String distinctCompany : distinctCompanies) {
originalCompanyList.remove(distinctCompany);
}
// 3.1 print duplicate elements
System.out.println("\n3. Duplicate elements in String[] array : \n");
originalCompanyList.forEach(System.out::println);
// 3.2 Duplicate element count
System.out.println("\nNumber of Duplicate elements = "
+ originalCompanyList.size());
}
}
Output:
1. Original String[] Array with duplicates :
Meta
Apple
Amazon
Netflix
Meta
Google
Apple
2. Unique elements in String[] array :
Meta
Apple
Amazon
Netflix
Google
Number of Unique elements = 5
3. Duplicate elements in String[] array :
Meta
Apple
Number of Duplicate elements = 2
2. Using Stream.filter() and Collections.frequency() methods
- Convert original Arrays into Set using collect(Collectors.toSet()) method which results into new Set with unique elements
- For finding duplicates, use Stream.filter() method by checking/verifying whether Collections.frequency() method returns value greater than 1 or not
- If it is greater than 1, then it means that there are duplicate elements present in the Original Arrays
- Finally, store those elements into another new Set using collect(Collectors.toSet()) method
FindDuplicatesUsingStreamFilterAndCollectionsFrequency.java
package in.bench.resources.java.stream;
import java.util.Arrays;
import java.util.Collections;
import java.util.Set;
import java.util.stream.Collectors;
public class FindDuplicatesUsingStreamFilterAndCollectionsFrequency {
public static void main(String[] args) {
// 1. String[] array
String[] companies = new String[]{
"Meta",
"Apple",
"Amazon",
"Netflix",
"Meta", // duplicate
"Google",
"Apple" // duplicate
};
// 1.1 print String[] array to console
System.out.println("1. Original String[] Array with duplicates : \n");
Arrays.stream(companies).forEach(System.out::println);
// 2. get unique elements after removing duplicates
Set<String> distinctCompanies = Arrays
.stream(companies)
.collect(Collectors.toSet());
// 2.2 print unique elements
System.out.println("\n2. Unique elements in String[] array : \n");
distinctCompanies.forEach(System.out::println);
// 2.3 Unique element count
System.out.println("\nNumber of Unique elements = "
+ distinctCompanies.size());
// 3. get duplicate elements
Set<String> duplicateCompanies = Arrays
.stream(companies)
.filter(company -> Collections.frequency(Arrays.asList(companies), company) > 1)
.collect(Collectors.toSet());
// 3.1 print duplicate elements
System.out.println("\n3. Duplicate elements in String[] array : \n");
duplicateCompanies.forEach(System.out::println);
// 3.2 Duplicate element count
System.out.println("\nNumber of Duplicate elements = "
+ duplicateCompanies.size());
}
}
Output:
1. Original String[] Array with duplicates :
Meta
Apple
Amazon
Netflix
Meta
Google
Apple
2. Unique elements in String[] array :
Netflix
Meta
Google
Apple
Amazon
Number of Unique elements = 5
3. Duplicate elements in String[] array :
Meta
Apple
Number of Duplicate elements = 2
3. Using Stream.filter() and Set.add() methods
- Create HashSet object to store/add unique elements
- For finding duplicates, use Stream.filter() method by adding elements into newly created HashSet object using add() method
- If it returns false then it means that there are duplicates present in the Original Arrays
- Finally, store those elements into another new Set using collect(Collectors.toSet()) method
- By doing this,
- Newly created HashSet object will contain only unique elements
- Filtered stream contains duplicate elements in an another Set
FindDuplicatesUsingStreamFilterAndSetAddMethod.java
package in.bench.resources.java.stream;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;
import java.util.stream.Collectors;
public class FindDuplicatesUsingStreamFilterAndSetAddMethod {
public static void main(String[] args) {
// 1. String[] array
String[] companies = new String[]{
"Meta",
"Apple",
"Amazon",
"Netflix",
"Meta", // duplicate
"Google",
"Apple" // duplicate
};
// 1.1 print String[] array to console
System.out.println("1. Original String[] Array with duplicates : \n");
Arrays.stream(companies).forEach(System.out::println);
// 2. create Set object to store unique elements
Set<String> uniqueCompanies = new HashSet<>();
// 3. get duplicate elements
Set<String> duplicateCompanies = Arrays
.stream(companies)
.filter(company -> !uniqueCompanies.add(company))
.collect(Collectors.toSet());
// 2.1 print unique elements
System.out.println("\n2. Unique elements in String[] array : \n");
uniqueCompanies.forEach(System.out::println);
// 2.2 Unique element count
System.out.println("\nNumber of Unique elements = " + uniqueCompanies.size());
// 3.1 print duplicate elements
System.out.println("\n3. Duplicate elements in String[] array : \n");
duplicateCompanies.forEach(System.out::println);
// 3.2 Duplicate element count
System.out.println("\nNumber of Duplicate elements = "
+ duplicateCompanies.size());
}
}
Output:
1. Original String[] Array with duplicates :
Meta
Apple
Amazon
Netflix
Meta
Google
Apple
2. Unique elements in String[] array :
Netflix
Meta
Google
Apple
Amazon
Number of Unique elements = 5
3. Duplicate elements in String[] array :
Meta
Apple
Number of Duplicate elements = 2
4. Using Collectors.toMap() and Math::addExact for counting duplicates
- Collectors.toMap() method can be used to convert Stream/List or Arrays of Stream into Map with actual Stream/List/Arrays elements being Key and their duplicate count as Value
- For Key,
- We will use Function.identity() method or Lambda expression (element -> element)
- For duplicate count, we can use Math::addExact
- Method Reference Math::addExact can be used to add/sum duplicates in the Integer form
FindDuplicateCountUsingCollectorsToMap.java
package in.bench.resources.java.stream;
import java.util.Arrays;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;
public class FindDuplicateCountUsingCollectorsToMap {
public static void main(String[] args) {
// 1. String[] array
String[] companies = new String[]{
"Meta",
"Apple",
"Amazon",
"Netflix",
"Meta", // duplicate
"Google",
"Apple" // duplicate
};
// 1.1 print String[] array to console
System.out.println("1. Original String[] Array with duplicates : \n");
Arrays.stream(companies).forEach(System.out::println);
// 2. get duplicate count using Map and Collectors.toMap()
Map<String, Integer> duplicateCountMap = Arrays
.stream(companies)
.collect(
Collectors.toMap(Function.identity(), company -> 1, Math::addExact)
);
// 2.1 print unique elements
System.out.println("\n2. Unique elements in String[] array : \n");
duplicateCountMap
.entrySet()
.stream()
.forEach(entry -> System.out.println(entry.getKey()));
// 2.2 print duplicate elements
System.out.println("\n3. Duplicate elements in String[] array : \n");
duplicateCountMap
.entrySet()
.stream()
.filter(entry -> entry.getValue() > 1)
.forEach(entry -> System.out.println(entry.getKey()));
// 2.3 print Map with duplicate count
System.out.println("\n4. Map Key as Company and Value as its duplicate count : \n");
duplicateCountMap.forEach(
(key, value) -> System.out.println("Key : " + key + "\t Count : " + value)
);
}
}
Output:
1. Original String[] Array with duplicates :
Meta
Apple
Amazon
Netflix
Meta
Google
Apple
2. Unique elements in String[] array :
Netflix
Google
Meta
Apple
Amazon
3. Duplicate elements in String[] array :
Meta
Apple
4. Map Key as Company and Value as its duplicate count :
Key : Netflix Count : 1
Key : Google Count : 1
Key : Meta Count : 2
Key : Apple Count : 2
Key : Amazon Count : 1
5. Using Collectors.groupingBy() method
- Collectors.groupingBy() method accepts 2 values,
- 1st input-argument can be used as Key
- 2nd input-argument can be used to store duplicate count as Value
- So basically Collectors.groupingBy() method used to convert Stream/List/Arrays into Map according to classification/category
- For Key,
- We will use Function.identity() method or Lambda expression (element -> element)
- For counting duplicates as Value, we can use either of the below methods,
- Collectors.counting() method
- Collectors.summingInt() method
FindDuplicateCountUsingGroupingByAndCounting.java
package in.bench.resources.java.stream;
import java.util.Arrays;
import java.util.Map;
import java.util.Set;
import java.util.function.Function;
import java.util.stream.Collectors;
public class FindDuplicateCountUsingGroupingByAndCounting {
public static void main(String[] args) {
// 1. String[] array
String[] companies = new String[]{
"Meta",
"Apple",
"Amazon",
"Netflix",
"Meta", // duplicate
"Google",
"Apple" // duplicate
};
// 1.1 print String[] array to console
System.out.println("1. Original String[] Array with duplicates : \n");
Arrays.stream(companies).forEach(System.out::println);
// 2. get unique elements
Set<String> distinctCompanies = Arrays
.stream(companies)
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()))
.entrySet()
.stream()
.map(Map.Entry::getKey)
.collect(Collectors.toSet());
// 2.1 print unique elements
System.out.println("\n2. Unique elements in String[] array : \n");
distinctCompanies.forEach(System.out::println);
// 3. get duplicate elements
Set<String> duplicateCompanies = Arrays
.stream(companies)
.collect(Collectors.groupingBy(Function.identity(), Collectors.summingInt(c -> 1)))
.entrySet()
.stream()
.filter(company -> company.getValue() > 1)
.map(Map.Entry::getKey)
.collect(Collectors.toSet());
// 3.1 print duplicate elements
System.out.println("\n3. Duplicate elements in String[] array : \n");
duplicateCompanies.forEach(System.out::println);
// 4. get duplicate count using Map
Map<String, Long> duplicateCount = Arrays
.stream(companies)
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));
// 4.1 print Map for duplicate count
System.out.println("\n4. Map Key as Company and Value as its duplicate count : \n");
duplicateCount.forEach(
(key, value) -> System.out.println("Key : " + key + "\t Count : " + value)
);
}
}
Output:
1. Original String[] Array with duplicates :
Meta
Apple
Amazon
Netflix
Meta
Google
Apple
2. Unique elements in String[] array :
Netflix
Google
Meta
Apple
Amazon
3. Duplicate elements in String[] array :
Meta
Apple
4. Map Key as Company and Value as its duplicate count :
Key : Netflix Count : 1
Key : Google Count : 1
Key : Meta Count : 2
Key : Apple Count : 2
Key : Amazon Count : 1
6. Using Map.getOrDefault() and Collection.forEach() methods
- Create HashMap object to store String element as Key and their respective duplicate count as Value
- Note: HashMap doesn’t allow duplicate Key
- Iterate through original Arrays and store/put element into newly created HashMap to get
- unique elements as Key
- their respective duplicate count as Value
- At the time of iterating original Arrays,
- For Key, store unique element from Arrays
- For Value, start with 1 as count and increment by 1 for each duplicate using Map‘s getOrDefault() method
FindDuplicateCountUsingMapGetOrDefaultAndForEach.java
package in.bench.resources.java.stream;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
public class FindDuplicateCountUsingMapGetOrDefaultAndForEach {
public static void main(String[] args) {
// 1. String[] array
String[] companies = new String[]{
"Meta",
"Apple",
"Amazon",
"Netflix",
"Meta", // duplicate
"Google",
"Apple" // duplicate
};
// 1.1 print String[] array to console
System.out.println("1. Original String[] Array with duplicates : \n");
Arrays.stream(companies).forEach(System.out::println);
// 2. create HashMap object
Map<String, Integer> duplicateCountMap = new HashMap<>();
// 2.1 iterate and store duplicate count into Map object
Arrays
.stream(companies)
.forEach(company -> duplicateCountMap.put((String)company, // Map key
duplicateCountMap.getOrDefault((String)company, 0) + 1) // Map value
);
// 2.2 print unique elements
System.out.println("\n2. Unique elements in String[] array : \n");
duplicateCountMap
.entrySet()
.stream()
.forEach(entry -> System.out.println(entry.getKey()));
// 2.3 print duplicate elements
System.out.println("\n3. Duplicate elements in String[] array : \n");
duplicateCountMap
.entrySet()
.stream()
.filter(entry -> entry.getValue() > 1)
.forEach(entry -> System.out.println(entry.getKey()));
// 2.4 print Map with duplicate count
System.out.println("\n4. Map Key as Company and "
+ "Value as its duplicate count : \n"
+ duplicateCountMap);
}
}
Output:
1. Original String[] Array with duplicates :
Meta
Apple
Amazon
Netflix
Meta
Google
Apple
2. Unique elements in String[] array :
Netflix
Meta
Google
Apple
Amazon
3. Duplicate elements in String[] array :
Meta
Apple
4. Map Key as Company and Value as its duplicate count :
{Netflix=1, Meta=2, Google=1, Apple=2, Amazon=1}
7. Use Map.merge() method and lambda for counting duplicates
- Create HashMap object to store String element as Key and their respective duplicate count as Value
- Note: HashMap doesn’t allow duplicate Key
- Use Map.merge() method to store/put into newly created HashMap to get
- unique elements as Key
- their respective duplicate count as Value
- At the time of iterating original Arrays,
- For Key, store unique element from Arrays
- For Value, start with 1 as count and use lambda expression (a, b) -> a + b for counting duplicates by adding/summing
FindDuplicateCountUsingMapMergeAndForEach.java
package in.bench.resources.java.stream;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
public class FindDuplicateCountUsingMapMergeAndForEach {
public static void main(String[] args) {
// 1. String[] array
String[] companies = new String[]{
"Meta",
"Apple",
"Amazon",
"Netflix",
"Meta", // duplicate
"Google",
"Apple" // duplicate
};
// 1.1 print String[] array to console
System.out.println("1. Original String[] Array with duplicates : \n");
Arrays.stream(companies).forEach(System.out::println);
// 2. create HashMap object
Map<String, Integer> duplicateCountMap = new HashMap<>();
// 2.1 iterate and store duplicate count into Map object
Arrays
.stream(companies)
.forEach(company -> duplicateCountMap.merge(company, 1, (a, b) -> a + b));
// 2.2 print unique elements
System.out.println("\n2. Unique elements in String[] array : \n");
duplicateCountMap
.entrySet()
.stream()
.forEach(entry -> System.out.println(entry.getKey()));
// 2.3 print duplicate elements
System.out.println("\n3. Duplicate elements in String[] array : \n");
duplicateCountMap
.entrySet()
.stream()
.filter(entry -> entry.getValue() > 1)
.forEach(entry -> System.out.println(entry.getKey()));
// 2.4 print Map with duplicate count
System.out.println("\n4. Map Key as Company and "
+ "Value as its duplicate count : \n"
+ duplicateCountMap);
}
}
Output:
1. Original String[] Array with duplicates :
Meta
Apple
Amazon
Netflix
Meta
Google
Apple
2. Unique elements in String[] array :
Netflix
Google
Meta
Apple
Amazon
3. Duplicate elements in String[] array :
Meta
Apple
4. Map Key as Company and Value as its duplicate count :
{Netflix=1, Google=1, Meta=2, Apple=2, Amazon=1}
Related Articles:
- Java 8 – How to find duplicate and its count in a Stream or List ?
- Java 8 – How to remove duplicates from ArrayList
- Java 8 – How to remove duplicates from LinkedList
- Java 8 – How to find duplicate and its count in an Arrays ?
- Java 8 – How to remove duplicate from Arrays ?
- Java 8 – Various ways to remove duplicate elements from Arrays
- Java 8 – How to remove an entry from HashMap by comparing keys
- Java 8 – How to remove an entry from HashMap by comparing values
References:
- https://docs.oracle.com/javase/8/docs/api/java/util/Arrays.html
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collector.html
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#distinct–
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#collect-java.util.stream.Collector-
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#counting–
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#groupingBy-java.util.function.Function-
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#summingInt-java.util.function.ToIntFunction-
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#toMap-java.util.function.Function-java.util.function.Function-
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#toList–
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#toSet–
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#filter-java.util.function.Predicate-
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#collect-java.util.function.Supplier-java.util.function.BiConsumer-java.util.function.BiConsumer-
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#forEach-java.util.function.Consumer-
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#map-java.util.function.Function-
- https://docs.oracle.com/javase/8/docs/api/java/util/Map.html
- https://docs.oracle.com/javase/8/docs/api/java/util/Map.html#forEach-java.util.function.BiConsumer-
- https://docs.oracle.com/javase/8/docs/api/java/util/Map.html#merge-K-V-java.util.function.BiFunction-
Happy Coding !!
Happy Learning !!