Java 8 – How to find duplicate and its count in an Arrays ?

In this article, we will discuss how to find and count duplicates in an Arrays in different ways

Find and count duplicates in an Arrays :

  1. Using Stream.distinct() method
  2. Using Stream.filter() and Collections.frequency() methods
  3. Using Stream.filter() and Set.add() methods
  4. Using Collectors.toMap() method and Method Reference Math::addExact for summation of duplicates
  5. Using Collectors.groupingBy() and Collectors.counting() methods
  6. Using Map.getOrDefault() and Collection.forEach() methods
  7. Using Map.merge() and Collection.forEach() methods and lambda Expression for counting duplicates

Let us discuss each one with example and description

1. Using Stream.distinct() method

  • Stream.distinct() method eliminates/removes duplicate from Original Arrays and store into new Arrays using toArray(String[]::new) method which results into unique elements in an Arrays
  • For finding duplicates,
    • Create a new List using original Arrays
    • Iterate through new List and remove elements by comparing elements in unique Arrays
    • Elements remaining in the new List will contain only duplicates

FindDuplicatesUsingStreamDistinctMethod.java

package in.bench.resources.java.stream;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

public class FindDuplicatesUsingStreamDistinctMethod {

	public static void main(String[] args) {

		// 1. String[] array
		String[] companies = new String[]{
				"Meta",
				"Apple",
				"Amazon",
				"Netflix",
				"Meta", // duplicate
				"Google",
				"Apple" // duplicate
		};


		// 1.1 print String[] array to console
		System.out.println("1. Original String[] Array with duplicates : \n");
		Arrays.stream(companies).forEach(System.out::println);


		// 2. get unique elements after removing duplicates
		String[] distinctCompanies = Arrays
				.stream(companies)
				.distinct()
				.toArray(String[]::new);


		// 2.1 print unique elements
		System.out.println("\n2. Unique elements in String[] array : \n");
		Arrays.stream(distinctCompanies).forEach(System.out::println);


		// 2.2 Unique element count
		System.out.println("\nNumber of Unique elements = " 
				+ distinctCompanies.length);


		// 3. create List<String> with original String[] array  elements
		List<String> originalCompanyList = new ArrayList<String>(
				Arrays.asList(companies));


		// 3. get duplicate elements
		for (String distinctCompany : distinctCompanies) {
			originalCompanyList.remove(distinctCompany);
		}


		// 3.1 print duplicate elements
		System.out.println("\n3. Duplicate elements in String[] array : \n");
		originalCompanyList.forEach(System.out::println);


		// 3.2 Duplicate element count
		System.out.println("\nNumber of Duplicate elements = " 
				+ originalCompanyList.size());
	}
}

Output:

1. Original String[] Array with duplicates : 

Meta
Apple
Amazon
Netflix
Meta
Google
Apple

2. Unique elements in String[] array : 

Meta
Apple
Amazon
Netflix
Google

Number of Unique elements = 5

3. Duplicate elements in String[] array : 

Meta
Apple

Number of Duplicate elements = 2

2. Using Stream.filter() and Collections.frequency() methods

  • Convert original Arrays into Set using collect(Collectors.toSet()) method which results into new Set with unique elements
  • For finding duplicates, use Stream.filter() method by checking/verifying whether Collections.frequency() method returns value greater than 1 or not
    • If it is greater than 1, then it means that there are duplicate elements present in the Original Arrays
    • Finally, store those elements into another new Set using collect(Collectors.toSet()) method

FindDuplicatesUsingStreamFilterAndCollectionsFrequency.java

package in.bench.resources.java.stream;

import java.util.Arrays;
import java.util.Collections;
import java.util.Set;
import java.util.stream.Collectors;

public class FindDuplicatesUsingStreamFilterAndCollectionsFrequency {

	public static void main(String[] args) {

		// 1. String[] array
		String[] companies = new String[]{
				"Meta",
				"Apple",
				"Amazon",
				"Netflix",
				"Meta", // duplicate
				"Google",
				"Apple" // duplicate
		};


		// 1.1 print String[] array to console
		System.out.println("1. Original String[] Array with duplicates : \n");
		Arrays.stream(companies).forEach(System.out::println);


		// 2. get unique elements after removing duplicates
		Set<String> distinctCompanies = Arrays
				.stream(companies)
				.collect(Collectors.toSet());


		// 2.2 print unique elements
		System.out.println("\n2. Unique elements in String[] array : \n");
		distinctCompanies.forEach(System.out::println);


		// 2.3 Unique element count
		System.out.println("\nNumber of Unique elements = " 
				+ distinctCompanies.size());


		// 3. get duplicate elements
		Set<String> duplicateCompanies = Arrays
				.stream(companies)
				.filter(company -> Collections.frequency(Arrays.asList(companies), company) > 1)
				.collect(Collectors.toSet());


		// 3.1 print duplicate elements
		System.out.println("\n3. Duplicate elements in String[] array : \n");
		duplicateCompanies.forEach(System.out::println);


		// 3.2 Duplicate element count
		System.out.println("\nNumber of Duplicate elements = " 
				+ duplicateCompanies.size());
	}
}

Output:

1. Original String[] Array with duplicates : 

Meta
Apple
Amazon
Netflix
Meta
Google
Apple

2. Unique elements in String[] array : 

Netflix
Meta
Google
Apple
Amazon

Number of Unique elements = 5

3. Duplicate elements in String[] array : 

Meta
Apple

Number of Duplicate elements = 2

3. Using Stream.filter() and Set.add() methods

  • Create HashSet object to store/add unique elements
  • For finding duplicates, use Stream.filter() method by adding elements into newly created HashSet object using add() method
    • If it returns false then it means that there are duplicates present in the Original Arrays
    • Finally, store those elements into another new Set using collect(Collectors.toSet()) method
  • By doing this,
    • Newly created HashSet object will contain only unique elements
    • Filtered stream contains duplicate elements in an another Set

FindDuplicatesUsingStreamFilterAndSetAddMethod.java

package in.bench.resources.java.stream;

import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;
import java.util.stream.Collectors;

public class FindDuplicatesUsingStreamFilterAndSetAddMethod {

	public static void main(String[] args) {

		// 1. String[] array
		String[] companies = new String[]{
				"Meta",
				"Apple",
				"Amazon",
				"Netflix",
				"Meta", // duplicate
				"Google",
				"Apple" // duplicate
		};


		// 1.1 print String[] array to console
		System.out.println("1. Original String[] Array with duplicates : \n");
		Arrays.stream(companies).forEach(System.out::println);


		// 2. create Set object to store unique elements
		Set<String> uniqueCompanies = new HashSet<>();


		// 3. get duplicate elements
		Set<String> duplicateCompanies = Arrays
				.stream(companies)
				.filter(company -> !uniqueCompanies.add(company))
				.collect(Collectors.toSet());


		// 2.1 print unique elements
		System.out.println("\n2. Unique elements in String[] array : \n");
		uniqueCompanies.forEach(System.out::println);


		// 2.2 Unique element count
		System.out.println("\nNumber of Unique elements = " + uniqueCompanies.size());


		// 3.1 print duplicate elements
		System.out.println("\n3. Duplicate elements in String[] array : \n");
		duplicateCompanies.forEach(System.out::println);


		// 3.2 Duplicate element count
		System.out.println("\nNumber of Duplicate elements = " 
				+ duplicateCompanies.size());
	}
}

Output:

1. Original String[] Array with duplicates : 

Meta
Apple
Amazon
Netflix
Meta
Google
Apple

2. Unique elements in String[] array : 

Netflix
Meta
Google
Apple
Amazon

Number of Unique elements = 5

3. Duplicate elements in String[] array : 

Meta
Apple

Number of Duplicate elements = 2

4. Using Collectors.toMap() and Math::addExact for counting duplicates

  • Collectors.toMap() method can be used to convert Stream/List or Arrays of Stream into Map with actual Stream/List/Arrays elements being Key and their duplicate count as Value
  • For Key,
    • We will use Function.identity() method or Lambda expression (element -> element)
  • For duplicate count, we can use Math::addExact
    • Method Reference Math::addExact can be used to add/sum duplicates in the Integer form

FindDuplicateCountUsingCollectorsToMap.java

package in.bench.resources.java.stream;

import java.util.Arrays;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;

public class FindDuplicateCountUsingCollectorsToMap {

	public static void main(String[] args) {

		// 1. String[] array
		String[] companies = new String[]{
				"Meta",
				"Apple",
				"Amazon",
				"Netflix",
				"Meta", // duplicate
				"Google",
				"Apple" // duplicate
		};


		// 1.1 print String[] array to console
		System.out.println("1. Original String[] Array with duplicates : \n");
		Arrays.stream(companies).forEach(System.out::println);


		// 2. get duplicate count using Map and Collectors.toMap()
		Map<String, Integer> duplicateCountMap = Arrays
				.stream(companies)
				.collect(
						Collectors.toMap(Function.identity(), company -> 1, Math::addExact)
						);


		// 2.1 print unique elements
		System.out.println("\n2. Unique elements in String[] array : \n");
		duplicateCountMap
		.entrySet()
		.stream()
		.forEach(entry -> System.out.println(entry.getKey()));


		// 2.2 print duplicate elements
		System.out.println("\n3. Duplicate elements in String[] array : \n");
		duplicateCountMap
		.entrySet()
		.stream()
		.filter(entry -> entry.getValue() > 1)
		.forEach(entry -> System.out.println(entry.getKey()));


		// 2.3 print Map with duplicate count
		System.out.println("\n4. Map Key as Company and Value as its duplicate count : \n");
		duplicateCountMap.forEach(
				(key, value) -> System.out.println("Key : " + key + "\t Count : " + value)
				);
	}
}

Output:

1. Original String[] Array with duplicates : 

Meta
Apple
Amazon
Netflix
Meta
Google
Apple

2. Unique elements in String[] array : 

Netflix
Google
Meta
Apple
Amazon

3. Duplicate elements in String[] array : 

Meta
Apple

4. Map Key as Company and Value as its duplicate count : 

Key : Netflix	 Count : 1
Key : Google	 Count : 1
Key : Meta	 Count : 2
Key : Apple	 Count : 2
Key : Amazon	 Count : 1

5. Using Collectors.groupingBy() method

  • Collectors.groupingBy() method accepts 2 values,
    • 1st input-argument can be used as Key
    • 2nd input-argument can be used to store duplicate count as Value
  • So basically Collectors.groupingBy() method used to convert Stream/List/Arrays into Map according to classification/category
  • For Key,
    • We will use Function.identity() method or Lambda expression (element -> element)
  • For counting duplicates as Value, we can use either of the below methods,
    1. Collectors.counting() method
    2. Collectors.summingInt() method

FindDuplicateCountUsingGroupingByAndCounting.java

package in.bench.resources.java.stream;

import java.util.Arrays;
import java.util.Map;
import java.util.Set;
import java.util.function.Function;
import java.util.stream.Collectors;

public class FindDuplicateCountUsingGroupingByAndCounting {

	public static void main(String[] args) {

		// 1. String[] array
		String[] companies = new String[]{
				"Meta",
				"Apple",
				"Amazon",
				"Netflix",
				"Meta", // duplicate
				"Google",
				"Apple" // duplicate
		};


		// 1.1 print String[] array to console
		System.out.println("1. Original String[] Array with duplicates : \n");
		Arrays.stream(companies).forEach(System.out::println);


		// 2. get unique elements
		Set<String> distinctCompanies = Arrays
				.stream(companies)
				.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()))
				.entrySet()
				.stream()
				.map(Map.Entry::getKey)
				.collect(Collectors.toSet());


		// 2.1 print unique elements
		System.out.println("\n2. Unique elements in String[] array : \n");
		distinctCompanies.forEach(System.out::println);


		// 3. get duplicate elements
		Set<String> duplicateCompanies = Arrays
				.stream(companies)
				.collect(Collectors.groupingBy(Function.identity(), Collectors.summingInt(c -> 1)))
				.entrySet()
				.stream()
				.filter(company -> company.getValue() > 1)
				.map(Map.Entry::getKey)
				.collect(Collectors.toSet());


		// 3.1 print duplicate elements
		System.out.println("\n3. Duplicate elements in String[] array : \n");
		duplicateCompanies.forEach(System.out::println);


		// 4. get duplicate count using Map
		Map<String, Long> duplicateCount = Arrays
				.stream(companies)
				.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));


		// 4.1 print Map for duplicate count
		System.out.println("\n4. Map Key as Company and Value as its duplicate count : \n");
		duplicateCount.forEach(
				(key, value) -> System.out.println("Key : " + key + "\t Count : " + value)
				);
	}
}

Output:

1. Original String[] Array with duplicates : 

Meta
Apple
Amazon
Netflix
Meta
Google
Apple

2. Unique elements in String[] array : 

Netflix
Google
Meta
Apple
Amazon

3. Duplicate elements in String[] array : 

Meta
Apple

4. Map Key as Company and Value as its duplicate count : 

Key : Netflix	 Count : 1
Key : Google	 Count : 1
Key : Meta	 Count : 2
Key : Apple	 Count : 2
Key : Amazon	 Count : 1

6. Using Map.getOrDefault() and Collection.forEach() methods

  • Create HashMap object to store String element as Key and their respective duplicate count as Value
  • Note: HashMap doesn’t allow duplicate Key
  • Iterate through original Arrays and store/put element into newly created HashMap to get
    • unique elements as Key
    • their respective duplicate count as Value
  • At the time of iterating original Arrays,
    • For Key, store unique element from Arrays
    • For Value, start with 1 as count and increment by 1 for each duplicate using Map‘s getOrDefault() method

FindDuplicateCountUsingMapGetOrDefaultAndForEach.java

package in.bench.resources.java.stream;

import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;

public class FindDuplicateCountUsingMapGetOrDefaultAndForEach {

	public static void main(String[] args) {

		// 1. String[] array
		String[] companies = new String[]{
				"Meta",
				"Apple",
				"Amazon",
				"Netflix",
				"Meta", // duplicate
				"Google",
				"Apple" // duplicate
		};


		// 1.1 print String[] array to console
		System.out.println("1. Original String[] Array with duplicates : \n");
		Arrays.stream(companies).forEach(System.out::println);


		// 2. create HashMap object
		Map<String, Integer> duplicateCountMap = new HashMap<>();


		// 2.1 iterate and store duplicate count into Map object
		Arrays
		.stream(companies)
		.forEach(company -> duplicateCountMap.put((String)company, // Map key
				duplicateCountMap.getOrDefault((String)company, 0) + 1) // Map value
				);


		// 2.2 print unique elements
		System.out.println("\n2. Unique elements in String[] array : \n");
		duplicateCountMap
		.entrySet()
		.stream()
		.forEach(entry -> System.out.println(entry.getKey()));


		// 2.3 print duplicate elements
		System.out.println("\n3. Duplicate elements in String[] array : \n");
		duplicateCountMap
		.entrySet()
		.stream()
		.filter(entry -> entry.getValue() > 1)
		.forEach(entry -> System.out.println(entry.getKey()));


		// 2.4 print Map with duplicate count
		System.out.println("\n4. Map Key as Company and "
				+ "Value as its duplicate count : \n" 
				+ duplicateCountMap);
	}
}

Output:

1. Original String[] Array with duplicates : 

Meta
Apple
Amazon
Netflix
Meta
Google
Apple

2. Unique elements in String[] array : 

Netflix
Meta
Google
Apple
Amazon

3. Duplicate elements in String[] array : 

Meta
Apple

4. Map Key as Company and Value as its duplicate count : 
{Netflix=1, Meta=2, Google=1, Apple=2, Amazon=1}

7. Use Map.merge() method and lambda for counting duplicates

  • Create HashMap object to store String element as Key and their respective duplicate count as Value
  • Note: HashMap doesn’t allow duplicate Key
  • Use Map.merge() method to store/put into newly created HashMap to get
    • unique elements as Key
    • their respective duplicate count as Value
  • At the time of iterating original Arrays,
    • For Key, store unique element from Arrays
    • For Value, start with 1 as count and use lambda expression (a, b) -> a + b for counting duplicates by adding/summing

FindDuplicateCountUsingMapMergeAndForEach.java

package in.bench.resources.java.stream;

import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;

public class FindDuplicateCountUsingMapMergeAndForEach {

	public static void main(String[] args) {

		// 1. String[] array
		String[] companies = new String[]{
				"Meta",
				"Apple",
				"Amazon",
				"Netflix",
				"Meta", // duplicate
				"Google",
				"Apple" // duplicate
		};


		// 1.1 print String[] array to console
		System.out.println("1. Original String[] Array with duplicates : \n");
		Arrays.stream(companies).forEach(System.out::println);


		// 2. create HashMap object
		Map<String, Integer> duplicateCountMap = new HashMap<>();


		// 2.1 iterate and store duplicate count into Map object
		Arrays
		.stream(companies)
		.forEach(company -> duplicateCountMap.merge(company, 1, (a, b) -> a + b));


		// 2.2 print unique elements
		System.out.println("\n2. Unique elements in String[] array : \n");
		duplicateCountMap
		.entrySet()
		.stream()
		.forEach(entry -> System.out.println(entry.getKey()));


		// 2.3 print duplicate elements
		System.out.println("\n3. Duplicate elements in String[] array : \n");
		duplicateCountMap
		.entrySet()
		.stream()
		.filter(entry -> entry.getValue() > 1)
		.forEach(entry -> System.out.println(entry.getKey()));


		// 2.4 print Map with duplicate count
		System.out.println("\n4. Map Key as Company and "
				+ "Value as its duplicate count : \n" 
				+ duplicateCountMap);
	}
}

Output:

1. Original String[] Array with duplicates : 

Meta
Apple
Amazon
Netflix
Meta
Google
Apple

2. Unique elements in String[] array : 

Netflix
Google
Meta
Apple
Amazon

3. Duplicate elements in String[] array : 

Meta
Apple

4. Map Key as Company and Value as its duplicate count : 
{Netflix=1, Google=1, Meta=2, Apple=2, Amazon=1}

References:

Happy Coding !!
Happy Learning !!

Java 8 - Sorting Arrays in Ascending and Descending order
Java 8 - How to Merge or Concatenate 2 Arrays using Stream API