Java 8 – Various ways to remove duplicate elements from Arrays

April 13, 2017 SJ Collection, Java 8 0

In this article, we will discuss various ways to remove duplicate elements from Arrays in Java i.e.;

Ways to remove duplicate elements from Arrays:

  1. Using List implemented classes
  2. Using Set implemented classes
  3. Using combination of both List & Set implemented classes
  4. Without using any collection classes
  5. Using Java 8 Streams

Let’s discuss one-by-one in detail with example/explanation

1. Using List implemented classes (i.e.; ArrayList class)

  1. Iterate through original Arrays to read duplicate elements
  2. Initialize ArrayList (i.e.; to store unique elements, after checking)
  3. While iterating String Array, check whether element already present in the unique list (created in step-2)
  4. If it doesn’t contains inside unique List, then add element to unique List
  5. Repeat step 3-4, until all elements of Arrays are compared and unique elements are stored inside List
  6. Convert unique list into Arrays using toArray() method
  7. Again, iterate through Arrays to print unique elements

RemoveDuplicateFromArraysUsingList.java

package in.bench.resources.java.arrays;

import java.util.ArrayList;
import java.util.List;

public class RemoveDuplicateFromArraysUsingList {

	// main() method - entry point for JVM
	public static void main(String[] args) {

		// initialize an Arrays with duplicate values
		String[] strArray = {
				"Sun",
				"Apple",
				"JBoss",
				"Whatsup",
				"Apple", // duplicate
				"BEA Weblogic",
				"JBoss" // duplicate
		};

		// invoke removeDuplicatesFromArray() with above initialized Arrays
		removeDuplicatesFromArray(strArray);
	}

	/**
	 * This method removes duplicate elements from Arrays
	 * using List class and finally prints unique elements
	 * @param strArray
	 */
	public static void removeDuplicatesFromArray(String[] strArray) {

		// Iterating using enhanced for-loop
		System.out.println("Original Arrays with duplicates:\n");
		for(String str : strArray) {
			System.out.println(str);
		}

		// initialize an empty ArrayList of String type
		List<String> uniqueList = new ArrayList<String>();

		// remove duplicate iterating through Arrays
		for(int index = 0; index < strArray.length; index++) {

			// check whether list contains duplicate, while iterating
			if(!uniqueList.contains(strArray[index])) {

				// if it is doesn't contains, then add to unique list
				uniqueList.add(strArray[index]);
			}
		}

		// convert unique List into Array using toArray() method
		strArray = uniqueList.toArray(new String[0]);

		// Iterating using enhanced for-loop
		System.out.println("\n\nUnique elements:\n");
		for(String str : strArray) {
			System.out.println(str);
		}
	}
}

Output:

Original Arrays with duplicates:

Sun
Apple
JBoss
Whatsup
Apple
BEA Weblogic
JBoss


Unique elements:

Sun
Apple
JBoss
Whatsup
BEA Weblogic

2. Using Set implemented classes (i.e.; HashSet class)

  1. Iterate through original Arrays to read duplicate elements
  2. Initialize HashSet (i.e.; to store unique elements, while iterating)
  3. While iterating String Array, just simply add elements to HashSet; because Set allows only unique items thereby removing duplicate items
  4. Convert Set into Arrays using toArray() method
  5. Again, iterate through Arrays to print unique elements

RemoveDuplicateFromArraysUsingSet.java

package in.bench.resources.java.arrays;

import java.util.HashSet;
import java.util.Set;

public class RemoveDuplicateFromArraysUsingSet {

	// main() method - entry point for JVM
	public static void main(String[] args) {

		// initialize an Arrays with duplicate values
		String[] strArray = {
				"Sun",
				"Apple",
				"JBoss",
				"Whatsup",
				"Apple", // duplicate
				"BEA Weblogic",
				"JBoss" // duplicate
		};

		// invoke removeDuplicatesFromArray() with above initialized Arrays
		removeDuplicatesFromArray(strArray);
	}

	/**
	 * This method removes duplicate elements from Arrays
	 * using Set class and finally prints unique elements
	 * @param strArray
	 */
	public static void removeDuplicatesFromArray(String[] strArray) {

		// Iterating using enhanced for-loop
		System.out.println("Original Arrays with duplicates:\n");
		for(String str : strArray) {
			System.out.println(str);
		}

		// initialize an empty HashSet of String type
		Set<String> set = new HashSet<String>();

		// iterate through Arrays to remove duplicates
		for(int index = 0; index < strArray.length; index++) {

			// add elements to HashSet, which doesn't allow duplicates
			set.add(strArray[index]);
		}

		// convert unique Set into Arrays using toArray() method
		strArray = set.toArray(new String[0]);

		// Iterating using enhanced for-loop
		System.out.println("\n\nUnique elements:\n");
		for(String str : strArray) {
			System.out.println(str);
		}
	}
}

Output:

Original Arrays with duplicates:

Sun
Apple
JBoss
Whatsup
Apple
BEA Weblogic
JBoss


Unique elements:

Apple
JBoss
Whatsup
BEA Weblogic
Sun

Difference between above 2 implemented approaches:

  • In the 1st way, we have to check manually for each items inside String Arrays with unique List and then add to List for each iteration
  • Whereas for Set implemented classes, we don’t need to do any comparison or checking because Set allows only unique items thereby removing duplicate items
  • In addition to this, 1st approach maintains insertion order whereas 2nd approach follows random order as HashSet internally uses hashing algorithm to store elements

3. Using both List & Set classes

  1. Iterate through original Arrays to read duplicate elements
  2. Convert Arrays into List; using Arrays’ asList(arrObj); method
  3. Add converted List object into HashSet using inter-conversion collection constructor; for removing duplicates
  4. Create new Arrays of required data-type
  5. Convert Set to Arrays (using newly created Arrays in step-4)
  6. Again, iterate through HashSet to print unique elements

RemoveDuplicateUsingListAndSet.java

package in.bench.resources.java.arrays;

import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

public class RemoveDuplicateUsingListAndSet {

	// main() method - entry point for JVM
	public static void main(String[] args) {

		// initialize an Arrays with duplicate values
		String[] strArray = {
				"Sun",
				"Apple",
				"JBoss",
				"Whatsup",
				"Apple", // duplicate
				"BEA Weblogic",
				"JBoss" // duplicate
		};

		// invoke removeDuplicatesFromArray() with above initialized Arrays
		removeDuplicatesFromArray(strArray);
	}

	/**
	 * This method removes duplicate elements from Arrays
	 * using List and Set classes and finally prints unique elements
	 * @param strArray
	 */
	public static void removeDuplicatesFromArray(String[] strArray) {

		// Iterating using enhanced for-loop
		System.out.println("Original Arrays with duplicates:\n");
		for(String str : strArray) {
			System.out.println(str);
		}

		// convert Arrays into List
		List<String> lst = Arrays.asList(strArray);

		// again convert List into Set, for removing duplicates
		// using inter-conversion constructor
		Set<String> set = new HashSet<String>(lst);

		// create new String[] with no. of elements inside Set
		String[] uniqueArr = new String[set.size()];

		// convert back Set into Arrays
		set.toArray(uniqueArr);

		// Iterating using enhanced for-loop
		System.out.println("\n\nUnique elements:\n");
		for(String uStr : uniqueArr) {
			System.out.println(uStr);
		}
	}
}

Output:

Original Arrays with duplicates:

Sun
Apple
JBoss
Whatsup
Apple
BEA Weblogic
JBoss


Unique elements:

Apple
JBoss
Whatsup
BEA Weblogic
Sun

Note : above discussed ways are very easy for development, but in coding interview they might ask about removing duplicates elements without using any Collection classes

4. Without using any collection classes

  1. Iterate through original Arrays to read duplicate elements
  2. Construct outer for-loop for iterating Arrays elements
  3. Construct inner for-loop for iterating Arrays elements starting from 2nd position (or 1st index)
  4. Within inner for-loop, check whether outer for-loop element with inner for-loop elements (while iterating)
  5. If comparison comes out to be true, then assign last element at this position (thereby replacing duplicate element)
  6. While doing step 5, decrement the size of Arrays by 1 as well as decrement inner for-loop count (this will give unique elements inside Arrays after all iterations are completed)
  7. Finally print Arrays elements again

RemoveDuplicateOfArrays.java

package in.bench.resources.java.arrays;

import java.util.Arrays;

public class RemoveDuplicateOfArrays {

	// main() method - entry point for JVM
	public static void main(String[] args) {

		// initialize an Arrays with duplicate values
		String[] strArray = {
				"Sun",
				"Apple",
				"JBoss",
				"Whatsup",
				"Apple", // duplicate
				"BEA Weblogic",
				"JBoss" // duplicate
		};

		// invoke removeDuplicatesFromArray() with above initialized Arrays
		removeDuplicatesFromArray(strArray);
	}

	/**
	 * This method removes duplicate elements from Arrays
	 * using 2 for-loops and finally prints unique elements
	 * @param strArray
	 */
	public static void removeDuplicatesFromArray(String[] strArray) {

		// Iterating using enhanced for-loop
		System.out.println("Original Arrays with duplicates:\n");
		for(String str : strArray) {
			System.out.println(str);
		}

		// get size of the Arrays using length property
		int sizeOfArrays = strArray.length;

		// remove duplicates, using 2 for-loops

		// outer for-loop
		for(int outForLoop = 0; outForLoop < sizeOfArrays; outForLoop++) {

			// inner for-loop
			for(int inForLoop = outForLoop + 1;
					inForLoop < sizeOfArrays; inForLoop++) {

				// check whether, it already contains this element
				if(strArray[outForLoop] == strArray[inForLoop]){

					// if elements are same, then replace with last item
					strArray[inForLoop] = strArray[sizeOfArrays-1];

					// at the same time, decrement size value by 1
					sizeOfArrays--;

					// also decrement inner for-loop
					inForLoop--;
				}
			}
		}

		// create new String[] to copy unique elements
		String[] uniqueStrArrays = Arrays.copyOf(strArray, sizeOfArrays);

		// Iterating using enhanced for-loop
		System.out.println("\n\nUnique elements:\n");
		for(String str : uniqueStrArrays) {
			System.out.println(str);
		}
	}
}

Output:

Original Arrays with duplicates:

Sun
Apple
JBoss
Whatsup
Apple
BEA Weblogic
JBoss


Unique elements:

Sun
Apple
JBoss
Whatsup
BEA Weblogic

5. Using Java 8 Streams

  1. Iterate through original Arrays to read duplicate elements
  2. Using Streams class, remove duplicate elements (i.e.; using distinct() method)
  3. At the same time, convert back to Object[] arrays by invoking toArray() method
  4. Finally print Arrays elements again

RemoveDuplicateUsingStreamsInJava8.java

package in.bench.resources.java.arrays;

import java.util.Arrays;

public class RemoveDuplicateUsingStreamsInJava8 {

	// main() method - entry point for JVM
	public static void main(String[] args) {

		// initialize an Arrays with duplicate values
		String[] strArray = {
				"Sun",
				"Apple",
				"JBoss",
				"Whatsup",
				"Apple", // duplicate
				"BEA Weblogic",
				"JBoss" // duplicate
		};

		// invoke removeDuplicatesFromArray() with above initialized Arrays
		removeDuplicatesFromArray(strArray);
	}

	/**
	 * This method removes duplicate elements from Arrays
	 * using Streams class in Java 8 and finally prints unique elements
	 * @param strArray
	 */
	public static void removeDuplicatesFromArray(String[] strArray) {

		// Iterating original Arrays using enhanced for-loop
		System.out.println("Original Arrays with duplicates:\n");
		for(String str : strArray) {
			System.out.println(str);
		}

		// convert to unique/distinct Arrays using Java 8 Streams class
		Object[] uniqueArrays = Arrays
				.stream(strArray)
				.distinct()
				.toArray();

		// Iterating unique Arrays using enhanced for-loop
		System.out.println("\n\nUnique elements:\n");
		for(Object obj : uniqueArrays) {
			System.out.println(obj.toString());
		}
	}
}

Output:

Original Arrays with duplicates:

Sun
Apple
JBoss
Whatsup
Apple
BEA Weblogic
JBoss


Unique elements:

Sun
Apple
JBoss
Whatsup
BEA Weblogic

Related Articles:

References:

Hope, you found this article very helpful. If you any suggestion or want to contribute any other way or tricky situation you faced during Interview hours, then share with us. We will include that code here.

Happy Coding !!
Happy Learning !!

Java 8 - Conversion of Arrays to List
Java - Remove duplicate elements from ArrayList

Related posts:

  1. Java 8 – Find First and Last entries in a Map or HashMap ?
  2. Java – Remove duplicate elements from ArrayList
  3. Java 8 – Find sum and average of a List or ArrayList ?
  4. Java 8 – Find First and Last elements in a Set or HashSet ?