[FIXED] java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.DistributedFileSystem

Issue

I want to run the below program,using the hadoop-3.0:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;


public class HDFSFileTest {
public static void main(String[]args) {
    try {
        String fileName="input/test.txt";
        Configuration conf =new Configuration();
        conf.set("fs.defaultFS","hdfs://localhost:9000");
    conf.set("fs.hdfs.impl","org.apache.hadoop.hdfs.DistributedFileSystem");
        FileSystem fs=FileSystem.get(conf);
        if(fs.exists(new Path(fileName))) {
            System.out.println("File exists!");
        }
        else {
            System.out.println("File not exists!");
        }
    }
    catch(Exception e){
        e.printStackTrace();
    }
}
}

But I get the exception when I execute the code in eclipse:

log4j:WARN No appenders could be found for logger 
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class 
org.apache.hadoop.hdfs.DistributedFileSystem not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2559)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3254)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3286)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3337)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3305)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:476)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:225)
at HDFSFileTest.main(HDFSFileTest.java:13)
Caused by: java.lang.ClassNotFoundException: Class 
org.apache.hadoop.hdfs.DistributedFileSystem not found
at 

I have checked the package hadoop-common-3.0.1.jar, hadoop-hdfs-3.0.1.jar,and there is no such class called

 org.apache.hadoop.hdfs.DistributedFileSystem

Solution

You can download the client core jar for the hadoop version and try running the code again. This sometimes occurs due to jar missing issue.

The second thing is that "fs.hdfs.impl" is deprecated.

You can have a look into :- https://community.hortonworks.com/questions/32800/where-can-i-find-fshdfsimpl-property.html

Answered By - Deepan Ram
Answer Checked By - Senaida (JavaFixing Volunteer)