A Technical I.T./DevOps Blog
Problem scenario
You use the “hdfs dfsadmin -report” command on your Hadoop NameNode server. You see that there is a small amount of “DFS Remaining.” You expect many more GB. You also see one datanode not listed under the “Live datanodes” section. All of your live datanodes are not appearing in hdfs,
Your /usr/local/hadoop/etc/hadoop/slaves file has the DNS name of a server you expect to be a datanode.
…
Problem scenario
How do you know if YARN or Hadoop’s NameNode services are running or not?
Solution
Use the “sudo jps” command. Some Linux users may not have sufficient permissions to install it.
If the command is not found, go to the option below for your distribution of Linux:
If you are running a RedHat derivative (e.g.,
…
Continue reading “How Do You Know If Yarn or Hadoop’s NameNode Services Are Running?”
Problem scenario
You need to use this following command on a CentOS/RedHat/Fedora Linux server in a /home/hadoop/ directory:
mvn package -Pdist,native -DskipTests -Dtar
The Maven build is failing. When you run the command with the “-e” flag for verbosity you see this:
[INFO] BUILD FAILURE
…
[WARNING] The requested profile “dist” could not be activated because it does not exist.
[WARNING] The requested profile “native” could not be activated because it does not exist.
…
Continue reading “How Do You Troubleshoot the Maven Build Failure of No “POM in this directory”?”
Problem scenario
When trying to run an hdfs command you see this message:
“WARN ipc.Client: Failed to connect to server: localhost/127.0.0.1:54310: try once and fail.
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:681)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:777)
at org.apache.hadoop.ipc.Client$Connection.access$3500(Client.java:409)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1542)
at org.apache.hadoop.ipc.Client.call(Client.java:1373)
at org.apache.hadoop.ipc.Client.call(Client.java:1337)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:787)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:398)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1700)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1436)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1433)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1433)
at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:64)
at org.apache.hadoop.fs.Globber.doGlob(Globber.java:269)
at org.apache.hadoop.fs.Globber.glob(Globber.java:148)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1685)
at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:326)
at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:235)
at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:218)
at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:103)
at org.apache.hadoop.fs.shell.Command.run(Command.java:165)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:315)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:378)
ls: Call From contint/10.10.10.10 to localhost:54310 failed on connection exception: java.net.ConnectException: Connection refused; …
Problem scenario
You want to add a file to Hadoop. You are trying to run a basic Hadoop command to copy a file into HDFS. You get this error: copyFromLocal: `hdfs://localhost:54310/user/…’: No such file or directory
How do you copy a file from your OS into HDFS?
Solution
Do one of the following:
Option 1. Run this command to create a new directory (substitute “jdoe” with the name of your user):
hdfs dfs -mkdir -p /user/jdoe/contint
# Now repeat your copy command
Option 2.
…
Continue reading “How Do You Copy a File into HDFS without the Error “No such file or directory”?”
Problem scenario
You run an hdfs command and you get this:
[Fatal Error] core-site.xml:2:6: The processing instruction target matching “[xX][mM][lL]” is not allowed.
17/09/25 04:21:00 FATAL conf.Configuration: error parsing conf core-site.xml
org.xml.sax.SAXParseException; systemId: file:/home/hadoop/hadoop/etc/hadoop/core-site.xml; lineNumber: 2; columnNumber: 6; The processing instruction target matching “[xX][mM][lL]” is not allowed.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2531)
…
Continue reading “How Do You Troubleshoot a Fatal HDFS Error?”
Problem scenario
You are looking for the Hadoop components’ versions. You run these commands:
hadoop version
hdfs version
yarn version
You notice the output is the same for each of the three commands above. You are not sure if Apache Spark has been installed. What do you do?
Solution
Run this command:
spark-submit –version
…
Continue reading “How Do You Know If Apache Spark Has Been Installed?”
Problem scenario
You have recently deployed Apache Spark to AWS. You see the EC-2 instances were created. But you cannot access them over the web UI (even over ports 4140, 8088, or 50070). You cannot access the instances via Putty. You changed your normal Security Group to allow TCP communication from your work station’s IP address. What should you do to connect to your new Spark instance for the first time?
…
Continue reading “How Do You Connect to Your Apache Spark Deployment in AWS?”
Problem scenario
You want to deploy Apache Spark to AWS. How do you do this?
Solution
1. Log into the AWS management console. Once in, go to this link.
2. Click “Create cluster” and then “Quick Create”
3. For Software Configuration, choose “Spark:…”
4. For “Security and access”, for EC2 key pair, choose the key pair you desire.
…
Continue reading “How Do You Deploy an Apache Spark Cluster in AWS?”
Advanced Analytics with Spark: Patterns for Learning from Data at Scale by Sandy Ryza, Uri Laserson, Sean Owen and Josh Wills
Agile Data Science: Building Data Analytics Applications with Hadoop by Russell Jurney
Apache Drill: The SQL query engine for Hadoop and NoSQL by Ted Dunning, Ellen Friedman, Tomer Shiran and Jacques Nadeau
Apache Flume: Distributed Log Collection for Hadoop -Second Edition by Steve Hoffman
Apache Hadoop YARN: Moving beyond MapReduce and Batch Processing with Apache Hadoop 2 (Addison-Wesley Data &
…