A Technical I.T./DevOps Blog
Problem scenario
You try to start an older version of Presto. It keeps failing with a variety of errors. You are using Python 3. Where can you find a launcher.py file that uses Python 3 for Apache Presto?
Solution
Go to this GitHub link.
…
Continue reading “Where Can You Obtain a Python 3 Version of launcher.py in Presto?”
Problem scenario
You are trying to run a hadoop command (to kick off a mapreduce job). But you get this error:
“Exception in thread “main” java.nio.file.AccessDeniedException: /home/jdoe/./mapper.py”
What should you do?
Solution (short version)
Change to a directory where the user can write files to. Retry the command.
Solution (long version)
Create a directory that is owned by the user and the group associated with the user that is running this command.
…
Problem scenario
You try to run a playbook. But you get a message like this: ‘fatal: … “AnsibleUndefinedVariable” ‘hadoop_env’ is undefined’. What should you do?
Root cause: a variable you defined (e.g., in a playbook or role) is not getting assigned when you run the playbook.
Possible solution #1
Find the vars directory or create it.
…
Continue reading “How Do You Troubleshoot the Ansible Error ‘”hadoop_env” is undefined’?”
Problem scenario
You want a script to install Apache Flink on any type of Linux (including CentOS/RHEL/Fedora, Debian/Ubuntu and SUSE). What should you do?
Solution
Prerequisites
When using Ubuntu/Debian or SUSE servers, 1 GB of RAM is sufficient. For CentOS/RHEL/Fedora, we recommend either more RAM or use virtual memory. To find a script that will create swap space,
…
Continue reading “How Do You Install Apache Flink on Any Type of Linux?”
Problem scenario
You want to install Apache Presto to try it out. How do you do this with a script that will work on Debian/Ubuntu, CentOS/RHEL/Fedora and/or SUSE?
Solution
Prerequisites
i. You have Python installed as “python” (not just python3). If you need assistance installing Python, see this posting. Verify that python –version works.
…
Continue reading “How Do You Install Apache Presto on Any Type of Linux?”
Problem scenario
You run sudo bash start-yarn.sh but you receive this message:
ERROR: Attempting to operate on yarn resourcemanager as root
ERROR: but there is no YARN_RESOURCEMANAGER_USER defined. Aborting operation.
What should you do?
Solution
1. Modify start-yarn.sh. Underneath the last section of comments, place three lines with the following text:
YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root
2. Modify stop-yarn.sh.
…
Problem scenario
You have Hadoop installed on a Linux machine. You want to install Apache Sqoop. What do you do?
Solution
Prerequisite
Verify one of these commands work:
hadoop dfs -ls
or
hdfs dfs ls
# If you need assistance installing Hadoop, see this posting.
Procedures
Download the installation media here: http://www.apache.org/dyn/closer.cgi/sqoop/
There are other sources.
…
Problem scenario
You run “hadoop version” but you receive this message “Error: Could not find or load main class org.apache.hadoop.util.VersionInfo”. What do you do?
Possible Solution #1
Use “sudo ” before the “hadoop version” command.
Possible Solution #2
Use “sudo -i ” before the “hadoop version” command.
Possible Solution #3
Use a different user.
…
Problem scenario
Hadoop is installed on Linux. But hadoop version and other hadoop commands are not working. What should you do?
Solution
Find the hadoop executable in a directory named bin. It is often “/usr/local/hadoop/bin/hadoop”. Ultimately you need to find the directory that houses this “bin.” has a subdirectory with “bin” and “hadoop” inside, run these two commands:
sudo find / -name hadoop -type f
whereis hadoop
Run these commands interactively where “/usr/local/hadoop” is the directory that is the parent of the subdirectory named “bin” that is the parent of the hadoop executable.
…
Problem scenario
You are administering Linux SUSE machines. You want to see if Hadoop is installed on them. The command hadoop version does not work.
Solution
Run this command:
sudo find / -name hadoop -type f
From the results above, you can probably find the file and path of the executable. It will likely not be in /var/ or /tmp/.
…