How Do You Solve This Problem “Error: Could not find or load main class org.apache.hadoop.util.VersionInfo”

Problem scenario
You run "hadoop version" but you receive this message "Error: Could not find or load main class org.apache.hadoop.util.VersionInfo". What do you do?

Possible Solution #1
Use "sudo " before the "hadoop version" command.

Possible Solution #2
Use "sudo -i " before the "hadoop version" command.

Possible Solution #3
Use a different user. Sometimes Hadoop is set up to be run under a different user (e.g., hduser).

Possible Solution #4
Re-install Hadoop. If you need assistance, see this posting.

How Do You Get Hadoop Commands to Work from Any Directory without Using the Full Path?

Problem scenario
Hadoop is installed on Linux. But hadoop version and other hadoop commands are not working. What should you do?

Solution
Find the hadoop executable in a directory named bin. It is often "/usr/local/hadoop/bin/hadoop". Ultimately you need to find the directory that houses this "bin." has a subdirectory with "bin" and "hadoop" inside, run these two commands:

sudo find / -name hadoop -type f
whereis hadoop

Run these commands interactively where "/usr/local/hadoop" is the directory that is the parent of the subdirectory named "bin" that is the parent of the hadoop executable. 

export HADOOP_PREFIX=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_HOME=$HADOOP_PREFIX
export HADOOP_HDFS_HOME=$HADOOP_PREFIX
export YARN_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_PREFIX/lib/native
export PATH=$PATH:$HADOOP_PREFIX/sbin:$HADOOP_PREFIX/bin
source ~/.bashrc

Test it. Run this hadoop version

Now to ensure this persists through a reboot, run this: sudo vi ~/.bashrc Append the lines below to the file (and then save the file):

export HADOOP_PREFIX=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_HOME=$HADOOP_PREFIX
export HADOOP_HDFS_HOME=$HADOOP_PREFIX
export YARN_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_PREFIX/lib/native
export PATH=$PATH:$HADOOP_PREFIX/sbin:$HADOOP_PREFIX/bin

How Do You Troubleshoot the Go Programming Error ‘non-name m1[“foo”] on left side of :=’?

Problem scenario
You are running a Go program. You see this message about "command-line-arguments":

non-name m1["foo"] on left side of :=

What should you do?

Possible Solution
Can you try to use "=" instead of ":="? This may fix your problem.

The := is a variable declaration and an assignment. If your map has been declared, individual values in it should be assigned with an "=" (equals sign).

How Do You Use a Function in Golang?

Problem scenario
You are trying to learn to program in Go. How do you use a function?

Solution
Prerequisite
This assumes that you have installed Golang; if you need assistance with this, see this posting.

Procedures
1. Create a file called b.go with this as the content:

package main

import "fmt"

// ContintFunction will display a message
func ContintFunction() {
        fmt.Println("Hello from Continual Integration")
}

func main() {
        ContintFunction()
}

2. Run it like this: go run b.go

How Do You Find out What Packages You Can Install That Are Related to a Certain Category when Using Ubuntu Linux?

Problem scenario
You want to find what Java packages are available on the configured repositories of the Linux server you are using. What do you do?

Solution
Run a command like this: apt-cache search java

You can replace "java" with a word of the package you want to install.

If you want to learn more about the package and have its exact name, run this command: apt-cache showpkg java-common

How Do You Find out What Packages You Can Install That Are Related to a Certain Category when Using Linux SUSE?

Problem scenario
You want to find what Java packages are available on the configured repositories of the Linux SUSE server you are using. What do you do?

Solution
Run this command: zypper se java

You can replace "java" with a word of the package you want to install.

In Python, How Do You Call a Bound Function as a New Thread with the thread Module?

Problem scenario
You want to write a program to call a bound function in a new thread. How do you do this?

Solution
Run this program (e.g., python foobar.py):

import _thread as thread
 class mighty:
   def cool():
     print("Cool.")
def contint():
   print("Hello!")   
if name == "main":
     foobar = thread.start_new_thread(mighty.cool, () )
     print("thread finished…exiting")

How Do You Generate a 100 GB Log File?

Problem scenario
You want to generate a large log file to use later on (e.g., for the Elastic Stack or Splunk). How do you create a 100 GB log file?

Solution
Find an example log to base the generation off. Run these commands to find a log that you would want to copy its format:

cd /var/log
ls -lh --sort=size
sudo tail foobar # where foobar is the name of the log file you want to sample

2. Draft a script like this, but substitute "foobar" with the name of the file you want to change.

for i in {1..1000}
  do cat /var/log/foobar >> /tmp/newfile.log
done

# The new file will be 1,000 times as large as the original log file.  
# You will want to change 1000 to a different value so the resulting file is 100 GB.

3. Name the script "cool.sh" and run it: bash cool.sh