How Do You Install Splunk on a Debian/Ubuntu Linux Server?

Problem scenario
You have a Debian or Ubuntu Linux server. You want to install a trial version of Splunk. What should you do?

Solution
1. Sign up for an account to get the installation media: https://www.splunk.com/en_us/download.html
2. Click on "Linux"
3. Obtain the .deb file.
4. Put the .deb file on the Linux server.
5. Install the file. The command will look something like this:
sudo dpkg -i splunk-7.3.2-c60db69f8e32-linux-2.6-amd64.deb

6. Run this after it is installed: sudo /opt/splunk/bin/splunk status
7. Read the license agreement and enter "y" if you can accept.
8. Enter a username and password at the prompts.
9. Run this command: sudo /opt/splunk/bin/splunk start
10. By default you will have a web service listening on the external IP address of the server. You can construct the URL like this: http://x.x.x.x:8000
11. Replace x.x.x.x with the external IP address of the server and open a web browser from a desktop.
12. Log in with the credentials you configured on the back-end (in step #8 above).

How Do You Troubleshoot the Message “ERROR: but there is no YARN_RESOURCEMANAGER_USER defined.”?

Problem scenario
You run sudo bash start-yarn.sh but you receive this message:

ERROR: Attempting to operate on yarn resourcemanager as root
ERROR: but there is no YARN_RESOURCEMANAGER_USER defined. Aborting operation.

What should you do?

Solution
1. Modify start-yarn.sh. Underneath the last section of comments, place three lines with the following text:

YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root

2. Modify stop-yarn.sh. Underneath the last section of comments, place three lines with the following text:

YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root

In most cases, you would not use sudo to start yarn. To learn how to set up a multi-node cluster of open source Hadoop that can be administered with a user without sudoer rights, see this posting.

How Do You Pass Parameters to a Function in Golang and Assign the Returned Value to a Variable?

Problem scenario
You want to pass parameters to a function and assign the returned value to a parameter using the Go programming language. What should you do?

Solution
Prerequisite
This assumes that you have installed Golang; if you need assistance with this, see this posting.

Procedures
1. Create a file called c.go with this as the content:

package main

import "fmt"

// Function that accepts parameters and returns an integer value:
func multiply(x int, y int) int {
        total := 0
        total = x * y
        return total
}

func main() {
        // Pass integers for an example
        newtotal := multiply(100, 50)
        fmt.Println(newtotal)
}

2. Run it like this: go run c.go

How Do You Install Sqoop?

Problem scenario
You have Hadoop installed on a Linux machine. You want to install Apache Sqoop. What do you do?

Solution
Prerequisite
Verify one of these commands work:
hadoop dfs -ls
or
hdfs dfs ls
# If you need assistance installing Hadoop, see this posting.

Procedures
Download the installation media here: http://www.apache.org/dyn/closer.cgi/sqoop/

There are other sources. Here is one example of the commands necessary to install Sqoop:

cd /tmp/
curl http://sourceFromAbove/sqoop-1.99.7-bin-hadoop200.tar.gz > /tmp/sqoop-1.99.7-bin-hadoop200.tar.gz
sudo -xzvf sqoop-1.99.7-bin-hadoop200.tar.gz
mv sqoop-1.99.7-bin-hadoop200 /usr/lib/sqoop
cd /usr/lib/sqoop

How Do You Troubleshoot the sonar-scanner CLI Not Working?

Problem scenario
You run sonar-scanner from the command prompt, but you receive an error. The error looks like one of the following:

"Task '-x' does not exist. Please use 'list' task to see all available tasks"

"You must define the folowing mandatory properties for 'Unknown': sonar.projectKey, sonar.sources"

What should you do to get sonar-scanner to work?

Solution
Run the sonar-scanner command from a local directory where you have sonar-project.properties.

How Do You Get Python to Compute How Long Something Took?

Problem scenario
You want to check the runtime duration of a section of Python code. How do you compute the amount of time something took in Python?

Solution
Write a program like this:

import datetime, time
t1 = datetime.datetime.now()
time.sleep(5)     # replace this line with the section of code you want to time
t2 = datetime.datetime.now()
t3 = t2 - t1
print("Time format is in hours:minutes:seconds:seconds_decimals")
print(t3)

How Do You Install Maven on Linux?

Problem scenario
How do you install Apache Maven on any distribution of Linux?

Solution
Prerequisite: You must have Java installed for Maven to work after it is installed. It may make sense to install Java before you install Maven. Understand that mvn commands will not work until Java has been installed. If you need assistance install Java, see this posting.

Procedures
This is based on Apache Maven 3.6.3.*

mavenversion=3.6.3
curl http://mirrors.advancedhosters.com/apache/maven/maven-3/$mavenversion/binaries/apache-maven-$mavenversion-bin.tar.gz > /tmp/apache-maven-$mavenversion-bin.tar.gz
cd /tmp/
sudo tar -xzvf apache-maven-$mavenversion-bin.tar.gz
sudo mv apache-maven-$mavenversion /opt/maven
export M2_HOME=/opt/maven
export M2=$M2_HOME/bin
export PATH=$M2:$PATH
source ~/.bashrc
ln -s /opt/maven/bin/mvn /bin/mvn
mvn -v

*You can change the directions if you want. You can get the latest installation media from here:
https://maven.apache.org/download.cgi

mavenscriptDownload

How Do You Solve This Problem “Error: Could not find or load main class org.apache.hadoop.util.VersionInfo”

Problem scenario
You run "hadoop version" but you receive this message "Error: Could not find or load main class org.apache.hadoop.util.VersionInfo". What do you do?

Possible Solution #1
Use "sudo " before the "hadoop version" command.

Possible Solution #2
Use "sudo -i " before the "hadoop version" command.

Possible Solution #3
Use a different user. Sometimes Hadoop is set up to be run under a different user (e.g., hduser).

Possible Solution #4
Re-install Hadoop. If you need assistance, see this posting.

How Do You Get Hadoop Commands to Work from Any Directory without Using the Full Path?

Problem scenario
Hadoop is installed on Linux. But hadoop version and other hadoop commands are not working. What should you do?

Solution
Find the hadoop executable in a directory named bin. It is often "/usr/local/hadoop/bin/hadoop". Ultimately you need to find the directory that houses this "bin." has a subdirectory with "bin" and "hadoop" inside, run these two commands:

sudo find / -name hadoop -type f
whereis hadoop

Run these commands interactively where "/usr/local/hadoop" is the directory that is the parent of the subdirectory named "bin" that is the parent of the hadoop executable. 

export HADOOP_PREFIX=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_HOME=$HADOOP_PREFIX
export HADOOP_HDFS_HOME=$HADOOP_PREFIX
export YARN_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_PREFIX/lib/native
export PATH=$PATH:$HADOOP_PREFIX/sbin:$HADOOP_PREFIX/bin
source ~/.bashrc

Test it. Run this hadoop version

Now to ensure this persists through a reboot, run this: sudo vi ~/.bashrc Append the lines below to the file (and then save the file):

export HADOOP_PREFIX=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_HOME=$HADOOP_PREFIX
export HADOOP_HDFS_HOME=$HADOOP_PREFIX
export YARN_HOME=$HADOOP_PREFIX
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_PREFIX/lib/native
export PATH=$PATH:$HADOOP_PREFIX/sbin:$HADOOP_PREFIX/bin