How Do You Troubleshoot the tar Error “Cannot execute remote shell: No such file or directory”?

Problem scenario
You are using the tar command. You are getting an error when you try to compress a file. This is the message:

"tar (child): Cannot execute remote shell: No such file or directory
Cannot open: Input/output error
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now"

What should you do?

Solution
Eliminate colons in the file. Even quotes around the file name's characters that have ":" colons will not work for some tar functions. We find that tar works fine when there are just letters, numerals, and periods ".".

How Do You Get Ubuntu/Debian Linux to Have Packages That Can Be Updated When the Upgrade Command Did Not Work?

Problem scenario
You are using Ubuntu or a Debian distribution of Linux. You log in and see this (the message of the day):

6 packages can be updated.
6 updates are security updates.

You then run this: sudo apt-get -y upgrade

But it does not work. You see this message:

"0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded."

What can you do to update your Linux server and not be warned about packages that can be updated?

Solution
Run this command: sudo apt -y upgrade

How Do You Get a Script to Execute Every Time You Log in as the User in Linux?

Problem scenario
You want a script to run whenever a user logs in (e.g., when a sudoer uses "su jdoe", you want a script to run before you assume the jdoe user). You are using Linux SUSE or Ubuntu or a Debian distribution. You found that having a script in /etc/profile.d/ does not work. What should you do?

Solution
To have a script automatically run for a given user, create a .bashrc file in the /home/jdoe/ directory (where jdoe is the username). For CentOS/RedHat/Fedora, place a script in /etc/profile.d/.

To have a script automatically run for a given user, create a .bashrc file in the /home/jdoe/ directory (where jdoe is the username). If one exists modify it. Make sure the .bashrc file has this line appended to it (where /home/ubuntu/foobar.sh is the complete path to the shell script):

/bin/bash /home/ubuntu/foobar.sh

For CentOS/RedHat/Fedora, place a script in /etc/profile.d/ (and do not be concerned about a .bashrc file).

How Do You Install Splunk on a Debian/Ubuntu Linux Server?

Problem scenario
You have a Debian or Ubuntu Linux server. You want to install a trial version of Splunk. What should you do?

Solution
1. Sign up for an account to get the installation media: https://www.splunk.com/en_us/download.html
2. Click on "Linux"
3. Obtain the .deb file.
4. Put the .deb file on the Linux server.
5. Install the file. The command will look something like this:
sudo dpkg -i splunk-7.3.2-c60db69f8e32-linux-2.6-amd64.deb

6. Run this after it is installed: sudo /opt/splunk/bin/splunk status
7. Read the license agreement and enter "y" if you can accept.
8. Enter a username and password at the prompts.
9. Run this command: sudo /opt/splunk/bin/splunk start
10. By default you will have a web service listening on the external IP address of the server. You can construct the URL like this: http://x.x.x.x:8000
11. Replace x.x.x.x with the external IP address of the server and open a web browser from a desktop.
12. Log in with the credentials you configured on the back-end (in step #8 above).

How Do You Troubleshoot the Message “ERROR: but there is no YARN_RESOURCEMANAGER_USER defined.”?

Problem scenario
You run sudo bash start-yarn.sh but you receive this message:

ERROR: Attempting to operate on yarn resourcemanager as root
ERROR: but there is no YARN_RESOURCEMANAGER_USER defined. Aborting operation.

What should you do?

Solution
1. Modify start-yarn.sh. Underneath the last section of comments, place three lines with the following text:

YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root

2. Modify stop-yarn.sh. Underneath the last section of comments, place three lines with the following text:

YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root

In most cases, you would not use sudo to start yarn. To learn how to set up a multi-node cluster of open source Hadoop that can be administered with a user without sudoer rights, see this posting.

How Do You Pass Parameters to a Function in Golang and Assign the Returned Value to a Variable?

Problem scenario
You want to pass parameters to a function and assign the returned value to a parameter using the Go programming language. What should you do?

Solution
Prerequisite
This assumes that you have installed Golang; if you need assistance with this, see this posting.

Procedures
1. Create a file called c.go with this as the content:

package main

import "fmt"

// Function that accepts parameters and returns an integer value:
func multiply(x int, y int) int {
        total := 0
        total = x * y
        return total
}

func main() {
        // Pass integers for an example
        newtotal := multiply(100, 50)
        fmt.Println(newtotal)
}

2. Run it like this: go run c.go

How Do You Install Sqoop?

Problem scenario
You have Hadoop installed on a Linux machine. You want to install Apache Sqoop. What do you do?

Solution
Prerequisite
Verify one of these commands work:
hadoop dfs -ls
or
hdfs dfs ls
# If you need assistance installing Hadoop, see this posting.

Procedures
Download the installation media here: http://www.apache.org/dyn/closer.cgi/sqoop/

There are other sources. Here is one example of the commands necessary to install Sqoop:

cd /tmp/
curl http://sourceFromAbove/sqoop-1.99.7-bin-hadoop200.tar.gz > /tmp/sqoop-1.99.7-bin-hadoop200.tar.gz
sudo -xzvf sqoop-1.99.7-bin-hadoop200.tar.gz
mv sqoop-1.99.7-bin-hadoop200 /usr/lib/sqoop
cd /usr/lib/sqoop

How Do You Troubleshoot the sonar-scanner CLI Not Working?

Problem scenario
You run sonar-scanner from the command prompt, but you receive an error. The error looks like one of the following:

"Task '-x' does not exist. Please use 'list' task to see all available tasks"

"You must define the folowing mandatory properties for 'Unknown': sonar.projectKey, sonar.sources"

What should you do to get sonar-scanner to work?

Solution
Run the sonar-scanner command from a local directory where you have sonar-project.properties.

How Do You Get Python to Compute How Long Something Took?

Problem scenario
You want to check the runtime duration of a section of Python code. How do you compute the amount of time something took in Python?

Solution
Write a program like this:

import datetime, time
t1 = datetime.datetime.now()
time.sleep(5)     # replace this line with the section of code you want to time
t2 = datetime.datetime.now()
t3 = t2 - t1
print("Time format is in hours:minutes:seconds:seconds_decimals")
print(t3)

How Do You Install Maven on Linux?

Problem scenario
How do you install Apache Maven on any distribution of Linux?

Solution
Prerequisite: You must have Java installed for Maven to work after it is installed. It may make sense to install Java before you install Maven. Understand that mvn commands will not work until Java has been installed. If you need assistance install Java, see this posting.

Procedures
This is based on Apache Maven 3.6.3.*

mavenversion=3.6.3
curl http://mirrors.advancedhosters.com/apache/maven/maven-3/$mavenversion/binaries/apache-maven-$mavenversion-bin.tar.gz > /tmp/apache-maven-$mavenversion-bin.tar.gz
cd /tmp/
sudo tar -xzvf apache-maven-$mavenversion-bin.tar.gz
sudo mv apache-maven-$mavenversion /opt/maven
export M2_HOME=/opt/maven
export M2=$M2_HOME/bin
export PATH=$M2:$PATH
source ~/.bashrc
ln -s /opt/maven/bin/mvn /bin/mvn
mvn -v

*You can change the directions if you want. You can get the latest installation media from here:
https://maven.apache.org/download.cgi

mavenscriptDownload