CONTINUAL INTEGRATION – Page 74 – A Technical I.T./DevOps Blog

How Do You Fix the Error “grep: memory exhausted”?

Problem scenario
You run a grep command but it fails. You get a message about "memory exhausted." What should you do?

Solution
Possible Solution #1
If you are comparing two files line-by-line, try reversing the order of the file arguments for the grep command. For example try these two versions:

grep -f a.txt b.txt grep -f b.txt a.txt

If the file in the left argument is smaller, we know it uses less memory.

Possible Solution #2
If you have enough disk space, add swap space. See this posting if you need more assistance.

Possible Solution #3
See How Do You Add Memory to a Server? for resizing the VM to add more memory.

A List of Terraform Books

How Do You Respond to a Vagrant Prompt for which Interface the Network Should Bridge to?

Problem scenario
You are running Vagrant to create a VM on Oracle VirtualBox. You are given a prompt after your "vagrant up" command:
"Which interface should the network bridge to?"
No option seems to work. You have used a variety of different names, GUIDs, syntaxes, etc. What should you do?

Solution
Use "1)" or "2)" or any of the integer values listed above the question. There should be something like this several lines above the question:

=> default: Available bridged network interfaces:
1) enp1s0
2) virbr0

What Is The Watch Mechanism in Kubernetes?

Question
What is the "watch mechanism" in Kubernetes?

Solution
A notification system from the API server to trigger changes in Kubernetes. In Kubernetes there are observable events. The watch mechanism is a system for listening for specific events to respond to. The Kubelet, the Scheduler, the kube-dns pod, and individual controllers are components that use the watch mechanism. The API server is a central component that watching components are aware of. (This was summarized from pages 319, 321, and 323 of Kubernetes in Action.)

If you want to see how the relevant --watch flag works, run this command:

kubectl get pods --watch

(This command was taken from 318 of Kubernetes in Action.)

To see various watch.go files, see this: https://github.com/kubernetes/kubernetes/search?q=filename%3Awatch.go

To see the source code of watch.go you for etcd you used to be able to go to this file: https://github.com/etcd-io/etcd/blob/master/clientv3/watch.go

How Do You Troubleshoot the pipenv error message “tomlkit.exception”?

Problem scenario
You try to run a pipenv command, but you get an error about "tomlkit.exception".

Solution
Copy the Pipfile to a new name (to back it up). Delete the original Pipfile. Try to run the command again. If it still fails, try removing pipenv by uninstalling it. Then reinstall pipenv. Then reboot the computer.

How Do You Install and Configure Apache Camel on a Linux Server?

Problem scenario
You want to install and configure Apache Camel from source on any distribution of Linux. What do you do?

Solution
You cannot. But there is a way to use Apache Camel as a test.

Background: It is not something that is can be purely installed-and-configured like other applications. Camel is a framework for integrating messaging solutions or microservice architecture. Camel can be used as a library within Spring Boot and other tools (according to https://camel.apache.org/). You can use Camel with Maven (or other tools). The following is how to use Camel with Maven as a proof-of-concept.

Prerequisites if you want to use the script below:
Your server has 4 GB of memory. If this memory is mostly from swap space, the installation will take several hours to complete. With two CPUs and 4 GB of RAM, it will take 40 to 60 minutes to complete. You have Java and Maven installed. If you need assisting installing Java, see this posting. If you need assisting installing Maven, see this posting.

Procedures
Run this script. (It was tested to work with SUSE Linux Enterprise Server 15 SP2, Ubuntu 20.04.1 LTS, and RHEL 8.3.)

# Written by www.continualintegration.com.
# We incorporated a command found here: https://stackoverflow.com/questions/11616835
# Last modified on 11/26/20.

camelversion=3.6.0

echo "Warning: This script will wipe out previous installations of Mavan and Java to create new ones.  It will also create a 4 GB swap space file on the hard drive."
sleep 5
echo "Installing the dependencies of Camel at this time..."
curl https://www.continualintegration.com/wp-content/uploads/2020/11/swapspace4G.txt > /tmp/swapspace4G.sh
curl https://www.continualintegration.com/wp-content/uploads/2020/11/javainstallscript.txt > /tmp/javainstallscript.sh
curl https://www.continualintegration.com/wp-content/uploads/2020/11/mavenscript.txt > /tmp/mavenscript.sh
sed -i 's/\r$//' /tmp/swapspace4G.sh
sed -i 's/\r$//' /tmp/javainstallscript.sh
sed -i 's/\r$//' /tmp/mavenscript.sh
bash /tmp/swapspace4G.sh
bash /tmp/javainstallscript.sh
bash /tmp/mavenscript.sh
echo "Installation of the dependencies should be complete now."
echo "********************************************************"
echo "Beginning the installation of Apache Camel.  This process on a server with 4 GB of RAM takes 30 to 60 minutes.  If it has 1 GB of RAM [and uses swap space], it may take several hours."

curl -Ls https://mirrors.sonic.net/apache/camel/apache-camel/$camelversion/apache-camel-$camelversion-src.zip > /tmp/apache-camel-$camelversion-src.zip
echo "Verify unzip is installed..."
apt -y install unzip
yum -y install unzip
zypper -n install unzip
echo "Two of the above commands should have failed.  Do not be alarmed if there were previous failures with 'commmand not found' messages"
cd /opt
cp /tmp/apache-camel-$camelversion-src.zip .
unzip apache-camel-$camelversion-src.zip 
echo "Now attempting to set the permissions of the new directory and its subdirectories"
gn=$(groups | awk '{print $1}')
un=$whoami
chown -R $un:$gn apache-camel-$camelversion
cd apache-camel-$camelversion
echo "This process may take hours to complete"
MAVEN_OPTS=-Xmx5120m
export MAVEN_OPTS=-Xmx5120m
mvn clean install -Pfastinstall
#mvn clean install -X
# The above stanza is commented out as a suggestion of something you could potentially run.

How Do You Troubleshoot an Error about sqlite3 Not Being Installed when It Is Actually Installed?

Problem scenario
You have installed sqlite3. It is a version above 3.22.0. But when you run "make deps" you see this error message:

checking for SQLITE… no
configure: error: Package requirements (sqlite3 >= 3.22.0) were not met:
No package 'sqlite3' found

What should you do?

Solution
Run this:
sudo apt -y install libsqlite3-dev

How Do You Get the NameNode Process to Start in Hadoop?

Problem scenario
You have run start-dfs.sh and start-yarn.sh. You have stopped all the Hadoop services too. When you run "jps", the NameNode is not showing up. You have tried a variety of different troubleshooting methods (including rebooting the NameNode). The NameNode has never worked correctly. You can delete all the data in the cluster because it never really worked. What should you do?

Solution
Run this but remember it will delete all your data:

hdfs namenode -format # Warning: this command will delete all your data in Hadoop

How Do You Troubleshoot the fdisk Error “cannot open /dev/sdaX: No such file or directory”?

Problem scenario
You have added a partition to Linux. When you type "df -h" you do not see evidence that it is available. You tried the "sudo lsblk" command, but you do not see the device. The "fdisk -l" command does show the new device and the size that is available. The command ll /dev/sd* does not show your new device.

When you run "fdisk /dev/sdaX", you get this: "fdisk: cannot open /dev/sdaX: No such file or directory"

How do you make it available?

Solution
Reboot the server.

How Do You Use an Array in Linux (Bash)?

Problem scenario
You are aware that arrays are a supported data type in Bash. You would like to iterate through some variables as arrays could enable complex logic. What do you do?

Solution
The array is zero-based. So the first element is referred to by index number 0.

continualarray=(dog cat hamster rabbit horse cow)
echo ${continualarray[2]}
echo ${continualarray[5]}

Create a file foobar.txt to test an array's line-reading capacity.

$ cat /tmp/foobar.txt
this is a sentence
another sentence is here.
some more text.
various words; another phrase.

Here is an example of the built-in readarray command:

readarray rows < /tmp/foobar.txt
echo ${rows[2]) # this will print out this:
some more text.