How Do You Get a Script with Yum Commands That Rely on a Public Website to Work when you have No Access to the Internet?

Problem scenario
You have a Bash script that cannot be modified that runs yum commands.  The script runs on a RedHat distribution of Linux and uses URLs with SSL (e.g., https://continualintegration.com).  The script expects a yum repository to be set up at this URL.  This script must run without access to the Internet due to enterprise security policies.  What do you do if you are behind enough security to not need SSL to be set up properly (given that you have no Internet access) and need to get the script to work immediately?

Solution
Mirror the URL resources on a server internally, and intentionally "spoof" the IP address to point DNS records for the URL to that internal yum server. 

1.  Go to the server that will run the script and change the /etc/hosts file so the FQDN of the URL will map to the IP address of the internal server.  For example, the /etc/hosts file of the server that will run the script should have an entry like this (assuming the script refers to https://continualintegration.com and the internal server that will serve as https://continualintegration.com is at 123.123.123.123):

123.123.123.123                             continualintegration.com

The steps will keep referring to 123.123.123.123 as the IP address of your internal yum repository server.

2.  Go to the yum repository server at 123.123.123.123.  Install Apache httpd server on this server.  Modify the httpd.conf file (find it with this command "find / -name httpd.conf").  Search for the word "DocumentRoot" in httpd.conf to change instances of the default "/var/www/html" path to the path of the .rpms. 

3.  Transfer the .rpms from the server on the Internet to the yum repository server at 123.123.123.123.  The destination directory should be the one you configured in the step above.   Start the Apache server (e.g., "apachectl start").

4.  Install createrepo on the 123.123.123.123 server if it has not been installed.  Use this command to configure the directory with .rpms to be a yum repository:  createrepo /path/to/rpms/
The above command will create a "repodata" directory in the directory you specified.  The command will also create a repomd.xml file in that "repodata" subdirectory.  These things allow the yum repository to work.

5.  Set up SSL on this yum repository server at 123.123.123.123.  Use this command:  yum install mod_ssl
For further configuration details of SSL, see this link.  

6.  On the server that will run the script, modify the /etc/yum.conf file.  Add this entry as the lowest stanza in the [main] section:  sslverify=false
(This is not a best practice.  But behind a firewall that blocks Internet access, it is a configuration that some Linux administrators use without too much risk.)

7.  Now you should be able to run the script.  The resolution of the FQDN of the URL should be to this new yum repository on your server.  The script will never know it did not use the real server on the Internet.

For more information about configuring yum clients see this external link.  It has details about retrieving RPMs for specific sources with SSL.

Software Firewall and Anti-Virus Program

Looking for a downloadable anti-virus or anti-spyware solution on Amazon?  You will not find this on Amazon: The latest Lavasoft Ad-Aware Total Security is an all-in-one solution for Windows computers.  Ad-Aware Total Security has a great OS firewall and strong anti-spyware capabilities. Its virus database and scanning technology provides great all-around protection.  To learn more about this under-rated security solution for Windows computers or to buy it, click on this link.

How Do You Solve The “No route to host” Error When You Try To Use The wget Command?

Problem scenario
You can ping a web server, but you cannot use the "wget https://www.continualintegration.com" command (with no quotes and where www.continualintegration.com is the URL or IP address of the web server).  You get the error result "No route to host."  How do you use wget to retrieve the files on the web server?

Solution
Try one of these four diagnostic methods to isolate the problem:

  1. Verify the firewall is not blocking web requests on the Apache server.  Use "ps -ef | grep firewalld" with no quotes to see if it is running.  If it is running, and you are in a situation where you can afford to turn it off, turn it off with "service firewalld stop" or "systemctl stop firewalld". 
  2. See if there is an intermediate proxy server or firewall between the two servers.  Use traceroute or some other network tool. 
  3. Use nmap -p 80 x.x.x.x (where x.x.x.x is the IP address of the Apache server). 
  4. Use "route -n" on the requesting server (the one you were issuing the wget command from).  The results will let you see if there is a default gateway, a subnet mask, and an IP address on your server.  If you are not receiving these, the problem should probably be solved by reconfiguring the network interface card correctly.

How Can Puppet’s Logging Stay With a Manifest That Has a Long Duration On A Windows Server?

Problem scenario:  Some manifests kick off a non-Puppet script that has a long duration.  By default the manifest will time out after 300 seconds.  That means the manifest will not run properly if it cannot complete in five minutes.  How do you run long-duration manifests?

Solution (Part 1 of 2):  Use the timeout => directive.  For example in the Puppet manifest, have a stanza like this:

...
timeout => 1200,
...

This sets the timeout to be 1,200 seconds.  For Linux Puppet Agents, this is all you need.  For Windows Puppet Agents, a second part is needed for the solution.

Solution (Part 2 of 2):  The following does not apply for Linux Puppet Agents (just Windows).  Puppet may initiate a PowerShell script (or batch script) but then Puppet itself quits logging/monitoring.  Besides not having logging throughout the scripts' run-time duration, you cannot monitor the process or even debug where it fails (assuming it does not run smoothly).  What is worse is that by default the manifest will time out after 300 seconds.  

For PowerShell scripts, have the command invoke start-process with the "-wait -NoNewWindow" flags.  For example,

...
command => 'c:\windows\system32\WindowsPowerShell\v1.0\powershell.exe start-process c:\windows\system32\WindowsPowerShell\v1.0\powershell.exe c:\temp\contint.ps1 -Wait -NoNewWindow',
require => File['c:\temp\contint.ps1'],
timeout => 1200,
...

Timeouts of 3600 or higher are acceptable.  The above command stanza does not need a "provider" designation because the absolute path to powershell.exe is in the command.

If a PowerShell script starts a Scheduled Task, the rest of the script will be evaluated.  Ordinarily the PowerShell execution does not pause to wait for the Scheduled Task to complete, and the rest of the script continues to run.  To ensure that the execution is controlled serially and strictly for Puppet to log accurately and synchronously the activity of the script, you can do something like this:

command => start-process (c:\windows\system32\schtasks.exe /run /tn 'NameOfST") -wait -nonewwindow
provider => powershell,

Having a Scheduled Task run another process has its advantages because remoting into a server does not provide a full interactive desktop environment.  (See this link for more information.)  You may want to delete the Scheduled Task if it is a one-time operation.  Then the Scheduled Tasks stay clean for future administration of the server.

How Do You Get The sar Utility to Work On Ubuntu?

Problem scenario:  On some servers you try this command: "apt-get -y install sar", but this command does not work.  On some servers with sar installed, you get an error when you run "sar".  The error says "Cannot open /var/log/sysstat/sa25: No such file or directory."

Background:  The sar utility comes from the acronym for system activity report.  It is useful in performance tuning situations.

Solution: If you type "man sar" and get a man page, then it is installed.  For these servers with a man page for "man sar," skip to the "Configuration" below.  If there is no man page for sar, issue this: apt-get -y install sysstat

Configuration:
1.a)  Edit /etc/default/sysstat.  1.b) Find the stanza that says "ENABLED='false'" and change the "false" to "true".  1.c) Save the changes and exit the file.
​
2)  Issue this command: systemctl stop sysstat  
If you get "bash: systemctl: command not found" see * below.

3)  Issue this command: systemctl start sysstat

* Use this command if "systemctl" is not a recognized command: service sysstat stop; service sysstat start

How Do You Print One Attribute (or Element) From The Results of a Built-in PowerShell Command?

Problem Scenario
You are writing a PowerShell script.  You want to extract one value from the results of a command.  You get a set of values all on one line.  It is difficult to parse just what you need. The output values for each attribute are separated by spaces.  Sometimes the output values are blank.  You want to reliably extract one property value from the command. You know that piping the output to an "sls" command will retrieve an entire row.  You could write the output to a file then parse the plain text file.  But you know that writing to the disk unnecessarily can cause disk contention.  You also need to delete the files unless they accumulate.  It is more elegant to leverage the PowerShell objects in memory and avoid writing to disk if possible.  How do you select one property and not the entire row?

Solution
Pipe the output to a "select -expandproperty nameOfProperty" command.  It will look something like this:

complex_command_here | select -ExpandProperty desiredProperty

Here is an example:  Get-PSDrive -Name C | select -ExpandProperty Provider

Replace "Provider" with the attribute you want.  Replace "Get-PSDrive -Name C" with the command that will return the results you want. If you use a variable assignment, you can then use conditional logic based on this sole variable.  This attribute can be the basis of more complex scripts.

In a Bash Script How Do You Assign a Variable a Value of a Complex Linux Expression?

Problem scenario:  You know how to assign variables in a Bash script. But how do you assign the output of a complex command such as this?

cat foo.txt | grep continual | grep integration | uniq

To push the output of some compound statement into a text file is simple.  Sometimes you want to redirect everything to a variable. How do you do this?

Solution:  Use $(expression)

For example:
varA=$(cat foo.txt | grep continual | grep integration | uniq)

Remember that there cannot be spaces around the equal sign.  Later call varA like this to evaluate it for its content: $varA

How Do You Configure Linux To Be Ready for Cloudera 5 (Hadoop)?

Two problem scenarios and solutions.

Problem scenario:  You installed Cloudera 5 (Hadoop) on CentOS 7.2 for the first time. You tried to start the Cloudera database service. But you get an error that it failed.

You run this: systemctl status cloudera-scm-server-db.service

The results include this fragment:  "  ... Failed to start LSB: Cloudera SCM Server's Embedded DB."

How do you start the Cloudera DB service?

Solution
Disable SELinux. 

  • If you find the file /etc/sysconfig/selinux, modify it.  Replace "SELINUX=enforcing" with "SELINUX=disabled".  Reboot the Linux server.  
  • If you cannot find the file, e.g., you are on RHEL 7.3, find /etc/selinux/config and modify it.  Replace "SELINUX=enforcing" with "SELINUX=disabled".  Reboot the Linux server.  

------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Problem scenario:  You installed Cloudera 5 (Hadoop) on Linux.  But you cannot reach the web UI from a regular Windows workstation from any port.  What should you do?

Solution
Shut off the firewall on the Linux server or reconfigure it.  You may want to research the consequences of this change before doing it.

How To Troubleshoot Mounting a File Share With The Error “mount: wrong fs type, bad option, bad superblock”?

Problem scenario:  You try to mount a file share with this command:
mount continualintegration.com:/export/path/to /mnt

You get this error:

" mount: wrong fs type, bad option, bad superblock on continualintegration.com:/export/path/to,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)
       In some cases useful info is found in syslog - try dmesg | tail"

Solution:

#1  Verify there is a space between the two arguments (continualintegration.com:/export/path/to and /mnt).

#2  This next step may install more packages than necessary.  But it should help eliminate the problem if you need a quick solution.

If you are running a Debian distribution of Linux, run this:

sudo apt-get install -y nfs-common cifs-utils nfs4-acl-tools nfs-utils

If you are running a RedHat distribution of Linux, run this:

sudo yum -y install nfs-common cifs-utils nfs4-acl-tools nfs-utils

If you are running a SUSE distribution of Linux, run this:

sudo zypper -n install nfs-common cifs-utils nfs4-acl-tools nfs-utils

How Do You See The Time Previous Commands Were Entered On A Linux Server?

Question:  How do you see the exact time when previous commands were entered on a Linux server when you use the "history" command?  

Background:  Linux by default logs the time and day when commands were issued.  (There is a limit to the number of previous commands that stay logged.)  The timestamps by default are not visible.

Solution:  For the session, until you log out, this will work:

HISTTIMEFORMAT="%d/%m/%y %T "

If you permanently want to see the timestamps associated with the commands entered even after a reboot, you execute this command once:

echo 'export HISTTIMEFORMAT="%d/%m/%y %T "' >> ~/.bash_profile ; source ~/.bash_profile

The above works on RedHat, SUSE, and Ubuntu.