How Can Puppet’s Logging Stay With a Manifest That Has a Long Duration On A Windows Server?

Problem scenario:  Some manifests kick off a non-Puppet script that has a long duration.  By default the manifest will time out after 300 seconds.  That means the manifest will not run properly if it cannot complete in five minutes.  How do you run long-duration manifests?

Solution (Part 1 of 2):  Use the timeout => directive.  For example in the Puppet manifest, have a stanza like this:

...
timeout => 1200,
...

This sets the timeout to be 1,200 seconds.  For Linux Puppet Agents, this is all you need.  For Windows Puppet Agents, a second part is needed for the solution.

Solution (Part 2 of 2):  The following does not apply for Linux Puppet Agents (just Windows).  Puppet may initiate a PowerShell script (or batch script) but then Puppet itself quits logging/monitoring.  Besides not having logging throughout the scripts' run-time duration, you cannot monitor the process or even debug where it fails (assuming it does not run smoothly).  What is worse is that by default the manifest will time out after 300 seconds.  

For PowerShell scripts, have the command invoke start-process with the "-wait -NoNewWindow" flags.  For example,

...
command => 'c:\windows\system32\WindowsPowerShell\v1.0\powershell.exe start-process c:\windows\system32\WindowsPowerShell\v1.0\powershell.exe c:\temp\contint.ps1 -Wait -NoNewWindow',
require => File['c:\temp\contint.ps1'],
timeout => 1200,
...

Timeouts of 3600 or higher are acceptable.  The above command stanza does not need a "provider" designation because the absolute path to powershell.exe is in the command.

If a PowerShell script starts a Scheduled Task, the rest of the script will be evaluated.  Ordinarily the PowerShell execution does not pause to wait for the Scheduled Task to complete, and the rest of the script continues to run.  To ensure that the execution is controlled serially and strictly for Puppet to log accurately and synchronously the activity of the script, you can do something like this:

command => start-process (c:\windows\system32\schtasks.exe /run /tn 'NameOfST") -wait -nonewwindow
provider => powershell,

Having a Scheduled Task run another process has its advantages because remoting into a server does not provide a full interactive desktop environment.  (See this link for more information.)  You may want to delete the Scheduled Task if it is a one-time operation.  Then the Scheduled Tasks stay clean for future administration of the server.

How Do You Get The sar Utility to Work On Ubuntu?

Problem scenario:  On some servers you try this command: "apt-get -y install sar", but this command does not work.  On some servers with sar installed, you get an error when you run "sar".  The error says "Cannot open /var/log/sysstat/sa25: No such file or directory."

Background:  The sar utility comes from the acronym for system activity report.  It is useful in performance tuning situations.

Solution: If you type "man sar" and get a man page, then it is installed.  For these servers with a man page for "man sar," skip to the "Configuration" below.  If there is no man page for sar, issue this: apt-get -y install sysstat

Configuration:
1.a)  Edit /etc/default/sysstat.  1.b) Find the stanza that says "ENABLED='false'" and change the "false" to "true".  1.c) Save the changes and exit the file.
​
2)  Issue this command: systemctl stop sysstat  
If you get "bash: systemctl: command not found" see * below.

3)  Issue this command: systemctl start sysstat

* Use this command if "systemctl" is not a recognized command: service sysstat stop; service sysstat start

How Do You Print One Attribute (or Element) From The Results of a Built-in PowerShell Command?

Problem Scenario
You are writing a PowerShell script.  You want to extract one value from the results of a command.  You get a set of values all on one line.  It is difficult to parse just what you need. The output values for each attribute are separated by spaces.  Sometimes the output values are blank.  You want to reliably extract one property value from the command. You know that piping the output to an "sls" command will retrieve an entire row.  You could write the output to a file then parse the plain text file.  But you know that writing to the disk unnecessarily can cause disk contention.  You also need to delete the files unless they accumulate.  It is more elegant to leverage the PowerShell objects in memory and avoid writing to disk if possible.  How do you select one property and not the entire row?

Solution
Pipe the output to a "select -expandproperty nameOfProperty" command.  It will look something like this:

complex_command_here | select -ExpandProperty desiredProperty

Here is an example:  Get-PSDrive -Name C | select -ExpandProperty Provider

Replace "Provider" with the attribute you want.  Replace "Get-PSDrive -Name C" with the command that will return the results you want. If you use a variable assignment, you can then use conditional logic based on this sole variable.  This attribute can be the basis of more complex scripts.

In a Bash Script How Do You Assign a Variable a Value of a Complex Linux Expression?

Problem scenario:  You know how to assign variables in a Bash script. But how do you assign the output of a complex command such as this?

cat foo.txt | grep continual | grep integration | uniq

To push the output of some compound statement into a text file is simple.  Sometimes you want to redirect everything to a variable. How do you do this?

Solution:  Use $(expression)

For example:
varA=$(cat foo.txt | grep continual | grep integration | uniq)

Remember that there cannot be spaces around the equal sign.  Later call varA like this to evaluate it for its content: $varA

How Do You Configure Linux To Be Ready for Cloudera 5 (Hadoop)?

Two problem scenarios and solutions.

Problem scenario:  You installed Cloudera 5 (Hadoop) on CentOS 7.2 for the first time. You tried to start the Cloudera database service. But you get an error that it failed.

You run this: systemctl status cloudera-scm-server-db.service

The results include this fragment:  "  ... Failed to start LSB: Cloudera SCM Server's Embedded DB."

How do you start the Cloudera DB service?

Solution
Disable SELinux. 

  • If you find the file /etc/sysconfig/selinux, modify it.  Replace "SELINUX=enforcing" with "SELINUX=disabled".  Reboot the Linux server.  
  • If you cannot find the file, e.g., you are on RHEL 7.3, find /etc/selinux/config and modify it.  Replace "SELINUX=enforcing" with "SELINUX=disabled".  Reboot the Linux server.  

------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Problem scenario:  You installed Cloudera 5 (Hadoop) on Linux.  But you cannot reach the web UI from a regular Windows workstation from any port.  What should you do?

Solution
Shut off the firewall on the Linux server or reconfigure it.  You may want to research the consequences of this change before doing it.

How To Troubleshoot Mounting a File Share With The Error “mount: wrong fs type, bad option, bad superblock”?

Problem scenario:  You try to mount a file share with this command:
mount continualintegration.com:/export/path/to /mnt

You get this error:

" mount: wrong fs type, bad option, bad superblock on continualintegration.com:/export/path/to,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)
       In some cases useful info is found in syslog - try dmesg | tail"

Solution:

#1  Verify there is a space between the two arguments (continualintegration.com:/export/path/to and /mnt).

#2  This next step may install more packages than necessary.  But it should help eliminate the problem if you need a quick solution.

If you are running a Debian distribution of Linux, run this:

sudo apt-get install -y nfs-common cifs-utils nfs4-acl-tools nfs-utils

If you are running a RedHat distribution of Linux, run this:

sudo yum -y install nfs-common cifs-utils nfs4-acl-tools nfs-utils

If you are running a SUSE distribution of Linux, run this:

sudo zypper -n install nfs-common cifs-utils nfs4-acl-tools nfs-utils

How Do You See The Time Previous Commands Were Entered On A Linux Server?

Question:  How do you see the exact time when previous commands were entered on a Linux server when you use the "history" command?  

Background:  Linux by default logs the time and day when commands were issued.  (There is a limit to the number of previous commands that stay logged.)  The timestamps by default are not visible.

Solution:  For the session, until you log out, this will work:

HISTTIMEFORMAT="%d/%m/%y %T "

If you permanently want to see the timestamps associated with the commands entered even after a reboot, you execute this command once:

echo 'export HISTTIMEFORMAT="%d/%m/%y %T "' >> ~/.bash_profile ; source ~/.bash_profile

The above works on RedHat, SUSE, and Ubuntu.

Puppet Needs To Use a Windows Network File Share: How Is This Done Consistently?

Problem Scenario:  You are using Puppet Master on Linux and Puppet Agent on Windows servers.  The manifest attempts to copy to or from a Windows network file share, but you are getting an error about the network path not existing.  The network file share has permissions that are open to Everyone.  What do you do?

Solution:  Puppet Agent will run PowerShell commands under the security context of the user who runs the "puppet agent -t -d" command.  If no one is logged in, the local system user will be the user whose security context is running the commands.  For the first time that the network file share is used from the Windows server, it is sometimes necessary to use a command like this: 

net use \\fileShareServer /USER:continual integration

Replace "fileShareServer" with the hostname of the file share.  Any username and any password will work because the file share is open to Everyone.  Note that there is no closing backslash after the fileShareServer name.  So the Puppet manifest should look like this:

...
command => 'net use \\\fileShareServer /USER:continual integration; echo "this is the manifest command" > \\\fileShareServer'
...

Three backslashes in a manifest on Linux will translate to two backslashes on a Windows server.  The main idea is that the "net use" command needs to run one time for initial use (and any credentials will suffice).

How Do You Control The Order Of When Puppet Manifests Are Applied to a Puppet Agent?

Problem scenario:  One Puppet manifest relies on another manifest to work.  Trying to apply them at the same time is not working.  The "require" and "before" keywords only seem to work for packages, exec, and file resources.  What do you do to order the manifests to satisfy dependency requirements of other manifests?

Solution:  The relationship dependency can be solved by the order in which they are applied.   The keyword "import" has been deprecated and made obsolete in many versions of Puppet.  The keyword "include" in the existing manifests (either one) will not solve the ordering problem.

The solution is to create a third module with its own manifest and call the existing manifests in the desired order from top to bottom with the "include" key word.  We'll call the module "continual" for this example.  Here is its manifest (init.pp):

continual {
   include integration
   include specialty
   }

The above manifest for the "continual" module will apply the "integration" manifest first.  The "integration" manifest is the init.pp file in the manifests directory of the module named "integration."   In the example above "specialty" is a Puppet module that requires "integration" to have been applied.  The "specialty" module will be applied after the "integration" module.

This solution is modular in that the original respective modules can be completely separate and unmodified.  This composite manifest applies the component manifests in the order to solve the dependency problems.

How Do You Use a Text-Based Web Browser on RedHat Linux?

Background:  Text-based browsers are fast.  Operations can be scriptable and repeatable.

Question:  How Do You Use a Text-Based Web Browser on RedHat Linux?

Prerequisites:  You have access to the Internet and root access.

Solution:  As root, enter these two commands (where "#" is the command prompt).

# yum-config-manager --enable rhui-REGION-rhel-server-extras rhui-REGION-rhel-server-optional
# yum -y install lynx

To test it out, run this:
lynx continualintegration.com

By default you will be prompted to accept cookies.  If privacy is a concern, this is a good thing.  Radio check buttons are toggled with the right arrow key.  This solution works on AWS instances of RedHat as long as you have access to the Internet.