How Do You Deploy Puppet Agent to an Ubuntu 16 Linux Server in AWS?

Updated on 4/15/18

Problem scenario
You installed Puppet Master on an AWS instance of Linux.  (See this link if you want to install Puppet Master on Ubuntu.  If you want to install Puppet Master on a Red Hat Enterprise Linux server, see this link). You now want an AWS instance of Ubuntu to be configured as a Puppet Agent node with open source Puppet version 5. You want to run a manifest to make sure that this new Puppet Agent node (i.e., server) is working with your Puppet Master server.  What do you do to install Puppet Agent on an Ubuntu server and configure it to work with Puppet Master?

Solution
This solution is for Ubuntu 16 Linux instances in AWS.  You could easily adapt this solution to other Ubuntu servers in different environments.  It explains how to deploy Puppet and apply a manifest (in combination with these directions) from scratch.  These directions have been tested to work with a Puppet Master server on either an AWS RHEL server or an AWS Ubuntu Linux server.

Prerequisite
Using AWS change the Security Group so that the inbound rules will allow connections from the Puppet Agent server.  One way of doing this is to find the internal IP addresses of the Puppet Agent server.  This Linux command should help you if ran on an Ubuntu Linux server in AWS:

ip addr show | grep inet | grep -v 127.0.0.1 | grep -v inet6 | cut -c 10-24 | awk -F "/" '/1/ {print $1}'

Using the AWS console configure the relevant AWS Security Group to allow an inbound connection from the IP address in the result above.  Disregard a trailing backslash and any CIDR or numbers to the right of the backslash (e.g., ignore any trailing "/xx").

Here is a detailed explanation of how to create an inbound connection in an AWS security group:

Go to Security Groups.  Find the relevant security group and click the "Inbound" tab.  Then click "Edit."  Click "Add Rule."  Then choose for "Type" in the dropdown menu "Custom TCP Rule."  For "Port Range" choose 8140.  For the "Source" drop down option, choose "Custom."  Enter the internal IP address of the Puppet agent like this:

x.x.x.x/32

Substitute x.x.x.x with the internal IP address (as found with the above "ip addr show" command).

Procedures
1.  On the Puppet Master server, run these four commands:

sudo puppet master
sudo puppet resource service puppet ensure=running enable=true
hostname -f
ip addr show | grep inet | grep -v 127.0.0.1 # remember the IP address result from this command

2.  On the Puppet Agent server, modify the /etc/hosts file.  It should have this stanza where x.x.x.x is the internal IP address from the Puppet Master server (the result of the last command ran in step #2) and "FQDNofPuppetMaster" is the FQDN of the Puppet Master server:

x.x.x.x puppet FQDNofPuppetMaster

3.  On the Puppet Agent node, create a file name puppetagent.sh in /tmp/.  Have the content of this file be the following:

#!/bin/bash
# Written by continualintegration.com
# This script will install Puppet agent on Debian 9 or Ubuntu 16.x
# This script will not work on Ubuntu 17.x

distro=$(cat /etc/*-release | grep NAME)

debflag=$(echo $distro | grep -i "ubuntu")
if [ -z "$debflag" ]
then   # If it is not Ubuntu, test if it is Debian.
  debflag=$(echo $distro | grep -i "debian")
  echo "determining Linux distribution..."
else
   echo "You have Ubuntu Linux!"
   curl -O https://apt.puppetlabs.com/puppetlabs-release-pc1-xenial.deb
   dpkg -i puppetlabs-release-pc1-xenial.deb
   apt -y update
   apt-get -y install puppet-agent
fi

if [ -z "$debflag" ]
then
  echo "You could do not have Debian Linux."
else
   echo "You are using Debian Linux."
 #  Commented out lines are provided as a reference in case you need a higher version.
 #  curl http://http.us.debian.org/debian/pool/main/p/puppet/puppet_5.4.0-2_all.deb > /tmp/puppet_5.4.0-2_all.deb
 #  dpkg -i /tmp/puppet_5.4.0-2_all.deb
 #  apt -y update
 #  apt-get -y install ruby-shadow ruby-deep-merge ruby init-system-helpers ruby-augeas puppet
   apt-get -y install puppet
fi

systemctl start puppet
systemctl enable puppet
ln -s /opt/puppetlabs/puppet/bin/puppet /usr/bin/puppet
puppet agent -t -d
#cd /tmp
#wget https://apt.puppetlabs.com/puppet5-release-xenial.deb
#dpkg -i puppet5-release-xenial.deb

4.  On the Puppet Agent node become root or assume sudoers privileges.  Run this command:

sudo bash /tmp/puppetagent.sh

5.  On the Puppet Agent node update this file: /etc/puppetlabs/puppet/puppet.conf
If you cannot find it there, use this command: sudo find / -name puppet.conf
The last line of the [main] section of this puppet.conf file should have this stanza (where FQDNofPuppetMasterserver with the result of the "hostname -f" command ran on the Puppet Master server (in step #1)):

server=FQDNofPuppetMasterserver

6.  Go to the Puppet Master server.  Run this command:  sudo puppet cert list

7.  Assuming the above had some output such as puppet.agent.continualintegration.com, run this command (but substitute puppet.agent.continualintegration.com with the FQDN that resulted from the command in step #6):

sudo puppet cert sign puppet.agent.continualintegration.com

Alternative step #7: Assuming that step #6 showed no other servers that you do not want signed, run this command:

sudo puppet cert sign --all

8.  Test it.  If you are using Puppet 5, see if you have /etc/puppetlabs/.  If the puppetlabs subdirectory was not created, your installation failed.  The "puppet" and "puppet master" commands may work however.  The command puppet -V will tell you what version you are using.  But the installation may be defective if you have not /etc/puppet/ or /etc/puppetlabs/ directory path.

a) On the Puppet Master server go to /etc/puppetlabs/manifests/.  (The directory path used to be /etc/puppet/manifests.)  Create site.pp with the following content:

  exec { 'somethingneat':
        command => '/bin/date > /tmp/continual.txt'
       }

b)  On the Puppet Agent node, run this:  sudo puppet agent -t -d

c)  Check the /tmp/ directory for the file named "continual.txt."

How Do You Have PHP Return an Informative Error Message When a SQL Command Returns No Results?

Problem scenario
You have a PHP web page that integrates with Postgres behind the scenes.  You find that when a SQL command returns nothing (e.g., when a table name is used and it is known to not exist in the underlying database), the PHP web page is blank.  You want the result to be more clear about what happened or potentially happened.  How do you handle an exception or commonly occurring SQL query with no results using PHP?  

Solution
This assumes you were using pg_query.  Remember that pg_query returns a boolean value if the query failed.  If credentials were not typed in correctly the result of the pg_query() will be a false.  

Successful pg_query executions will return non-false statements -- but the value will not necessarily be "TRUE."  Failed executions will return false.

The solution is to introduce logic by testing the result of the pg_query() with an if statement.

...
$cc = pg_query($query);
if($cc) {} else {echo  '<td>' . "Query failure.  Did you enter the correct credentials?  Was the relevant table name typed correctly?  Was the user the owner of the database or table involved?  Were certain variables hard-coded that prevent the operation you attempted?" . '</td>';}

How Do You Install Puppet Master on an AWS Instance of Ubuntu 16?

Updated 12/1/17

Problem scenario
You need to install open source Puppet version 5 on a server to become a Puppet Master.  You have an AWS Ubuntu Linux server.  You only have five minutes to do it.  How do you install Puppet Master immediately?

Solution
1.  Create puppetinstaller.sh in the /tmp directory.  It should have the following lines (stop before you get to step #2):

#!/bin/bash
# Written by continualintegration.com

apt-get update
apt-get -y install ntp
echo '
server 0.us.pool.ntp.org
server 1.us.pool.ntp.org
server 2.us.pool.ntp.org
server 3.us.pool.ntp.org' >> /etc/ntp.conf

service ntp restart
cd ~ && wget https://apt.puppetlabs.com/puppet5-nightly/puppet5-nightly-release-xenial.deb
dpkg -i puppet5-nightly-release-xenial.deb
apt-get update
apt-get -y install puppetserver puppet-agent

2. Run this command:  bash /tmp/puppetinstaller.sh

3.  Verify it worked by running this command:  puppet -V
      You are done with the installation when you see the numeric version from the above command.

4.  This is an optional configuration step.Mentally decide how much RAM you want Puppet Master application to have.  (In small development environments it is possible the role of Puppet Master competes with another purpose of the server.)  If you are not sure how much RAM you have, try run this command:  cat /proc/meminfo | grep Mem

Create this file or modify the existing one:  /etc/default/puppetserver
In this file put a stanza like one of the following, but if the file exists modify a single stanza you see that is like it:

JAVA_ARGS="-Xms512m -Xmx512m"

JAVA_ARGS="-Xms1g -Xmx1g"

JAVA_ARGS="-Xms2g -Xmx2g"

# The above stanzas would limit Puppet Master's RAM usage at 512 MB, 1 GB and 2 GB respectively.

#  If you want to increase the amount of virtual memory, you can use this posting for directions on how to create swap space.  If you want more RAM, you can upgrade the flavor of the AWS EC-2 instance (aka virtual server) by following this link.

If you want to see directions like these for installing Puppet Master on a Debian server in GCP, see this posting.

How Do You Create a Table inside a Specific Postgres Database?

Problem scenario
You want to create a table in a specific Postgres database.  You want to use a command line interface (not a GUI) for this task.  (The GUI version of these directions can be found here.)  What are the SQL commands to do this?

Solution
This assumes you have a role (a username and password) to connect to the Postgres database with permissions to create tables.

1.  Connect to the Postgres instance (e.g., either through a web front-end or the command line interface).  If you are unsure on how to do this, see this link. If you use an desktop application to connect to the instance, skip step #2.

2.  Connect to the database using a command like this:
\c nameofdatabase \\where nameofdatabase is the name of the database you want

3.  Run a SQL [DDL] command like this (where cool is the name of the table and column1 is the name of the first column in the table):  create table cool(column1 int);

If you are not sure what database you are connected to, run this command:
SELECT current_database();

The database you connect to with a GUI application is chosen at the time you configure the connection.

How Do You List the Tables of a Specific Postgres Database?

Problem scenario
You want to see the names of the tables of a specific Postgres database.  You know the Postgres database name.  How do you do list its tables?

Solution
This assumes you have a role (a username and password) to connect to the Postgres database.

1.  Connect to the Postgres database (e.g., either through a web front-end or the command line interface).  If you are unsure on how to do this, see this link.

2.  Connect to the database with a command like this:
\c nameofdatabase \\where nameofdatabase is the name of the database you want

3.  Run this SQL command:
\dt

How Do You Change a Firewall Rule in Google Cloud Platform?

Problem scenario
You want to create an exception to your GCP firewall to allow connectivity from a different workstation or server.  What should you do?

Solution
1.  In the upper right hand corner of the console, click the icon with three horizontal bars.  This is the "Products and Services" button.  Then go to "VPC Network" -> "Firewall Rules."

2.  Click "Create Firewall Rule."

3.  Fill out the required options.

4.  Click Create.

How Do You Install Npm and/or node.js on a AWS Instance of RHEL?

Problem scenario
Using a RHEL server in AWS, you try this command "npm -v", and it shows that npm is not installed.  You want to use the npm utility or you want Node.js to be installed on your RedHat version 7 Linux server in AWS.  You may have tried various yum commands but received errors about dependencies.  Various other scripts fail to get npm to be a recognized command.  How do you install npm and/or Node.js without changing yum repositories and without using the wget command?

Solution
Warning: This solution takes approximately 30 minutes, but it does not require interaction.  Do not try this solution on a sever with less than 1 GB of RAM. If you want extra swap space, you can see this posting. If you want to install npm on a different distribution of Linux, see this posting.

1.  Create a file named "npminstaller.sh" (with no quotes) with the following content:

version=8.1.3
#If version changes to 9, update the cd node-v8* command below
cd /opt/
curl http://nodejs.org/dist/v$version/node-v$version.tar.gz > node-v$version.tar.gz
tar xzvf node-v$version.tar.gz
cd node-v8*
yum -y install gcc gcc-c++
./configure
make
make install

2.  Run the script with sudo bash npminstaller.sh with no quotes.

How Do You Find You out the AWS Instance ID from inside the OS of a Server?

Problem scenario
You are using AWS via web browser while using multiple Putty sessions are you logged into several AWS servers (aka AWS instances).  You can find the host names and internal IP addresses of the servers when you are logged into via the command line of the OS itself.  From the AWS console you can find external IP addresses and external FQDNs.  You do not know how to correlate these instances (as the information readily available in Putty at the OS level is independent of the information available via the AWS console).  You want to terminate one instance.  You want to know which instance ID (in the AWS console) corresponds to which OS you are using.  How do you find out which server is which?  

Solution
This solution does not require you to install the wget utility.  From a command prompt in Linux, run this command:

curl http://169.254.169.254/latest/meta-data/instance-id

The result of the above command will provide the instance ID.  This is readily available in the AWS console. Yes, it looks unusual to use a specific IP address in a URL. We believe the command should always work on an EC-2 instance -- regardless of your Security Group.

How Do You Fix the Ansible Problem “Failed to connect to the host via ssh: percent_expand: unknown key %C\r\n”, “unreachable”: true”?

Problem scenario 
When you run an Ansible playbook on the control node that is not running on a Mac (e.g., you are running Ansible on RedHat, Ubuntu or SUSE), you get this error: "Failed to connect to the host via ssh: percent_expand: unknown key %C\r\n", "unreachable": true".   How do you debug your playbook to get it to run?

Root cause
​The problem may not be in the playbook.   It may be in the ansible.cfg file if the ansible.cfg file was created for Ansible to work on a Mac OS. 

Solution
Find the ansible.cfg file.  Back it up.  Comment out the stanza with "ansible-ssh-%%C". This stanza is appropriate for Mac OS deployments of Ansible. 

How Do You View Files on a Flash Drive with Your Android Phone or Tablet Computer?

Problem scenario
You attached a USB stick to your Android tablet with a cable like this (male micro USB plug to female USB port).  The USB drive has files on it that you want to view.  How do you view these files?

Solution
1.  In Android, go to Settings -> Storage -> Mount USB Storage.

2.  Next, go to Click the Google Apps Icon (six squares in a circle).  It should take you to Apps/Widgets.  Go to File Manager -> All files -> USB Storage.  

3.  Now you can browse to the files you wish.