How Do You Troubleshoot the Ansible Error “this task includes an undefined variable”?

Problem scenario
You are running an Ansible playbook, but you get the error "this task includes an undefined variable."  Every variable seems to be defined.  What is wrong?

Possible solution #1
Do one of the variables rely on an index of server names?  If the playbook orchestrates something that responds to Ansible values on other servers and one server is not reachable, you could get this error.  In the "Gathering Facts" stage, was each server connected to?  If one server was temporarily down, an indexing of server names and facts thereof may have not happened.  Thus you could have received an error about an undefined variable.

Possible solution #2
Use the "gather_facts: no" stanza under the module throwing the problem.  Variables derived from facts can be unnecessary.  Configuring the playbook to not gather facts is a workaround.  But this option reduces the number of potential problems that can keep the playbook from running.

How Do You Get Vagrant Working on a Debian/Ubuntu Server in AWS?

Problem scenario
You are using Ubuntu 16.x in AWS.  You want to get Vagrant working to create VMs in AWS.  What do you do?

Solution
1.  Install Vagrant.  If you need assistance see this link.

2.  Install gem and various other dependencies.  To do this run these three commands:  

sudo apt-get -y update
sudo apt-get -y install ruby-dev libffi-dev build-essential dh-autoreconf
sudo gem install ffi -v 1.9.14

3.  Install the vagrant-aws plugin:  vagrant plugin install vagrant-aws

4.  Retrieve a Vagrant box image with this command:

vagrant box add dummy https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box

5.  Create a Vagrant file with these three commands:

mkdir contint-demo
cd contint-demo
vagrant init

6.  Go to the ~/.ssh directory for the user that will run Vagrant.

7.  Create a file with the same name as the AWS key pair you will use with Vagrant.  This should be an existing key pair with AWS used to connect to your VMs as specified in the last step when you manually launch a new EC2 instance.  Let's say this key pair is called "foobar".  You should create a file named foobar in the ~/.ssh/ directory.  The content of foobar should be these three lines (but replace YourAWSAccessKeyID and YourAWSSecretAccessKey accordingly):

[default]
AWSAccessKeyId=YourAWSAccessKeyID
AWSSecretKey=YourAWSSecretAccessKey

To find the AWS Access Key ID and AWS Secret Access Key, in the AWS console, click on your name in the upper right hand corner. Click on "My Security Credentials."  Click "Create New Access Key." Then click "Show Access Key."

8.  The Vagrantfile must be replaced. Here is a template with six values to replace.

require 'vagrant-aws'
Vagrant.configure('2') do |config|
    config.vm.box = 'dummy'
    config.vm.provider 'aws' do |aws, override|
    aws.access_key_id = "value1"
    aws.secret_access_key = "value2"
    aws.keypair_name = 'value3'
    aws.instance_type = "t2.nano"
    aws.region = 'value4'
    aws.ami = 'value5'
    aws.security_groups = ['value6']
    override.ssh.username = 'ubuntu'
    override.ssh.private_key_path = '~/.ssh/value3'
  end
end

Change value1, value2, value3, value4, value5 and value6.  You may also want to change "t2.nano" and "ubuntu".

value1 and value2 would be determined in step 7 above.

value3 is the AWS Key Pair name.  For the ASS key pair name, this is a reference to the key pair to log into the instance OS with.  The last step of creating a VM in AWS gives you the opportunity to select the key pair.  If you want to test your machine's ability to connect with the Access Key ID and Secret Access Key, you may want to install the AWS CLI (sudo apt install awscli).  You can then run "aws configure".  This can test your network access, ensure that the time on your machine is configured properly etc.  This way if there is a problem with Vagrant connecting, you know the fundamentals are in place.  One advantage of testing like this gives you the ability to see the key pairs (with "aws ec2 describe-key-pairs").

value4 is the region name (e.g., us-west-1).  

value5 is the AMI ID.  
For the AMI (or image) ID, you could log into AWS and start to launch a new instance.  The AMI ID is visible in the first step when selecting an image.  

For value6, it should be fairly obvious what to change the Security Group name to.  With the AWS Console you can determine the Security Group name.  

9.  Run this command:  vagrant up --provider=aws --debug

10.  This will create VMs in the region that you provided for value4 in step #8.  You will want to make sure these are destroyed after you run the command to not incur an expense.  You can destroy them from the Vagrant host with this command:  vagrant destroy default

How Do You Clone a Git Repository when You Are Being Prompted for a Password and Do Not Know What It Is?

Problem scenario
You have a Git repository on one server.  You want to clone it to another server.  When you try to clone it, you are prompted for a password.  What do you do?

Solution
On the destination server, run this (where x.x.x.x is the IP address of the server with the Git repo):

git clone git@x.x.x.x:root/goodrepo.git

If you are prompted for a password, enter the password for the user git.  You could use a different user in your subsequent "git clone" command.  (Such a user must have permissions to read the .git file.  The user is specified immediately before the "@" symbol above.)

Alternatively if you can change the password for the git user, go to the back-end of the server with the git repository.  Run this:  sudo passwd git

How Do You Get awk to Use a Colon or a Semicolon as a Field Separator?

Problem scenario
You run a command such as this, with the goal of printing the number of occurrences of "foo":

grep -icR foo * | awk '{fs=":"; print $2}'

It does not do what you want it to do.  You want the field separator to be a colon.  It is not working.  How do you print the element after a ":" (colon) with an awk utility?

Solution
Technically awk is a programming language; but it is also a utility.  Use this command instead:

grep -icr foo *  | awk --field-separator=":" '{print $2}'

The syntax above uses awk in a different way to recognize the ":" (colon) as a field separator.  With the "--field-separator" syntax above, replacing the colon with a semicolon will allow you use to use a semicolon as a field separator too.

How Do You Set up AWS CloudFront?

Problem scenario
You want to test out a CDN in AWS.  What do you initially deploy CloudFront?

Solution

Prerequisite

This assumes that you have an S3 bucket with read permissions.  To do this, log into the Amazon S3 console and create a bucket.  Upload files and configure the permissions to grant read permissions to anyone.  

Procedures
AWS CloudFront has "Distributions."  You want to create a CloudFront Distribution.

1.  Log into the CloudFront console with this link.

2.  Click "Create Distribution"

3.  For this example, click "Get Started" under "Web" (as the delivery method for your content).

4.  Fill out the fields.  Not every field is mandatory.  If you are not sure click the "i" icon for more information on what to put in each field.  If you provide a basic S3 bucket of files (and not a website link hosted by an S3 bucket), CloudFront will take 15 minutes to deploy.  A website file (e.g., an AngularJS or HTML file) will make things faster.  But an S3 bucket is acceptable to use.

5.  Click "Create Distribution" in the lower right hand corner.

6.  You will see the "CloudFront Distributions" page with the Status "In Progress."  Eventually the Status will change to "Deployed."  If you configured CloudFront to use a regular S3 bucket by itself (and not a specific web page file), be prepared to wait 15 minutes.

7.  When the status is "Deployed", you can test your links.  Copy the domain name, e.g., d3lnudekzjqklzuwx.cloudfront.net and construct a URL to a file of your choice.  If you uploaded a file named foobar.txt to your S3 bucket (that is now part of the CloudFront CDN), construct the URL as follows:

http://d3lnudekzjqklzuwx.cloudfront.net/foobar.txt

8.  Open a web browser.  Go to the URL described in step #7.

How Do You Use the MongoDB Command Line?

Problem scenario
You want to enter the MongoDB shell to enter commands interactively.  What do you do?

Solution
1.  Run this command:  /usr/bin/mongo

If the command is not found, do two things:
  a)  Run this: sudo find / -name mongo | grep bin
  b)  Run the mongo file found above (as an executable).

2.  From the prompt ">" enter a command such as this:

db

How Do You Troubleshoot the Logstash (or Elastic Stack) Error “logging.log4j.core.appender.RollingFileAppender”?

Problem scenario
You try to start Logstash but you get this error: "main ERROR Could not create plugin of type class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: "

What should you do?

Solution
Do one of the two options below.

#1  Solution (rather simple, for a one-time fix)
Do not start the process with the root or some other regular user.  Start the Logstash service with a user dedicated for Elastic Stack.  There should be a user that is dedicated for Elastic Search and Logstash.  Become that user:

sudo su -
su elasticuser

(Replace "elasticuser" with the user dedicated for Elastic Stack.)

Now start the logstash service.

#2  Solution (more complex, for a permanent solution)
1.  You may want to create a logstash.service file in /etc/systemd/system/.  Here is a template of the file; replace esu with the Elastic Stack user; replace esg with the Elastic Stack group; replace "/opt/logstash/config/" with the path to the startup.options file; and replace other options as needed:

[Unit]
Description=logstash

[Service]
Type=simple
User=esu
Group=esg

EnvironmentFile=/opt/logstash/config/startup.options
ExecStart=/opt/logstash "-f" "/opt/logstash/config/logstash-simple.conf"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

2.  You will need to run this command after you create it (or reboot the server): sudo systemctl daemon-reload

3.  Now you can run this command: sudo systemctl start logstash

Starting Logstash this way will always start with the user context of the dedicated ElasticStack user.

What Do You Do If SonarQube Has No Logs and the Service Will Not Start?

Problem scenario
You try to start the SonarQube service.  It does not turn on or stay on.  You go to the logs directory in the "sonar" subdirectory on the Linux server.  Every time you run "sudo systemctl start sonar", there are no corresponding log entries.  You may find no logs at all.  What should you do to troubleshoot SonarQube to start when the logs are not helping you?

Solution
1.  Have the permissions changed on the files where it is installed?  Has the ownership or group of the files and the subdirectories of the Sonarqube installation changed?

If the user that runs the "sonar" service does not have the ability to write to the logs directory, you will not get proper information in the logs.  Check what Linux user and group owns the "sonar" subdirectory.  Run these two commands (where /path/of/parentdir/ is the path of the directory that houses the "sonar" subdirectory):

cd /path/of/parentdir/
ll

2.  If you do indeed find that the sonar subdirectory is owned by a user and group that is not special or dedicated for running the sonarqube service, run a command such as this:

chown -R sonar:sonargroup /path/of/parentdir/sonar

(If you need to find what the sonargroup is, try this: Log in as the sonar user or become root and run "su sonar".  Then run the groups command to see what user the sonar user has.)

3.  Try running these commands:
sudo systemctl enable sonar
sudo systemctl start sonar

How Do You Troubleshoot the Ansible Problem “IOError: [Errno 13] Permission denied…”?

Problem scenario
You are running an Ansible playbook, but you get an error like this:  "IOError: [Errno 13] Permission denied..."

How do you get the playbook to work?

Solution
If the playbook has a destination for a file and that file is already there on the managed node, this error may occur. 

1.  Go to the managed node. 
2.  Consider these four possible solutions:

  • Change the destination of the file in the playbook.
  • Move the file that is already in the destination file's place on the managed node to a different location.
  • Delete the file from the managed node.
  • Change the permissions of the destination directory on the managed node (either manually or via the Ansible playbook).

How Do You Write a Hello World Program in Lua?

Problem scenario
You want to run a basic Lua program as a test.  What do you do?

Solution
Prerequisite

You must have installed the Lua compiler.  If you are using a CentOS/RHEL/Fedora server, try this link.

Procedures
1.  Create a file called contint.lua with the following line:
print("This is a Hello World program")

2.  Run it with this command: lua contint.lua