The Apparent Story Behind OpenStack Component Log File Names

AWS used to have a separate API package from their CLI package; now the two are the same.  OpenStack components' command line interface activity is captured in a log when proper thresholds are set (e.g., nova.conf, glance.conf etc.) and later met.  CLI activity (which bypasses Horizon) as well as GUI API activity (e.g., mouse clicks on buttons in Horizon) is written to a file named api.log.  The location of this api.log file follows this convention: /var/log/OpenStackComponentName/api.log (where "OpenStackComponentName" is nova, neutron, glance, or swift, etc.).

The maker of the preeminent F5 load balancer has a corresponding technical cyber community named DevCentral.  One of the cofounders of this community, Jeff Browning, believes there is a salient difference between the CLI and API technologies.  (You can read his article here.)   Taking Glance as an example, it seems that he would disagree with OpenStack naming the Glance log api.log when it captures Glance CLI activity. 

On the other hand, enterprise, TCP/IP networking author Ivan Pepelnjak views APIs and CLIs as very similar concepts.   (See this link for Pepelnjak's opinions on similarities between APIs and CLIs.)  It appears that he may agree with the design of OpenStack and naming the log api.log when it captures CLI activity as well as API activity for a given OpenStack component.  So maybe the short name api.log is better than a compound word like api-cli.log.  Logging to one file can allow for greater centralization of logs which can aid troubleshooting for often unruly technologies like components of OpenStack.

Apparently there is a reason that the OpenStack foundation has file with the name api.log capture CLI and API activity.  According to OpenStack Operations Guide, written by Fifield, Fleming, Gentle, Hochstein, Proulx, Towes, and Topjian and published in May of 2014, there are plans to eliminate CLIs from bypassing the APIs.  Command line tools will eventually invoke API-based tools (along with Horizon).  Right now, the CLI commands can bypass API endpoints.  Knowing that API-based commands will be used for all OpenStack operations should eliminate the confusion about the name of the log files.

How To Install Docker on an AWS Instance of RedHat Linux

Update on 11/20/20: We recommend you see How Do You Install the docker.service File on RHEL 8.x? instead of the directions below. (Many times this command will work: sudo yum -y install docker )

Update on 9/10/19: These directions below work to install Docker version 17.05.0-ce on RHEL 8.x.

These "directions" to install Docker on RedHat Enterprise Linux include a script and how to run it.  The script was designed to install Docker 1.12 on an AWS instance of RHEL 7.x.  This script requires that your AWS RHEL server is in a security group that has access to the Internet.  The script takes approximately 10 minutes to run.  But bandwidth and resources on your instance may vary.  The benefit of these directions is that they will work when you may not run the "yum-config-manager" command.  The script below uses CentOS packages which are not ideal for RHEL.  If you can run the "yum-config-manager" command, see these alternative directions.  (If you want a variety of different directions for installing Docker, please see this posting.)

Step #1  Log in and become root.
Step #2  Run these commands:

cd /tmp/
touch installer.sh
chmod u+x installer.sh
vi installer.sh 
 #then paste everything below up to and including the 'echo "docker run -it centos /bin/bash"' line

#!/bin/bash
#Written by continualintegration.com
yum -y update
echo '
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/7/
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg' > /etc/yum.repos.d/docker.repo
yum -y install docker-engine
groupadd docker
usermod -aG docker ec2-user
service docker start
echo "run 'docker run hello-world' to test it out"
echo "Afterward, you can run "
echo "docker run -it ubuntu bash"
echo "or you can run"
echo "docker run -it centos /bin/bash"
echo "Log out and log back in"

Step #4  Save the file.
Step #5  Run the script with a command like this: ./installer.sh
Step #6  Test it with a command like this:  docker run hello-world
(You may need to log out and log back in.  Instead of the "chmod" command above, you could run it with "sudo bash".)
Step #7 Enter a docker container with a command like this:  docker run -it centos /bin/bash

(If you want to install Docker on a RHEL VM running in GCP, see this posting.)

We recommend you see How Do You Install the docker.service File on RHEL 8.x? instead of the directions above.

How To Install Docker on an AWS Instance of Ubuntu

Updated 12/26/18
Problem scenario

You want to install Docker.  You can make changes to the Ubuntu Linux repositories so you can use apt-get to install it.  How do you install Docker on an AWS EC-2 instance?

Solution
There is an alternative set of directions if you click on this link here (if you would prefer not to configure the apt-get repositories).  These directions below to install Docker on Ubuntu Linux include a script and how to run it (but the script below will make a change to the repositories that the Linux server uses).  The script was designed to install Docker 1.12 on an AWS instance of Ubuntu 14.04, 16.04 or 18.04.  These directions assume that the user that will run Docker commands is named "ubuntu"; the "usermod" stanza associates this user with the group named "docker".  Please note that these directions do work for installing Docker on an Azure instance of Ubuntu too!  For AWS this script requires that your Ubuntu Linux server is in a security group that has access to the Internet.  The script takes approximately three minutes to run in AWS.  It may or may not take longer in Azure.  Bandwidth and resources on your instance may vary.  (If you want a variety of different Docker installation directions, see this posting.)

Step #1  Log in and become root.
Step #2  Run these commands:

cd /tmp/
touch installer.sh
chmod u+x installer.sh   # 
this is unnecessary if you prefer to run the script with the "sudo bash installer.sh" command
vi installer.sh

Step #3  Enter this text into the file from "#!/bin/bash" (the first line) to the line '"echo "docker run -it ubuntu bash""' (the last line):

#!/bin/bash
apt-get update
apt-get install -y apt-transport-https ca-certificates
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
echo 'deb https://apt.dockerproject.org/repo ubuntu-trusty main' > /etc/apt/sources.list.d/docker.list
echo 'deb http://cz.archive.ubuntu.com/ubuntu trusty main' >> /etc/apt/sources.list.d/docker.list
apt-get update
apt-cache policy docker-engine
apt-get --assume-yes install linux-image-extra-$(uname -r) linux-image-extra-virtual
apt-get --assume-yes install docker-engine
sudo service docker start
sudo groupadd docker
sudo usermod -aG docker ubuntu
docker run hello-world
echo "Try running this command below to enter a Docker container:"
echo "docker run -it ubuntu bash"

Step #4  Save the changes. 
Step #5  Run the script (./installer.sh).  (An alternative way is to use "sudo bash installer.sh".)
Step #6  If you ran the above script when you were logged in with the "ubuntu" username, you will need to log out and log back in.  Otherwise start a new terminal session with the "ubuntu" user. Then run this command with no quotes to enter the Docker container:  "docker run -it ubuntu bash"

What is TCP port 8080 typically used for?

Question
What is TCP port 8080 typically used for?

Answer
Jenkins, Docker, NodeJS, Apache Ambari, Apache Marathon, Apache Tomcat, Amazon Web Services' Elastic Load Balancer, JBoss Application Server, GitLab, M2MLogger (remote monitoring), InfoSphere BigInsights Console (IBM's proprietary Hadoop and Spark solution), JasperReports (because of Apache Tomcat), remote management of physical routers, and enterprise network proxy services all commonly use port 8080. 

In part, taken from Learning AWS, reference.com, Adobe's website (the old website was http://help.adobe.com/en_US/livecycle/9.0/securityHardening.pdf), Jaspersoft's website, and IBM's website (http://www.ibm.com/support/knowledgecenter/SSPT3X_2.1.2/com.ibm.swg.im.infosphere.biginsights.install.doc/doc/c0057855.html).

AWS S3 Problem: “The specified key does not exist.” How do you solve this?

Problem scenario:  You are trying to upload a file to S3 but get an error.  You use a command like this:
curl -T /path/to/sourcefile.java http://s3.amazonaws.com/my.bucket/sourcefile.java
You get an error in XML that includes "The specified key does not exist."

Solution:  Check the path to your source file.  Verify your source file is there with the name you expect it to be.

How Do You Troubleshoot the Error “eve.exceptions.ConfigException: DOMAIN dictionary missing or wrong.”?

Problem scenario
You are trying to use Python Eve.  You are using Python's interactive command prompt, and you run these two commands:

from eve import Eve
app = Eve()

But you get this error:

"File "/usr/local/lib/python2.7/dist-packages/eve/flaskapp.py", line 138, in __init__
    self.validate_domain_struct()
  File "/usr/local/lib/python2.7/dist-packages/eve/flaskapp.py", line 269, in validate_domain_struct
    raise ConfigException('DOMAIN dictionary missing or wrong.')
eve.exceptions.ConfigException: DOMAIN dictionary missing or wrong."

What should you do?

Solution
Run these three commands instead of the two above:

contint = {'MONGO_HOST': 'localhost', 'MONGO_PORT': 27017, 'MONGO_DBNAME': 'the_db_name', 'DOMAIN': {'contacts': {}}}
from eve import Eve
app = Eve(settings=contint)

# Now you can run app.run()

We based this solution from this page.

How Do You Copy Many Files from Your Linux Server into a Docker Container?

Problem scenario
You want to move copies of all the files in a given subdirectory (we'll say /tmp/a/ for an example) on a Docker host to go into a Docker container.  

Solution
From the server use the command "docker ps" to find the container ID.  Now that you have the container ID, you can write a script to move copies of the files into the Docker container.  Write this Bash script (with its final word being "done"):

#!/bin/bash
fir="docker cp "
las=" d387209ce13:/tmp/cont/"

for filename in /tmp/a/*; do
           full=$fir$filename$las
           eval $full
           done

Use this command: chmod u+x nameOfScript.sh
Change the "d387209ce13" your the container ID.  Change the /tmp/cont/ to the desired destination directory path in the Docker container. Run it (e.g., ./nameOfScript.sh) in the directory on the Docker host that has all the files you want copied (the location of the source files).  For future reference, the eval command is very useful.  Shell scripts can compose complex bash expressions and ultimately execute them easily with the eval command.

How To Install Docker on an AWS Instance of SUSE Linux?

Updated on 12/26/18

These directions show how to install Docker on SUSE Linux. These commands are for installing Docker 1.x on an AWS instance of SUSE 15.  These directions require that your AWS SUSE Linux server is in a security group that has access to the internet.  (If you want to see more directions for install Docker on different distributions of Linux in different public clouds, see this posting.)

Step #1  Log into Linux SUSE.
Step #2  Run these commands (but replace "jdoe" with the user that will run the Docker commands):

     sudo zypper -n in docker
sudo usermod -aG docker jdoe  
     sudo systemctl start docker

Step #3  If you are logged in as jdoe (or whichever user you used in step #2), log out.  Otherwise log into a different terminal as jdoe.
Step #4  Enter the docker container with a command like this: docker run -it ubuntu bash

What is the Order of Execution of a Puppet Manifest?

Problem scenario:  When someone uses a .pp file (either through a puppet agent call from a Puppet Agent node or via a puppet apply on a Puppet Master server), what order can a human DevOps Engineer expect the order to be?

Answer:  The execution order of a given manifest is the top of the manifest is executed and Puppet works its way down from there.  ContinualIntegration.com tested and verified this, and Puppet Labs purports this is correct.  However, PuppetLabs does recommend using metaparameters (reserved words that ensure the order of resource declaration fulfillment).  The word "before" and "require" can ensure that a given resource declaration will not happen outside of a desired sequence (before and after another operation respectively).  The syntax for these words is like this:

before => Exec["nameOfExecResourceDeclaration"]  # An operation with this specifier will always happen before nameOfExecResourceDeclaration happens.

require => Exec["nameOfExecResourceDeclaration"]  # This ensures an operation will not happen until nameOfExecResourceDeclaration happens.

Violating the syntax above will cause an error when the Puppet Agent runs such as "could not retrieve catalog from remote server...invalid relationship ... use correct format."