How Do You Deploy Artifactory in a Docker Container?

Problem scenario
You want to install and configure Artifactory in a Docker container on Linux.  How do you do this?

Solution
Prerequisites
i.  You must have Docker installed.  If you need assistance, click on the link for your distribution of Linux:

ii.  You must have Docker compose installed.  If you need assistance, see this posting.

Procedures

1.  Create a script in /tmp/ called artdock.sh with the content found in this file.

2.  Run the script like this: sudo bash /tmp/artdock.sh

3.  Open a web browser and go to the external IP address of the server (e.g., Docker host).  (To find this IP address from the back-end, run "curl icanhazip.com").  

4.  When looking at the web browser, close the initial pop up window after it loads.  To login, use the default username and password of "admin" and "password".

How Do You Install Docker on Debian 9 in GCP?

Problem scenario
You are running Debian 9 in GCP.  You want to install Docker on this server.  What should you do?

Solution
1.  Create a script in /tmp/ call installer.sh with the following lines of content:

#!/bin/bash
apt-get -y update
apt-get install -y apt-transport-https ca-certificates wget software-properties-common
wget https://download.docker.com/linux/debian/gpg
apt-key add gpg
echo "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee -a /etc/apt/sources.list.d/docker.list
apt-get -y update
apt-cache policy docker-ce
apt-get -y install docker-ce

2.  Run this command:  sudo bash /tmp/installer.sh

How Do You Install Redis on an Ubuntu 16.x or an 18.x Server in AWS?

Updated 12/26/18

Problem scenario

You want to deploy (install and configure) Redis (an in-memory data store) to an Ubuntu 16.x or an 18.x server.  What do you do?

Solution
These directions were based on this external posting as they appeared on 3/25/18.  

1.  Install some dependencies with these two commands:

sudo apt-get -y update
sudo apt-get -y install build-essential tcl

2.  Download the source media with this command:
curl http://download.redis.io/redis-stable.tar.gz > /tmp/redis-stable.tar.gz

3.  Install Redis with these commands:

sudo cp /tmp/redis-stable.tar.gz /opt/redis-stable.tar.gz
cd /opt/
sudo tar xzvf redis-stable.tar.gz
cd redis-stable
sudo make
sudo make test 
sudo make install

4.a.  Configure Redis with these commands: 

sudo mkdir /etc/redis
sudo cp /opt/redis-stable/redis.conf /etc/redis
sudo vi /etc/redis/redis.conf  
# find the uncommented "supervised" stanza and change the word "no" to "systemd"

4.b.  Configure Redis by creating a redis.service file.

sudo vi /etc/systemd/system/redis.service  # To create this file

# Insert these lines below to include "[Unit]" (as the first line) and 

"WantedBy=multi-user.target" (as the last line):

[Unit]
Description=Redis In-Memory Data Store
After=network.target

[Service]
User=redis
Group=redis
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecStop=/usr/local/bin/redis-cli shutdown
Restart=always

[Install]
WantedBy=multi-user.target

5.  Run these four commands to create the proper user, directory, and permissions settings:

sudo adduser --system --group --no-create-home redis
sudo mkdir /var/lib/redis
sudo chown redis:redis /var/lib/redis
sudo chmod 770 /var/lib/redis

6.  Run these two commands to start up Redis:

sudo systemctl start redis
sudo systemctl status redis

7.  Verify the installation with this command:  redis-server --version

How Do You Install Docker on Different Distributions of Linux in Different Public Clouds or on an On-Premise Server?

Problem scenario
You want to install Docker.  You want directions for installing it with and without making changes to how packages are installed on the Linux server.  You want directions that apply to Linux servers in AWS, Azure, or GCP.  You want directions that can apply to CentOS/RHEL, SUSE, or Ubuntu.  What do you do?

Solution
Find the link below that is the most appropriate for your needs.

How To Install Docker on an AWS Instance of RedHat Linux  (These directions use CentOS packages.)
How Do You Install Docker on an AWS Instance of RHEL?  (These directions use RHEL packages.)
How To Install Docker on an AWS Instance of SUSE Linux?
How To Install Docker on an AWS Instance of Ubuntu  (These directions do make changes to the Debian package repositories of your Linux server and use dockerproject.org and Ubuntu.com for the installation media.)
How Do You Install Docker on an AWS Instance of Ubuntu?  (These directions do not make changes to the Debian package repositories of your Linux server.)
How Do You Install Docker-Compose on a Linux Server?
How Do You Install Docker on a RHEL VM in Azure?
How Do You Install Docker on a GCP VM Running RHEL?
How Do You Install Docker on Debian 9 in GCP?



To run a Docker container as a non-root user, see this posting.

How Do You Troubleshoot the Cassandra Problem about “Connection refused”?

Problem Scenario
You have installed Apache Cassandra on Linux.  It is deployed as a single node configuration (not a cluster).  When you run "cqlsh" you get this error:

"Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})"

How do you troubleshoot cqlsh when you get this message ""Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})"?

Solution
Possible solution #1
Try running this (but substitute x.x.x.x with internal IP address of the Cassandra server): cqlsh x.x.x.x

Possible solution #2
Modify /etc/cassandra/default.conf/cassandra.yaml so the rpc_address stanza uses the FQDN of the Cassandra server (as found with a hostname -f command).  Restart the Cassandra services with these commands:

sudo systemctl daemon-reload
sudo systemctl enable cassandra
sudo systemctl start cassandra

Possible solution #3
If you get cqlsh to work sometimes and not others, check /var/log/cassandra for clues.  Search for the word "error" in the .log files.  You may need to increase the amount of virtual memory or add RAM to your server.  Click this link for more information on how to add virtual memory (aka swap space).  If the server is in AWS click here for directions on how to resize it.  If the server is in GCP click here for directions on how to resize it.

Problem solution #4
You may want to see if a software firewall such as ufw or firewalld is running.  You may want to check if iptables is blocking the port.  For further network diagnosing, see this posting.  In our experience we do not see activity on port 9042 when Cassandra is properly listening and ready for a cqlsh command.

Possible solution #5
Deploy Cassandra again.  Remove the files that were deployed and created, and start over.  If you need directions for deploying Cassandra to a new, fresh Linux server, click on the link that corresponds with your distribution:

Debian or Ubuntu
CentOS/RHEL/Fedora
SUSE

How Do You Install kubectl on a CentOS/RHEL/Fedora of Linux server?

Problem scenario
You want the kubectl command to work on a server running a Red Hat distribution of Linux.  How do you install kubectl on a CentOS/RHEL/Fedora Linux server?

Solution
#1  This assumes that you have access to the internet.  You can do it with these three commands:

cd /tmp

curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl

sudo mv kubectl /usr/local/bin/

(An alternative way with a .repo file that is more secure would be to use steps #1 through #3 of How Do You Install Kubernetes on an AWS Instance of RedHat Linux? and then run "sudo yum -y install kubectl". )

#2  For kubectl commands to work, you will need a .kube directory in your home directory.  You may want to run this: mkdir ~/.kube

#3  Inside this .kube directory, you will need a file following this naming convention: config-foobar
Replace foobar with the name of your cluster.

#4  The contents of this config-foobar file should be something like this if you are using AWS:

apiVersion: v1
clusters:
- cluster:
    server: <endpoint-url>
    certificate-authority-data: <base64-encoded-ca-cert>
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: aws
  name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws-iam-authenticator
      args:
        - "token"
        - "-i"
        - "<cluster-name>"
         - "-r"
         - "<role-arn>"

#5  If you are not using AWS, you will need to modify the above template to the best of your ability.  If you are using AWS, follow these four steps (each quoted from https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html ):

"Replace the <endpoint-url> with the endpoint URL that was created for your cluster."  (It will appear under the "API server endpoint" value.)

"Replace the <base64-encoded-ca-cert> with the certificateAuthority.data that was created for your cluster."  (It will appear under the "Certificate Authority" value.)

"Replace the <cluster-name> with your cluster name."

Replace the <role-arn> with the value for the "Role ARN" as the web UI shows.

#6  You will need to run this command (but replace foobar with the name of your cluster):

export KUBECONFIG=$KUBECONFIG:~/.kube/config-foobar

(You may want to do this: "cd ~/.kube" and run "ls -lh".)

You may want to see these directions too.

How Do You Deploy Graylog in a Docker Container?

Problem scenario
You want to set up (install and configure) Graylog from a Docker container.  What do you do?

Solution
1.  Verify you have a least two processors with this command:  cat /proc/cpuinfo | grep -ic processor

Was the output two or more?  If so, proceed.  If not, get a server with two processors.  If you are using an AWS EC-2 instance, see this link to add a processor.  If you are using a GCP virtual server, see this posting for adding a processor.

2.  Install Docker.  If you need help with this, see this posting.

3.  Run these four commands (where x.x.x.x in the final command below is the external IP address found with a "curl icanhazip.com" command):

docker pull graylog2/server

docker run --name contint-mongo -d mongo:2

docker run --name contint-elasticsearch -d elasticsearch:2 elasticsearch -Des.cluster.name="graylog"

docker run --link contint-mongo:mongo --link contint-elasticsearch:elasticsearch -p 9000:9000 -e GRAYLOG_WEB_ENDPOINT_URI="http://x.x.x.x:9000/api" -d graylog2/server

4.  Wait a few minutes, and then open a web browser.  Go to "http://x.x.x.x:9000/" (with no quotes), where x.x.x.x is the external IP address of the Linux server.  The default credentials to the web UI of Graylog are as follows:

username: admin
password: admin

What is DevOps and Why Do People Want to Adopt It?

What is DevOps?
DevOps, among other things, is a business cultural shift that affects how labor is divided at different companies that adopt it. At the enterprise level, a company that embraces DevOps culture may have teams of employees focused on builds and releases or automation. These teams may be integrated with software engineers, systems administrators, quality assurance analysts, database administrators and security experts. A reduction itself in departments within I.T. in an organization may be a movement toward DevOps practices. Departments and specializations may be seen as factors that isolate professionals rather than integrate (for collaboration and building security into technology products) from a DevOps consultant's perspective. Some people say DevOps is automating what you can program (having programs being designed by automatic processes such as those in operations) and programming what you can automate (creating automation that is sophisticated for operations tasks).

Why do people and businesses want to use DevOps?
DevOps is a force multiplier. Code can be generated dynamically that adheres to a number of quality guidelines in an efficient way. Code itself can be instrumented (e.g., with a metaclass in a DevOps-favorite language of Python) and have the ability to manipulate or leverage itself.* This can enable greater observability, measurability, and quality assurance.

One reason to use DevOps is to develop software or I.T. solutions that are designed securely and highly performant. A second reason is to use Agile, or to be efficient with software development in fulfilling frequently-changing business requirements; to bring software (as a service through a website) to the market as quickly as possible, DevOps can help. A third reason is to moralize the employees by giving them more interesting work.

Ultimately development involves security and performance concerns and timing is critical for the delivery a new product to the marketplace. By dividing labor into specific categories that are inches wide and miles deep, the costs associated with the constraints from other teams can go unappreciated. Technology professionals frequently do not have visibility into their teammates' productivity obstacles. There often is not enough time to learn of different work blockers. Processes can be dependent on one workers' knowledge if a company shuns integration in favor of isolation (as exemplified in the seminal book The Phoenix Project). If the employees can achieve an optimal amount of communication (neither excessive nor insufficient), sharing responsibilities has a multitude of advantages.

The traditional division of responsibilities of I.T. development and operations can make people not work together. Rather than have programmers programming and systems administrators administering servers, companies mature in their DevOps adoption have people doing some duties of both programming and systems administration.

There are several goals that an enterprise has. By sharing broad responsibilities, each person has a greater appreciation for product and platform security. By leveraging tools and methods across specialized I.T. disciplines, organizations can perform more effectively. Internal tools can be built as an investment to help the business maximize its productivity. Having individuals leverage (and repurpose) multiple, unisolated tools can lead to efficiencies and creative utilizations that a highly departmentalized businesses would be ignorant of.

An enterprise may want to adopt DevOps culture which would involve "full stack developers" to engineer platforms and leverage programming to automate infrastructure tasks. At the individual level a DevOps practitioner may fulfill duties as a professional I.T. generalist. DevOps professionals may engage in activities as diverse as coding, installing and configuring operating systems, optimizing containers, creating or fixing a CI/CD pipeline, extending configuration management tools to deploy code or infrastructure patches rapidly, testing code, automating processes related to quality assurance, managing databases, or adjusting networking devices to enhance security. Harley Shaiken said "[s]killed work can be satisfying because it is varied, challenging, and creative." (This quote was taken from on page 113 of Disabling Professions, a book by several authors.) In The Mythical Man-Month, or more specifically "No Silver Bullet", the author Fred Brooks indicates that job growth could be limited for functional specialists in a matrixed team (pages 222 and 305). Some DevOps advocates have pushed for adopting it because they personally dislike the overly-repetitive tasks of highly specialized positions. Automation and well-built internal tools can lead toward greater quality control of a company's product line.

To comply with confusing regulations, you may want to use DevOps. Sarbanes Oxley encourages utilization of DevOps as developers must live in a development environment (according to this external site).

Some sources are skeptical of generalists handling the work of specialists.** Arguably Sarbanes Oxley discourages DevOps as generalists cannot write code and have access to production if the application deals with financial reporting. Some companies error on the side of caution and design jobs based on the regulations from a risk-averse attitude.

"For software developers, the big problem comes in Separation of Duties. Basically, SarbOx says that a software developer should not have ANY kind of write access to production systems that can affect the financial reporting of the corporation." (This previous quote was taken from Toolbox.com.) Automation capabilities and scaling capabilities of DevOps help make compliance achievable (with regulations and laws beyond just Sarbanes Oxley).

DevOps engineers and consultants generally could not have learned their trade at school (as of 2019). Ivan Illich said (on page 66 of The Right to Useful Unemployment) "education turns into the major generator of a disabling division of labour." The DevOps movement enhances the productive potential of many professionals. DevOps engineers tend to earn more than software engineers or systems administrators.

Shareholders, governing boards, and executives of businesses usually determine the extent of DevOps adoption more than the employees do. The DevOps Handbook indicates that a business' profitability is directly proportional to the degree that it adopts DevOps.


*Many implementations of DevOps have nothing to do with Python's metaclasses. It is just an example.

The term metaprogramming refers to the potential for a program to have knowledge of or manipulate itself. Python supports a form of metaprogramming for classes called metaclasses.

Taken from https://realpython.com/python-metaclasses/

To learn more about metaclasses in Python, see page 173 of Expert Python Programming, Third Edition.
** This article says that generalists can make mistakes.

How Do You Install Golang on to Any Type of Linux?

Updated on 10/18/19

Problem scenario
You want to do some coding in the Go programming language.  You want a script to install Golang on any distribution of Linux (e.g., CentOS/Red Hat Enterprise Linux/Fedora, Debian/Ubuntu, or SUSE).  What do you do?

Solution
1.  Create a file in /tmp/ called go.sh with the following content:

version=1.10.8  # Change this version as needed
curl https://storage.googleapis.com/golang/go$version.linux-amd64.tar.gz > /tmp/go$version.linux-amd64.tar.gz
cp /tmp/go$version.linux-amd64.tar.gz /usr/local/go$version.linux-amd64.tar.gz
cd /usr/local/
tar xzvf go$version.linux-amd64.tar.gz
#echo "export PATH=""$""PATH:/usr/local/go/bin" >> ~/.profile
echo "export PATH=""$""PATH:/usr/local/go/bin" > /etc/profile.d/go.sh
echo "You will have to log out and log back in for go to work."
echo "You can use the 'go version' command to determine what version was installed"

2.  Run it like this:  sudo bash /tmp/go.sh

How Do You Demonstrate Recursion in a Sorting Algorithm Written in Python?

Problem scenario
You want to use recursion in Python in a sorting algorithm. How do you do this?

Solution
Save this program as contintsort.py and run it python contintsort.py. The code closer to the top is in Python 2, and the code closer to the bottom is in Python 3.

# Written by www.continualintegration.com
# Usage is self-explanatory. Just run the program. python contintsort.py
# No extra files are needed.
# This demonstrates a sorting algorithm.  It runs in O(n^2) time in worst case scenarios.
# It runs in O(n) time in best case scenarios (when the data is already sorted).
# It accepts user input as long as the input is an integer.

import random, re  # We need random to generate random integers.  We need re to do "regular expressions"
# Regular expressions are methods of searching one pattern of text with another.

print "If you just press enter with the prompts below, you will get random integers assigned."
print "Enter an integer or press enter to get one assigned at random."
print " "

first = raw_input("Enter the first number: ")
second = raw_input("Enter the second number: ")
third = raw_input("Enter the third number: ")
fourth = raw_input("Enter the fourth number: ")
fifth = raw_input("Enter the fifth number: ")
sixth = raw_input("Enter the sixth number: ")
seventh = raw_input("Enter the seventh number: ")
eighth = raw_input("Enter the eighth number: ")
ninth = raw_input("Enter the ninth number: ")
tenth = raw_input("Enter the tenth number: ")
coolList = [first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth]

for i in range(10):
  if coolList[i] == "":
    coolList[i] = random.randint(1,1001) # Assign if a blank was entered
  else:
    b = coolList[i] # Assign input to a variable.
    a = (re.search('[a-zA-Z]', b)) # Check for non-numerical characters that were inputted.
    if a: coolList[i] = random.randint(1,1001) # Assign random int if alphabet chars were entered
    else: coolList[i] = int(coolList[i]) # make input an integer (cast input to be an integer)

def swapper(counter, interListb, flag):   # Define a function that requires three parameters when called.
  if (flag == 'exitnow'):
     print "************************************"  # This function will end if the flag was set to exit.
  else:
     x = interListb[(counter)]
     y = interListb[(counter + 1)]
     if (y < x):   # swap and then verify previous pair in list is ordered after swap
       interListb[(counter + 1)] = x
       interListb[(counter)] = y
       counter = counter -1  # Now that there was a swap, let's see of the pair to the left of swapped number is in order.
       if (counter < 1):
           counter = 0 # Do not go outside the index.  Go to the left-most item in the list.
       swapper(counter, interListb, flag) # Make recursive calls to check adjacent pairs in the list moving to the left.
     else:          # do not swap. do move to the right in the list to continue checking pairs.
       var1 = (len(interListb) - 2) # Counter will be incremented later. Lists are 0 based. Length counts the 0th (first) item.
       if counter >= var1:  # The right-most item was reached.  Set a flag to exit the recursive calling of swapper.
           flag = 'exitnow' # Exit recursion with the next call.
       counter = counter + 1
       swapper(counter, interListb, flag) # Make recursive calls to check adjacent pairs in the list moving to the right.
  return interListb # Return the list outside of the recursion.  In Python you do not return something inside a recursive call.

qq = swapper(0, coolList, 'enter') # This invokes swapper for the first time.
print qq

Here is the program in Python 3:

# Written by www.continualintegration.com
# Usage is self-explanatory. Just run the program. python contintsort.py
# No extra files are needed.
# This demonstrates a sorting algorithm.  It runs in O(n^2) time in worst case scenarios.
# It runs in O(n) time in best case scenarios (when the data is already sorted).
# It accepts user input as long as the input is an integer.

import random, re  # We need random to generate random integers.  We need re to do "regular expressions"
# Regular expressions are methods of searching one pattern of text with another.

print("If you just press enter with the prompts below, you will get random integers assigned.")
print("Enter an integer or press enter to get one assigned at random.")
print(" ")

first = input("Enter the first number: ")
second = input("Enter the second number: ")
third = input("Enter the third number: ")
fourth = input("Enter the fourth number: ")
fifth = input("Enter the fifth number: ")
sixth = input("Enter the sixth number: ")
seventh = input("Enter the seventh number: ")
eighth = input("Enter the eighth number: ")
ninth = input("Enter the ninth number: ")
tenth = input("Enter the tenth number: ")
pre_list = [first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth]

for i in range(10):
  if pre_list[i] == "":
    pre_list[i] = random.randint(1,1001) # Assign if a blank was entered
  else:
    b = pre_list[i] # Assign input to a variable.
    a = (re.search('[a-zA-Z]', b)) # Check for non-numerical characters that were inputted.
    if a: pre_list[i] = random.randint(1,1001) # Assign random int if alphabet chars were entered
    else: pre_list[i] = int(pre_list[i]) # make input an integer (cast input to be an integer)

def swapper(counter, intermediate_list, flag):   # Define a function that requires three parameters when called.
  if (flag == 'exitnow'):
     print("************************************")  # This function will end if the flag was set to exit.
  else:
     x = intermediate_list[(counter)]
     y = intermediate_list[(counter + 1)]
     if (y < x):   # swap and then verify previous pair in list is ordered after swap
       intermediate_list[(counter + 1)] = x
       intermediate_list[(counter)] = y
       counter = counter -1  # Now that there was a swap, let's see of the pair to the left of swapped number is in order.
       if (counter < 1):
           counter = 0 # Do not go outside the index.  Go to the left-most item in the list.
       swapper(counter, intermediate_list, flag) # Make recursive calls to check adjacent pairs in the list moving to the left.
     else:          # do not swap. do move to the right in the list to continue checking pairs.
       var1 = (len(intermediate_list) - 2) # Counter will be incremented later. Lists are 0 based. Length counts the 0th (first) item.
       if counter >= var1:  # The right-most item was reached.  Set a flag to exit the recursive calling of swapper.
           flag = 'exitnow' # Exit recursion with the next call.
       counter = counter + 1
       swapper(counter, intermediate_list, flag) # Make recursive calls to check adjacent pairs in the list moving to the right.
  return intermediate_list # Return the list outside of the recursion.  In Python you do not return something inside a recursive call.

sorted_list = swapper(0, pre_list, 'enter') # This invokes swapper for the first time.
print(sorted_list)