How Do You Deploy LAMP with Ubuntu Linux?

Problem scenario
You want to deploy the LAMP stack on a Debian/Ubuntu distribution of Linux. You want to install and the four core technologies that compose the LAMP stack (Linux, Apache web server, MariaDB/MySQL, and PHP).   How do you do this?

Solution #1 (with MySQL)
To install the OS, Ubuntu Linux, see this link. To install MySQL run these commands:

sudo apt-get -y update
sudo apt-get -y install php apache2 mysql-server

# Provide a password for the mysql root user when prompted.
# To log into the mysql database for the first time, use this command:
# mysql -u root -p  # You will be prompted to enter a password.

Solution #2 (with MariaDB)
If you wanted to use MariaDB and assuming Ubuntu is already installed, run these commands:
sudo apt-get -y update
sudo apt-get -y install php apache2 mariadb-server

How Do You Tell if There is a Firewall on Linux?

Problem scenario
You want to see if there is an operating system, software or host-based firewall on your Linux machine.  What do you do?

Solution
Run the following six commands for clues:

sudo ps -ef | grep firewalld
sudo systemctl status firewalld
sudo ps -ef | grep ufw
sudo ufw status
sudo iptables -L   #*
sudo nmap -P0 127.0.0.1

If you are running the server in a public cloud, remember that there are Security Groups in AWS, Network Security Groups (NSGs) in Azure, and firewall rules in GCP that could block connectivity.  The above is mostly for determining if there is a running an OS-level, software firewall.

* Here is an example of output of the iptables -L command when iptables is not blocking anything:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

How Do You Add a New Node to a Hadoop Cluster?

Problem scenario
You have a multi-node cluster of Hadoop.  You want to add a new data node.  What do you do?

Solution
1. a)  Log into the server that will be the new DataNode.  Do these things until you get to step 2.

b)  Install Hadoop on the new DataNode.  If you do not know how, see this posting.

c)  The DataNode server must be configured to resolve the NameNode's hostname.  Modify the new DataNode's /etc/hosts file if you do not have a DNS server on your network for this server that will be the DataNode.  The modification the /etc/hosts file should get (if there is no DNS server for it) should be an entry for the NameNode server.  

d)i.  You need to modify three .xml files on the DataNode. These files are core-site.xml, mapred-site.xml, and hdfs-site.xml.  To do these modifications, see steps 6 through 8 on this set of instructions.

d)ii.  Look at the port used in core-site.xml.  Verify this port is open from the DataNode to the NameNode.  If you do not know how to test and the DataNode is running Windows, see this posting.  If you do not know how to test and the DataNode us running Linux, install nmap and use this command: nmap -p xxxxx FQDNofNameNode # where xxxxx is the port number and FQDNofNameNode is the FQDN of the NameNode.

e)  Create an hduser account.  (You'll need this for step 2e below.)  Create an hadoop group name.  Ensure that the hduser can ssh into the local machine.  If you do not know how to do all these things, see steps 2 and 3 in this posting.  

f)  Run these two commands:

cd /app
sudo chown hduser:hadoop hadoop

2. a)  Log into the NameNode server.  Do these things:  

b)  The NameNode server must be configured to resolve the DataNode's hostname.  Modify the new NameNode's /etc/hosts file if you do not have a DNS server on your network for the NameNode.  The modification the /etc/hosts file should get (if there is no DNS server for it) should be an entry for the DataNode server.  

c)  On the NameNode server, append the new DataNode's hostname to this file /usr/local/hadoop/etc/hadoop/workers

d)  Modify the hdfs-site.xml file.  vi /usr/local/hadoop/etc/hadoop/hdfs-site.xml  # Update the dfs.replication integer value to be the sum of the servers incluing the NameNode and all DataNodes (including the one you added).

e)  Configure passwordless SSH authentication from the NameNode's hduser account to the server that will be the DataNode.  See this posting if you do not know how.  Test the ability to passwordless SSH.

f) From the NameNode run ssh hduser@DataNodeHostName # where DataNodeHostName is the host name of the DataNode.  This tests it will work.

g)  Run these commands (but note that it will delete all the data in your cluster) as the hduser (or designated Linux username for running such commands):

hdfs namenode -format
# When prompted with this:
# "Re-format filesystem in Storage Directory /app/hadoop/tmp/dfs/name ? (Y or N)"

# choose "Y" (with no quotes) and press enter
bash /usr/local/hadoop/sbin/start-dfs.sh

How Do You Use ECR in AWS?

Problem scenario
You have heard about Amazon Elastic Container Registry.  You want to configure a Docker registry using an AWS PaaS offering.  How do you use ECR?

Solution
1.  Log into the AWS web console.
2.  Go here: https://console.aws.amazon.com/ecs/home#/repositories
3.  Click "Get started" if you have never created a repository before.  If you have created a repository, skip to the next step.
4.  Click "Create Repository"
5.  Enter text in the "Repository name" field.
6.  Click "Next step".  
7.  Search for the word "region" on the webpage.  Make a note of the region that you are in (e.g., us-west-2).
8.  You should see something like this: "Successfully created repository"

Make a note of tha a string in this format:
3827461381.dkr.ecr.us-west-2.amazonaws.com/contint

9.a.  Go to the back-end of a Linux server.  To connect to this repository, install and configure AWS CLI.  If you need assistance with this, see this posting.

9.b Then run this command from that server, but substitute us-east-6 with the region found in step #7:
(aws ecr get-login --no-include-email --region us-east-6)

10.  The very long results from the above command is a draft of another command.  Compose this draft of a command after "sudo " (with no quotes).  Run this "sudo ..." command.

11.  You are done and now ready to upload images.  You may want to see this posting.

How Do You Prevent a Process from Ending Due to a User Logging Off?

Problem scenario
You want a process to keep running on a Linux server.  But when the user who starts it logs off, the process ends.  How do you allow the user to log off but obviate the logging off from terminating (through signaling termination) of that process you want to remain running?

Solution
Have the user use the nohup command to execute the important process.  As an example imagine that the process that is important is running "script.sh."  Here is how to use nohup:

nohup bash script.sh

You may get a message like this in response:  "nohup: ignoring input and appending output to 'nohup.out'"

This is normal.  However, the process may appear to hang. So it is often advisable to send the command to the background with an "&" symbol (because commands that involve a user logging off tend to have a long duration):

nohup bash script.sh &

If the user session, the one that ran the "nohup" command (like either of the above), ends (e.g., the user logs off or a system administrator kills the session), the process for script.sh will continue.  If script.sh is a long-running and important process, the nohup command can be of great help.

How Do You Create a New Project in GitLab?

Problem scenario
You want to test out GitLab.  How do you create your first project?

Solution
Prerequisite
This assumes you have GitLab installed; if you do not know how, see this posting.

Procedures
1.  Log into GitLab via the web UI.  (If you do not have credentials and you set it up, the web UI for GitLab should have prompted you to enter a new password twice.  This password is for the "root" user.)

2.  Click the icon that looks like a document for "Create a project". 

3.  Enter a name in the field for "Project name". Set the visibility level to "Public" for non-sensitive files and to make testing Gitlab easy.  Click "Create project".  

4.  At the top of the screen it may say "You won't be able to pull or push project code via SSH until you add an SSH key to your profile."  Click the "add an SSH key" portion.

5.  Go to the back end (the character prompt) of the Linux server with GitLab.  If you have not created an id_rsa.pub file for the Linux user that will interact with GitLab, log into Linux as that user (or use sudo su jdoe, assuming jdoe is the user).  Then run this command:

ssh-keygen -t rsa -P "" # Accept the default location

6.  Obtain a copy of the content of the Linux user's id_rsa.pub file that will interact GitLab.  Paste it into the web browser for the field that is "Key." 

7.  Click "Add Key" in the web UI.  Now you have a project in GitLab.

Happy International Data Privacy Day!

In 2009 the U.S. Congress decided to recognize Data Privacy Day for years in the future (nationaldaycalendar.com).  We appreciate a day to promote privacy.  We are in the works to support encrypted email on this website.

Data Privacy Day "is celebrated every January 28 and is an international effort to promote the importance of data privacy" (taken from https://www.us-cert.gov/ncas/current-activity/2017/01/24/Data-Privacy-Day-Events).  It started when Convention 108 was signed on January 28, 1981; this was "the first legally binding international treaty dealing with privacy and data protection" (taken from https://staysafeonline.org/data-privacy-day/about/).  Enumerated rights are not always enforced.  Cisco and Intel are leading sponsors (https://staysafeonline.org/data-privacy-day/about/).  

You may want to go to this website that is sponsored by the National Cyber Security Alliance.  It has links that individuals as well as businesses can use to learn about what they can do to protect their privacy.  For software and other tools to help you secure your computer, you may want to see this link.  If you have people walk by your monitor, you may want a privacy shield.  It can obfuscate your screen to passers-by. 

Recently cryptocurrency has given people a new way to transact business with many privacy benefits.  To purchase a variety of cryptocurrency with U.S. dollars, you may want to try Changelly.  You may also want to purchase a book on the subject.  It is advisable to keep your cryptocurrency in an offline storage format given the publicity of recent of platform hackings.  Amazon has accessories to help you do this. 

Some books to help you learn about privacy include the following:  

You may want to see our links on intrusion detection and intrusion prevention systems.

How Do You Install GitLab on Debian or Ubuntu Linux?

Problem scenario
You want to do deploy (install and configure) GitLab on a Debian Linux server.  What do you do?

Solution
1.  Get a Debian or Ubuntu Linux server with either 3.5 GB of RAM or at least 1.5 GB of RAM and 2 GB of virtual memory.  If you need to resize an AWS instance, see this posting.  If you need to resize a GCP instance, see this posting.  If you need directions for creating virtual memory (aka swap space), see this posting.

2.  Run these commands from the backend of the Linux server:

sudo apt-get -y update
sudo apt-get install -y curl openssh-server ca-certificates

sudo apt-get install -y postfix
# Choose "Ok" to the first prompt you see (your only option).
# Select the "Internet Site" option
# Accept the default options continually by pressing Enter to each one. In some cases there will be only one option anyway.

3.  Run this command:

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash

4.  Run a modified version of command.  Substitute x.x.x.x with the external IP address of your server or the URL you want to access via a web browser when the process is finished, append ":yyyy", with no quotes, to the "x.x.x.x" where yyyy is the non-default port you want GitLab to operate on:

sudo EXTERNAL_URL="http://x.x.x.x:yyyy" apt-get install gitlab-ee

5.  Go to the web UI.  You will be prompted to create a new password.  Remember this password.

6.  Log into the web UI.  For the user, use "root" with no quotes.  Use the password you entered in the step above.

How Do You Troubleshoot the GitLab Integration Webhook Error “Permission you need to have (but didn’t) hudson.model.Hudson”?

Problem scenario
You are trying to configure GitLab to work with Jenkins (so the two are integrated to enable  CI, continual integration, in your environment).  When you test a connection of an integration  webhook from GitLab to reach out to Jenkins you receive this error:

"Hook executed successfully but returned HTTP 403... You are authenticated as anonymous Groups that you are in: Permission you need to have (but didn't) hudson.model.Hudson.Read...which is implied by hudson.security.Permission.GenericRead..."

How do you solve this problem?

Solution

1.  In Jenkins install the Build Authorization Token Root Plugin if it is not already installed.  If it is installed, skip to step #5.
2.  To install it log into Jenkins. 
3.  Go to Plugin Manager -> Available. 
4.  Check the box for "Build Authorization Token Root Plugin."
5.  Check the button "Install without restart."
6.  Go to the Jenkins job in this scenario. 
7.  Click "Configure" -> "Build Triggers". 
8.  Check the option "Trigger builds remotely..." option.  
9.  Insert an arbitrary string in the "Authentication Token" field.  
10.  Click Save.
11.  Copy the URL for the job and draft it according to this format:  

http://JENKINS_URL/buildByToken/build?job=JOBNAME&token=TOKENNAME

Substitute JENKINS_URL for the URL to your Jenkins server (e.g., with port 8080).  Substitute JOBNAME for the name of the job.  Substitute TOKENNAME with the arbitrary string you entered above in the "Authentication Token" field.

12.  Copy this URL and paste it in the GitLab field for the URL in the integration webhook. 
13.  Test the connection again.

How Do You Create Swap Space on a Linux Server Running in the Cloud?

Problem scenario
You have sufficient hard disk space.  But your applications are memory constrained on your Linux server.  You cannot add more RAM.  How do you create swap space on your Linux server (to create virtual memory)?

Solution
Overview
We consider virtual memory to be a hybrid of RAM "and disk space that running processes can use. Swap space is the portion of virtual memory that is on the hard disk, used when RAM is full." The quoted section was taken from StackOverflow.

Procedures
These directions to have been tested to work on a Google Cloud Platform Debian server, an AWS Ubuntu server, and an AWS RedHat Enterprise Linux (RHEL) server.  (If you want a script to do this, see this posting.) They will work on a server in Azure too.

Warning: This will create a swap space that can be read by other processes.  You may want to use the optional step at the end to make it more secure.

1.a.  This is optional.  You may want to run this command:

top | grep Swap # This is an optional step.  Cancel out after you see a line of results.

1.b.  If you are running SUSE or a RedHat derivative of Linux (CentOS/RHEL/Fedora), run this:
sudo dd if=/dev/zero of=/mnt/2GB.swap count=2048 bs=1MiB

If you are running Debian/Ubuntu Linux, run this:
sudo fallocate -l 2G /mnt/2GB.swap

2.  Run these commands:
sudo mkswap /mnt/2GB.swap
sudo swapon /mnt/2GB.swap

3.  Update the /etc/fstab to have this line (i.e., append it to the file as the lowest line):
/mnt/2GB.swap  none  swap  sw 0  0

4.  Update the /etc/sysctl.conf file to have this stanza at the very end:

vm.swappiness=10

5.  Run this command:  sudo swapon -s

6.  Optional command to make the swap space more secure:  sudo chmod 600 /mnt/2GB.swap

7.  Optional command to check your work and compare it to the first command above:
top | grep Swap # This is an optional step.  Cancel out after you see a line of results.

The above directions were based on RackSpace directions here.