How Do You Use ECR in AWS?

Problem scenario
You have heard about Amazon Elastic Container Registry.  You want to configure a Docker registry using an AWS PaaS offering.  How do you use ECR?

Solution
1.  Log into the AWS web console.
2.  Go here: https://console.aws.amazon.com/ecs/home#/repositories
3.  Click "Get started" if you have never created a repository before.  If you have created a repository, skip to the next step.
4.  Click "Create Repository"
5.  Enter text in the "Repository name" field.
6.  Click "Next step".  
7.  Search for the word "region" on the webpage.  Make a note of the region that you are in (e.g., us-west-2).
8.  You should see something like this: "Successfully created repository"

Make a note of tha a string in this format:
3827461381.dkr.ecr.us-west-2.amazonaws.com/contint

9.a.  Go to the back-end of a Linux server.  To connect to this repository, install and configure AWS CLI.  If you need assistance with this, see this posting.

9.b Then run this command from that server, but substitute us-east-6 with the region found in step #7:
(aws ecr get-login --no-include-email --region us-east-6)

10.  The very long results from the above command is a draft of another command.  Compose this draft of a command after "sudo " (with no quotes).  Run this "sudo ..." command.

11.  You are done and now ready to upload images.  You may want to see this posting.

How Do You Prevent a Process from Ending Due to a User Logging Off?

Problem scenario
You want a process to keep running on a Linux server.  But when the user who starts it logs off, the process ends.  How do you allow the user to log off but obviate the logging off from terminating (through signaling termination) of that process you want to remain running?

Solution
Have the user use the nohup command to execute the important process.  As an example imagine that the process that is important is running "script.sh."  Here is how to use nohup:

nohup bash script.sh

You may get a message like this in response:  "nohup: ignoring input and appending output to 'nohup.out'"

This is normal.  However, the process may appear to hang. So it is often advisable to send the command to the background with an "&" symbol (because commands that involve a user logging off tend to have a long duration):

nohup bash script.sh &

If the user session, the one that ran the "nohup" command (like either of the above), ends (e.g., the user logs off or a system administrator kills the session), the process for script.sh will continue.  If script.sh is a long-running and important process, the nohup command can be of great help.

How Do You Create a New Project in GitLab?

Problem scenario
You want to test out GitLab.  How do you create your first project?

Solution
Prerequisite
This assumes you have GitLab installed; if you do not know how, see this posting.

Procedures
1.  Log into GitLab via the web UI.  (If you do not have credentials and you set it up, the web UI for GitLab should have prompted you to enter a new password twice.  This password is for the "root" user.)

2.  Click the icon that looks like a document for "Create a project". 

3.  Enter a name in the field for "Project name". Set the visibility level to "Public" for non-sensitive files and to make testing Gitlab easy.  Click "Create project".  

4.  At the top of the screen it may say "You won't be able to pull or push project code via SSH until you add an SSH key to your profile."  Click the "add an SSH key" portion.

5.  Go to the back end (the character prompt) of the Linux server with GitLab.  If you have not created an id_rsa.pub file for the Linux user that will interact with GitLab, log into Linux as that user (or use sudo su jdoe, assuming jdoe is the user).  Then run this command:

ssh-keygen -t rsa -P "" # Accept the default location

6.  Obtain a copy of the content of the Linux user's id_rsa.pub file that will interact GitLab.  Paste it into the web browser for the field that is "Key." 

7.  Click "Add Key" in the web UI.  Now you have a project in GitLab.

Happy International Data Privacy Day!

In 2009 the U.S. Congress decided to recognize Data Privacy Day for years in the future (nationaldaycalendar.com).  We appreciate a day to promote privacy.  We are in the works to support encrypted email on this website.

Data Privacy Day "is celebrated every January 28 and is an international effort to promote the importance of data privacy" (taken from https://www.us-cert.gov/ncas/current-activity/2017/01/24/Data-Privacy-Day-Events).  It started when Convention 108 was signed on January 28, 1981; this was "the first legally binding international treaty dealing with privacy and data protection" (taken from https://staysafeonline.org/data-privacy-day/about/).  Enumerated rights are not always enforced.  Cisco and Intel are leading sponsors (https://staysafeonline.org/data-privacy-day/about/).  

You may want to go to this website that is sponsored by the National Cyber Security Alliance.  It has links that individuals as well as businesses can use to learn about what they can do to protect their privacy.  For software and other tools to help you secure your computer, you may want to see this link.  If you have people walk by your monitor, you may want a privacy shield.  It can obfuscate your screen to passers-by. 

Recently cryptocurrency has given people a new way to transact business with many privacy benefits.  To purchase a variety of cryptocurrency with U.S. dollars, you may want to try Changelly.  You may also want to purchase a book on the subject.  It is advisable to keep your cryptocurrency in an offline storage format given the publicity of recent of platform hackings.  Amazon has accessories to help you do this. 

Some books to help you learn about privacy include the following:  

You may want to see our links on intrusion detection and intrusion prevention systems.

How Do You Install GitLab on Debian or Ubuntu Linux?

Problem scenario
You want to do deploy (install and configure) GitLab on a Debian Linux server.  What do you do?

Solution
1.  Get a Debian or Ubuntu Linux server with either 3.5 GB of RAM or at least 1.5 GB of RAM and 2 GB of virtual memory.  If you need to resize an AWS instance, see this posting.  If you need to resize a GCP instance, see this posting.  If you need directions for creating virtual memory (aka swap space), see this posting.

2.  Run these commands from the backend of the Linux server:

sudo apt-get -y update
sudo apt-get install -y curl openssh-server ca-certificates

sudo apt-get install -y postfix
# Choose "Ok" to the first prompt you see (your only option).
# Select the "Internet Site" option
# Accept the default options continually by pressing Enter to each one. In some cases there will be only one option anyway.

3.  Run this command:

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash

4.  Run a modified version of command.  Substitute x.x.x.x with the external IP address of your server or the URL you want to access via a web browser when the process is finished, append ":yyyy", with no quotes, to the "x.x.x.x" where yyyy is the non-default port you want GitLab to operate on:

sudo EXTERNAL_URL="http://x.x.x.x:yyyy" apt-get install gitlab-ee

5.  Go to the web UI.  You will be prompted to create a new password.  Remember this password.

6.  Log into the web UI.  For the user, use "root" with no quotes.  Use the password you entered in the step above.

How Do You Troubleshoot the GitLab Integration Webhook Error “Permission you need to have (but didn’t) hudson.model.Hudson”?

Problem scenario
You are trying to configure GitLab to work with Jenkins (so the two are integrated to enable  CI, continual integration, in your environment).  When you test a connection of an integration  webhook from GitLab to reach out to Jenkins you receive this error:

"Hook executed successfully but returned HTTP 403... You are authenticated as anonymous Groups that you are in: Permission you need to have (but didn't) hudson.model.Hudson.Read...which is implied by hudson.security.Permission.GenericRead..."

How do you solve this problem?

Solution

1.  In Jenkins install the Build Authorization Token Root Plugin if it is not already installed.  If it is installed, skip to step #5.
2.  To install it log into Jenkins. 
3.  Go to Plugin Manager -> Available. 
4.  Check the box for "Build Authorization Token Root Plugin."
5.  Check the button "Install without restart."
6.  Go to the Jenkins job in this scenario. 
7.  Click "Configure" -> "Build Triggers". 
8.  Check the option "Trigger builds remotely..." option.  
9.  Insert an arbitrary string in the "Authentication Token" field.  
10.  Click Save.
11.  Copy the URL for the job and draft it according to this format:  

http://JENKINS_URL/buildByToken/build?job=JOBNAME&token=TOKENNAME

Substitute JENKINS_URL for the URL to your Jenkins server (e.g., with port 8080).  Substitute JOBNAME for the name of the job.  Substitute TOKENNAME with the arbitrary string you entered above in the "Authentication Token" field.

12.  Copy this URL and paste it in the GitLab field for the URL in the integration webhook. 
13.  Test the connection again.

How Do You Create Swap Space on a Linux Server Running in the Cloud?

Problem scenario
You have sufficient hard disk space.  But your applications are memory constrained on your Linux server.  You cannot add more RAM.  How do you create swap space on your Linux server (to create virtual memory)?

Solution
Overview
We consider virtual memory to be a hybrid of RAM "and disk space that running processes can use. Swap space is the portion of virtual memory that is on the hard disk, used when RAM is full." The quoted section was taken from StackOverflow.

Procedures
These directions to have been tested to work on a Google Cloud Platform Debian server, an AWS Ubuntu server, and an AWS RedHat Enterprise Linux (RHEL) server.  (If you want a script to do this, see this posting.) They will work on a server in Azure too.

Warning: This will create a swap space that can be read by other processes.  You may want to use the optional step at the end to make it more secure.

1.a.  This is optional.  You may want to run this command:

top | grep Swap # This is an optional step.  Cancel out after you see a line of results.

1.b.  If you are running SUSE or a RedHat derivative of Linux (CentOS/RHEL/Fedora), run this:
sudo dd if=/dev/zero of=/mnt/2GB.swap count=2048 bs=1MiB

If you are running Debian/Ubuntu Linux, run this:
sudo fallocate -l 2G /mnt/2GB.swap

2.  Run these commands:
sudo mkswap /mnt/2GB.swap
sudo swapon /mnt/2GB.swap

3.  Update the /etc/fstab to have this line (i.e., append it to the file as the lowest line):
/mnt/2GB.swap  none  swap  sw 0  0

4.  Update the /etc/sysctl.conf file to have this stanza at the very end:

vm.swappiness=10

5.  Run this command:  sudo swapon -s

6.  Optional command to make the swap space more secure:  sudo chmod 600 /mnt/2GB.swap

7.  Optional command to check your work and compare it to the first command above:
top | grep Swap # This is an optional step.  Cancel out after you see a line of results.

The above directions were based on RackSpace directions here.

How Do You Upgrade the Flavor of Your EC2 Instance with Only 15 Minutes of down Time?

Problem scenario
The needs for a server have changed.  You now want your AWS EC-2 virtual server to have more CPU and RAM, but you do not want to  migrate to a new instance.  You can only afford 15 minutes of down time.  What should you do to resize the server's capacity  to have more processors and greater memory?

Solution
These basic steps will work if you want to downgrade your server too (to save money).

1.  This is an optional step if the instance is running Linux (as a "sanity check").  From the instance, check the amount of RAM and number of processors with these commands:
free -mh
cat /proc/cpuinfo | grep processor | tail -n 1

2.  The following steps are performed from the AWS console (except the last step which is optional).  In the AWS console go to the EC2 instance screen.
3.  Check the radio button on the left column for the instance (the row that has information about the instance you want to modify).
4.  Go to Actions -> Instance State -> Stop.
5.  When it has stopped go to Actions -> Instance Settings -> Change Instance Type.
6.  In the pop-up choose the Instance Type you want (e.g.,  m4.xlarge).  Click "Apply."
7.  Go to Actions -> Instance State -> Start.
8.  To the next prompt, click "Yes, Start."
9.  You are done. This is an optional step if the instance is running Linux.  From the instance, check the amount of RAM and number of processors with these commands:
free -mh
cat /proc/cpuinfo | grep processor | tail -n 1

How Do You Install Jenkins 2.x on Debian Linux in Google Cloud Platform?

Problem scenarios
#1  You have a Debian server in GCP with roughly 0.592 GB of RAM.  You want to install Jenkins on it.  How do you do this?

OR

#2  You have a nano flavor of an EC2 instance running Ubuntu Linux (i.e., 0.5 GB of RAM).  You want to install Jenkins on it.  How do you install Jenkins 2.x on Ubuntu Linux in AWS?  (If you do not want to install a "apt-transport-https" package, but you can run "apt-get upgrade", then see this posting.)

Solution
(The directions were tested separately in both AWS and GCP.)

1.  Run these commands from the command line prompt of the Debian Linux server:

sudo apt-get -y update 
sudo apt-get -y install apt-transport-https
wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -
echo deb https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list
sudo apt-get -y update
sudo apt-get -y install jenkins
sudo systemctl start jenkins
curl http://icanhazip.com # to find the external IP address of the server

2.  Go back to your workstation and open a web browser.  Go to this URL where x.x.x.x is the IP address obtained from the curl command above:  http://x.x.x.x:8080

Ensure (e.g., in the GCP console) that the relevant firewall rule allows for inbound connections over port 8080 (unless you configured a different port) from your workstation.  If you are not sure how, see this posting.  If you are running the server in AWS, ensure that the Security Group allows for connectivity.

3.  In the web browser, enter the initial password.  You may need to use sudo to access the file from the Linux back-end after you see the path to the file is shown in the web browser.  You will then click choose "Install Suggested Plugins."  Enter the fields as requested for the "First Admin User."  Click "Save and Finish."  You are done.

FYI
A server with 0.5 GB of RAM will be sufficient for Jenkins as a development test server.  If you will put a considerable load on Jenkins, you will need a server with more memory.  To resize a server in AWS, see this posting.  To resize a server in GCP, see this posting.

How Do You Get Jenkins 2.x Running on Linux to Do Code Deployments to Windows Servers?

Problem scenario
You are trying to set up a CI/CD pipeline in your heterogeneous enterprise server network.  You want to push down files (integrate code) from your Jenkins 2.x server running on Linux to your Windows 2016 Servers.  How do you get builds to be controlled by Jenkins on Linux to be deployed to other Windows servers?

Prerequisites
This solution assumes that Jenkins is running on Linux.  If you have Debian Linux, see this posting.  If you have Ubuntu Linux, see this posting.

Solution
1.  Install cygwin on the Windows server so it is ready for SSH.  If you do not know how to do this, see this posting.  Cygwin is covered under the GNU General Public License.   

2.  On the Linux server run this:  sudo su jenkins

3.  Set up SSH with the Windows server so that the Jenkins user can passwordlessly connect to the Windows server.  If you do not know how to configure passwordless SSH, see this posting and prepare the Windows server for SSH via its Cygwin command prompt application.

4.  On the Linux server run these commands:

cd /home
mkdir jenkins
sudo chown jenkins jenkins
sudo chgrp jenkins jenkins
cd jenkins
mkdir .ssh
chmod 700 .ssh

5.  Create an authorized_keys file in the .ssh directory.  The content should be that of an id_rsa.pub file.  This id_rsa.pub file should belong to a local user of the server you want to SSH into (e.g., a Windows server with Cygwin on it).  This content should become the basis of the authorized_keys file to reside in the .ssh directory from the above step.

6.  Run these commands from the .ssh directory in step #4:

sudo chmod 600 authorized_keys
sudo chown jenkins authorized_keys
sudo chgrp jenkins authorized_keys

7.  Go to the Jenkins web UI and log in.

8.  Go to "New Item" and enter a name.  Then click "Freestyle project."  Then click "Ok."

9.  Go to "Add build step" -> "Execute Shell". 

10.  Insert a command such as this (where jdoe is the user and x.x.x.x is the IP address of the Windows server):

scp /tmp/basic.txt jdoe@x.x.x.x:/home/jdoe/basic.txt

11.  Click "Save."

12.  Schedule the build or choose the option to "Build now."