How Do You Troubleshoot an Error with a Certificate When You Are Trying to Configure a Docker Registry?

Problem scenario
You get an error about a certificate not working with your host/server name.  One of the following apply to your situation:

When you use "docker login" you get this message:
"Error response from daemon: x509: certificate is not valid for your server"

OR

When you try to use curl to the Docker registry server you get this error:
curl: (51) SSL: certificate subject name  does not match target host name

You have copied over the latest .crt files to the client server where you are running commands to get the errors above.  What do you do to not get a message about a certificate name not being valid with a server name?

Solution
What did you use for the CN or Common Name when you created the .crt file?  When creating a new SSL certificate, you generally configure the Common Name interactively from a prompt when running an openssl command.  Alternatively you add a Common Name with a complex string as part of a flag option when you run the openssl command.  If the name you passed was a server name other than the FQDN of the Docker registry server or you passed some username value for the Common Name, then that is likely the problem.  

The following prompt generally needs the FQDN of the Docker registry server:

"Common Name (e.g. server FQDN or YOUR name) []"

Recreate the .crt file. Use openssl commands to recreate your .crt file.  Try using the FQDN of the Docker registry server for this "Common Name" field.

The Histories Behind the American Thanksgiving(s)

People who come to this blog, and homeschooled people, often learn from Google or other search engines.  The woman who wrote the song "Mary Had a Little Lamb," Sarah Josepha Hale, was homeschooled herself.  She made two historic contributions, one was musical and the other was political.  She influenced Abraham Lincoln to make Thanksgiving a national holiday for many years before it was officially recognized.  Wikipedia reports that she was from New England where people regularly celebrated Thanksgiving for years before it was was proclaimed as the third national holiday in 1863.  George Washington proclaimed the celebration of Thanksgiving in a repeated yet ad hoc (per year) fashion (Business Insider).

The story behind what is commonly regarded as the first Thanksgiving was the peaceful meeting between Native Americans and English colonists who settled in Plymouth, MA.  There is a living history museum for Plimoth Plantation in Massachusetts to help people learn more.  The spelling "Plimoth" was acceptable in the 1600s, and that is why the institution uses it (Boston.com).

According to the National Park Service website, Thanksgiving was originally celebrated by Jamestown settlers in Virginia (before Plymouth was settled by English people).  We know that most knowledge of Thanksgiving lends itself to the history in New England with the Pilgrims.  If you visit the Jamestown Settlement in Virginia, another living history museum, you may meet a historical guide who will claim that the story of the Pilgrims (starting in 1620 C.E.) overshadows the earlier story of Jamestown (starting in 1607 C.E.) because of the Civil War.  In other words celebrating Thanksgiving nationwide may have been Lincoln's ploy to help save the Union by honoring the North's history.  The North's history included ending slavery well before the Civil War even started.  Indeed some people think the aversion to the Jamestown Settlement story could have been due to slavery (Big Think).  Several sources say that Thanksgiving was originally celebrated in Virginia.

To learn more about the controversies concerning the history of Thanksgiving and whether it was originally celebrated in Virginia or Massachusetts, feel free to read the following articles:

There are Native American groups who explain a sinister side of the history of Thanksgiving.  To read more about them you may want to see this Oyate.org site; there used to be a page at http://www.manataka.org/page269.html that had more information. We encourage you to read more and formulate your own opinion about Thanksgiving.  Here is a list of books on the subject:

A Great and Godly Adventure: The Pilgrims and the Myth of the First Thanksgiving
Chronicles of American Indian Protest
Good Newes from New England: A True Relation of Things Very Remarkable at the Plantation of Plimoth in New England
Mourt's Relation: A Journal of the Pilgrims at Plymouth
Of Plymouth Plantation
Thanksgiving and Black History, an article
Thanksgiving: The Biography of an American Holiday
Thanksgiving: The True Story
The Thanksgiving Book: A Companion to the Holiday Covering Its History, Lore, Traditions, Foods, and Symbols...
Three Visitors to Early Plymouth: Letters about the Pilgrim Settlement in New England during its First Seven Years

How Do You Install Docker on a RHEL VM in Azure?

Problem scenario
You have a Red Hat Enterprise Linux server in Azure.  How do you install Docker on it?

Solution
Run these three commands:
sudo yum -y install docker
sudo groupadd docker
sudo usermod -aG docker ec2-user

FFR
For directions for install Docker to an AWS instance of RHEL, see this link.  If you cannot use a yum command to install Docker but want to install it on a Red Hat derivative distribution of Linux, see this posting.

What is DevSecOps (or DevOpsSec, SecDevOps, or rugged DevOps)?

Updated 2/10/22

Answer
DevSecOps is the integration of security practices with DevOps.  DevOps may be a culture of automation (as Stackify says), a blend of disciplines (development and operations), or a key word in a job title (e.g., DevOps Engineer or DevOps Architect).  DevSecOps seeks to rapidly integrate security measures into development and operations and avoid security specialists working in separate and relatively isolated teams.  By having security-minded professionals in the same team, the benefits of automation can help security, and security can protect rapidly-growing server environments with Agile development.

To learn more about DevSecOps, see the following five links:

  • The DevSecOps Manifesto is here.
  • An independent definition of DevSecOps is here.
  • SANS produced a whitepaper on DevSecOps.
  • Sonatype has a PowerPoint presentation here.
  • ZDNet has an article defining DevSecOps.
  • CA Technologies used to have an article defining DevSecOps here: https://automic.com/blog/just-what-devsecops-ara

In 2012 Gartner called what we think of DevSecOps as DevOpsSec (https://blogs.gartner.com/neil_macdonald/2012/01/17/devops-needs-to-become-devopssec/).  This term persists, and there are actually two more synonymous terms in use: SecDevOps and rugged DevOps.  To learn about the term Rugged DevOps (as it is sometimes capitalized), see this link.  A leading security website, CSOonline.com, describes three of the terms (DevOpsSec, DevSecOps, and SecDevOps, but not the rugged DevOps term).  A book published in 2016 clarifies that there are four different terms for the infusion of security practices into DevOps: "Whether you call it SecDevOps, DevSecOps, DevOpsSec, or Rugged DevOps..." (page 5 of DevOpsSec by Jim Bird, Oreilly Press 2016).  Another credible source confirms that there are four terms that mean the same thing (Sqreen.io).  Consider the following findings (as of 11/22/17):

  • A search for "devsecops" on Indeed.com found 192 jobs and on Dice.com found 31 jobs. 
  • A search for "secdevops" on Indeed.com found 38 jobs and on Dice.com found 3 jobs.
  • A search for "devopssec" on Indeed.com found 11 jobs and on Dice.com found 1 job.

Consider the updated findings (as of 9/25/20):

  • A search for "devsecops" on Indeed.com found 4,797 jobs and on Dice.com found 95 jobs. 
  • A search for "secdevops" on Indeed.com found 234 jobs and on Dice.com found 6 jobs.
  • A search for "devopssec" on Indeed.com found 16 jobs and on Dice.com found 0 jobs.

Consider the updated findings (as of 2/10/22):

  • A search for "devsecops" on Indeed.com found 19,002 jobs and on Dice.com found 110 jobs. 
  • A search for "secdevops" on Indeed.com found 536 jobs and on Dice.com found 3 jobs.
  • A search for "devopssec" on Indeed.com found 20 jobs and on Dice.com found 0 jobs.

Clearly varieties of the terms exist, and there has been a tremendous growth of this concept in the past few years. The term "rugged devops" is not discernibly searchable like the other three terms.  Which of the four options should be used for the infusion of security practices into DevOps?  Here is an excerpt from a third source, a relevant TechBeacon article, acknowledging all four terms:  "People call it DevSecOps, SecDevOps, DevOpsSec, and even rugged DevOps. How can we have so many different terms to describe the exact same thing?  This gives us a hint as to the disconnect that exists within security in DevOps. It’s still the wild west [sic]. There is no standard that defines security for DevOps, and the chances of a standard ever developing is small because different organizations are doing things their own way, and can’t even agree on a standard name. And while there is a standard for the secure development lifecycle (ISO/IEC 27034-1), few organizations are ever validated against it . [sic]"

To clarify, DevOpsSec, SecDevOps, DevSecOps, and rugged DevOps are each the same thing: they are the integration of security practices with DevOps.

Arguably DevOps necessarily incorporates security with quality assurance (or QA), development, and operations; DevSecOps or synonyms such as SecDevOps, DevOpsSec, or rugged DevOps, could all be considered redundant to the term DevOps.  One difference that is undeniable in three of these terms is the three letters Sec which explicitly reminds people about security. 

Gene Kim, who is a DevOps celebrity (if such a thing exists in September 2020), started to make a name for himself when he co-founded a security software company called Tripwire.  Tripwire's main product is a host-based intrusion detection system that is focused on finding changes in files themselves.  (Host-based intrusion detection systems, HIDSes, look for intrusions on a server, by monitoring files and processes.  They do not monitor a TCP/IP network like Snort or other network-based intrusion detection systems.)  Tripwire's system can also be called security information and event management, or SIEM.

This host-based intrusion detection system approach to security concerns itself with minute changes in files.  This scrutiny of files is characteristic of code versioning systems (such as Subversion or Git) used in CI/CD pipelines.  These pipelines are a significant part of what DevOps is -- integrating development with operations.  DevOps engineers can design CI/CD pipelines to have processes triggered on the act of a new file being uploaded or committed to a repository.  A typical CI/CD pipeline will initiate a build upon the event of an existing file being modified. 

In the mid 1990s when the Tripwire security product came out, no one could have known the role configuration management tools (such as Puppet, Chef, Ansible, and SaltStack) would play in changing files across large numbers of enterprise servers.  Similarly at this time no one could have known the importance of future tools such as Git, Jenkins, or the DevOps movement itself.  Gene Kim became an expert at file changes and automating an event (such as logging or notification) with his security software business.  By the time DevOps became a common word, he was incredibly well-prepared for the I.T. industry to begin a transformation toward focusing on automating processes upon a file being changed.  No one would disagree that Gene Kim is one of the most well-known figures in the DevOps realm today and will be one of the biggest names in the history of the DevOps movement.   His rise to fame came from expertise of a SEIM product.  This story of Gene Kim corroborates the idea that security is necessarily infused in regular DevOps culture (with no "sec" in the word).

The opinion that DevOps necessarily includes traditional practices of development, operations, QA, and security, is substantiated by the following list (when taking the first one into account with the other individual items):

  • Devops.com shows how QA, development and operations are part of DevOps.  Consider those three areas comprising DevOps with the four links below.
  • Wired.com explains how DevOps integrates security.
  • SANS has a whitepaper that explains how DevOps helps security work.  Specifically pages 5, 10, and 11 of this paper corroborate the view that DevOps inherently incorporates DevSecOps.
  • Ron Wilson who has authored an article on DevOps.com maintains that DevOps includes security in this link.
  • OWASP sponsored a discussion from Helen Bravo that was entitled "DevOps & Security: It's Happening. Right Now."  The slides were published in 2014, you can see them by going to this link.

If you want to read more on the topic of DevSecOps, see one of the following books:
DevSecOps All-Inclusive Self-Assessment
DevOpsSec by Jim Bird, Oreilly Press 2016
Hands-On Security in DevOps
Securing DevOps: Security in the Cloud

What Is a “data swamp”?

Question
What is a "data swamp"?

Answer
A data swamp is best defined as a severely degraded data lake.  The term data swamp connotes poor governance and negligent management that caused a data lake to gradually lose its value.  A data swamp is data lake that was once useful but through negligent utilization can no longer be used by even highly talented analytics professionals.  Data swamps can be improved and restored to regular, functional data lakes.  There is a gray area between a poor data lake and data swamp that is degraded with a low degree of severity. 

IBM suggests that there is a continuum from data swamp (least valuable) to data lake to data reservoir (most valuable).  Page 122 of this IBM document uses all three terms in a way that alludes to a continuous series of these three data concepts.  IBM's website refers to a "well-managed and governed data lake" being the same as a data reservoir.   Gartner makes a distinction between data lake and data reservoir too (infocus.emc.com).  Other companies besides IBM and Gartner refer to "data lake" and "data reservoir" interchangeably.

According to an article on TeraData's website that was published in late 2016, the biggest mistake that people are making  with data lakes is "poor governance."  Expect to hear or read the phrase "data swamp" as you continue to work with big data.  For further information about data swamps, data lakes, and data reservoirs, see the links below:

What Is the Difference between Erasure Coding and Negative Coding?

Question
Erasure coding versus negative coding: what is the difference?

Answer
In the verbose output of a Hadoop or mapred job, you may have seen "INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path:"

Erasure coding is a fault-tolerant storage mechanism.  Like replication the usable capacity of the disks participating in such a mechanism is not fully utilized as in a RAID 0 configuration.  (The aggregate amount of storage of disks in RAID 0 is the sum of the individual disks' capacity.)  Also like replication, one disk, and potentially more, could fail while the data written to the storage system with erasure coding is still safe.  To learn more about erasure coding, see this link from the Storage Networking Industry Association.

Negative coding is a programming technique where code is rewritten to preserve its functionality with fewer lines of code.  A benefit to programmers is that the code base is more concise.  With fewer lines of code, the program is potentially more readable and less intimidating to scrutinize.  A benefit from a computing hardware perspective is that the code base has a smaller footprint in storage (as it resides in a code versioning repository) and uses less RAM when in memory.  For more information on negative coding, see this external link.

"The real hero of programming is the one who writes negative code." -Douglas McIlroy
This quote was taken from here. McIlroy advanced the use of pipes at Bell Labs (the origin of Unix); to read more about this, see this link. He also championed Unix philosophy of having modular components in the 1960s (page 79 of this pdf).

Are There Similarities or Differences between Negative Coding and Throw-away Prototyping?

Question
What is the difference between negative coding and throw-away prototyping?

Answer
"Plan to throw one away..." -Frederick P. Brooks, Jr. (Taken from page 266 of The Mythical Man-Month).

Both negative coding and throw-away prototyping involve eliminating lines of code.  But this is the only similarity they have.  Negative coding is contrasted from throw-away prototyping in that the former is a good practice for production systems and long-term use whereas the latter is designed to be temporary.  Negative coding is the rewriting of code to make it more concise.  Rather than adding lines of code the programmer removes them (thus the word negative) while retaining a program's functionality. 

Throw-away prototyping is the development of a disposable program or programs to have a customer communicate feedback.  Throw-away prototyping is for proof-of-concept ideas and to work closely with the customer for crystallizing the requirements.  To take a customer's requirements and design a superficial solution with a high-level language such as Python while being aware that the code will never be used beyond a verification test is an example of a throw-away prototype.  Throw-away prototyping is the process of creating a symbolic application in the earliest stages of a software design project with the intent that this initial program is both disposable and merely symbolic (unsuitable for a production system due to performance or security reasons).  An illustrative representation of a program can be pithy in that it is the basis of communicating an understanding of many small requirements by a software designer to a customer to confirm or deny satisfaction with the prototype. 

Iterative development may involve a series of disposable experiments.  Planning on discarding a program allows developers to forgo time-consuming security, performance and "best practice" concerns initially.  It also invites greater candor from the customer as the illustrative "application" is intentionally abortive.  Developers will then invest time from the beginning of their real solution to handle the multitude of tedious obligations only after the feasibility of the project has been confirmed.  Thus they can begin a full software development lifecycle in a language such a C, replete with proper testing given the confidence bestowed by the customer's approval of the illustrative prototype.  The first line of code will have been written with the specific requirements in mind that will ultimately be able to handle a production load robustly and securely.  To learn more about throw-away prototyping, see page 136 of The Internet Encyclopedia, Volume 3 by Hossein Bidgoli published by John Wiley & Sons in 2004. It can be inappropriate from a business perspective for developers, in some instances, to present a throwaway to a customer (as described on page 116 of The Mythical Man-Month).

Given the rapid profusion of Docker containers, the disposable concept is very popular in the software world today. "[Docker's] architectural philosophy centers around atomic or throwaway containers." (This was taken from page 3 of Docker Up & Running by Matthias & Kane, O'Reilly Press, 2015.)

Throw-away prototyping is also referred to as revolutionary prototyping. (Slide 31 of this presentation confirms that these terms are synonymous; another source proving their equivalent meaning is here.)  It seems preferable to hyphenate the term "throw-away" compared to the unhyphenated variation "throwaway."  The reason is that two-word, or compound, adjectives that are not well-established nor frequently used should be hyphenated (Grammarbook.com).  Additionally a 2004 reference to "throw-away prototyping" uses the hyphenated version (Revolutionary vs. Evolutionary Rapid Prototyping: Balancing Software Productivity and HCI Design Concerns. Center of Excellence in Command, Control, Communications and Intelligence (C3I), by S.P. Overmyer, published by George Mason University, 4400 University Drive, Fairfax, Virginia).  This 2004 article also states that revolutionary prototyping is a synonym of throw-away prototyping.

Douglas McIlroy, head of research at Bell Labs when Unix was being created, said "[o]f course mass production, in the sense of limitless replication of a prototype, is trivial for software." (The sources of these statements are here and page 79 of this pdf). He also said "[t]he real hero of programming is the one who writes negative code." (This quote was taken from here.) McIlroy advanced the use of pipes at Bell Labs (the origin of Unix); to read more about this, see this link. He also championed Unix philosophy of having modular components in the 1960s (page 79 of this pdf).

How Do You Create Kubernetes Pods with a .YAML File That Use a Docker Image from a Google Container Registry Instead of Docker Hub?

Problem scenario
You have a .yaml configuration file that works with the kubectl command to create a deployment.  You want to change the source image from the hub.docker.com to a Google container source.  How do you configure the .yaml file to obtain the image from a Google container registry instead of Docker Hub?

Your .yaml file has these four lines (among other lines):

  spec:
      containers:
      - image: mysql
        name: mysql

Solution
You have to identify the URL from gcr.io first.  Once you have that, substitute the "mysql" on the "- image:" stanza to this new URL.  Here is how the .yaml file's four lines would look:

  spec:
      containers:
      - image: gcr.io/google_containers/mysql-galera:e2e
        name: mysql

For Future Reference
Assuming the .yaml file was complete and named contint.yaml, you could now create a Cluster with a command like this:

kubectl create -f contint.yaml

To delete the cluster created with such a file, just use a command like this:

kubectl delete -f contint.yaml

How Do You Enter a Docker Container?

Problem scenario
You have a Docker host that is running Linux (the server with Docker containers).  You want to run commands inside a Docker container.  How do you get into a Docker container?

Solution
Prerequisite
You need to know the container ID of a running container.  To find it run this: docker ps -a

Procedures
Run this command:
docker exec -it <container ID> bash

You may want to see this posting.  To view a list of Docker books, see this link.

How Do You Configure SSH Passwordless Authentication between an AWS EC-2 Instance and an Azure Virtual Machine?

Problem scenario
You have a Linux server in AWS and another Linux server in Azure. You want to use scp without a password to transfer files rapidly.  How do you configure passwordless SSH authentication between the two servers?

Solution
1.  Run this command on each server:

ssh-keygen -t rsa -P ""

# Accept the default prompt by pressing enter.

2.i.  Run this command on the AWS instance (replace "ubuntu" with "ec2-user" if the AWS instance is a RHEL server):

cat /home/ubuntu/.ssh/id_rsa.pub

2.ii.  Append the output (of the above command) to this file where "george" is a username on the Azure server:

/home/george/.ssh/authorized_keys

# Create the authorized_keys file if it does not yet exist.  RedHat servers do not necessarily have an authorized_keys file after ssh-keygen is run.   If you have to create it run this command (from the .ssh directory): sudo chmod 600 authorized_keys

3.i.  Run this command on the Azure instance where "george" is a username on the Azure server:

cat /home/george/.ssh/id_rsa.pub

3.ii.  Append the output (of the above command) to this file on the AWS server (replace "ubuntu" with "ec2-user" if the AWS instance is a RHEL server):

/home/ubuntu/.ssh/authorized_keys

# Create the authorized_keys file if it does not yet exist.

4.  Ensure that port 22 is not blocked via an Azure Network Security Group from the external IP address of the AWS EC-2 instance.  To find the external IP address, use the AWS Console and go to the EC2 Dashboard.  Alternatively from the backend you can run "curl http://icanhazip.com".  Do not be alarmed in nmap commands show port 22 is blocked.  Do not be alarmed if you cannot ping the Azure VM from the AWS instance.

5.  Ensure that inbound ports 22 and 80 are not blocked in an AWS Security Group from the external IP address of the Azure VM.  To find the external IP address, use the Azure Portal and go to resources and click on the VM.  Alternatively from the backend you can run "curl http://icanhazip.com".  

(Technically port 80 does not need to be opened.  For nmap results to work from the Azure VM, port 80 needs to be opened.  It is merely one port with one IP address.  You can keep it blocked if you do not need to use the nmap command for troubleshooting.)

6.  From the Azure instance, use this command:

ssh ubuntu@<IP address of server>    # replace "ubuntu" with "ec2-user" if the AWS instance is running Azure

# If you are prompted to continue connecting, choose "yes."

7.  From the AWS instance, use this command:

ssh george@<IP address of Azure instance>    # replace "george" with the username on the Azure instance

# If you are prompted to continue connecting, choose "yes."

# We hope you found these directions succinct yet thorough, and very clear.