How Do You Create a config File for Kubernetes Manually?

Problem scenario
You want to manually create a config file for Kubernetes.  (You do not want to run aws eks update-kubeconfig --name foobar.)  How do you do this?

Solution
These directions are most suited for Amazon EKS.  But they could be adapted for clusters running in other environments (not just AWS).  If you are using GCP, go to the very bottom of the directions and skip the rest.

1.  Go to your home directory: cd ~
2.  Create a .kube file: mkdir .kube
3.  Go inside this directory: cd .kube
4.  Replace "foobar" with the name of your cluster, and run this: vi config-foobar
5.  Paste the following inside it:

apiVersion: v1
clusters:
- cluster:
    server: <endpoint-url>
    certificate-authority-data: <base64-encoded-ca-cert>
  name: <role-arn>
contexts:
- context:
    cluster: <role-arn>
    user: <role-arn>
  name: <role-arn>
current-context: <role-arn>
kind: Config
preferences: {}
users:
- name: <role-arn>
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
        - "token"
        - "-i"
        - "<cluster-name>"
         command: aws-iam-authenticator

6.  Follow these four steps (each quoted from https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html ), and get the values you need by going to this link https://us-east-2.console.aws.amazon.com/eks/home? in a web UI and then click on the cluster for which you want to configure and use.

  • Replace the <endpoint-url> with the endpoint URL that was created for your cluster.  (It will appear under the "API server endpoint" value.)
  • Replace the <base64-encoded-ca-cert> with the certificateAuthority.data that was created for your cluster.  (It will appear under the "Certificate Authority" value.)
  • Replace the <cluster-name> with your cluster name.
  • Replace the <role-arn> with the value for the "Role ARN" as the web UI shows.

For GCP
To automatically generate a new kubeconfig file when you are using GCP's Kubernetes, run this command: gcloud container clusters get-credentials "CONTINT"

Replace "CONTINT" with the name of the cluster you want the kubeconfig file to be for.

How Do You Troubleshoot the AWS CLI Error “Traceback … import pkg_resources”?

Problem scenario
You run an "aws" command but you receive an error message such as this:

Traceback (most recent call last):
  File "/usr/bin/aws", line 4, in <module>
    import pkg_resources
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 3007, in <module>

What should you do?

Solution
1.  Install pip.  If you need assistance, see this posting (especially the "Problem scenario" paragraph with hyperlinks based on distribution families of Linux).

2.  Do run this command:
sudo pip install colorama rsa s3transfer ruamel.yaml botocore

3.  Try the aws command again.  If it fails, install a different, older version of the awscli.  You can go to /bin/ and run "sudo mv aws-cli... aws-bak".  Then see this posting for details about installing the AWS CLI.

How Do You Create an IAM User Account in AWS to Run AWS CLI Commands and Log into the AWS Web UI to Manage EKS?

Problem scenario
You want to use Amazon EKS.  You tried to run an "aws eks" command but got an error about the root user not being allowed to perform the operation.  How do you create an IAM user account in AWS to run AWS CLI commands for EKS and log into the AWS web UI and view EKS clusters?

Solution
Prerequisite
This assumes you have a group that has the correct policies to enable EKS manipulation.  If you do not know how to do this, see the posting: How do you create a group for AWS users that can manage a Kubernetes cluster running in AWS?

Procedures
Follow How Do You Create an IAM User Account in AWS to Run AWS CLI Commands and Log into the AWS Web UI?  With step #10, choose the group for EKS.

How Do You Create a Group for AWS Users That Can Manage a Kubernetes Cluster Running in AWS?

Problem scenario
You use EKS to create a Kubernetes cluster.  How do you create a group for AWS user accounts to enable them to manage Kubernetes clusters in AWS?

Solution
1.  Log into the AWS web console.
2.  Go here: https://console.aws.amazon.com/iam/home?
3.  Go to Groups on the left.
4.  Click the "Create New Group" button.
5.  Enter a name (e.g., coolgroup).
6.  Click "Next Step".
7.  In the "Attach Policy" section, check the boxes for these five policies:
AmazonEKSClusterPolicy, ElasticLoadBalancingFullAccess, AmazonECS_FullAccess, AmazonEKSServicePolicy, AmazonECSTaskExecutionRolePolicy 
8.  Click "Next Step".
9.  Click "Create Group".

How Do You Create an IAM User Account in AWS to Run AWS CLI Commands and Log into the AWS Web UI?

Problem scenario
How do you create an AWS user that can run AWS CLI commands and log into the AWS web console?

Solution
1.  Log into the AWS web console.
2.  Go here: https://console.aws.amazon.com/iam/home?
3.  Go to Users on the left.
4.  Click the "Add User" button.
5.  Enter a name (e.g., jdoe).  
6.  Check the boxes for both "Programmatic access" and "AWS Management Console access".
7.  For the "Console Password", choose the option for the "Custom Password".
8.  If you are giving these credentials to someone else, keep the option checked for "Require password reset".  (Otherwise you may want to uncheck it.)
9.  Click "Next: Permissions" in the lower right-hand corner.
10.  Check the option for the group that you want.  (If you want directions for how to create a group, see How Do You Create a Group for AWS Users That Can Manage a Kubernetes Cluster Running in AWS? as an example.)  Then click "Next: Add Tags" in the lower right-hand corner.
11.  It is optional to add tags.  Click "Next: Review" in the lower right-hand corner.
12.  Click "Create User".
13.  Find the AWS Management Console URL.  This is important for the user to log in going forward.
14.  You may want to copy and/or save the secret access key at this time.

What Is a Shared Library in Jenkins?

Question
You have heard about shared libraries in the context of Jenkins Pipelines.  What is a Jenkins Pipeline shared library?

Answer
It is something that extends a Jenkins Pipeline (1).  A library is often composed of one or more of the following Roman numeral items (sources include (2) (3)):

  i.  Groovy-based DSL.
  ii. Groovy code and compiled programs.
  iii.  Java code and compiled programs.
  iv.  variables and other artifacts possibly in directories named /vars, /src, and /resources.  

There are three requirements for defining a shared library; these are a name, a code repository, and a version (1).  The version can refer to a branch in a Git repository (1).

One big benefit of using a shared library is that it makes other Jenkins Pipelines easier to maintain by keeping one group of common files, configuration settings and variables in one place (1).  Abstracting out the shared values and files you create an opportunity to have individual Jenkins Pipelines be of minimal sizes. (1)

The term shared library is used in the context of the C programming language.  To learn more about this see this link.

Citations
(1)  https://jenkins.io/doc/book/pipeline/shared-libraries/
(2)  https://medium.com/devopslinks/a-hacky-hackers-guide-to-jenkins-scripted-pipelines-part-4-dd49fcb0d62
(3)  https://automatingguy.com/2017/12/29/jenkins-pipelines-shared-libraries/

How Do You Create an Amazon Resource Name Role in AWS?

Problem scenario
You want to create an ARN Role in AWS.  What do you do?

Solution
1.  Log into the AWS web console and go here:  https://console.aws.amazon.com/iam/home?#/roles
2.  Click "Create role" (a blue button).  
3.  The type of trusted entity (at the top) should be "AWS Service" (a button like option near the top).  
4.  Click the hyperlink that you desire.  (For a role to that will create Kubernetes clusters, click "EKS").
5.a. Underneath the "Select your use case" section, click on "EKS".
5.b.  Click "Next: Permissions" (a blue button).
6.  Click "Next: Tags" (a blue button).  
7.  At your option you may enter a key name.  Click the "Next: Review" blue button in the lower right-hand corner.
8.  Type in a "Role name" in the appropriate field.  Click the "Create role" blue button in the lower right-hand corner.

How Do You Run Docker commands without sudo?

Problem scenario
For security reasons, you do not want to have to use "sudo" to elevate your privileges when you run docker commands.  What should you do?

Solution
1.  Run these commands as the user that will be able to run docker commands:
sudo groupadd docker
sudo gpasswd -a $USER docker
newgrp docker

2.  Install Docker.  See this posting if you need directions.

3.  Run these commands:
sudo systemctl enable docker
sudo systemctl start docker

4.  Reboot the server.  

5.  You are done.  Going forward you should be able to run docker commands without sudo.  You can test it by running this: docker run hello-world

How Do You Deploy a Kubernetes Cluster into AWS with a RHEL Server?

Problem scenario
You want to create a Kubernetes cluster in AWS via a RHEL server.  What should you do?

Solution
This will deploy additional resources in your AWS account (not on to the RHEL server). It will cost money to follow these directions.

Prerequisites
i.  Install and configure the AWS CLI (or awscli). If you need help, see this posting.

ii.  Install kubectl.  If you need assistance, follow step #1 only of this posting.

iii.  Install kops (Kubernetes Operations command) by running these commands:

curl -Lk https://github.com/kubernetes/kops/releases/download/1.8.1/kops-linux-amd64 > /tmp/kops-linux-amd64

chmod +x /tmp/kops-linux-amd64

sudo mv /tmp/kops-linux-amd64 /usr/local/bin/kops

iv.  You need to have SSH keys set up.  You want to have a .pub file in your ~/.ssh/ directory. The directions below assume that your .pub file is id_rsa.pub; adjust the directions below when they refer to this file if you have a different name for it.

If you do not know how to create this file, run this command and press enter twice:

ssh-keygen -t rsa -P ""

Procedures
(We created these directions based on a Linoxide.com article.)

1.a.  Create a script called "aws.sh" in /tmp/ with the following lines:

#!/bin/bash
aws iam create-group --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonRoute53FullAccess --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/IAMFullAccess --group-name kops

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonVPCFullAccess --group-name kops

aws iam create-user --user-name kops

aws iam add-user-to-group --user-name kops --group-name kops

aws iam create-access-key --user-name kops
#This comment is the last line. EOF.

1.b.  Run this: source /tmp/aws.sh
The "source" command is necessary if all your AWS CLI commands on the system will work in the current shell (which is often the case if you followed the AWS configure directions available elsewhere on this website).

2.  Run these five commands (the fifth is a multi-line command) interactively (but replace us-west-9 with the region of your choice, e.g., us-east-1; to see the options available, click here):

export KOPS_CLUSTER_NAME=contint.k8s.local
rn=us-west-9

aws s3api create-bucket --bucket ${KOPS_CLUSTER_NAME}-state --region $rn --create-bucket-configuration LocationConstraint=$rn

export KOPS_STATE_STORE=s3://$KOPS_CLUSTER_NAME-state

# This command will place in the "a" zone of the region (i.e., us-west-9) the cluster:

kops create cluster \
 --name=${KOPS_CLUSTER_NAME} \
 --zones=$rn"a" \
 --master-size="t2.micro" \
 --node-size="t2.medium" \
 --node-count="3" \
 --ssh-public-key="~/.ssh/id_rsa.pub"

3.  You are done. To confirm the cluster was created, run this command:  kops get cluster

Here are possible commands (but substitute "contint" with the name you entered before ".k8s.local" in first command in step #2):

 * list clusters with: kops get cluster
 * edit this cluster with: kops edit cluster contint2.k8s.local
 * edit your node instance group: kops edit ig --name=contint2.k8s.local nodes
 * edit your master instance group: kops edit ig --name=contint2.k8s.local master-us-west-2a

What Is The Difference between AWS CodeDeploy and AWS CodeCommit?

Question
AWS has PaaS services called CodeDeploy and CodeCommit.  What are the differences between these?

Answer
"AWS CodeDeploy is a fully managed deployment service that automates software deployments to a variety of compute services such as Amazon EC2, AWS Fargate, AWS Lambda, and your on-premises servers."  (Taken from https://aws.amazon.com/codedeploy/.)

"AWS CodeCommit is a fully-managed source control service that hosts secure Git-based repositiories."  (Taken from https://aws.amazon.com/codecommit/.)

AWS CodeCommit is a repository of code.  CodeDeploy may use a repository of code in its automation of deploying software to various AWS components and [EC-2] servers.