A Technical I.T./DevOps Blog
Problem scenario
You are trying to run a command that consumes a JSON file. You get an error such as "Expecting ',' delimiter: line …column" or "No JSON object could be decoded". What do you do?
Possible Solution #1
You may have an extra closing brace that you need to eliminate.
Possible Solution #2
Use https://jsonlint.com
Problem scenario
You want to use a Python function decorator. What do you do?
Solution
Background: "A decorator in Python is a function that takes another function as its argument, and returns yet another function." (According to this source.)
Procedures
Run this program:
def enhanced_multiply(paramfunc):
def contint(c,d):
print("I am going to multiply",c,"and",d)
return paramfunc(c,d)
return contint # Here is where contint is elegantly & immediately invoked.
@enhanced_multiply
def multiply(a,b):
return a*b
x = multiply(3,8)
print(x)Notice the @enhanced_multiply has the "@" syntax. This decorates the function.
Note: If you place a variable assignment or simple print statement between the "@enhanced_multiply" and "def multiply(a,b):", you'll get an error like this:
"SyntaxError: invalid syntax"
We bring this up so you understand how function decorators are invoked; one line with an "@" symbol is directly above a function call. The lower line will not be invoked in an undecorated way in other parts of your program. The "multiply" function is permanently bound to the decorator; to learn more, see this posting.
The decorator in this example enhances the printout of the invocation of the function in the example above. The decorator in this example helps beautify things, but it does not affect the processing of the program. A function decorator is very similar to a function for most purposes.
Problem scenario
You want a script to install Apache Flink on any type of Linux (including CentOS/RHEL/Fedora, Debian/Ubuntu and SUSE). What should you do?
Solution
Prerequisites
When using Ubuntu/Debian or SUSE servers, 1 GB of RAM is sufficient. For CentOS/RHEL/Fedora, we recommend either more RAM or use virtual memory. To find a script that will create swap space, see this posting.
Procedures
(These procedures were tested on 2/3/22.)
For this file below we suggest naming it flink.sh and placing it in /tmp/. Run this script as sudo (e.g., sudo bash flink.sh):
#!/bin/bash
# Written by www.continualintegration.com
flinkversion=1.14.3
distro=$(cat /etc/*-release | grep NAME)
debflag=$(echo $distro | grep -i "ubuntu")
if [ -z "$debflag" ]
then # If it is not Ubuntu, test if it is Debian.
debflag=$(echo $distro | grep -i "debian")
echo "determining Linux distribution..."
else
echo "You have Ubuntu Linux!"
fi
rhflag=$(echo $distro | grep -i "red*hat")
if [ -z "$rhflag" ]
then #If it is not RedHat, see if it is CentOS or Fedora.
rhflag=$(echo $distro | grep -i "centos")
if [ -z "$rhflag" ]
then #If it is neither RedHat nor CentOS, see if it is Fedora.
echo "It does not appear to be CentOS or RHEL..."
rhflag=$(echo $distro | grep -i "fedora")
fi
fi
if [ -z "$rhflag" ]
then
echo "...still determining Linux distribution..."
else
echo "You have a RedHat distribution (e.g., CentOS, RHEL, or Fedora)"
yum -y install java-1.8.0-openjdk* nc # install nc for initial testing only.
JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk
echo 'export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk' >> ~/.bashrc
source ~/.bashrc
source /etc/environment
echo "In our experience Red Hat family of servers need more than 1 GB of memory. The other distros of Linux don't have this need."
echo "Your server may or may not have ample memory (either in pure RAM or with swap space). This message is just an FYI."
echo "For a script to create virtual memory, try this link: "
echo "https://www.continualintegration.com/miscellaneous-articles/how-do-you-write-a-bash-script-to-create-virtual-memory-in-the-size-of-2-gb-on-a-linux-server/"
echo " "
fi
if [ -z "$debflag" ]
then
echo "...still determining Linux distribution..."
else
echo "You are using either Ubuntu Linux or Debian Linux."
apt-get -y update # This is necessary on new AWS Ubuntu servers.
apt -y install openjdk-11-jre-headless unzip
fi
suseflag=$(echo $distro | grep -i "suse")
if [ -z "$suseflag" ]
then
if [ -z "$debflag" ]
then
if [ -z "$rhflag" ]
then
echo "*******************************************"
echo "Could not determine the Linux distribution!"
echo "Installation aborted. Nothing was done."
echo "******************************************"
exit
fi
fi
else
zypper -n install java-1_8_0-openjdk
fi
curl -Ls https://dlcdn.apache.org/flink/flink-1.14.3/flink-1.14.3-bin-scala_2.12.tgz > /tmp/flink-$flinkversion-bin-scala_2.12.tgz
cd /tmp/
tar -xzvf flink-$flinkversion-bin-scala_2.12.tgz
mv flink-$flinkversion /opt/flink
#./bin/start-cluster.sh # This is commented out in case you want to customize it before you start it.
ip=$(curl icanhazip.com)
complete=$ip:8081
echo "To start Apache Flink, do the following:"
echo "cd /opt/flink"
echo "Then run: sudo ./bin/start-cluster.sh"
echo "-----------------------------------------"
echo 'Next, after you have started the solo-server Flink "cluster"...'
echo "Open a web browser and go to http://"$complete" to test the installation of Flink on this server"
echo "(This assumes that port 8081 is open)"Problem scenario
You are running a Python program or running a command from the Python 3 interpreter. You get this error:
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'flask'
What should you do to use Flask with Python 3?
Solution
Prerequisite
You need to have pip3 installed. You can do it with this if you are using Debian/Ubuntu Linux:
sudo apt-get -y install python3-pipProcedures
Install Flask with pip3. Here is the command you should run: pip3 install flask
Problem scenario
You are trying to run a pip command. But you get "/bin/python3.7: bad interpreter: Permission denied". What should you do about these "bad interpreter" errors in response to having run pip commands?
Solution
Use "sudo" before the pip command. It should allow you to run the pip command. You may need to be a different user or obtain sudoer rights. If you need assistance with obtaining sudoer rights, see this posting.
Problem scenario
You want to list the Route Tables in your AWS account (under VPC). You are using Python and Boto3. What should you do?
Solution
Run this program with Python 3:
import boto3
ec2 = boto3.client('ec2')
rts = ec2.describe_route_tables()
print(rts)Problem scenario
You try to compile a Java program with a javac command. You get this error:
special.java:39: error: cannot find symbol
symbol: class foo
location: class bar
What should you do?
Possible solution
In some cases you need a keyword "new", but in other cases you need to not have the keyword "new".
Are you using the word "new" to the right of an equals sign "="? For example, does the offending line in your Java program look like this?
String [][] var1 = new foo();
Try eliminating the keyword "new". Then see if it compiles. There are many potential causes. To read more, see this website or see this page.
Problem scenario
You have found an AWS CLI command that shows you output consistent with the console. You run this:
aws ec2 describe-vpc-peering-connection --region us-west-2 | grep pcx-abcd1234
The results show you a peering connection called pcx-abcd1234
You run this from a Python3 interpreter prompt:
import boto3
foo = boto3.resource('ec2')
foo.describe_vpc_peering.connections()
print(foo)You then search the output for pcx-abcd1234. You do not see it. Using different methods and syntax, you can never see pcx-abcd1234 in Boto3.
Using the AWS CLI, you can readily find pcx-abcd1234. Using the web console you can find it too.
You can find other peering connections with Boto3. One peering connection is elusive however.
Why would Boto3 results be discrepant from the CLI?
Solution
In Python, you need a stanza like this with the region_name and the profile setting to make it consistent with your AWS CLI:
boto3.setup_default_session(profile_name="foobar", region='us-west-2')
If you want to see what your AWS CLI is using, see the .aws/config file. If the profile and region are not the same between the AWS CLI and Boto3, you could get seemingly inconsistent results in the output regarding VPC components.
Problem scenario
You are now an AWS administrator for an existing company. You want to find out what the password requirements are (e.g., minimum length, maximum duration, complexity requirements etc.) for IAM users. What do you do to find the minimum length?
Solution
Log into the AWS console. Go to IAM -> Account Settings.
Problem scenario
You have read about manifests in books or articles that pertain to DevOps, but are not sure what they are. What is a manifest in the I.T. field?
Answer
Manifest: Syntactic text, usually a file with a variety of details, that is used for the purpose of convergence towards a desired state.
The definition above applies to a Puppet manifest or a Kubernetes manifest. A Puppet manifest is a .pp file with the desired state configuration listed using Puppet's domain-specific language. For more information about Kubernetes manifests, see this Kubernetes website article or this internal posting. In I.T., but not with DevOps specifically, a manifest may be defined differently. Here are some examples:
Merriam Webster's dictionary has different definitions for the noun "manifest". One considers a manifest to be a process of manifesting. This is similar to Kubernetes or Puppet manifests when they are being processed. Another definition of manifest is "a list of passengers or an invoice of cargo" on a transportation vehicle. This is similar to the HTML5 manifest.