A Technical I.T./DevOps Blog
Problem scenario
In an Ubuntu-based Docker container, how do you install MariaDB and start the MariaDB service?
Solution
Run these commands:
sudo apt-get -y updatesudo apt-get -y install vim
sudo apt-get install mariadb-server
sudo service mysql start
Question
What and when was the first reference to cloud automation operations?
Answer
The cloud (e.g., the SaaS model or public cloud offerings) allow for new types of automation in computing. Before electricity existed, pneumatic and hydraulic automations were among the oldest automations in history. The only man-made clouds back then were smoke or steam clouds.
Homer's Iliad refers to cloud automation. This appears to the be earliest use of such an idea (a gate), possibly made of water moisture such as a cloud, would open without any intervention. The Iliad is believed to have been composed in the mid 750s BCE (Scientific American). The ancient story describes automation in two different passages.
"Observe the freedom of the imagery by which the gate, though said to be a cloud in 751, is made to creak." (Homer, The Iliad, book 5 line 749).
As of 2018 we do not think that tripods are relevant to cloud automation operations. However The Iliad refers to "automatically travelling tripods." (Homer, The Iliad, book 18 line 376). Pneumatic, hydraulic and supernatural automation could have involved clouds in oral history passed down as folklore in ancient times. But we know of no record related to automated cloud operations before The Iliad was written.
Speaking of ancient operations related to computers, the world's largest undocumented computer is Stonehenge (according to page 163 of The Mythical Man-Month).
If you want to learn more about when non-cloud automation became an important concept to business outside of just cloud automation, see this posting.
Question
In the Context of Nginx, What Is the Difference between the default.conf and nginx.conf Files?
Answer
For general Nginx purposes, the nginx.conf file specifies the location of the Nginx logs and the names of the log files themselves.
For general Nginx purposes, the default.conf handles many more configuration items than nginx.conf. The default.conf file will allow you to configure a non-standard port (other than port 80) and specifies if Nginx will act as a load balancer or a web server. If you choose to configure the Nginx deployment as an HTTP load balancer (distributing incoming connections to backend web servers), this default.conf file will allow you to configure how the traffic is distributed. If you configure Nginx to be a web server, the location of the index.html file can be configured with default.conf.
Problem scenario
You are trying to run a bash/shell script but you get an error message that is difficult to troubleshoot. It looks something like this:
myscript.sh: line 4: syntax error near unexpected token `curl'
myscript.sh: line 4: `curl 'http://10.10.0.1' '
You thoroughly analyzed the line of the script that is called out in the error. What is the root cause?
Solution
Did you define a function in the script? The true problem may not be on the line that is called out in the error message. If you defined a function like this "coolfunction{", this will cause the above problem. The error message is misleading. You need a space after the text "coolfunction" but before the opening brace "{".
Problem scenario
You have deployed the LAPP stack. You want a basic and meaningful message to be presented to a user of the web application after a SQL command has been run. Currently a user can click on buttons to run SQL commands behind the scenes. Messages that are returned to the user may be meaningless like "Resource id #4" or "Resource id #5." You want to augment the functionality so at least a rudimentary message (with some meaning) is returned to the user. This way he/she will know if the query was successful or not. How do you do this?
Solution
Assuming that you are invoking SQL commands with the pg_query command (see this link to deploy the LAPP stack), assign the output to a variable like this:
$varcomplete = pg_query($query);
Then add these two lines of code to your PHP page that is displayed after the user clicks the button to invoke a SQL command:
if ($varcomplete != '') {$varcomplete = "The query was successful";}
else {$varcomplete = "WARNING. The query failed!!!";}
Problem scenario
You are trying to install Snort on Ubuntu. You have installed hwloc (because you know the command hwloc-info works). But when you try to run a configure script (e.g., configure_cmake.sh), you get this error:
"-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
CMake Error at cmake/FindHWLOC.cmake:6 (message):
ERROR! hwloc library not found.
Get it from https://www.open-mpi.org/projects/hwloc/
Call Stack (most recent call first):
cmake/include_libraries.cmake:6 (find_package)
CMakeLists.txt:18 (include)"
How do you troubleshoot this "Could not find PkgConfig: Missing:" error?
Solution
Install several packages with the following command:
sudo apt-get -y install libxext-dev libpng-dev libimlib2-dev libglew-dev libxrender-dev libxrandr-dev libglm-dev
# If one or two of the above is not found, do not be alarmed. The above command may still solve the problem.
Updated 1/4/19
Question
You have heard of Azure's Serverless Function Apps. What are they and what do they do?
Answer
A Serverless Function App is a PaaS solution. As of 1/4/19 they support .NET or JavaScript code as if the code was run on a Windows environment. They will eventually support Java/Linux.
You can run a program without a server using Azure. It could potentially be accessible on the web or through other connectors to different Azure servers. You do not need to be concerned about the systems administration on the back-end. See this posting if you want to use an Azure Function.
Problem scenario
You want to deploy the LAMP stack on a Debian/Ubuntu distribution of Linux. You want to install and the four core technologies that compose the LAMP stack (Linux, Apache web server, MariaDB/MySQL, and PHP). How do you do this?
Solution #1 (with MySQL)
To install the OS, Ubuntu Linux, see this link. To install MySQL run these commands:
sudo apt-get -y update
sudo apt-get -y install php apache2 mysql-server
# Provide a password for the mysql root user when prompted.
# To log into the mysql database for the first time, use this command:
# mysql -u root -p # You will be prompted to enter a password.Solution #2 (with MariaDB)
If you wanted to use MariaDB and assuming Ubuntu is already installed, run these commands:sudo apt-get -y update
sudo apt-get -y install php apache2 mariadb-server
Problem scenario
You want to see if there is an operating system, software or host-based firewall on your Linux machine. What do you do?
Solution
Run the following six commands for clues:
sudo ps -ef | grep firewalld
sudo systemctl status firewalld
sudo ps -ef | grep ufw
sudo ufw status
sudo iptables -L #*
sudo nmap -P0 127.0.0.1If you are running the server in a public cloud, remember that there are Security Groups in AWS, Network Security Groups (NSGs) in Azure, and firewall rules in GCP that could block connectivity. The above is mostly for determining if there is a running an OS-level, software firewall.
* Here is an example of output of the iptables -L command when iptables is not blocking anything:
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Problem scenario
You have a multi-node cluster of Hadoop. You want to add a new data node. What do you do?
Solution
1. a) Log into the server that will be the new DataNode. Do these things until you get to step 2.
b) Install Hadoop on the new DataNode. If you do not know how, see this posting.
c) The DataNode server must be configured to resolve the NameNode's hostname. Modify the new DataNode's /etc/hosts file if you do not have a DNS server on your network for this server that will be the DataNode. The modification the /etc/hosts file should get (if there is no DNS server for it) should be an entry for the NameNode server.
d)i. You need to modify three .xml files on the DataNode. These files are core-site.xml, mapred-site.xml, and hdfs-site.xml. To do these modifications, see steps 6 through 8 on this set of instructions.
d)ii. Look at the port used in core-site.xml. Verify this port is open from the DataNode to the NameNode. If you do not know how to test and the DataNode is running Windows, see this posting. If you do not know how to test and the DataNode us running Linux, install nmap and use this command: nmap -p xxxxx FQDNofNameNode # where xxxxx is the port number and FQDNofNameNode is the FQDN of the NameNode.
e) Create an hduser account. (You'll need this for step 2e below.) Create an hadoop group name. Ensure that the hduser can ssh into the local machine. If you do not know how to do all these things, see steps 2 and 3 in this posting.
f) Run these two commands:
cd /app
sudo chown hduser:hadoop hadoop
2. a) Log into the NameNode server. Do these things:
b) The NameNode server must be configured to resolve the DataNode's hostname. Modify the new NameNode's /etc/hosts file if you do not have a DNS server on your network for the NameNode. The modification the /etc/hosts file should get (if there is no DNS server for it) should be an entry for the DataNode server.
c) On the NameNode server, append the new DataNode's hostname to this file /usr/local/hadoop/etc/hadoop/workers
d) Modify the hdfs-site.xml file. vi /usr/local/hadoop/etc/hadoop/hdfs-site.xml # Update the dfs.replication integer value to be the sum of the servers incluing the NameNode and all DataNodes (including the one you added).
e) Configure passwordless SSH authentication from the NameNode's hduser account to the server that will be the DataNode. See this posting if you do not know how. Test the ability to passwordless SSH.
f) From the NameNode run ssh hduser@DataNodeHostName # where DataNodeHostName is the host name of the DataNode. This tests it will work.
g) Run these commands (but note that it will delete all the data in your cluster) as the hduser (or designated Linux username for running such commands):
hdfs namenode -format
# When prompted with this:
# "Re-format filesystem in Storage Directory /app/hadoop/tmp/dfs/name ? (Y or N)"
# choose "Y" (with no quotes) and press enter
bash /usr/local/hadoop/sbin/start-dfs.sh