How Do You Troubleshoot the Hadoop NameNode Service Not Running Properly?

Problem scenario
The Hadoop NameNode service won't start.  You run start-dfs.sh and it starts the SecondaryNameNode and the DataNode.  It won't start the NameNode.

Solution
Assumption: This solution only works if you are running the start-dfs.sh as a sudo user or the root user itself.

Procedures
Verify you can ssh with root to the localhost.  Run this command twice without exiting from the first session:  ssh root@localhost

If you cannot do that, verify the SSH service is running.  Verify you set up SSH keys for passwordless authentication for root.  You will need to be able to run these in succession without exiting them:

ssh root@localhost
ssh root@localhost

What Is an ARM in Modern Information Technology Vernacular?

Question
What Is an ARM in Modern Information Technology Vernacular?

Answer / Disambiguation
In I.T. jargon the term ARM can refer to a number of different things.  Here are some potential definitions of this acronym in random order:

#1  Azure Resource Manager is a component of Azure.  It can be interacted with in the console if you log into a Azure via a web browser.  ARM Templates are JSON files that can programmatically create servers, NICs, subnets etc.

#2 a) ARM can stand for an Advanced RISC Machine.  In I.T. RISC stands for Reduced Instruction Set Computing.  RISC technology is not limited to mobile devices.  IBM's AIX Unix servers leverage RISC architecture.  Sources include TechTarget and Wikipedia. 

#2 b) ARM could refer to the global technology company referred to as ARM Limited or ARM Holdings.

#2 c) Acorn RISC Machine.

#3  Application Response Measurement which is "an open standard published by the Open Group for monitoring and diagnosing performance bottlenecks within complex enterprise applications that use loosely-coupled designs or service-oriented architectures."  This quote was taken from Wikipedia's article on the subject.

#4  Asynchronous Response Mode.  This term is rare however.

#5  Adaptive and Reflective Middleware.   An annual workshop on this topic has been happening since 2004.  To learn more about the workshop itself see the Cyber Physical Systems Virtual Organization website.

How Do You Create a Linux Server in Azure with PowerShell When You Keep Getting an Error “Status Code: 400…Bad Request”?

Problem scenario
You are trying to create a Linux server in Azure with the Azure Tools for PowerShell.  You receive this message:

"Destination path for SSH public keys is currently limited to its default value due to a known issue in Linux provisioning agent."

You see  "StatusCode: 400 ... ReasonPhrase: Bad Request."

What do you do to create an Ubuntu 16.x server and not get this error?

Solution
Assuming you were using a .pub file with SSH when you received the error above, this solution below will work.  Create a VM without using a .pub file.  You have to enter the username and password interactively to get around the need for a .pub file being in a .ssh folder.

Here is a script that will create an Ubuntu 16.x Linux machine with 4 GB of RAM.  Just save the script then run it and answer the prompts.  You will need to manually enter your Azure credentials when prompted with a pop up after the script starts.

# Usage instructions
# Do five things before running this script:
# 1.  Change the password "ch@ngePassword" below
# 2.  Change the $resourceGroup name from "contIntGroup" to the one you want.  
# 3.  Change the location "eastus" unless that is what you want.
# 4.  Change the vmName variable from "contIntVM" to the name you want.
# 5.  Save the script locally.  It cannot run as an untitled draft.
# 6.  Optional: Change the IP addresses associated with the subnet mask and separately virtual network to something you want.
# 7.  Optional: You may want to change the names of the virtual network and subnet too.

# This script was modified from a copy taken
# from https://docs.microsoft.com/en-us/azure/virtual-machines/scripts/virtual-machines-windows-powershell-sample-create-vm

# This script creates a new resource group, a subnet mask and a NIC for the Linux server too.

write-host "*****************************"
write-host " "
write-host "ALERT: This script will require interaction.  It was designed that way."
write-host 'First you will be prompted to enter your Azure credentials to interactively login with a GUI.  This script therefore launches a web browser.'
write-host " "
write-host 'The subsequent GUI prompt will have the header "cmdlet Get-Credential at command pipeline..."'
write-host 'For this prompt, enter the credentials you want your Linux server to have.'
write-host "*****************************"
# Variables for common values
Login-AzureRmAccount
$resourceGroup = "contIntGroup"
$location = "eastus"
$vmName = "contIntVM"

# Definer user name and blank password
#$securePassword = ConvertTo-SecureString 'ch@ngePassword' -AsPlainText -Force
#$cred = New-Object System.Management.Automation.PSCredential ("contintuser", $securePassword)

$creda = Get-Credential

# Create a resource group
New-AzureRmResourceGroup -Name $resourceGroup -Location $location

# Create a subnet configuration
$subnetConfig = New-AzureRmVirtualNetworkSubnetConfig -Name contIntSubnet -AddressPrefix 192.168.1.0/24

# Create a virtual network
$vnet = New-AzureRmVirtualNetwork -ResourceGroupName $resourceGroup -Location $location `
  -Name ContIntNET -AddressPrefix 192.168.0.0/16 -Subnet $subnetConfig

# Create a public IP address and specify a DNS name
$pip = New-AzureRmPublicIpAddress -ResourceGroupName $resourceGroup -Location $location `
  -Name "mypublicdns$(Get-Random)" -AllocationMethod Static -IdleTimeoutInMinutes 4

# Create an inbound network security group rule for port 22
$nsgRuleSSH = New-AzureRmNetworkSecurityRuleConfig -Name ContIntNetworkSecurityGroupRuleSSH  -Protocol Tcp `
  -Direction Inbound -Priority 1000 -SourceAddressPrefix * -SourcePortRange * -DestinationAddressPrefix * `
  -DestinationPortRange 22 -Access Allow

# Create a network security group
$nsg = New-AzureRmNetworkSecurityGroup -ResourceGroupName $resourceGroup -Location $location `
  -Name myNetworkSecurityGroup -SecurityRules $nsgRuleSSH

# Create a virtual network card and associate with public IP address and NSG
$nic = New-AzureRmNetworkInterface -Name contIntNic -ResourceGroupName $resourceGroup -Location $location `
  -SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id -NetworkSecurityGroupId $nsg.Id

# Create a virtual machine configuration
$vmConfig = New-AzureRmVMConfig -VMName $vmName -VMSize Standard_D1 | `
Set-AzureRmVMOperatingSystem -Linux -ComputerName $vmName -Credential $creda | ` #-EnablePasswordAuthentication
Set-AzureRmVMSourceImage -PublisherName Canonical -Offer UbuntuServer -Skus 16.04-LTS -Version latest | `
Add-AzureRmVMNetworkInterface -Id $nic.Id

# Create a virtual machine
New-AzureRmVM -ResourceGroupName $resourceGroup -Location $location -VM $vmConfig

In the future we want to re-write these directions to use write-verbose instead of write-host. We want to warn people that write-verbose is recommended instead of write-host (according to this posting).

How Do You Troubleshoot the Error “WARN conf.Configuration: bad conf file: element not < property >”?

Problem scenario
You are trying to start HDFS.  But you get this error: "WARN conf.Configuration: bad conf file: element not <property>"

What should you do?

Solution
Look at mapred-site.xml, hdfs-site.xml, and core-site.xml files.  Verify you have a <property> tag in each file.

cat /path/to/mapred-site.xml | grep -i property
cat /path/to/hdfs-site.xml | grep -i property
cat /path/to/core-site.xml | grep -i property

If there is no <property> tag and closing </property> tag in each one of these three files, you may receive this error above.

Look for something like this:

<configuration>
 <name>
...
 </name>
</configuration>

The above is an example of a missing <property> tag.

How Do You Install an Older Version of Jenkins (e.g., 1.x) on Ubuntu Linux?

Problem scenario
Your enterprise's policies are to install a version of Jenkins that is lower than the latest.  How do you install Jenkins 1.651.1 on Ubuntu 16.04?

Solution
1.  Install Java.  See this posting if you do not know how.
2.  Run these three commands to install Jenkins 1.651.1 (but you can chance the version whichever one is available):

curl -L https://pkg.jenkins.io/debian-stable/binary/jenkins_1.651.1_all.deb > /tmp/jenkins_1.651.1_all.deb
sudo apt-get -y install daemon
sudo dpkg -i /tmp/jenkins_1.651.1_all.deb

How Do You Install Maven, openjdk, libssl-dev, build-essential, and pkgconf on a RedHat 7.X or CentOS 6.X Linux Server?

Problem scenario
You are using a RedHat derivative distribution of Linux (e.g., CentOS/RHEL/Fedora).  To eliminate a Hadoop error, you need to install a variety of packages.  How do you install Maven, openjdk, libssl-Dev, build-Essential, and pkgconf?

Solution
To install Maven, see this article.

For the rest of the packages, run these commands:

sudo yum -y install java-1.8.0-openjdk-devel openssl-devel cmake protobuf

sudo yum -y groupinstall 'Development Tools'

curl ftp://rpmfind.net/linux/fedora/linux/updates/25/x86_64/p/pkgconf-1.3.7-1.fc25.x86_64.rpm > pkgconf-1.3.7-1.fc25.x86_64.rpm

curl ftp://rpmfind.net/linux/fedora/linux/updates/25/x86_64/l/libpkgconf-1.3.7-1.fc25.x86_64.rpm > libpkgconf-1.3.7-1.fc25.x86_64.rpm

sudo rpm -ivh *.rpm

How Do You Pronounce Microsoft Azure?

Problem scenario
You want to know how to discuss Microsoft's public cloud offering. You do not know how to pronounce "Azure." How should it be pronounced?

Solution
It is pronounced "azh-er" according to Dictionary.com. To hear it pronounced you can click on the link above. Another related video is here.

How Do You Troubleshoot a Chef Client That Has the Results of “knife client list” as “404 not found”?

Problem scenario
You set up Chef server and Chef client on Linux servers.  On the Chef client server you run this command:

knife client list

You get this as the output:

ERROR: The object you are looking for could not be found
Response: <!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta http-equiv="X-UA-Compatible" content="IE=EmulateIE7" />
  <title>Chef - 404 Not Found</title>
  <link media="all" rel="stylesheet" type="text/css" href="/css/all.css" />
  <!--[if lt IE 7]><link rel="stylesheet" type="text/css" href="/css/lt7.css" /><![endif]-->
</head>
<body>
  <div class="header-block">
    <div id="header">
      <strong class="logo"><a href="https://www.chef.io">Chef</a></strong>
    </div>
  </div>
  <div id="wrapper">
    <div id="main">
      <div class="mybox">
        <div id="content">
          <h1>404 - Not Found</h1>
          <p>Sorry, I can't find what you are looking for.</p>
        </div>
      </div>
    </div>
  </div>
  <div class="footer-block">
    <div id="footer">
      <div class="mybox">
      </div>
      <div class="footer-bottom">
        <span>&copy; 2010&thinsp;&ndash;&thinsp;2017 Chef Software, Inc. All Rights Reserved</span>
      </div>
    </div>
  </div>
</body>
</html>

What do you do?

Solution
Find knife.rb.   (Use "sudo find / -name knife.rb".)   In this file find the "chef_server_url".

Verify the end of the URL has "/organizations/nameOfOrganization" (with no quotes where "nameOfOrganization" is the name of your organization as configured on the Chef server).  Some directions (or instructions) on different websites do not make it clear that the URL must have this last part, with the FQDN of the Chef server as the first part.

If the URL does not have "/organizations/nameOfOrganization", the above error may occur when running "knife client list". If this posting has not helped you, you may also want to see this posting.

How Do You Create a Certificate Signing Request (CSR) File and Set up an HTTPS Server for Basic Testing?

Problem scenarios
This is a two-in-one posting.  The same solution works for two different problem scenarios.

Problem scenario A
You want to create a certificate signing request permissions file (a csr.pem) to learn more about it.  You also want to try to use HTTPS as a test.  How do you do these things?

Problem scenario B
You want to implement a solution with TLS.  You want to test out an example to be able to know what it is like beyond theory and reading about it.  You find many articles on the internet to be old.  How do you use TLS?

Solution
1.  Install Node.JS on a Linux server.  You need a Linux server that is not blocking port 8000 with at least 1 GB of RAM.  Use these directions to install Node.   You do not need a web server (like Apache web server or Nginx) to be installed.  

2.  Go to any given directory on the Linux server.  Run these four commands and respond to their prompts as you see fit*:

sudo openssl genrsa -out key.pem
sudo openssl req -new -key key.pem -out csr.pem
sudo openssl x509 -req -days 9999 -in csr.pem -signkey key.pem -out cert.pem
sudo rm csr.pem

3.  In the same directory create a new file named test.js with the following content:

var https = require('https');
var fs = require('fs');

var options = {
  key: fs.readFileSync('key.pem'),
  cert: fs.readFileSync('cert.pem')
};

var a = https.createServer(options, function (req, res) {
  res.writeHead(200);
  res.end("hello world\n");
}).listen(8000);

// Code taken from https://docs.nodejitsu.com/articles/HTTP/servers/how-to-create-a-HTTPS-server/

4.  Run this command "node test.js"

5.  Open a second terminal to your Linux server.  Run this command:  curl -k https://127.0.0.1:8000

You'll see "hello world".  You are now done. Be advised that the -k flag defeats the ssl certificate requirement. From the client server you could upload the .crt file to /etc/ssl/certs/ca-certificates.crt. Then the -k flag should not be necessary with the curl command to access the JavaScript web page.

6.  Optional step if you do not want to do step #5:
You can open a web browser and go to this url (where x.x.x.x is the external IP address of the Linux server):  https://x.x.x.x:8000
In Chrome you will have to click "Advanced" to accept an exception as the certificate will not be valid.

You'll see "hello world".

*  OpenSSL (or openssl commands) use TLS.  The openssl man page (as taken from a Linux server) says this: "OpenSSL is a cryptography toolkit implementing the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) network protocols and related cryptography standards required by them."