How Do You Troubleshoot the Spark-Shell Error “A JNI error has occurred”?

Problem scenario
You run spark-shell in a Debian distribution of Linux (e.g., Ubuntu) but you receive this error:

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 64
        at java.util.jar.JarFile.match(java.base@9-internal/JarFile.java:983)
        at java.util.jar.JarFile.checkForSpecialAttributes(java.base@9-internal/JarFile.java:1017)
        at java.util.jar.JarFile.isMultiRelease(java.base@9-internal/JarFile.java:399)
        at java.util.jar.JarFile.getEntry(java.base@9-internal/JarFile.java:524)
        at java.util.jar.JarFile.getJarEntry(java.base@9-internal/JarFile.java:480)
at jdk.internal.util.jar.JarIndex.getJarIndex(java.base@9-internal/JarIndex.java:114)
        at jdk.internal.loader.URLClassPath$JarLoader$1.run(java.base@9-internal/URLClassPath.java:640)
        at jdk.internal.loader.URLClassPath$JarLoader$1.run(java.base@9-internal/URLClassPath.java:632)
        at java.security.AccessController.doPrivileged(java.base@9-internal/Native Method)
        at jdk.internal.loader.URLClassPath$JarLoader.ensureOpen(java.base@9-internal/URLClassPath.java:631)
        at jdk.internal.loader.URLClassPath$JarLoader.<init>(java.base@9-internal/URLClassPath.java:606)
        at jdk.internal.loader.URLClassPath$3.run(java.base@9-internal/URLClassPath.java:386)
        at jdk.internal.loader.URLClassPath$3.run(java.base@9-internal/URLClassPath.java:376)
        at java.security.AccessController.doPrivileged(java.base@9-internal/Native Method)
        at jdk.internal.loader.URLClassPath.getLoader(java.base@9-internal/URLClassPath.java:375)
        at jdk.internal.loader.URLClassPath.getLoader(java.base@9-internal/URLClassPath.java:352)
        at jdk.internal.loader.URLClassPath.getResource(java.base@9-internal/URLClassPath.java:218)
        at jdk.internal.loader.BuiltinClassLoader$3.run(java.base@9-internal/BuiltinClassLoader.java:463)
        at jdk.internal.loader.BuiltinClassLoader$3.run(java.base@9-internal/BuiltinClassLoader.java:460)
        at java.security.AccessController.doPrivileged(java.base@9-internal/Native Method)
        at jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(java.base@9-internal/BuiltinClassLoader.java:459)
        at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base@9-internal/BuiltinClassLoader.java:406)
        at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base@9-internal/BuiltinClassLoader.java:364)
        at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(java.base@9-internal/ClassLoaders.java:184)
        at java.lang.ClassLoader.loadClass(java.base@9-internal/ClassLoader.java:419)
        at sun.launcher.LauncherHelper.loadMainClass(java.base@9-internal/LauncherHelper.java:585)
        at sun.launcher.LauncherHelper.checkAndLoadMain(java.base@9-internal/LauncherHelper.java:497)

How do you solve this?

Solution
Use Java version 8 not version 9.  

Option 1
One option would be to uninstall Java (version 9) and reinstall Java (version 8).  (You can view this posting for installing Java; make sure you make necessary changes so you install version 8.)

Option 2
If you have both installed and you are using Ubuntu, you can use this command:

sudo update-alternatives --config java

# To the prompt you will see, respond with the integer associated with Java 8 and press enter.

How Do You Install Apache Spark on Any Type of Linux?

Problem scenario
You want a generic script that can install open source Apache Spark on Debian/Ubuntu, CentOS/RedHat/Fedora or SUSE distributions of Linux.  How do you do this?

Solution
1.  Create a script such as this in /tmp/ (e.g., /tmp/spark.sh).

#!/bin/bash
# Written by www.continualintegration.com

sparkversion=2.2.1  # Change this version as necessary

distro=$(cat /etc/*-release | grep NAME)

debflag=$(echo $distro | grep -i "ubuntu")
if [ -z "$debflag" ]
then   # If it is not Ubuntu, test if it is Debian.
  debflag=$(echo $distro | grep -i "debian")
  echo "determining Linux distribution..."
else
   echo "You have Ubuntu Linux!"
fi

rhflag=$(echo $distro | grep -i "red*hat")
if [ -z "$rhflag" ]
then   #If it is not RedHat, see if it is CentOS or Fedora.
  rhflag=$(echo $distro | grep -i "centos")
  if [ -z "$rhflag" ]
    then    #If it is neither RedHat nor CentOS, see if it is Fedora.
    echo "It does not appear to be CentOS or RHEL..."
    rhflag=$(echo $distro | grep -i "fedora")
    fi
fi

if [ -z "$rhflag" ]
  then
  echo "...still determining Linux distribution..."
else
  echo "You have a RedHat distribution (e.g., CentOS, RHEL, or Fedora)"
  yum -y install java-1.8.0-openjdk*
  JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk
  echo 'export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk' >> ~/.bashrc
  source ~/.bashrc
  source /etc/environment
fi

if [ -z "$debflag" ]
then
  echo "...still determining Linux distribution..."
else
   echo "You are using either Ubuntu Linux or Debian Linux."
   apt-get -y update # This is necessary on new AWS Ubuntu servers.
   apt -y install openjdk-8-jre-headless
fi

suseflag=$(echo $distro | grep -i "suse")
if [ -z "$suseflag" ]
then
  if [ -z "$debflag" ]
  then
    if [ -z "$rhflag" ]
      then
      echo "*******************************************"
      echo "Could not determine the Linux distribution!"
      echo "Installation aborted. Nothing was done."
      echo "******************************************"
      exit
    fi
  fi
else
   zypper -n install java-1_8_0-openjdk
fi

curl http://ftp.wayne.edu/apache/spark/spark-$sparkversion/spark-$sparkversion-bin-hadoop2.7.tgz > /bin/spark-$sparkversion-bin-hadoop2.7.tgz

cd /bin
tar -zxvf spark-$sparkversion-bin-hadoop2.7.tgz
mkdir -p /usr/local/spark
mv /bin/spark-$sparkversion-bin-hadoop2.7/* /usr/local/spark

echo 'PATH="$PATH":/usr/local/spark/bin/' > /etc/profile.d/spark.sh

echo 'To test the installation, run "/usr/local/spark/bin/spark-shell"'

2.  Run it like this:

sudo bash /tmp/spark.sh

How Do You Install OSSEC on Any Type of Linux?

Problem scenario
You have Debian/Ubuntu, RedHat (including CentOS and Fedora), and SUSE distributions of Linux.  You want to install OSSEC on each server (to protect them with host-based intrusion detection systems, IDSes).  You want to use the same script to install OSSEC on each server. How do you do this?

Solution
1.  Create a script such as this /tmp/ossec.sh.

#!/bin/bash
# Written by www.continualintegration.com

ossecversion=3.1.0  # Change this version as necessary

filename=`basename "$0"`
numlines=$(wc -l $filename | awk '{print $1}')
if [ 83 -lt $numlines ]    # change 83 to the correct number of lines if script grows
then
  echo "line numbers too long.  Were extra lines introduced accidentally during a cut and paste?"
  echo "****************EXITING**********************************"
  exit
else
  echo "Script running..."
fi

distro=$(cat /etc/*-release | grep NAME)

debflag=$(echo $distro | grep -i "ubuntu")
if [ -z "$debflag" ]
then   # If it is not Ubuntu, test if it is Debian.
  debflag=$(echo $distro | grep -i "debian")
  echo "determining Linux distribution..."
else
   echo "You have Ubuntu Linux!"
fi

rhflag=$(echo $distro | grep -i "red*hat")
if [ -z "$rhflag" ]
then   #If it is not RedHat, see if it is CentOS or Fedora.
  rhflag=$(echo $distro | grep -i "centos")
  if [ -z "$rhflag" ]
    then    #If it is neither RedHat nor CentOS, see if it is Fedora.
    echo "It does not appear to be CentOS or RHEL..."
    rhflag=$(echo $distro | grep -i "fedora")
    fi
fi

if [ -z "$rhflag" ]
  then
  echo "...still determining Linux distribution..."
else
  echo "You have a RedHat distribution (e.g., CentOS, RHEL, or Fedora)"
  yum -y install httpd unzip wget gcc php sendmail python-inotify
fi

if [ -z "$debflag" ]
then
  echo "...still determining Linux distribution..."
else
   echo "You are using either Ubuntu Linux or Debian Linux."
   apt-get -y update # This is necessary on new AWS Ubuntu servers.
   apt -y install build-essential gcc make apache2 libapache2-mod-php7.0 php7.0 php7.0-cli php7.0-common apache2-utils unzip

wget sendmail inotify-tools
fi

suseflag=$(echo $distro | grep -i "suse")
if [ -z "$suseflag" ]
then
  if [ -z "$debflag" ]
  then
    if [ -z "$rhflag" ]
      then
      echo "*******************************************"
      echo "Could not determine the Linux distribution!"
      echo "Installation aborted. Nothing was done."
      echo "******************************************"
      exit
    fi
  fi
else
   echo "You have Linux SUSE."
   zypper -n install gcc make apache2 apache2-mod_php7 php7 apache2-utils unzip
fi

cd /tmp
wget https://github.com/ossec/ossec-hids/archive/$ossecversion.tar.gz
tar -zxvf $ossecversion.tar.gz
mv /tmp/ossec-hids-$ossecversion /bin/ossec   #ossec-hids-$ossecversion
echo "go to /bin/ossec (with a cd) and run 'sudo bash install.sh'"
echo "Then follow the interactive text menu prompts after that.  It should be self-explanatory."
echo "For a proof-of-concept, it is easiest if you do not configure email alerts or an SMTP notification server."

2.  Run the script with this command: sudo bash ossec.sh

How Do You Install Apache Solr on Any Type of Linux?

Problem scenario
You want a generic script that can install open source Apache Solr on Debian/Ubuntu, CentOS/RedHat/Fedora or SUSE distributions of Linux.  How do you do this with the same bash script?

Solution
1.  Create a script such as this in /tmp/ (e.g., /tmp/solr.sh).

#!/bin/bash
# Written by www.continualintegration.com

solrversion=7.2.1  # Change this version as necessary

distro=$(cat /etc/*-release | grep NAME)

debflag=$(echo $distro | grep -i "ubuntu")
if [ -z "$debflag" ]
then   # If it is not Ubuntu, test if it is Debian.
  debflag=$(echo $distro | grep -i "debian")
  echo "determining Linux distribution..."
else
   echo "You have Ubuntu Linux!"
fi

rhflag=$(echo $distro | grep -i "red*hat")
if [ -z "$rhflag" ]
then   #If it is not RedHat, see if it is CentOS or Fedora.
  rhflag=$(echo $distro | grep -i "centos")
  if [ -z "$rhflag" ]
    then    #If it is neither RedHat nor CentOS, see if it is Fedora.
    echo "It does not appear to be CentOS or RHEL..."
    rhflag=$(echo $distro | grep -i "fedora")
    fi
fi

if [ -z "$rhflag" ]
  then
  echo "...still determining Linux distribution..."
else
  echo "You have a RedHat distribution (e.g., CentOS, RHEL, or Fedora)"
  yum -y install java-1.8.0-openjdk* lsof
  JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk
  echo 'export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk' >> ~/.bashrc
  source ~/.bashrc
  source /etc/environment
fi

if [ -z "$debflag" ]
then
  echo "...still determining Linux distribution..."
else
   echo "You are using either Ubuntu Linux or Debian Linux."
   apt-get -y update # This is necessary on new AWS Ubuntu servers.
   apt -y install openjdk-9-jre-headless lsof #default-jdk g++ ant
apt -y install openjdk-11-jre-headless
fi

suseflag=$(echo $distro | grep -i "suse")
if [ -z "$suseflag" ]
then
  if [ -z "$debflag" ]
  then
    if [ -z "$rhflag" ]
      then
      echo "*******************************************"
      echo "Could not determine the Linux distribution!"
      echo "Installation aborted. Nothing was done."
      echo "******************************************"
      exit
    fi
  fi
else
   zypper -n install java-1_8_0-openjdk lsof
fi

curl http://apache.mesi.com.ar/lucene/solr/$solrversion/solr-$solrversion.tgz > /bin/solr-$solrversion.tgz

cd /bin
tar -xvf solr-$solrversion.tgz
mkdir -p /usr/local/solr
mv /bin/solr-$solrversion/* /usr/local/solr
chmod o+w -R /usr/local/solr/bin/
chmod o+w -R /usr/local/solr/server/

echo 'PATH="$PATH":/usr/local/solr/bin/' > /etc/profile.d/solr.sh

echo 'To finish the process, run this command:'
echo 'export PATH="$PATH":/usr/local/solr/bin/'

echo "Run '/usr/local/solr/bin/solr version' to see if solr is installed."
echo "Run '/usr/local/solr/bin/solr start' to start it"

How Do You Create a GitLab API Token?

Problem scenario
You have GitLab and Jenkins set up already.  You want to use a GitLab API token in Jenkins.  How do you create a GitLab API token?

Solution
1.  Log into the GitLab web UI.  (If you need directions on setting up GitLab, see this posting.)
2.  In the upper righthand corner, go to the circle icon next to an icon of a down arrow.  Click on it and go to "Profile" (which should appear underneath this icon).
3.  Click the pencil icon in the upper righthand portion of the screen.  When you hover over this icon, it should say "Edit."
4.  Find "Access Tokens" on the lefthand side of the screen.
5.  Type an arbitrary name in the "Name" field.
6.  Enter a date the token will expire.
7.  Check the radio boxes under "Scopes" as you see fit.
8.  Click "Create personal access token."
9.  Save the alphanumeric string under "Your New Personal Access Token".  You will not be able to see it again.

How Do You Write a Python Program to Simulate a Dial-by-Name directory?

Updated on 9/19/19 to use Python 3

Problem scenario
You have a list of first and last names in a file.  You want a program to extract the last names based on the first two or three letters.  You want to allow the user to dial by last name (to enter phone keys that correspond to letters of last names).  You want the user to enter a sequence of numbers that will correspond to the first two or three letters of the person's last name. 

How do you write a Python program that reads a text file on the same server and then prompts the user to dial by name with a touchtone keypad ("2" corresponds to "abc", "3" corresponds to "def", etc.) to return potential matches by last name?

Solution

1.  Create a file named list.txt that has names in this format (with no blank lines at the top or bottom):

Allen, Paul
Ballmer, Steve
Bell, Gordon
Berners-Lee, Tim
Bosack, Leonard
Brin, Sergey
Ellison, Larry
Gates, Bill
Jobs, Steve
Hopper, Grace
Lerner, Sandy
Li, Bruce
Moore, Gordon
Page, Larry
Torvalds, Linus
Turing, Alan
Watson, Thomas
Wozniack, Steve
Zuckerberg, Mark

2.  Use this as the program:

# This is for Python 3. It was written by continualintegration.com
# It requires that a file named list.txt be in the same directory.  
""" Written by continualintegration.com.  Usage instructions:
    This program needs to be in the same directory as a file called list.txt.
    You would run the program like this: "python phone.py"
    The content of list.txt can be as follows:

Allen, Paul
Ballmer, Steve
Bell, Gordon
Berners-Lee, Tim
Bosack, Leonard
Brin, Sergey
Ellison, Larry
Gates, Bill
Jobs, Steve
Hopper, Grace
Lerner, Sandy
Li, Bruce
Moore, Gordon
Page, Larry
Torvalds, Linus
Turing, Alan
Watson, Thomas
Wozniack, Steve
Zuckerberg, Mark
"""

def phone():
  pad = """This is the keypad.  Press a number key that corresponds to the letter of the name.
  Enter a minimum of two and a maximum of three numbers.

  1
  2 ABC
  3 DEF
  4 GHI
  5 JKL
  6 MNO
  7 PQRS
  8 TUV
  9 WXYZ
  0"""

  print(" ")
  print(pad)
  first = input("Please enter the first three letters of the person's last name: ")

  if len(first) < 2:
    print("you entered too few numbers")
    print("Please try again")
    return 'DO_OVER_FLAG'

  if len(first) > 3:
    print("you entered too many numbers")
    print("Please try again")
    return 'DO_OVER_FLAG'

  if first[0] == '0':
    print("0 is not a valid key.")
    print("Please try again.")
    return 'DO_OVER_FLAG'

  if first[0] == '1':
    print("1 is not a valid key.")
    print("Please try again.")
    return 'DO_OVER_FLAG'

  if first[0].isalpha():
    print("enter numbers only")
    print("Please try again.")
    return 'DO_OVER_FLAG'

  if first[1].isalpha():
    print("enter numbers only")
    print("Please try again.")
    return 'DO_OVER_FLAG'

  if len(first) == 3:
    if first[2].isalpha():
      print("enter numbers only")
      print("Please try again.")
      return 'DO_OVER_FLAG'

  print("This was valid input")
  return first


def telparser(astring):
  if astring[0] == '2':
    firstletter = 'ABC'
  if astring[0] == '3':
    firstletter = 'DEF'
  if astring[0] == '4':
    firstletter = 'GHI'
  if astring[0] == '5':
    firstletter = 'JKL'
  if astring[0] == '6':
    firstletter = 'MNO'
  if astring[0] == '7':
    firstletter = 'PQRS'
  if astring[0] == '8':
    firstletter = 'TUV'
  if astring[0] == '9':
    firstletter = 'WXYZ'

  if astring[1] == '2':
    secondletter = 'ABC'
  if astring[1] == '3':
    secondletter = 'DEF'
  if astring[1] == '4':
    secondletter = 'GHI'
  if astring[1] == '5':
    secondletter = 'JKL'
  if astring[1] == '6':
    secondletter = 'MNO'
  if astring[1] == '7':
    secondletter = 'PQRS'
  if astring[1] == '8':
    secondletter = 'TUV'
  if astring[1] == '9':
    secondletter = 'WXYZ'

  if len(astring) > 2:
    if astring[2] == '2':
      thirdletter = 'ABC'
    if astring[2] == '3':
      thirdletter = 'DEF'
    if astring[2] == '4':
      thirdletter = 'GHI'
    if astring[2] == '5':
      thirdletter = 'JKL'
    if astring[2] == '6':
      thirdletter = 'MNO'
    if astring[2] == '7':
      thirdletter = 'PQRS'
    if astring[2] == '8':
      thirdletter = 'TUV'
    if astring[2] == '9':
      thirdletter = 'WXYZ'
    return firstletter, secondletter, thirdletter
  else:
    return firstletter, secondletter


def finder(content):
  goodlist = []
  finallist = []
  prebook=open('list.txt', 'r')
  book = prebook.read()
  postbook = book.split('\n')
  postbook = postbook[:-1]
  d00 = str(content[0][0])
  d01 = str(content[0][1])
  d02 = str(content[0][2])
  if len(content[0]) > 3:
    d03 = str(content[0][3])

  for item in postbook:
    item0 = str(item[0])
    if d00.upper() == item0.upper():    # Test first letter of last name against the first character of first three letters o
      goodlist.append(item)
    elif d01.upper() == item0.upper():
      goodlist.append(item)
    elif d02.upper() == item0.upper():
      goodlist.append(item)
    else:
      if len(content[0]) > 3:
        if d03.upper() == item0.upper():
          goodlist.append(item)

  for item in goodlist:
    item1 = str(item[1])
    d10 = str(content[1][0])
    d11 = str(content[1][1])
    d12 = str(content[1][2])
    if len(content[1]) > 3:
      d13 = str(content[1][3])


    if d10.upper() == item1.upper():    # Test second letter of last name against the second character of first three letters
      finallist.append(item)
    elif d11.upper() == item1.upper():
      finallist.append(item)
    elif d12.upper() == item1.upper():
      finallist.append(item)
    else:
      if len(content[1]) > 3:
        if d13.upper() == item1.upper():
          finallist.append(item)
  if len(finallist) == 0:
    finallist.append("No names matched that entry.")


  return finallist


try:      # It is easy to forget that the program needs an extra file.
  toprint = phone()
except FileNotFoundError:
  print("")
  print("*********************************")
  print("FATAL ERROR")
  print("You need to create a list.txt file that is in the same directory as this program.")
  print("See this web page: https://www.continualintegration.com/miscellaneous-articles/how-do-you-write-a-python-program-to-s
  print("Step #1 of the solution in the web page above will help you.")
  print("The program will now exit.")
  quit()

while toprint == 'DO_OVER_FLAG':
  toprint = phone()


intermed = telparser(toprint)

r = finder(intermed)
print("We searched for matches and we found ...")
for x in r:
  print(x)

      

Here is a Python 2 version (but we know it does not work as well):

# Written by continualintegration.com
""" Written by continualintegration.com.  Usage instructions:
    This program needs to be in the same directory as a file called list.txt.
    The content of list.txt can be as follows:

Allen, Paul
Ballmer, Steve
Bell, Gordon
Berners-Lee, Tim
Bosack, Leonard
Brin, Sergey
Ellison, Larry
Gates, Bill
Jobs, Steve
Hopper, Grace
Lerner, Sandy
Li, Bruce
Moore, Gordon
Page, Larry
Torvalds, Linus
Turing, Alan
Watson, Thomas
Wozniack, Steve
Zuckerberg, Mark

# The term "MARK" in the program is just a designated value for logic.
# Do not confuse it with Mr. Zuckerberg's first name.
# Use "python phone.py" to run this program.

"""
import re

def phone():
  pad = """This is the keypad.  Press a number key that corresponds to the letter of the name.
  Enter a minimum of two and a maximum of three numbers.

  1
  2 ABC
  3 DEF
  4 GHI
  5 JKL
  6 MNO
  7 PQRS
  8 TUV
  9 WXYZ
  0"""

  print pad
  first = raw_input("Please enter the first three letters of the person's last name: ")

  prebook=open('list.txt', 'r')
  book = prebook.read()
  postbook = book.split('\n')
  global i;
  i = -1

  def presearch(digit, i):   #Digit is the key they entered.
    intermediate = []
    finvar = "initial"
    if (digit == '0'):
      finvar = "***Invalid key error!!!  0 has no letters associated with it. ABORTIVE ERROR"  
    elif digit == '1':
      finvar = "***invalid key error!!!  1 has no letters associated with it. ABORTIVE ERROR."  
    elif digit == '2':
      prevar1 = search('A', i)
      prevar2 = search('B', i)
      prevar3 = search('C', i)
      if (prevar1 == []):
        if (prevar2 == []):
          if (prevar3 == []):
            if (i == 0): place = 'first'
            if (i == 1): place = 'second'
            if (i == 3): place = 'third'
        finvar = '***No name found that starts with an "A", "B", or "C" in the ' + place + ' letter.'
      if (prevar1 != []): intermediate.extend(prevar1)
      if (prevar2 != []): intermediate.extend(prevar2) # Append would preserve groupings by first letter
      if (prevar3 != []): intermediate.extend(prevar3)
      finvar = intermediate

    elif digit == '3':
      prevar1 = search('D', i)
      prevar2 = search('E', i)
      prevar3 = search('F', i)
      if (prevar1 == []):
        if (prevar2 == []):
          if (prevar3 == []):
            if (i == 0): place = 'first'
            if (i == 1): place = 'second'
            if (i == 3): place = 'third'
            finvar = '***No name found that starts with an "D", "E", or "F" in the ' + place + ' letter.'
      if (prevar1 != []): intermediate.extend(prevar1)
      if (prevar2 != []): intermediate.extend(prevar2) # Append would preserve groupings by first letter
      if (prevar3 != []): intermediate.extend(prevar3)
      finvar = intermediate

    elif digit == '4':
      prevar1 = search('G', i)
      prevar2 = search('H', i)
      prevar3 = search('I', i)
      if (prevar1 == []):
        if (prevar2 == []):
          if (prevar3 == []):
            if (i == 0): place = 'first'
            if (i == 1): place = 'second'
            if (i == 3): place = 'third'
            finvar = '***No name found that starts with an "G", "H", or "I" in the ' + place + ' letter.'
      if (prevar1 != []): intermediate.extend(prevar1)
      if (prevar2 != []): intermediate.extend(prevar2) # Append would preserve groupings by first letter
      if (prevar3 != []): intermediate.extend(prevar3)
      finvar = intermediate

    elif digit == '5':
      prevar1 = search('J', i)
      prevar2 = search('K', i)
      prevar3 = search('L', i)
      if (prevar1 == []):
        if (prevar2 == []):
          if (prevar3 == []):
              if (i == 0): place = 'first'
              if (i == 1): place = 'second'
              if (i == 3): place = 'third'
              finvar = '***No name found that starts with an "J", "K", or "L" in the ' + place + ' letter.'
      if (prevar1 != []): intermediate.extend(prevar1)
      if (prevar2 != []): intermediate.extend(prevar2) # Append would preserve groupings by first letter
      if (prevar3 != []): intermediate.extend(prevar3)
      finvar = intermediate

    elif digit == '6':
      prevar1 = search('M', i)
      prevar2 = search('N', i)
      prevar3 = search('O', i)
      if (prevar1 == []):
        if (prevar2 == []):
          if (prevar3 == []):
              if (i == 0): place = 'first'
              if (i == 1): place = 'second'
              if (i == 3): place = 'third'
              finvar = '***No name found that starts with an "M", "N", or "O" in the ' + place + ' letter.'
      if (prevar1 != []): intermediate.extend(prevar1)
      if (prevar2 != []): intermediate.extend(prevar2) # Append would preserve groupings by first letter
      if (prevar3 != []): intermediate.extend(prevar3)
      finvar = intermediate

    elif digit == '7':
      prevar1 = search('P', i)
      prevar2 = search('Q', i)
      prevar3 = search('R', i)
      prevar4 = search('S', i)
      if (prevar1 == []):
        if (prevar2 == []):
          if (prevar3 == []):
            if (prevar4 == []):
                if (i == 0): place = 'first'
                if (i == 1): place = 'second'
                if (i == 3): place = 'third'
                finvar = '***No name found that starts with an "P", "Q", "R", or "S" in the ' + place + ' letter.'
      if (prevar1 != []): intermediate.extend(prevar1)
      if (prevar2 != []): intermediate.extend(prevar2) # Append would preserve groupings by first letter
      if (prevar3 != []): intermediate.extend(prevar3)
      if (prevar4 != []): intermediate.extend(prevar4)
      finvar = intermediate

    elif digit == '8':
      prevar1 = search('T', i)
      prevar2 = search('U', i)
      prevar3 = search('V', i)
      if (prevar1 == []):
        if (prevar2 == []):
          if (prevar3 == []):
            if (i == 0): place = 'first'
            if (i == 1): place = 'second'
            if (i == 3): place = 'third'
            finvar = '***No name found that starts with an "T", "U", or "V" in the ' + place + ' letter.'
      if (prevar1 != []): intermediate.extend(prevar1)
      if (prevar2 != []): intermediate.extend(prevar2) # Append would preserve groupings by first letter
      if (prevar3 != []): intermediate.extend(prevar3)
      finvar = intermediate

    elif digit == '9':
      prevar1 = search('W', i)
      prevar2 = search('X', i)
      prevar3 = search('Y', i)
      prevar4 = search('Z', i)
      if (prevar1 == []):
        if (prevar2 == []):
          if (prevar3 == []):
            if (i == 0): place = 'first'
            if (i == 1): place = 'second'
            if (i == 3): place = 'third'
            finvar = '***No name found that starts with an "W", "X", "Y", or "Z" in the ' + place + ' letter.'
      if (prevar1 != []): intermediate.extend(prevar1)
      if (prevar2 != []): intermediate.extend(prevar2) # Append would preserve groupings by first letter
      if (prevar3 != []): intermediate.extend(prevar3)
      if (prevar4 != []): intermediate.extend(prevar4)
      finvar = intermediate

    else: print "You cannot enter letters.  You must enter numbers only."
    if finvar == []: finvar = "***No names were found."
    return finvar

  def search(letter, pos):  # Take two parameters a string of 0 to 2 characters and the letter it is searching for
    potential = []
    for line in book.splitlines():
      if (line[pos] == letter):
        potential.append(line)
    return potential

  def secondkeyreducer(shortlist, seckey): #should rename to getnames
    potentiala = []
    ee = len(shortlist)
    x = 0
    while (x < ee):
      name = shortlist[x] # name in shortlist:
      if name == '*': x = x + ee # get out of while loop if no names were found
      tt = eliminator(name, seckey, 1)
      if (tt == 'MARK'): potentiala.append(name)
      elif (tt == 'FAILED'): quit()
      else: print "" #"not adding a name because of 2nd or 3rd key"
      x = x + 1
    return potentiala

  def thirdkeyreducer(shortlist, thirdkey): #should rename to getnames
    potentiala = []
    ee = len(shortlist)
    x = 0
    while (x < ee):
      name = shortlist[x] # name in shortlist:
      tt = eliminator(name, thirdkey, 2)
      if (tt == 'MARK'): potentiala.append(name)
      elif (tt == 'FAILED'): quit()
      else: print "" #"not adding a name because of 2nd or 3rd key"
      x = x + 1
    if potentiala == []:
      print "No names were found by the spelling of that name."
      phone()    
    else: print "" # potentiala #print all potential matches
    return potentiala

  def eliminator(name, seckey, itera):
    val = 'REMOVE'
    if name != '*':
      if seckey == '0': val = 'FAILED'
      if seckey == '1': val = 'FAILED'
      if seckey == '2':
        if name[itera].upper() in ['A', 'B', 'C']: val = 'MARK'
      if seckey == '3':
        if name[itera].upper() in ['D', 'E', 'F']: val = 'MARK'
      if seckey == '4':
        if name[itera].upper() in ['G', 'H', 'I']: val = 'MARK'
      if seckey == '5':
        if name[itera].upper() in ['J', 'K', 'L']: val = 'MARK'
      if seckey == '6':
        if name[itera].upper() in ['M', 'N', 'O']: val = 'MARK'
      if seckey == '7':
        if name[itera].upper() in ['P', 'Q', 'R', 'S']: val = 'MARK'
      if seckey == '8':
        if name[itera].upper() in ['T', 'U', 'V']: val = 'MARK'
      if seckey == '9':
        if name[itera].upper() in ['W', 'X', 'Y', 'Z']: val = 'MARK'
    return val

  inivar = first[0]
  firstlist = presearch(inivar, 0)
  cc = len(firstlist)

  if len(first) < 2:
    print "you entered too few numbers"
    print "try again"
    phone()

  if len(first) > 3:
    print "you entered too many numbers"
    print "try again"
    phone()

  if first[0].isalpha():
    print "enter numbers only"
    print "Call back again."
    phone()

  if first[1].isalpha():
    print "enter numbers only"
    print "Call back again."
    phone()

  if len(first) == 3:
    if first[2].isalpha():
      print "enter numbers only"
      print "Call back again."
      phone()

  yt = first[1]
  secondlist = secondkeyreducer(firstlist, yt)

  if len(first) == 3:
     zt = first[2]
     thirdlist = thirdkeyreducer(secondlist, zt)
     return thirdlist
  else:
    return secondlist

toprint = phone()
print toprint 

How Do You Create a VM in Google Cloud Platform Using a Command Line or an API?

Problem scenario
You want to create a server in Google Cloud Platform using an API or command line.  How do you do this?

Solution
1.  Log into Google Cloud Platform.  
2.  Click the hamburger icon in the upper left hand corner (the icon with three horizontal bars stacked onto each other).
3.  Go to Compute Engine -> VM Instances
4.  Click "Create Instance"
5.  Fill out the options you want.  Under "Boot disk" click "Change."  Remember that Windows Core has no GUI desktop.  It is a command line version of Windows (with just a character prompt).  For a more traditional Windows server, you may want to use the Windows Server 2016 with the Desktop Experience.
6.  Beneath the "Create" button look for text that says "Equivalent REST or command line".  Click the "REST" or "command line" hyperlink.  For "REST" you will be given a REST request with JSON parameters.  You can modify these parameters as you see fit when you go to run the command.  For the "command line" option you will be given a "gcloud command line with parameters."  You can modify these parameters as you see fit when you go to run the command.
7.  Optional step:  Click Create.  (This creates the VM using GUI operations.)

When There Is A “connection refused” Message for Graylog, What Might Be the Problem?

Problem scenario
Using PowerShell you test TCP/IP connectivity over port 9000 to your Graylog instance.  You use these commands:

$tcp = New-Object System.Net.Sockets.TcpClient
$tcp.connect('x.x.x.x', '9000')

You get "Exception calling "Connect" with "2" argument(s): "N connection could be made because the target maching actively refused it."   

You know there is no firewall blocking port 9000.  Using nmap you test TCP/IP connectivity over port 9000 to your Graylog instance. You find the port is close and not blocked.  You try to use curl http://x.x.x.x:9000.  You get this message:  "curl: (7) Failed to connect to x.x.x.x port 9000: Connection refused"

What could be wrong?

Solution
You have insufficient processor power on the Docker host.  On the Docker host, run the "top" command.  If you see loads above 2, then you appear to have insufficient processors for deploying Graylog.  You need a server with significantly more processors for Graylog to run smoothly.

How Do You Troubleshoot an Empty Multi-node Hadoop Cluster?

Problem scenario
One or more of the following is happening:
   1) There are 0 DataNodes in your Hadoop cluster according to an error message
   2) There is 0 B configured as capacity (as shown from a "hdfs dfsadmin -report" command).
   3)  There is one fewer DataNode in your Hadoop cluster than you expect.
   4)  You run "hdfs dfsadmin -report | grep Hostname" and do not see a node that has its DataNode service (as seen with the jps command) started and stopped from the NameNode with corresponding start-dfs.sh and stop-dfs.sh script runs.

What should you do?

Possible solution
Warning: This will delete all data in your Hadoop cluster.

1.  Go to the node that should be a DataNode (it could be the NameNode itself), do the following three steps:
     1.1  Delete everything in /app/hadoop/tmp (including subdirectories).
     1.2  Are you ready to delete all the content/data in your cluster? If so, run this without the "#" mark: # hdfs namenode -format #all your data will be deleted permanently!!!
     1.3  bash /usr/local/hadoop/sbin/start-dfs.sh

If you are still having problems, you may want to view this posting.

How Do You Troubleshoot This Problem “Access denied for user ‘zabbix’@’localhost’ (using password: YES)”?

Problem scenario
The web UI for Zabbix shows the Zabbix server is not running.  In "Status of Zabbix" there is a parameter "Zabbix server is running" and it has a value of "No."  To investigate on the back-end of the Zabbix server you run this command: tail -f /var/log/zabbix/zabbix_server.log

You see this error: "Access denied for user 'zabbix'@'localhost' (using password: YES)"

What should you do?

Solution
Examine the zabbix_server.conf file.  Run this command:  sudo vi /etc/zabbix/zabbix_server.conf

Verify the DBName=, DBUser, and DBPassword stanzas.  Typos in these assignments or if one is commented out could cause the problem you encountered.  Beware of near matches like "zabbix" and not "zabbixdb" or "zabbixuser".  Incorrect info in this file could cause the error you are seeing.