A Technical I.T./DevOps Blog
We worked very hard to write an original twelve-question Big Data Quiz. Please do not expect another posting (like we normally post) for several days. Please be sure to check out the Big Data Quiz!
…
Continue reading “Announcement: Big Data Quiz Now Available!”
Problem scenario
An IP address is pingable from one Linux server. On this server there are no Docker containers running. A traceroute reveals that this IP address is one hop away from your server. How do you find out what this IP address belongs to?
Solution
Use this command: ip addr show
You may also want to use this command:
…
Continue reading “How Do You Find an IP Address That Is Mysterious and Elusive in Your Network?”
Big Data Quiz
1. What does EDH stand for?
a. Enterprise Data Hub
b. Extract Develop Hadoop
c. Extract Decide Haul
d. Extract Data Hadoop
2. Gartner, Informatica and MapR think “data lakes” should be referred to as what?
a. data warehouses
b. data dams
c. data mills
d. data reservoirs
3. MapReduce is to Hadoop as ___________ is to Spark
a.
…
Problem scenario
You have Ansible installed as a control server (a centralized server to push down configurations to other servers). You want to use it to manage another server. How do you configure the other server to be a managed node? In other words, how do you configure Ansible to push configuration changes down to servers?
Solution
Prerequisite
Ansible must be installed.
…
Continue reading “How Do You Set up Ansible for the First Time?”
Problem scenario
You want to install Jenkins on an Ubuntu Linux server without running an “apt-get upgrade” command. (You are ok with installing a “apt-transport-https” package.) What should you do?
Solution
See this posting because the directions work for Ubuntu in AWS and Debian in GCP.
(If you can run “apt-get upgrade”, and you do not want to install “apt-transport-https”,
…
Continue reading “How Do You Install Jenkins 2.x on Ubuntu Linux in AWS?”
Problem scenario
You run spark-shell in a Debian distribution of Linux (e.g., Ubuntu) but you receive this error:
Error: A JNI error has occurred, please check your installation and try again
Exception in thread “main” java.lang.ArrayIndexOutOfBoundsException: 64
at java.util.jar.JarFile.match(java.base@9-internal/JarFile.java:983)
at java.util.jar.JarFile.checkForSpecialAttributes(java.base@9-internal/JarFile.java:1017)
at java.util.jar.JarFile.isMultiRelease(java.base@9-internal/JarFile.java:399)
at java.util.jar.JarFile.getEntry(java.base@9-internal/JarFile.java:524)
at java.util.jar.JarFile.getJarEntry(java.base@9-internal/JarFile.java:480)
at jdk.internal.util.jar.JarIndex.getJarIndex(java.base@9-internal/JarIndex.java:114)
at jdk.internal.loader.URLClassPath$JarLoader$1.run(java.base@9-internal/URLClassPath.java:640)
at jdk.internal.loader.URLClassPath$JarLoader$1.run(java.base@9-internal/URLClassPath.java:632)
at java.security.AccessController.doPrivileged(java.base@9-internal/Native Method)
at jdk.internal.loader.URLClassPath$JarLoader.ensureOpen(java.base@9-internal/URLClassPath.java:631)
at jdk.internal.loader.URLClassPath$JarLoader.<init(java.base@9-internal/URLClassPath.java:606)
at jdk.internal.loader.URLClassPath$3.run(java.base@9-internal/URLClassPath.java:386)
at jdk.internal.loader.URLClassPath$3.run(java.base@9-internal/URLClassPath.java:376)
at java.security.AccessController.doPrivileged(java.base@9-internal/Native Method)
at jdk.internal.loader.URLClassPath.getLoader(java.base@9-internal/URLClassPath.java:375)
at jdk.internal.loader.URLClassPath.getLoader(java.base@9-internal/URLClassPath.java:352)
at jdk.internal.loader.URLClassPath.getResource(java.base@9-internal/URLClassPath.java:218)
at jdk.internal.loader.BuiltinClassLoader$3.run(java.base@9-internal/BuiltinClassLoader.java:463)
at jdk.internal.loader.BuiltinClassLoader$3.run(java.base@9-internal/BuiltinClassLoader.java:460)
at java.security.AccessController.doPrivileged(java.base@9-internal/Native Method)
at jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(java.base@9-internal/BuiltinClassLoader.java:459)
at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(java.base@9-internal/BuiltinClassLoader.java:406)
at jdk.internal.loader.BuiltinClassLoader.loadClass(java.base@9-internal/BuiltinClassLoader.java:364)
at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(java.base@9-internal/ClassLoaders.java:184)
at java.lang.ClassLoader.loadClass(java.base@9-internal/ClassLoader.java:419)
at sun.launcher.LauncherHelper.loadMainClass(java.base@9-internal/LauncherHelper.java:585)
at sun.launcher.LauncherHelper.checkAndLoadMain(java.base@9-internal/LauncherHelper.java:497)
How do you solve this?
…
Continue reading “How Do You Troubleshoot the Spark-Shell Error “A JNI error has occurred”?”
Problem scenario
You want a generic script that can install open source Apache Spark on Debian/Ubuntu, CentOS/RedHat/Fedora or SUSE distributions of Linux. How do you do this?
Solution
1. Create a script such as this in /tmp/ (e.g., /tmp/spark.sh).
#!/bin/bash
# Written by www.continualintegration.com
sparkversion=2.2.1 # Change this version as necessary
distro=$(cat /etc/*-release | grep NAME)
debflag=$(echo $distro | grep -i “ubuntu”)
if [ -z “$debflag” ]
then
…
Continue reading “How Do You Install Apache Spark on Any Type of Linux?”
Problem scenario
You have Debian/Ubuntu, RedHat (including CentOS and Fedora), and SUSE distributions of Linux. You want to install OSSEC on each server (to protect them with host-based intrusion detection systems, IDSes). You want to use the same script to install OSSEC on each server. How do you do this?
Solution
1. Create a script such as this /tmp/ossec.sh.
#!/bin/bash
# Written by www.continualintegration.com
ossecversion=3.1.0
…
Continue reading “How Do You Install OSSEC on Any Type of Linux?”
Problem scenario
You want a generic script that can install open source Apache Solr on Debian/Ubuntu, CentOS/RedHat/Fedora or SUSE distributions of Linux. How do you do this with the same bash script?
Solution
1. Create a script such as this in /tmp/ (e.g., /tmp/solr.sh).
#!/bin/bash
# Written by www.continualintegration.com
solrversion=7.2.1 # Change this version as necessary
distro=$(cat /etc/*-release | grep NAME)
debflag=$(echo $distro | grep -i “ubuntu”)
if [ -z “$debflag” ]
then
…
Continue reading “How Do You Install Apache Solr on Any Type of Linux?”
Problem scenario
You have GitLab and Jenkins set up already. You want to use a GitLab API token in Jenkins. How do you create a GitLab API token?
Solution
1. Log into the GitLab web UI. (If you need directions on setting up GitLab, see this posting.)
2. In the upper righthand corner, go to the circle icon next to an icon of a down arrow.
…