CONTINUAL INTEGRATION – Page 224 – A Technical I.T./DevOps Blog

How Do You Troubleshoot a CentOS Server Not Being Able to Reach the Internet or outside Your Network?

Problem scenario: From a CentOS server, you try to ping an IP address outside of your network (e.g., 8.8.8.8). You get "connect: Network is unreachable" as the response. Your server has an IP address and subnet mask. What is wrong?

Solution
1. Ensure that the network has a default gateway. If the network equipment that connects the server to the network has no gateway to outside the network, these directions will not be able to help. If you know the server is on a network that has access to other networks (e.g., there is a router that can route traffic to the Internet), proceed to the next step.

2. If you are using CentOS, go to /etc/sysconfig/network-scripts/ and find the relevant interface file (e.g., ifcfg-enp0s3). If you are not sure of which file governs the interface for the server's default gateway, see these directions.

3. Ensure that this file as a gateway value. For example, a stanza with the word "GATEWAY=" is needed. Here is an example (but remember your IP address will likely be different)

GATEWAY=10.0.2.2

Ask your network administrator what the default gateway's IP address should be. If you have another server on the network, you could potentially find the default gateway of that server to learn what it is.

If you happen to be using Oracle VirtualBox on a Windows host, three potential IP addresses may work as a default gateway. This assumes that you have a CentOS 7.3 guest that has its Network Adapter configured to be attached to "NAT" with the "Cable Connected" option box being checked.

These three working default gateways can be 10.0.2.2, 10.0.2.3, and 10.0.2.4.

4. Once the GATEWAY stanza is configured properly in the interface file (in /etc/sysconfig/network-scripts/), run this command:

service network restart

Now you should be able to ping an IP address outside of your network (e.g., on the Internet).

A List of Apache Spark Books

99 Apache Spark Interview Questions for Professionals by Yogesh Kumar
Advanced Analytics with Spark: Patterns for Learning from Data at Scale by Juliet Hougland, Uri Laserson, Sean Owen, Sandy Ryza and Josh Wills
Apache Spark 2 for Beginners by Rajanarayanan Thottuvaikkatumana
Apache Spark in 24 Hours, Sams Teach Yourself by Jeffrey Aven
Apache Spark for Data Science Cookbook by Padma Priya Chitturi
Apache Spark for Java Developers by Sumit Kumar and Sourav Gulati
Apache Spark Graph Processing by Rindra Ramamonjison
Apache Spark Interview Question & Answers by Naman Goel (Editor)
Apache Spark Machine Learning Blueprints by Alex Liu
Apache Spark Machine Learning Cookbook by Siamak Amirghodsi
Apache Spark Scala Interview Questions: Shyam Mallesh by Shyam Mallesh
Big Data Analytics with Spark: A Practitioner's Guide to Using Spark for Large Scale Data Analysis by Mohammed Guller
Big Data: Principles and best practices of scalable realtime data systems by Nathan Marz and James Warren
Big Data SMACK: A Guide to Apache Spark, Mesos, Akka, Cassandra, and Kafka by Raul Estrada and Isaac Ruiz
Data Algorithms: Recipes for Scaling Up with Hadoop and Spark by Mahmoud Parsian
Deep Learning: A Practitioner's Approach by Josh Patterson and Adam Gibson
Fast Data Processing with Spark - Second Edition by Krishna Sankar and Holden Karau
High Performance Spark: Best Practices for Scaling and Optimizing Apache Spark by Holden Karau and Rachel Warren
Learning Apache Spark 2.0 by Asif Abbasi
Learning Real Time processing with Spark Streaming by Sumit Gupta
Learning Spark: Analytics With Spark Framework by Joseph Moore
Learn Spark In A DAY: The Ultimate Crash Course to Learning the Basics of Spark In No Time (Spark, Spark Course, Spark Development, Spark Books, Spark for Beginners) by Acodemy
Learning Spark: Lightning-Fast Big Data Analysis by Holden Karau, Andy Konwinski, Patrick Wendell and Matei Zaharia
Learning Spark Streaming: Best Practices for Scaling and Optimizing Apache Spark by Francois Garillot
Machine Learning with Spark - Tackle Big Data with Powerful Spark Machine Learning Algorithms by Nick Pentreath
Mastering Apache Spark by Mike Frampton
Mastering Apache Spark 2.0 - Second Edition by Romeo Kienzler
Pro Spark Streaming: The Zen of Real-Time Analytics Using Apache Spark by Zubair Nabi
Spark: Big Data Cluster Computing in Production by Ilya Ganelin, Ema Orhian, Kai Sasaki and Brennon York
Python Data Science Essentials - Second Edition by Alberto Boschetti and Luca Massaron
Spark Cookbook by Rishi Yadav
Spark for Data Science Cookbook by Padma Priya Chitturi
Spark for Python Developers by Amit Nandi
Spark GraphX in Action by Michael Malak and Robin East
Spark in Action by Petar Zecevic and Marko Bonaci
Spark Tutorials with Scala: The Beginner's Guide by Todd McGrath

How Do You Test If a Variable Is a String or a Numeric Data Type in Ruby?

Problem scenario
You want to test if a given variable is a Numeric or String. These are built-in data types that Ruby has.

What should you do?

Solution
Append ".is_a? Numeric" or ".is_a? String" to variable. For an example, here is code that explicitly prints out if the data type is a Numeric or String.

var1 = 123
an = var1.is_a? Numeric
as = var1.is_a? String
puts "var1 is a numeric: " + (an ? "true" : "false")
puts "var1 is a string: " + (as ? "true" : "false")
puts "---Now for a new variable---"
var2 = 'continual'
bn = var2.is_a? Numeric
bs = var2.is_a? String
puts "var2 is a numeric: " + (bn ? "true" : "false")
puts "var2 is a string: " + (bs ? "true" : "false")

nixCraft at cyberciti.biz Has a Number of Great Articles

Today there was a poignant posting about I.T. certifications. Consider donating to money when you go to cyberciti.biz or the nixCraft Patreon website. The content has helped many professionals over the years.

How Do You Configure Two Linux Guest Servers Running on Oracle VirtualBox to Have Network Connectivity with Each Other?

Problem scenario: On a Windows host you have Oracle VirtualBox. You want to establish TCP/IP connectivity with the two guest CentOS Linux virtual servers. When you try to ping one from the other, you get "destination host unreachable." The servers have Internet connectivity. How do you get them to communicate with each other via a networking protocol?

Solution
On each guest server, do the following:

1. Using the GUI of the Oracle VirtualBox guest server, go to Devices -> Network Settings.

a. Changed the drop down for "Attached to:" to be "Internal Network."
b. Make sure "Promiscuous Mode" is set to "Allow VMs" or "Allow all"
c. Click "OK."

2. Back up this file on both guest servers: /etc/sysconfig/network-scripts/ifcfg-enp0s3

3. Modify /etc/sysconfig/network-scripts/ifcfg-enp0s3

a) Make sure the "BOOTPROTO=" stanza is "BOOTPROTO=static"
b) This is just an example to get the two Linux guests to work. Other IP addresses may work.

For one guest, use the following entries underneath "BOOTPROTO=static":

    IPADDR=10.0.2.15
    NETMASK=255.255.255.0
    GATEWAY=10.0.2.2

For the other guest, use the following entries underneath "BOOTPROTO=static":

    IPADDR=10.0.2.17
    NETMASK=255.255.255.0
    GATEWAY=10.0.2.2

4. After you modify the file, issue this from the command line: service network restart
5. Now one guest should be able to ping or ssh to the other via the 10.0.2.1x IP address configured above.

Optional reading
How Do You Connect to a VM in Oracle VirtualBox from a Desktop?

How Do You Find the Amount of RAM Your Linux Server Has in GB?

Problem scenario
You want to find the total RAM a Linux server has. You do not want the number to be in kilobytes. You want human-readable output. What do you do?

Solution
Run this command:

free -g | grep Mem | awk '{print $2 " gigabytes of RAM total (sum of in use and available combined)"}'

If the amount of GB is less than 0.5, the result of the above command will show as 0 GB of memory. In that case, you can find out how many megabytes of memory the server has with this command:

free -m | grep Mem | awk '{print $2 " megabytes of RAM total (sum of in use and available combined)"}'

How Do You Install PostgreSQL on an EC2 SUSE (AWS Instance of Linux SUSE)?

Problem scenario
You want to install Postgres on Linux SUSE. This AWS server is in a security group with the ability to get to the Internet. How do you install PostgreSQL on it?

Solution
1. Log into the SUSE server with the default ec2-user. The default user for AWS SUSE instances used to be root. But now it is ec2-user.

2. Run these five commands (taken from https://wiki.postgresql.org/wiki/Zypper_Installation):

zypper addrepo -t YUM http://packages.2ndquadrant.com/postgresql-z-suse/zypper/sles-11sp3-s390x pg

zypper refresh

zypper addrepo -t YUM http://packages.2ndquadrant.com/postgresql-z-suse/zypper/sles-11sp3 pgsrc

zypper refresh

zypper install postgresql94-server

3. Now you are ready to start the Postgres service. Run this command:
/sbin/service postgresql start

4. You are done. For future knowledge, AWS security group rules have a "PostgreSQL" type that allow connections to servers over port 5432. If you are configuring a SQL front end from a Windows desktop to your SUSE instance, this rule in your security group is necessary.

How Do You Write a Backend PHP Script to Query Postgres and Create a Database Name “foobar”?

Problem scenario
You are running Ubuntu Linux, and you want to create a .php script to be ran at the command prompt like this:

php good.php

You want this to open a PosgreSQL connection and issue a DDL command to create a database. This is not a PHP web page.

Solution
This solution is only slightly more complicated than using PHP to run a basic SQL query in a Postgres database.

1. Go to the command prompt of PSQL. These commands may help you:

sudo -i -u postgres psql

2. Create a role from the psql command prompt. Here is an example:

create role orangeuser;

3. Alter the role to have the login trait and superuser attribute.

alter role orangeuser login superuser; \q

4. Install php5-pgsql or php7.0-pgsql. It depends on your version of PHP. Use this command to find out what version of PHP you have: php --version

To install the php pgsql package on Ubuntu, you should run a command like one of these as root:
apt-get install php5-pgsql apt-get install php7.0-pgsql

5. Create a .php file (e.g., cool.php) with the following as the content:

/* This assumes your server with PHP has access to the server with Postgres.  The IP address in the script is the local loopback.  You may have to change the IP address for this to work.  Naturally you will have to change the database name, the user and password in the PHP script to the credentials for your Postgres instance. */

<?php
   $host        = "host=127.0.0.1";
   $port        = "port=5432";
   $dbname      = "dbname=contint";
   $credentials = "user=jdoe password=softpassword";

   $db = pg_connect( "$host $port $dbname $credentials"  );
   pg_query("CREATE DATABASE foobar");

?>

6. Run it like this: php cool.php

If you want to write a front-end web page to execute a SQL command, see this posting:
How Can You Accept User Input with PHP and Use It in a SQL Command?

How Can PHP Accept Data from a Form and Assign It to Variables for Future Retrieval?

Problem scenario
You want a web page to request user input. After the user clicks "Submit" you want the values to be assigned to variables for future usage (e.g., an algorithm could retrieve the values to manipulate them). How do you use PHP to assign input from a web page to be variables on the back end?

Solution
You need two files. The first one will be a form to obtain data from an external source (user-entered data). You need a PHP web page like this (cool.php) to obtain data passed in:

<form action="contint.php" method="post">
Username: <input type="text" name="username" /><br />
Password: <input type="text" name="password" /><br />
<input type="submit" name="submit" value="Submit me!" />
</form>

//Change contint.php to the name of the second .php file you will use.

Here is the content of the second .php file you will use (e.g., contint.php)

<?php
if ($_POST) {
echo '<pre>';
$var1 = $_POST["username"];
$var2 = $_POST["password"];
echo '</pre>';
}
?>

//Now $var1 is the value the user entered in "Username."
//Now $var2 is the value the password entered in "Password."

How Do You Troubleshoot the PHP Error “role is not permitted to log in” So Your Script Can Log into a Postgres Database?

Problem scenario
You are trying to use a .php script on the back end to connect to your PostgreSQL database. However, you get an error like this when you try to run it:

'PHP Warning: pg_connect(): Unable to connect to PostgreSQL server: FATAL: role "funuser" is not permitted to log in in /tmp/d.php on line 7'

How can your .php script log into the Postgres database?

Solution
The user must have the ability to log in. Log into the PostgreSQL database. Run this command (where "funuser" is the database role that is hardcoded in the PHP script for the login to work):

alter role funuser login;