What Is The Difference between a VPC and a Subnet?

Question
You have heard about VPCs and subnets. What are the differences between these two?

Answer
A subnet is a subdivision of a network. Internal to an enterprise network, you may divide the network into different addresses with gateways, routers, bridges and subnet masks. The subnet mask (in CIDR (classless inter-domain routing) notation may be /28, /24, /10 etc.

A VPC ordinarily has its own NACLs and firewalls associated with it; these are usually part of the public cloud's built-in features. For AWS the firewall is called a Security Group. In AWS you can configure a VPC's availability zone, give the VPC a name, add service endpoints, add route tables, configure peering connections (with other VPCs) and select a hardware tenancy type (either shared or dedicated). In AWS a VPC will have a public subnet. VPCs also exist in IBM's public cloud, Azure, and GCP.

How Do You Write a Python Program without the import Statement That Palindromically Tests a String in Four Lines of Code?

Problem scenario
You want to avoid importing any modules in Python. You want to have a way to test if a string is a palindrome is four lines of code. What do you do?

Solution

def is_palindrome_pythonic(s):
    return all(a == b for a, b, in zip(
        map(str.lower, filter(str.isalnum, s)),
        map(str.lower, filter(str.isalnum, reversed(s)))))

# The above four lines of code were taken from page 78 of Elements of Programming Interviews in Python.

print(is_palindrome_pythonic("aaaabaabaa"))

To buy the cited book, click here.

What Are Guardrails in I.T.?

Question
What are guardrails in the I.T. industry?

Answer
A measure, technology or authentication policy in place to prevent problems in an enterprise network or with a application. With guardrails, a server or web application is protected from hackers or accidental misuse. It is a common metaphor based on highway guardrails. To learn more, see this posting:
https://devops.com/building-great-cloud-security-guardrails/

There is a specific product called GUARDRAILS that detects, fixes and prevents "security issues in your applications, open source dependencies and cloud platforms." taken from https://docs.guardrails.io/docs/en/what-is-guardrails

How Do You Use Python to Retrieve Information about Bitcoin Transactions?

Problem scenario
You want to use Python to get data from BTC transactions. You do not want to use "import requests". What do you do?

Solution
Prerequisite
This assumes you have installed the Bitcoin pypi package: pip install bitcoin
If you need assistance installing pip, see this posting for Ubuntu or this posting for a Red Hat derivative. (You may need to use pip3 instead of pip.)

Procedures

  1. Enter the Python command line (with running the "python" or "python3" command).
  2. Run these two commands (but substitute the "3N8PfUXqXZzzFDxoPPgRiaiepziXpvYrpb" with the Bitcoin address of your choice**):
from bitcoin import *
history("3N8PfUXqXZzzFDxoPPgRiaiepziXpvYrpb")

** To find more Bitcoin addresses, go here and look for hyperlinks of similar alphanumeric strings:
https://www.blockchain.com/btc/address/3N8PfUXqXZzzFDxoPPgRiaiepziXpvYrpb

How Do You Use an Amazon API Gateway?

Problem scenario
You want to use the AWS PaaS offering for exposing an API and having it be able to scale and handle lots of users easily. How do you use Amazon API Gateway?

Solution
Overview: This is a proof-of-concept. You could use different API types (e.g., web socket APIs for persistent applications like a chat window).

Prerequisite
This assumes you have a working Lambda function configured and you know its name.

Procedures

  1. Log into the AWS Console.
  2. Go here: https://console.aws.amazon.com/apigateway/main/
  3. Go to HTTP API and click on "Build".
  4. For "Integrations", choose "Lambda". Find your desired lambda for the "Lambda function" drop-down menu.
  5. For "API name", enter whatever name you want. Click "Next".
  6. For "Configure routes", you may want to accept the defaults and click "Next".
  7. For "Define stages", you may want to keep the defaults and click "Next".
  8. Review and click "Create".
  9. Copy the "Invoke URL" hyperlink.
  10. Copy it into a Notepad or text edit (e.g., https://abcd1234.execute-aps.us-west-2.amazonaws.com/ ). Append the Lambda's name to this URL (e.g., append "foobar" to the URL).
  11. Open a web browser and go to the URL that you constructed in step #10. (The output in the browser should match what happens when you test the Lambada via the "Test" tab's "Invoke" button. When you click "Invoke" in the web console, that should show you a result in the AWS Console; you may need to expand "Details" to see it completely.)

How Do You Troubleshoot Discrepant nmap Results Based on The sudo Command?

Problem scenario
You are using nmap. You try the exact same command with sudo and without sudo. Why are these commands providing different results?

Here are examples of running the exact same command, once with "sudo" and once without:

jdoe@ciserver:~$ sudo nmap -p 9090 192.168.2.25
Starting Nmap 7.80 ( https://nmap.org ) at 2022-01-05 09:56 EST
Nmap scan report for 192.168.2.25
Host is up (0.00056s latency).

PORT     STATE  SERVICE
9090/tcp closed zeus-admin
MAC Address: 08:00:27:32:27:2C (Oracle VirtualBox virtual NIC)

Nmap done: 1 IP address (1 host up) scanned in 0.25 seconds
jdoe@ciserver:~$ nmap -p 9090 192.168.2.25
Starting Nmap 7.80 ( https://nmap.org ) at 2022-01-05 09:56 EST
Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 0.07 seconds

Why is there a discrepancy in the output?

Solution/Answer
Use the -vv flag and you will learn that the difference is that with sudo the nmap command uses an "ARP Ping Scan" as opposed to a regular "Ping Scan." To learn why these are so different, see https://nmap.org/book/man-host-discovery.html.

See also this posting or Possible Solution #4 and the "**" at the end of this posting.

How Do You Use Nomics to a Retrieve Cryptocurrency Prices?

Problem scenario
You want to use Nomics to retrieve cryptocurrency prices. What do you do?

Prerequisite
Get an API key. It is free here: https://p.nomics.com/pricing#free-plan
(You do not need to provided credit card information to obtain one.)

Procedures
Run this Python program but replace YOUR_KEY_HERE with the key:

import requests
resp = requests.get("https://api.nomics.com/v1/prices?key=YOUR_KEY_HERE&format=json")
for item in resp.json():
if item['currency'] in ["BTC", "ETH", "XRP", "LTC", "XLM", "XMR"]:
print(item)

If you want to earn cryptocurrency by just learning more, try Coinbase; it is ideal for Americans. For Europeans, this company Iconomi.com can allow you to buy crypto or learn more.

How Does an ssh Command Work in Detail?

Question
You want an overview of how SSH works because it seems confusing. You want to comprehend how it works or know exactly what happens when you run an SSH command like ssh jdoe@acme.com. Where can you find a precise explanation on using ssh as well as some background information on SSH?

Solution
SSH is named after the secure shell way of remotely connecting to another server using encryption. You can use ssh commands to enter a shell or remotely execute ad hoc commands. The "ssh" command, and the tools that leverage SSH such as scp and sftp, come from the OpenSSH suite (https://www.openssh.com/). SSH supports GUI forwarding via X11 (according to this external posting).

When an SSH command is executed on a client, a binary file is found (often in /usr/bin/ssh) because of the $PATH environment variable. Then a TCP/IP request is made involving a hostname or an IP address. Routing happens starting from the client, and a request is ultimately made on the server with the address provided. This connection happens over port 22 by default; it can be configured to use a different IP TCP port number.

When this happens, the known_hosts file (normally in .ssh) on the server will look at the "fingerprint" of the client server. If the fingerprint is recognized, then authentication will proceed to happen as normal. If the fingerprint is not recognized, the client will be prompted to continue or abort the connection process. The fingerprint is a string of colon-separated hexadecimal pairs created when the public key is generated. This process helps the client from logging into a different server accidentally.

To see the details of an SSH operation, you can use the --v, --vv, or --vvv flags when you run the SSH command. Once the connection has been established, the user on the server can be different from the user on the client.

The identity file (aka the private key) is often in the .ssh directory of the user of the client. It often is named id_rsa. If it was configured with a passphrase*, the user on the client side will need to enter a passphrase interactively to authenticate to the server. In some configurations of SSH, only the keys need to authenticate and no password is needed. In other configurations of SSH, only a password needs to be entered correctly and no special file on the client side needs to be configured. The ssh-keygen command can create identity files; this utility comes with the OpenSSH suite. With SSH, everything is encrypted between the client and the server including the passphrase for the authentication process. However SSH can work without a passphrase.

To improve the speed of SSH (e.g., for Ansible or Hadoop), you could potentially do five things with the ssh_config file. We highly recommend you back up this file before modifying it! You could not use IPv6; you could turn off DNS lookups on the server of an SSH connection; you could reuse an existing SSH connection on the client; and you could use passwordless authentication. (The first four optimization techniques came from https://www.tecmint.com/speed-up-ssh-connections-in-linux/.)

You could also use the -o flag to override the ssh_config file and pass various directives to speed up, on an ad hoc basis, SSH commands. Finally you could eliminate SSH logging by changing the sshd_config file on the server. The ssh_config and/or sshd_config files can be found in the .ssh directory.

On the subject of optimizing the performance of SSH, encryption and decryption via SSH utilize CPU (according to an external site and other postings on the internet). (It is rare that the bottleneck is with the CPU and not the network in the context of optimizing SSH according to this external posting.)

The ssh-keygen command can generate a private and public key pair. Normally an id_rsa.pub will be the public key. The RSA stands for Rivest Shamir Adelman, a specific type of asymmetric cryptography that SSH supports; there are other types that SSH supports too.

The session key in SSH is symmetric, but it only lasts for the session itself. To read more, see this medium.com article.

The logging of the SSH process will normally happen in /var/log/auth.log for Ubuntu and Debian distributions of Linux. For CentOS/RHEL/Fedora, the file would be /var/log/secure.

SSH does use a three way handshake (SYN from the client to the server, SYN-ACK from the server to the client, and ACK from the client back to the server). To read more about this, see: https://www.inetdaemon.com/tutorials/internet/tcp/3-way_handshake.shtml

To harden SSH, ensure you are using a software firewall or have protection from a hardware firewall. Use the hosts.deny file to not allow connections from wide ranges of IP addresses. If you want tips on hardening a server in general, see this posting.

Use "man ssh" or "man ssh_config" for more information.

For even more information about the details of SSH see these three postings:

See also this book SSH, The Secure Shell.

For troubleshooting SSH, you may want to see these postings:

*A passphrase can accept spaces whereas a password may not accept them.

How Do You Retrieve Cryptocurrency Data for One Day in the Past Using Python?

Problem scenario
You want to find the historic data for a given cryptocurrency using Python. You want to print out the low, high, open and close prices on a specific day. How do you get data for one specific date?

Solution

Prerequisite
This assumes you have installed pandas: pip install pandas

Procedures
Run this program:

"""
Usage instructions: Change the "2016-12-25" the value (assigned to "date_to_see") to the day you want to view.
Change "BTC" to the symbol of the cryptocurrency you want to examine.
You may or may note want to change "USD" or "Coinbase"

Written in March of 2021 by www.continualintegration.com

It was adapted from these postings:
https://towardsdatascience.com/cryptocurrency-analysis-with-python-buy-and-hold-c3b0bc164ffa
https://towardsdatascience.com/cryptocurrency-analysis-with-python-macd-452ceb251d7c?gi=12b983021ac2

"""


import pandas as pd
import requests
from datetime import datetime

from_symbol = 'BTC'
to_symbol = 'USD'
exchange = 'Coinbase'      # "Bitstamp" and "Coinbase" are valid among others.
datetime_interval = 'day'
date_to_see="2016-12-25"

def get_filename(from_symbol, to_symbol, exchange, datetime_interval, download_date):
    return '%s_%s_%s_%s_%s.csv' % (from_symbol, to_symbol, exchange, datetime_interval, download_date)

def download_data(from_symbol, to_symbol, exchange, datetime_interval):
    supported_intervals = {'minute', 'hour', 'day'}
    print('Downloading %s trading data for %s %s from %s' %
          (datetime_interval, from_symbol, to_symbol, exchange))
    base_url = 'https://min-api.cryptocompare.com/data/histo'
    url = '%s%s' % (base_url, datetime_interval)    
    params = {'fsym': from_symbol, 'tsym': to_symbol,
              'limit': 2000, 'aggregate': 1,
              'e': exchange}
    request = requests.get(url, params=params)
    data = request.json()
    return data

def convert_to_dataframe(data):
    df = pd.json_normalize(data, ['Data'])
    df['datetime'] = pd.to_datetime(df.time, unit='s')
    df = df[['datetime', 'low', 'high', 'open',
             'close', 'volumefrom', 'volumeto']]
    return df

def filter_empty_datapoints(df):
    indices = df[df.sum(axis=1) == 0].index
    print('Filtering %d empty datapoints' % indices.shape[0])
    df = df.drop(indices)
    return df

def file_creator(from_symbol, to_symbol, exchange, datetime_interval):
    data = download_data(from_symbol, to_symbol, exchange, datetime_interval)
    df = convert_to_dataframe(data)
    df = filter_empty_datapoints(df)
    latest_date = datetime.now().date().isoformat() 
    filename = get_filename(from_symbol, to_symbol, exchange, datetime_interval, latest_date)
    print('The oldest day in the range obtained for ' + from_symbol + ' was ' + str(df['datetime'].min()))
    df.to_csv(filename, index=False)


def get_filename(from_symbol, to_symbol, exchange, datetime_interval, download_date):
    return '%s_%s_%s_%s_%s.csv' % (from_symbol, to_symbol, exchange, datetime_interval, download_date)

def read_dataset(filename):
    df = pd.read_csv(filename)
    df.datetime = pd.to_datetime(df.datetime) # change to datetime
    df = df.set_index('datetime') 
    df = df.sort_index() # sort by datetime
    return df


file_creator(from_symbol, to_symbol, exchange, datetime_interval)

latest_date = datetime.now().date().isoformat() 
df_btc = read_dataset(get_filename(from_symbol, to_symbol, exchange, datetime_interval, latest_date))
btc_date = open((get_filename(from_symbol, to_symbol, exchange, datetime_interval, latest_date)), 'r')
get_top = btc_date.read()[0:-1]


print("This prints out data for one cryptocurrency on one day.  It downloads a file first.")
print(" ")
print("Here is data for " + from_symbol + " for " + date_to_see + ":")

print(get_top.split('\n')[0])
for line in get_top.split():
    if line.split(",")[0] == date_to_see:
        print(line)

For aggregate data over a date range, see this posting.

If you sign up for Coinbase for the first time, you often can get up to $200 in free coins; it is ideal for Americans. For Europeans, the platform/company Iconomi.com can allow you to buy crypto or learn more.