How Do You Troubleshoot the Python Error “json.decoder.JSONDecode..”Expecting property name enclosed in double quotes…”?

Problem scenario
You are running a Python program that uses "import json". You get this error: "json.decoder.JSONDecode.."Expecting property name enclosed in double quotes…" You are not allowed to use bson, but you can use other Python packages. What should you do?

Solution
This solution only works if you can eliminate single quotes in the content to be serialized (or translated into JSON). This will get rid of single quotes indiscriminately.

Where "foobar" is the variable with the string you want to turn into JSON, do this:

import re, json
modifiedfoobar = foobar.replace("'", '"')   # (1)
latestagefoobar = json.dumps(modifiedfoobar)
finalfoobar = json.loads(latestagefoobar)

# (1) this eliminates single quotes. This may cause you problems if single quotes were part of the necessary content.
# If the string had single quotes instead of double quotes, this will save you from installing/importing bson.
# Often double quotes can be used instead of single quotes, but not always. If they are not interchangeable cannot, you may need bson.

How Do You Copy a File into an S3 Bucket using the AWS CLI?

Problem scenario
How do you copy a file from a Linux server with AWS CLI to an S3 bucket?

Solution
Draft a command like this but replace /tmp/good.txt with the path and name of the source file you are copying, and replace "foobar" with the name of the s3 bucket you are copying to (as the destination):

aws s3 cp /tmp/good.txt s3://foobar/

Run the command.

How Is Traffic Routed when a Browser Goes to a URL and the Website is Powered by a Kubernetes Cluster?

Question
You have been asked to explain how an external request for a website is routed and ultimately fulfilled by a pod in Kubernetes via an ingress controller. When a web browser downloads a website via HTTP and the website is running from a Kubernetes cluster, how does an individual container provide the HTML and/or data? How is external traffic routed to an underlying pod in a Kubernetes Cluster?

Answer
Assuming that external traffic can reach an ingress controller in Kubernetes, the short version of the answer may be the way ingress controllers are configured. (The long answer, or any answer, should account for variation in the process as there is not one way traffic is routed to a pod in a Kubernetes cluster.)

There are three types of HTTP routing: host-based, path-based and header-based (according to https://dzone.com/articles/the-three-http-routing-patterns-you-should-know). We think header-based routing with Kubernetes is exceptionally rare based on this SO posting. If you use an ALB with Kubernetes, you can use host-based routing (according to this Amazon posting).

A browser will make a GET request for a URL. The data will be retrieved via a DNS server based on the top-level domain (e.g., .com of the URL). The request will be routed to the appropriate domain (e.g., continualintegration). The next intermediate step would be to land on a load balancer or reverse proxy.*

External traffic will then go to either an Ingress or a Service of Kubernetes. Services pass the traffic to an Endpoints -- not to pods directly (according to page of 325 of Kubernetes in Action). (Technically the Endpoints resource should be plural.) Endpoints are separate resources (as opposed to being subcomponents of Services) according to page 133 of Kubernetes in Action.**

Ingress resources may use services without forwarding the traffic to the Service; Ingress controllers select pods to fulfill HTTP requests (according to page 145 of Kubernetes in Action). Ingress controllers can expose external traffic to underlying Kubernetes services themselves or send external traffic directly to pods and bypass Services (pages 143 through 146 of Kubernetes in Action and this). (An ambassador can also send traffic directly to pods and bypass kube-proxy on a worker node according to this external site.) Kubernetes Services can be available for external traffic without an Ingress forwarding such traffic (page 140 of Kubernetes in Action). To read about the advantages of a Service over an Ingress resource, you may want to view this posting.

Ingress controllers can be externally accessible with a public IP address (without a load balancer or reverse proxy). DNS can facilitate the resolution of an FQDN. During a request to a website over HTTP, once the request gets to the ingress controller, the traffic will be directed to a service either via a label selector or an IP address.

Assuming ingress controllers are used in Kubernetes, and assuming there was YAML in their creation, the "rules" stanzas in the "spec" section of the YAML will designate host stanzas (for routing to subdomains) and corresponding specific paths which map to backend services (a list of one or more) that are the target "services" that will fulfill the request.

Here is a YAML example for an ingress controller with label selectors that will route traffic destined for certain hosts (either continualintegration.foobar.com or weird.continualintegration.com) to corresponding services (either service1 or service2).

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-host
spec:
  rules:
  - host: "continualintegration.foobar.com"
    http:
      paths:
      - pathType: Prefix
        path: "/bar"
        backend:
          service:
            name: service1
            port:
              number: 80
  - host: "weird.contintualintegration.com"
    http:
      paths:
      - pathType: Prefix
        path: "/foo"
        backend:
          service:
            name: service2
            port:
              number: 80

Services can either have labels or IP addresses. It is possible to create a Service with no valid Endpoints resource (as explained here), but it would likely not be useful. Endpoints are separate Kubernetes resources (according to page 133 of Kubernetes in Action). Endpoints can be created manually if you do not create Services with labels. Routes to Endpoints can be done via IP addresses or label selectors that a Kubernetes service is configured with (pages 102 and 105 of Kubernetes Patterns by Bilgin Ibryam and Roland Huß (O'Reilly). Copyright 2019 Bilgin Ibryam and Roland Huß, 978-1-492-05028-5). Endpoints define what will be exposed in a pod (according to the front cover of Kubernetes in Action).

Internal networking in a Kubernetes cluster (e.g., among the worker nodes) is done via a Container Network Interface plugin (as opposed to Network Address Translation).

Load balancers can send traffic to different endpoints; load balancers require hardware to implement (according to page 114 of Kubernetes Patterns by Bilgin Ibryam and Roland Huß (O'Reilly). Copyright 2019 Bilgin Ibryam and Roland Huß, 978-1-492-05028-5). Pods can become available or unavailable based on readiness probes. kube-dns and kube-proxy are components that play a role in internal traffic routing inside a Kubernetes cluster. kube-proxy on each worker node helps balance the traffic load among the containers providing a given service (page 21 of Kubernetes in Action).

To read more about networking and Kubernetes, see this external page or this Dzone page. To learn more about how internal Kubernetes traffic, see this posting. For networking specific to GKE specifically see this Google documentation posting.

Further reading:

If Kubernetes is using a service mesh, then many aspects of the routing may be very different; see this posting for more details. You may also want to see these postings: Itnext.io or blog.getambassador.io.

To learn more about how a workstation connects to a website so a user can browse it, see this posting.

You may want to read these external pages for greater knowledge:

If you are asked in an interview "how does Kubernetes networking work?", we think you should mention three things: 1) kube-proxy on the worker nodes plays a big role for load distribution and/or routing traffic to Pods 2) the YAML file for creating the Pods will have a rules/spec stanzas will have a "paths:" section (for text like /foo or /foo/bar) and this ensures the paths of the URL route to different URLs 3) a CNI plugin does internal networking for the cluster.

* If no FQDN is used, there will be no DNS resolution, but the web browser may download a page. If an IP address is used in the address bar of the web browser, the IP address via a router's routing tables will be looked up. Naturally a web request involves typing in a URL (or IP address) on a web browser. This request will be resolved by a TLD (e.g., if the URL had a .com, .org, or .gov in it). The HTTP request via the originating web browser after it gets to the TLD is then routed to a website (possibly a reverse proxy). The resolution could happen via Route53 (if the DNS name was registered in AWS) to an ELB. The reverse proxy or ELB in these previous sentences could direct the traffic to an ingress controller created in Kubernetes. Ultimately the packets from an underlying pod would be sent back to the requesting web browser. After the routing happens packets will be assembled in the web browser. When refreshing a web browser (as opposed to issuing a curl), you will normally get the web page from the same exact pod every time because "Services work at the connection level" (according to page 140 of Kubernetes in Action).

** One type of Service that routes traffic is NodePort. This method bypasses security features of Kubernetes (according to inext.io website). NodePort is best for non-HTTP traffic.

What is the Document Object Model?

Question
You have read about the DOM. What is the Document Object Model and what do the three words mean in this phrase?

Answer
"The Document Object Model (DOM) is an abstract representation of an HTML document that can be queried and manipulated through its API." This quote was taken from page 62 of The Web Application Hacker's Handbook 2nd Edition.

An HTML document is "a file containing hypertext markup language" (taken from 0:07 of the video here).

The word object is an instance of a class (1). The word model refers to the combination of the structure and the behavior of the object (1). The structure and behavior of the object are not persisted via the DOM (2).

(1) 'The name "Document Object Model" was chosen because it is an "object model" in the traditional object oriented design sense: documents are modeled using objects, and the model encompasses not only the structure of a document, but also the behavior of a document and the objects of which it is composed. In other words, the nodes in the above diagram do not represent a data structure, they represent objects, which have functions and identity.' This quote was taken from this W3.org web page.

(2) 'The Document Object Model is not a way of persisting objects to XML or HTML. Instead of specifying how objects may be represented in XML, the DOM specifies how XML and HTML documents are represented as objects, so that they may be used in object oriented programs.' This quote was taken from W3.org web page.

How Do You Troubleshoot “ssh connect to host x.x.x.x port 22: No route to host” on Oracle VirtualBox on a VirtualBox VM?

Problem scenario
You are using Oracle VirtualBox. You get “ssh connect to host x.x.x.x port 22: No route to host”. What do you do?

Solution
In Oracle VirtualBox, go to Devices -> Network -> Network Settings. Is it attached to “Internal Network”? Try attaching it to “Bridged Adapter” instead.

How Do You Troubleshoot the Python Program Problem “FileNotFoundError: [Errno 2] No such file or directory”?

Problem scenario
Your Python program is trying to invoke a Bash command. But you get an error like this: "FileNotFoundError: [Errno 2] No such file or directory". What should you do?

Possible Solution #1 Use the shell=True command if your environment is secure. This is not a recommended practice from a security perspective. We have found that this with shell=True syntax can make the error go away:

subprocess.Popen(variableofcommand, shell=True)

Only do the above if you know it is ok with the systems administrator and/or programmers. Some businesses would not allow the above.

Possible Solution #2 We have found that this can happen with the Bash command is composed in a variable. If you hard code the Bash command in the subprocess.Popen(foobar) syntax (where foobar is where you place the bash command with words enclosed with double quotes and followed by commas), the command may work.

Possible Solution #3 The environment variables that the Python program may be different from what you expect. As an intermediate troubleshooting step, you may want the program to invoke something like this /bin/env > /tmp/results.txt (provided that you have no sensitive data in the env environment variable such that the /tmp/ directory should not have). You may find that you need to use full paths to the executables of the Bash command.

Possible Solution #4 Use a Python SDK instead of the subprocess/Popen/Bash commands. Sometimes what you are trying to do will have an SDK associated with it. This option is not always feasible however; sometimes there are no SDKs for what you are trying to do.

Possible Solution #5 If you use the absolute path to the command (e.g., /usr/bin/nmap) and keep getting the error, a space in the arguments can cause the problem. The solution is to put the variable of the command into tokens in a list like this:

This format could cause the problem:
comm_to_beex = ["ls -lh"]

This could fix the problem:
comm_to_beex = ["ls", "-lh"]

subprocess.run(commtobex, stdout=subprocess.PIPE, text=True)

For running Bash commands with Python 3, this external page has more information.

If you want to purchase a Python book, see this page.

How Do You Get an Android Phone to Filter Ring Notifications Based on the Calling Number?

Problem scenario
You only want your Android phone to ring if certain phone numbers call you. You have "blocked unknown callers" on your Android phone with certain settings. You went to the Phone app -> three vertical dots -> Settings -> Block numbers -> Block unknown callers

But you still get calls, or your phone still rings, when unknown callers call you. You tried rebooted or using Hiya's caller ID and spam protection. Nothing seems to work to keep. How do you get your phone to remain silent except for callers from certain numbers?

Solution

  1. Go to your phone's settings (usually a sprocket icon in the upper right hand corner). (You may find the Settings app in other locations).
  2. Go to "Notifications"
  3. Toggle "Do not disturb" if it was not already on.
  4. Click on "Do not disturb" and go to "Allow exceptions".
  5. Make sure "Alarms" and "Media sound" are on.
  6. Click "Calls from" near the top and choose "Favorite contacts only".
  7. Then go to your contacts. If you want to receive a notification from a given caller add this contact's phone number. Then click the yellow star icon for this contact. (This will designate the number as a favorite contact.) Now only calls from your favorite contacts (contacts with a yellow star), will make your phone ring. (The favorites will show up at the top of the Contacts section.)

What Does RDS Mean in the Context of Databases?

Question
You have seen the initialism RDS. What does this term mean?

Answer
Normally it is a reference to AWS' PaaS offering for relational databases called Amazon Relational Database Service.

The term RDS can refer to Reference Data Services in the Microsoft realm (e.g., Azure or SQL Server). Here is more information.