How Do You Troubleshoot the AWS CLI Error Regarding IllegalLocationConstraintException?

Problem scenario
You run a command like this:

aws s3api create-bucket --bucket coolnameforbucket

But you get this error:
"An error occurred (IllegalLocationConstraintException) when calling the CreateBucket operation: The unspecified location constraint is incompatible for the region specific endpoint this request was sent to."

How do you create a bucket with the aws s3api command?

Possible Solution #1
Run a command such as this, but replace "us-west-9" with the region of your choice:

aws s3api create-bucket --bucket my333zzz-bucketz --region us-west-9 --create-bucket-configuration LocationConstraint=us-west-9

Possible Solution #2
Use aws configure and accept the first two defaults. Then change the region to "us-east-1" or the region of your choice.

Possible Solution #3
Do not use the AWS CLI. Use the web console.

How Do You Use Google’s Cloud Pub/Sub with Python?

Problem scenario
You want to use a Data Analytics or a Big Data tool that publishes messages and subscribes to listening to messages being published. You know GCP has a Pub/Sub tool. You know it supports synchronous and asynchronous messaging. How do you use it with Python?

Solution

  1. Log into GCP via the web UI.
  2. Go here: https://console.cloud.google.com/cloudpubsub/
  3. Click "Create Topic".
  4. Enter a name and click "Create".
  5. In a different browser tab, go here:
    https://console.cloud.google.com/projectselector/appengine/create?lang=flex_python&st=true&_ga=2.217969685.-287673474.1542719370
  6. Click "Select." Choose an existing project. If there is none, click "Create" instead of "Select".
  7. This assumes that you have set up the App Engine and in cloud shell cloned a repository. If you need assistance, see this posting How do you use GCP's App Engine?
  8. From the cloud shell run this command:
cd python-docs-samples/appengine/flexible/pubsub

9.a. Draft these commands, but substitute contint_topic with your topic name, contint_subscription with your subscription name, PROJECT_ID with the project ID*, contint_token123 with a password like "token" of your choice:

gcloud pubsub topics create contint_topic
gcloud pubsub subscriptions create contint_subscription \
     --topic contint_topic \
     --push-endpoint \
     https://PROJECT_ID.appspot.com/pubsub/push?token=contint_token123 \
     --ack-deadline 10

9.b. FYI: The push endpoint will use the above to verify requests.

10.a. Modify the app.yaml in this directory. (If it is not there, find this file: appengine/flexible/pubsub/app.yaml)
10.b. Edit it such that it looks like this (with contint_topic and contint_token123 being references to the placeholder values in the previous step):

env_variables:
     PUBSUB_TOPIC: contint_topic
     # This token is used to verify that requests originate from your
     # application. It can be any sufficiently random string.
     PUBSUB_VERIFICATION_TOKEN: contint_token123

11. Run these commands:

pip install protobuf google-cloud-bigquery google-cloud-pubsub
export GOOGLE_CLOUD_PROJECT=[your-project-id]
export PUBSUB_VERIFICATION_TOKEN=[your-verification-token]
export PUBSUB_TOPIC=[your-topic]
python main.py

12. In the upper right-hand corner of the GCP console there should be an icon that when you hover over it, a message is displayed "Web preview". Click this icon, then choose "Preview on port 8080".

13. A new tab in the web browser should appear. You should see an opportunity to enter text in the website. Type in some text and click "Submit Query". Go back to the cloud shell. You should see some lines of activity for this "POST" and "GET" activity.

14. You are now done.

*If you want to find the ID of your project, from the GCP web UI, go to "My First Project" or something similar at the top middle part of the screen. Click on it. The pop up should show the project IDs according to their names.

How Do You Install RabbitMQ Server on Any Distribution of Linux?

Problem scenario
You want to install RabbitMQ on any distribution of Linux using the same script. How do you write such a script?

Solution

1.a. If you are using SUSE or a Red Hat deriviative of Linux (e.g., CentOS/RHEL/Fedora), go to step #2.

1.b. If you are using Debian or Ubuntu Linux, read the following. There may be manual steps (often three steps with Ubuntu 18.04 in AWS). If prompted for "Configuring grub-pc" you are asked "What do you want to do about modified configuration file grub?", choose "keep local version currently installed". If you are prompted with "A new version of /boot/grub/menu.lst is available, but the version installed currently has been locally modified…What do you want to do about menu.lst?", choose "keep local version currently installed".

2. Create a file in /tmp/ called rabbit.sh with the following lines:

rabbitmqversion=3.6.15

distro=$(cat /etc/*-release | grep NAME)

debflag=$(echo $distro | grep -i "ubuntu")
if [ -z "$debflag" ]
then # If it is not Ubuntu, test if it is Debian.
debflag=$(echo $distro | grep -i "debian")
echo "determining Linux distribution…"
else
echo "You have Ubuntu Linux!"
fi

rhflag=$(echo $distro | grep -i "red*hat")
if [ -z "$rhflag" ]
then #If it is not RedHat, see if it is CentOS or Fedora.
rhflag=$(echo $distro | grep -i "centos")
if [ -z "$rhflag" ]
then #If it is neither RedHat nor CentOS, see if it is Fedora.
echo "It does not appear to be CentOS or RHEL…"
rhflag=$(echo $distro | grep -i "fedora")
fi
fi

if [ -z "$rhflag" ]
then
echo "…still determining Linux distribution…"
else
echo "You have a RedHat distribution (e.g., CentOS, RHEL, or Fedora)"
yum -y install gcc zip elixir
makeversion=4.2
curl -Lk http://ftp.gnu.org/gnu/make/make-$makeversion.tar.gz > /tmp/make-$makeversion.tar.gz
cp /tmp/make-$makeversion.tar.gz /bin/
cd /bin
tar -zxvf make-$makeversion.tar.gz
cd make-$makeversion
./configure
make
make install
cp make /bin/
# install Erlang
curl -Lk https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm > /tmp/epel-release-latest-7.noarch.rpm
rpm -ivh /tmp/epel-release-latest-7.noarch.rpm
yum -y install erlang
curl -Lk https://www.rabbitmq.com/releases/rabbitmq-server/v$rabbitmqversion/rabbitmq-server-$rabbitmqversion.tar.xz > /tmp/rabbitmq-server-$rabbitmqversion.tar.xz
cp /tmp/rabbitmq-server-$rabbitmqversion.tar.xz /bin/
cd /bin
tar xf rabbitmq-server-$rabbitmqversion.tar.xz
cd rabbitmq-server-$rabbitmqversion
make
make install
fi

if [ -z "$debflag" ]
then
echo "…still determining Linux distribution…"
echo "But if the distribution was part of the Red Hat family, you can ignore the above message."
else
echo "You are using either Ubuntu Linux or Debian Linux."
apt -y update
apt -y install make xsltproc zip erlang-runtime-tools erlang-tools erlang-dev erlang-base erlang-ssl python erlang-nox erlang-src
apt -y upgrade
curl -Lk https://www.rabbitmq.com/releases/rabbitmq-server/v$rabbitmqversion/rabbitmq-server-$rabbitmqversion.tar.xz > /tmp/rabbitmq-server-$rabbitmqversion.tar.xz
cp /tmp/rabbitmq-server-$rabbitmqversion.tar.xz /bin/
cd /bin
tar xf rabbitmq-server-$rabbitmqversion.tar.xz
echo '{application, 'rabbit_common', [
{description, "Modules shared by rabbitmq-server and rabbitmq-erlang-client"},
{vsn, "$rabbitmqversion"},
{id, ""},
{modules, ['app_utils','code_version','credit_flow','delegate','delegate_sup','ec_semver','ec_semver_parser','file_handle_cache','file_handle_cache_stats','gen_server2','mirrored_supervisor','mnesia_sync','mochijson2','mochinum','mochiweb_util','pmon','priority_queue','rabbit_amqqueue_common','rabbit_auth_backend_dummy','rabbit_auth_mechanism','rabbit_authn_backend','rabbit_authz_backend','rabbit_backing_queue','rabbit_basic_common','rabbit_binary_generator','rabbit_binary_parser','rabbit_cert_info','rabbit_channel_common','rabbit_command_assembler','rabbit_control_misc','rabbit_core_metrics','rabbit_data_coercion','rabbit_error_logger_handler','rabbit_event','rabbit_exchange_type','rabbit_framing_amqp_0_8','rabbit_framing_amqp_0_9_1','rabbit_heartbeat','rabbit_log','rabbit_misc','rabbit_msg_store_index','rabbit_net','rabbit_nodes_common','rabbit_password_hashing','rabbit_pbe','rabbit_policy_validator','rabbit_queue_collector_common','rabbit_queue_master_locator','rabbit_resource_monitor_misc','rabbit_runtime_parameter','rabbit_ssl_options','rabbit_types','rabbit_writer','supervisor2','vm_memory_monitor','worker_pool','worker_pool_sup','worker_pool_worker','time_compat','ssl_compat','ets_compat','rand_compat','time_compat','ssl_compat','ets_compat','rand_compat']},
{registered, []},
{applications, [kernel,stdlib,compiler,syntax_tools,xmerl,recon]},
{env, []},
%% Hex.pm package informations.
{maintainers, [
"RabbitMQ Team info@rabbitmq.com",
"Jean-Sebastien Pedron jean-sebastien@rabbitmq.com"
]},
{licenses, ["MPL 1.1"]},
{links, [
{"Website", "http://www.rabbitmq.com/"},
{"GitHub", "https://github.com/rabbitmq/rabbitmq-common"}
]},
{build_tools, ["make", "rebar3"]},
{files, [
"erlang.mk",
"git-revisions.txt",
"include",
"LICENSE*",
"Makefile",
"rabbitmq-components.mk",
"README",
"README.md",
"src",
"mk"
]}
]}.
' > ./rabbitmq-server-$rabbitmqversion/deps/rabbit_common/ebin/rabbit_common.app

echo '%%
%% %CopyrightBegin%
%%
%% Copyright Ericsson AB 2013-2016. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%
%% %CopyrightEnd%
%%

-ifndef(ssl_api).
-define(ssl_api, true).

-include("ssl_cipher.hrl").

%% Visible in API
-export_type([connect_option/0, listen_option/0, ssl_option/0, transport_option/0,
prf_random/0, sslsocket/0]).

%% Looks like it does for backwards compatibility reasons
-record(sslsocket, {fd = nil, pid = nil}).

-type sslsocket() :: #sslsocket{}.
-type connect_option() :: socket_connect_option() | ssl_option() | transport_option().
-type socket_connect_option() :: gen_tcp:connect_option().
-type listen_option() :: socket_listen_option() | ssl_option() | transport_option().
-type socket_listen_option() :: gen_tcp:listen_option().

-type ssl_option() :: {versions, ssl_record:ssl_atom_version()} |
{verify, verify_type()} |
{verify_fun, {fun(), InitialUserState::term()}} |
{fail_if_no_peer_cert, boolean()} | {depth, integer()} |
{cert, Der::binary()} | {certfile, path()} | {key, Der::binary()} |
{keyfile, path()} | {password, string()} | {cacerts, [Der::binary()]} |
{cacertfile, path()} | {dh, Der::binary()} | {dhfile, path()} |
{user_lookup_fun, {fun(), InitialUserState::term()}} |
{psk_identity, string()} |
{srp_identity, {string(), string()}} |
{ciphers, ciphers()} | {ssl_imp, ssl_imp()} | {reuse_sessions, boolean()} |
{reuse_session, fun()} | {hibernate_after, integer()|undefined} |
{alpn_advertised_protocols, [binary()]} |
{alpn_preferred_protocols, [binary()]} |
{next_protocols_advertised, list(binary())} |
{client_preferred_next_protocols, binary(), client | server, list(binary())}.

-type verify_type() :: verify_none | verify_peer.
-type path() :: string().
-type ciphers() :: [ssl_cipher:erl_cipher_suite()] |
string(). % (according to old API)
-type ssl_imp() :: new | old.

-type transport_option() :: {cb_info, {CallbackModule::atom(), DataTag::atom(),
ClosedTag::atom(), ErrTag::atom()}}.
-type prf_random() :: client_random | server_random.

-endif. % -ifdef(ssl_api).' > /bin/rabbitmq-server-$rabbitmqversion/deps/rabbit_common/src/ssl_api.hrl

cd rabbitmq-server-$rabbitmqversion
make
make install

fi

suseflag=$(echo $distro | grep -i "suse")
if [ -z "$suseflag" ]
then
if [ -z "$debflag" ]
then
if [ -z "$rhflag" ]
then
echo "" echo "Could not determine the Linux distribution!" echo "Installation aborted. Nothing was done." echo "*"
exit
fi
fi
else
echo "this is for SUSE"
zypper -n update
zypper -n install gcc make autoconf ncurses-devel xsltproc
otpversion=20.2
curl -Lk https://github.com/erlang/otp/archive/OTP-$otpversion.tar.gz > /tmp/OTP-$otpversion.tar.gz
cp /tmp/OTP-$otpversion.tar.gz /bin/
cd /bin
tar -zxvf OTP-$otpversion.tar.gz
cd otp-OTP-$otpversion
./otp_build autoconf
./configure
make
make install
cp ./bin/erl /bin
cp -i /usr/local/bin/erlc /bin
curl -Lk https://www.rabbitmq.com/releases/rabbitmq-server/v$rabbitmqversion/rabbitmq-server-$rabbitmqversion.tar.xz > /tmp/rabbitmq-server-$rabbitmqversion.tar.xz
cp /tmp/rabbitmq-server-$rabbitmqversion.tar.xz /bin/
cd /bin
tar xf rabbitmq-server-$rabbitmqversion.tar.xz
cd rabbitmq-server-$rabbitmqversion
make
make install
fi

3. Run the script like this: sudo bash /tmp/rabbit.sh

4. After script is done, run this command:

sudo bash /bin/rabbitmq-server-3.6.15/deps/rabbit/scripts/rabbitmq-server

How Is Input Passed into the Python Program in HackerRank.com?

Problem scenario
You wrote a Python program on a Linux machine, and it works by interactively prompting a user for input. You want it to work for HackerRank, but you are not sure how standard input works in their platform. You use this line of code in your program: 'input("Enter your input here: ")' to ask the user for input. This does not work in HackerRank.

You tried rewriting your program to accept parameters when you run it like this:

python foobar.py blue

In foobar.py you have "sys.argv[1]" to accept the "blue" argument.

You have two versions of a Python program that accept input (either interactively or when the program is run). You want a third version to work for HackerRank. How do you get a Python program to accept HackerRank input?

Solution
Eliminate the text inside the parentheses of "input()".

Example of an incorrect way to ask for input for Python programs in HackerRank:

input("Enter your input here: ")

Example of the correct way to ask for input for Python programs in HackerRank:

input()

By not interactively prompting the user for input, the HackerRank platform will not process the prompt as final output. The program accepts input, but it is predefined -- behind-the-scenes. When you press "Run Code", you will see what is sent to the prompt. The input, or test case, is not sent when the Python program is run; it happens immediately afterward. Printing output to the screen from a Linux terminal is normally fine to prompt a human user for input -- but not with the HackerRank platform. Such output (or text prompt) will be perceived as output and tested accordingly for a pass or fail for a Sample Test Case.

Look at the code lines on the left when doing a HackerRank coding exam. Does the sequence increment by 1 each time? If not, is there an arrow nearby to expand the code?

For more information, view this: https://www.hackerrank.com/challenges/input/forum
Or view this: https://www.hackerrank.com/challenges/python-raw-input/tutorial

How Do You Get wget to Work when curl Commands Are Successful?

Problem scenario
You have a URL that works with a curl command (with the -X POST option). You need to get wget to do a POST like curl. You are having trouble getting wget to work. wget returns 403 forbidden errors, but the destination address is the same. It seems as though the source server is different and potentially causing the problem (e.g, by virtue of its IP address). You do not think, but are not sure, that Jenkins has an authentication mechanism to differentiate the source IP address or other attributes about the source of the HTTP invocation to the URL.

What should you do?

Solution
Use the --post-data flag with two empty quotes like this:
wget --post-data '' http://URL

The wget command is significantly different from curl when it comes to doing detailed REST API troubleshooting. By specifying the --post-data flag and passing an empty value in quotes, you force the POST function as opposed to using a GET with wget.

How Do You Find What Context Your Kubernetes Configuration Is Using?

Problem scenario
You are trying to fix a Kubernetes problem. You read that you need to check the context to solve your problem. How do you determine what context you are using?

Solution
Run this command:
kubectl config current-context

Or try this command:
kubectl config get-contexts

Another command that may be interesting is this one:
kubectl config view | grep -i context

How Do You Troubleshoot the kubectl Error “Unable to connect to the server: dial tcp: lookup ,,, no such host”?

Problem scenario

You run this command:
kubectl get cluster-info

You receive this error message:
"Unable to connect to the server: dial tcp: lookup ABCDEFGHIJKLMNOPQRSTUVWXYZ123456789.yl4.FQDN.com on 172.55.44.33: no such host"

Solution

Possible solution #1
Ask yourself these three questions:

  1. Have you recently destroyed a Kubernetes cluster?
  2. Is the intended Kubernetes cluster running?
  3. Have there been changes to networking or firewall configurations thereby blocking the server with kubectl to the cluster itself?

Possible solution #2
If you want to modify the config file that governs the kubectl settings, do the following:

cd ~
cd .kube
Back up the config file in this directory (e.g., cp config /home/jdoe/config.bak)
Edit the config file in this directory.

Possible solution #3
See this posting.

How Do You Troubleshoot a Kubernetes Dashboard Message about a 403 Forbidden Error?

Problem scenario
You created an EKS cluster. But when you open a web browser and go to the API endpoint, you get a 403 forbidden error.

The web browser shows something like this:

kind "Status"
apiVersion "v1"
metadata {}
status "Failure"
message "forbidden: User \"system:anonymous\" cannot get path \"/\""
reason "Forbidden"
details {}
code 403

How do you access the web UI in Kubernetes for the first time?

Possible Solution
The API endpoint as seen in the AWS Console for EKS is not a complete URL that you can browse to. Append a URL path. For example, append "/api" to the base URL (provided in the AWS Console).

Try appending "apis/extensions/v1beta1" to the base URL too.

You may also want to see this posting.

What Does the Phrase Cloud Native Mean?

Question
What does the phrase "Cloud Native" mean?

Answer
It is a descriptive term that that means having the qualities of cutting-edge cloud technologies. In our experience it connotes using microservices and containers primarily with open source technologies. In some ways it is cloud 2.0. Originally cloud computing referred to SaaS and possibly IaaS or PaaS, but now cloud computing is referring to CaaS and KaaS too.

The Cloud Native Computing Foundation is well-respected and has this definition. Stackify finds variations in the definition of "cloud native." We have found several more.

Oracle's definition denotes a need for code without regard for the cloud itself:
"Cloud native technologies are characterized by the use of containers, microservices, serverless functions, development pipelines, and infrastructure expressed as code."

IBM's definition also suggests that cloud computing is not necessary, but IBM makes no mention of code for something to be cloud-native:

What is cloud-native?
    Cloud-native refers less to where an application resides and more to how it is built and deployed.
    A cloud-native application consists of discrete, reusable components known as microservices that are designed to integrate into any cloud environment.
    These microservices act as building blocks and are often packaged in containers.
    Microservices work together as a whole to comprise an application, yet each can be independently scaled, continuously improved, and quickly iterated through automation and orchestration processes.
    The flexibility of each microservice adds to the agility and continuous improvement of cloud-native applications.

"Cloud native" (with no hyphen) is used interchangeably with Kubernetes in
Kubernetes Patterns (page three of "Kubernetes Patterns by Bilgin Ibryam and Roland Huß (O'Reilly). Copyright 2019 Bilgin Ibryam and Roland Huß, 978-1-492-05028-5").

One blogger writes '“Cloud Native” is the name of a particular approach to designing, building and running applications based on infrastructure-as-a-service combined with new operational tools and services like continuous integration, container engines and orchestrators.'

How Do You Troubleshoot a Bitbucket Webhook Invocation to Start a Jenkins Build?

Problem scenario
You have created a webhook in Bitbucket to call a Jenkins URL. You create the triggering event in the repository (e.g., by pushing code or creating a pull request). The build associated with the Jenkins URL does not start.

You manually tested the Jenkins URL, bypassing Bitbucket, with curl -X POST. The invocation successfully starts the build.

In the Bitbucket logs you see "Connection refused" or "403 Forbidden" errors when the triggering event happens to the given repository (configured with the webhook).

It seems as though the source server is different and potentially causing the problem (e.g, by virtue of its IP address). You do not think, but are not sure, that Jenkins has an authentication mechanism to differentiate the source IP address or other attributes about the source of the HTTP invocation to the URL.

What should you do to get Bitbucket to call the same URL?

Solution

Install and configure a Jenkins plugin that facilitates the integration with Bitbucket. Configure a Bitbucket server system-wide (in the Manage Jenkins section) of the Jenkins server.

Bitbucket with a manually-configured Jenkins URL can lead to 403 Forbidden errors that are not easily reproducible in other contexts. The Bitbucket invocation of the URL is not like a curl POST command or a wget command with respect to some crucial details of the REST API functionality.