How Do You Troubleshoot “Connect timeout on endpoint https://ssm.us-west1.amazonaws.com…”?

Problem scenario
You try to run an aws ssm command. But you get an error about a connection timing out. What should you do?

Solution
Find the EC-2 server's VPC and subnet. Go to VPC in the AWS console. Go to the Subnet section. Find the Route Table associated with the subnet for the EC-2 server. Make sure that the "Destination" field accommodates the IP address of the ssm.us-west1.amazonaws.com hostname. To find what IP addresses the ssm.us-west1.amazonaws.com resolve to, ping the hostname several different times manually. If you find the first octet is 43, then create a rule like this: 43.0.0.0/8 as the destination and have that traffic go to a NAT gateway. Now the EC-2 server should be able to route traffic to the ssm.us-west1.amazonaws.com URL. In our experience sending all traffic outward via 0.0.0.0/32 does not work very well.

This may not be a best practice as any traffic bound for an IP address that starts with the given octet will be sent outward through the NAT gateway. If you can do special configuration (e.g., forwarding) to limit the IP address resolution of the ssm.us-west1.amazonaws.com, that would be advisable.

How Do You Insure Your Bitcoin/Cryptocurrency Holdings?

Problem scenario
You have a sizable cryptocurrency position. You are concerned that if you lose it, you will not be able to get it back. What should you do?

Solution
This is not legal advice.

  1. Self-insure. Ensure you use multi-factor authentication if you use an online platform. If you need a username, a password, and access to a separate physical device (such as a smart phone for a passphrase or code number), it should be difficult to hack. If you do not use an online platform, it may be wise; you can use a cryptocurrency wallet, which is arguably more secure. One such wallet is Trezor. The Trezor wallet has the highest rating on CryptoCompare.com.
  2. Play it safe. Use Coinbase, Gemini Exchange or BitGo for a potential online provider. According to this article, they provide some insurance to the cash holdings. We do not know how much is insured and what the terms of making a claim are. So do be careful. But we do not recommend shunning investing in cryptocurrency either. Staying with a reputable company with a risky undertaking helps mitigate loss in general. (Coinbase often offers up to $200 in free coins when you sign up for the first time.)
  3. Read more about this topic. These next articles, as of July 2020, give some insight into a new market for insurance:

4. Try to find a wealthy person or company that can do customized insurance. A manuscripted insurance policy may not be run-of-the-mill, but they can be created. You may be able to get protection in case you lose your coins.

5. The laws surrounding I.T. and cryptocurrency change regularly. You may want to find an attorney for the state you live in to discuss how recourse can work.


Update on 11/24/23

HODL refers to Hold On for Dear Life. HODLers do not time the market or actively trade.

Protecting the recovery seed is essential for getting your cryptocurrency if your wallet is damaged. There are cryptocurrency wallet recovery plates made with titanium and steel capsules that can help protect your cold seed storage for your crypto wallet.

To learn more about insurance of digital assets, see the following links:


For Europeans, the platform/company Iconomi.com can allow you to buy crypto or learn more; they may or may not have a way to ensure the holdings.

What are Recommended Practices vs. “Best Practices”?

Background
Facebook's engineering culture during its earlier days of rapid growth was "move fast and break things." But the Harvard Business Review says that this era is over.

Problem scenario
You are concerned about recommended practices and "best practices." You want to be productive and leave no security gaps in your systems that you design/install or allow bugs in your code. You want logging and monitoring to be appropriate. What is the difference between recommended practices and "best practices?"

Solution
"There is an exception to every rule." - Many different sources.

"A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines. With consistency a great soul has simply nothing to do." -Ralph Waldo Emerson

Superlatives can be appropriate in various contexts. There is nothing wrong when a radio station advertisement is informing the audience that it plays only the best new music. But in business writing, superlatives can make someone seem less credible (according to this external website).

Marketing material and advertisements try to induce a potential customer into a sale. It sounds assuring if a vendor claims to always use "best" practices. Journalists for the industry seem to like the phrase too. The adjective "best" can be defined as "excelling all the others" (according to page 116 of Merriam-Webster's Dictionary 11th Edition).

A well-respected book on modern American English (Garner's Modern American Usage on page 918) defines superlative as "[t]he form of an adjective or adverb used to compare at least three things and show that one has a quality above or below the others." As an example the book cites the word best as the superlative to good. Therefore a "best" practice is indicating that two alternative practices are available. (The book The Careful Writer concurs that there are three degrees when someone uses a superlative.) The adjective "preferred" would be pleonastic if applied to something that is the "best".

After we consulted with The Careful Writer and Garner's Modern American Usage we would be disinclined to characterizing as many practices as "best" because there may be only one other worse way to accomplish a given I.T. goal (i.e., consider a business use case with numerous restrictions and there are exactly two options). Nonprofitcopywriter.com and mediasurvival.com suggest the word "best" should be avoided in writing. The noun "practice" can be defined as a customary course of action. It is a best practice to use proper grammar when writing.

Best practices may include "prohibitions without reasons" and the systematizing of human interactions (such as creating a Jira story when there is a problem). Nietzsche would have hated best practices. The Portable Nietzsche says that he said "prohibitions without reasons" do more harm than good (on page 68). The will toward a system was dishonest according to Nietzsche (as stated in the first chapter of Twilight of the Idols). He also thought that a will toward ignorance was strong (as he describes in Beyond Good and Evil).

Kant defines categorical imperatives as commands or moral laws all persons must follow, regardless of their desires or extenuating circumstances.

https://www.tc.columbia.edu/institutional-review-board/irb-blog/categorical-imperatives-and-the-case-for-deception-part-i/

Best practices purport a categorical truth. Humans all want a heuristic to avoid the toil of thinking. By definition there is nothing superior to the best. When deciding on how to engineer something in I.T., there are usually more than two choices (but there could be just one or two). From selecting a technology to adopting an indentation and formatting style in programming, there are a multitude of choices.

It’s also too easy to call something the “best”–have you really had a chance to test all other options before concluding that this particular thing or person surpasses them?

https://www.fastcompany.com/3068563/four-words-and-phrases-to-avoid-when-youre-trying-to-sound

Extremism has its disadvantages and absolutes can be uncommon in endeavors with people. Take for example The Agile Manifesto; it gives positive treatment to a set of desirable items juxtaposed to their common alternatives. But it specifically says that there is value in the less preferred alternatives themselves. Consider the concept of "best practices" when you read the those four sentences of the Agile Manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

agilemanifesto.org

Agile deprioritizes processes, documentation and following a plan. That does not sound like it advocates "best practices." Other sources (such as this one) say it is a best practice (an advisable process) to document, and follow a plan. It is considered a best practice to have a test plan for software products (according to Perforce) or a plan with a budget for software projects (also according to Perforce). However Agile itself was created to follow a path defined by best practices (according to devops.com). Many sources say it is a best practice to use Agile, but that would tend to say it is a best practice to not use best practices.

Valuable software tends to have many features, and sales professionals would say it is a best practice to give the market all that it demands. But in Eric S. Raymond's The Cathedral and the Bazaar there is praise for what is sometimes called New Jersey style software (page 225). This is the concept that "worse is better" because minimal functionality brings with it performance, simplicity and security. As technology gets increasingly sophisticated in modern times, demands to push software to its limits remain as high as ever.

Some independent software consulting companies purport to write sophisticated, complex code while always adhering to best practices. Here is an example of a blog post (by a well-respected author) that tends to contradict "best practices" for software architecture in favor of avoiding complexity:

Software architecture best practices, enterprise architecture patterns, and formalized ways to describe systems are all tools that are useful to know of and might come in handy one day. But when designing systems, start simple and stay as simple as you can.

https://blog.pragmaticengineer.com/software-architecture-is-overrated/

Another authority concurs with simplicity. One of the most influential database pioneers Jim Gray was aware that many books tend to prescribe complex solutions as best practices. He said this (in "Transaction Processing: Concepts and Techniques"):

Don't be fooled by the many books on complexity or by the complex and arcane algorithms you find in this book or elsewhere. Although there are no textbooks on simplicity, simple systems work and complex don't.

Page 326 of Elements of Programming Interviews in python)

Programmer and author Jon Bentley agrees:

don't write a big program when a little one will do

Page 29 of Programming Pearls

There is a saying that perfect is the enemy of good. (This quote originally comes from the first two lines of a Voltaire poem. You can use translate.google.com to see for yourself what the first two sentences mean in English.) In pursuit of perfection one may do a very poor job. There is evidence suggesting that no software project should be perfect and striving for perfection has its disadvantages. "Like it or not vulnerabilities exist in the software and networks that the world depends on from day to day. It's simply an inevitable result of the fast pace of software development. New software is often successful at first, even if there are vulnerabilities." This quote was taken from the book Hacking The Art of Exploitation 2nd Edition, page 451.

The author goes on to describe (on page 452) that the legal system is slow and ineffective in the context of computing. In some cases the law criminalizes learning activities with respect to networking. The author refers to Hans Christian Andersen's story "The Emperor's New Clothes" (which is an allegory illustrating the downsides to social conformity and pretending to go along with commonly accepted facts). So this I.T. security author (of Hacking The Art of Exploitation 2nd Edition) says that even the law can be counterproductive and alludes to the community being in denial. Clearly the author rejects the notion of absolutes in the world of I.T. security.

"We should forget about small efficiencies, say about 97 percent of the time: premature optimization is the root of all evil." (Donald Knuth)

https://www.forbes.com/sites/theyec/2019/12/27/planning-for-growth-without-premature-optimization/?sh=75e02ef27f6d

Other authors and advertisement creators that use the term "best practice" in I.T. may not have thought through what they were saying or possibly they felt pressure to conform to utilize this sometimes meaningless phrase of "best practice".

Best practices are not compatible in some instances. Here is one example where sources disagree on whether or not it is a best practice to normalize a relational database. If adhering to a best practice makes a system less secure, then best practices fail to be the best for security. To see an example of an approach that is referred to a as a best practice by some and a violation of best practices by other sources, see this posting.

Most professionals in I.T. would defer to Jez Humble and David Farley when it comes to best practices surrounding code versioning. In Continuous Delivery (on page 388) they say it is imperative to "keep absolutely everything in version control." While this book was published in 2011, it still has relevance in 2022. An established industry standard in today's world is the laudable 12 Factor App. It says:

The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally;

Taken from https://12factor.net/config

We know that a best-practice from one source may conflict with a widely-heeded practice in another source. On December 6, 2021, most people would have said that Amazon uses best practices. Amazon is considered to set a standard for other companies. It needs no citation to assert that AWS best practices were commonly promulgated in 2021 by various third party vendors. On December 7, 2021 AWS had a major outage.

Here is a rhetorical question: is it as simple as always using best practices?

The term "best practice" is an ascertainable and singular practice that is strictly better than other practices. It gives a false sense of security. Many companies purport to always use best practices on their websites and advertisements. Always using best practices would be tantamount to being perfect. The feasibility of this is self-explantory.

There may be legitimate times to disregard "best" practices to allocate resources for hardening a system from a pragmatic perspective. A Cloudera Guru says that best practices vary depending on the scenario:

It is usually a best practice not to transform…

https://community.cloudera.com/t5/Support-Questions/Real-time-streaming-using-Oracle-logs/td-p/160908

The adverb "usually" modifies the definition of a best practice. At a minimum best practices sometimes have applicability and sometimes do not have applicability. Prepositions such as "if" and "when" are common in discussions of best practices. Other qualifiers enable people to use the phrase "best practices." Their applicability hinges on context -- even for the people who prescribe them. Here is an example of a conditioned "best practice":

There are several, basic “best practices” that all businesses should follow if granting employees access to the corporate network through a VPN.

https://www.focusdatasolutions.com/best-practices-for-vpn-use-in-your-small-business/

Given the complexity of I.T. today, we cannot state with confidence that every so-called "best practice" will be compatible with other applicable "best practice" you may find. There may be no feasibility to such an undertaking.

We think that I.T. security is under-rated and many projects have been under-funded. Accordingly we think that I.T. professionals should think in terms of "recommended practices" instead of "best practices." Nothing in I.T. is so simple to be framed as the "best" in 2022 without being qualified or conditioned on a specific context (such as a use case).

Many "best practices" are merely specific conventions that are commonly implemented by corporations (e.g., configuring parameters for automatic desktop screen lockouts or periodic credential challenges on a Linux server). Some authorities will describe a best practice that is irrelevant based on your specific utilization of the tool or technology. Therefore some "best practices" can be safely ignored. Given temporal and financial constraints, no reasonable person would recommend they be adhered to when their need is obviated. The culture of "move fast and break stuff" may be a hyperbole to highlight the benefits of experimentation. Clearly it could be used in ways that would compromise the security of a data center. No one can refute that experimentation is a "best practice" (because of The DevOps Handbook, launchdarkly.com, the prevalence of the DevOps movement).

There is skepticism in the industry; many authors criticize "best practices." Consider the following examples:

  • The website visualstudiomagazine.com in 2016 had an article about why a system architect hated "best practices".
  • MIT had a magazine article in 2018 that said best practices should only be implemented in moderation.
  • The Massaro consulting website specifically stated that there are times and contexts when someone should avoid using best practices.
  • Miller Klein Associates Ltd. (an industrial strategy consulting company) said to not use the term "best practice" and to look for situations when practices don't apply.
  • Forbes said that there are downsides to best practices and that "best practices" do not even exist.

Recommended practices are those that are usually suitable. They are advisable but not always necessary. Not every practice in I.T. has been documented much less agreed upon.

Modern enterprise networks and software requirements can be very complex with nuances in design and a multitude of trade-offs inherent in the underlying architectural decisions. We think it is best to pursue recommended practices when engineering systems and/or writing code. Do not be seduced into ignoring the complexity of I.T. security. The DevOps Handbook says that a myth in the I.T. industry is "[s]ystems will be safe if people comply with the procedures they have been given." (page 360, see also [1]). There are contexts where "best practices" do not exist or should be avoided. We recommend that you strive to have a secure environment and not be content with merely following "best practices."

[1] Denis Besnard, Erik Hollnagel. I want to believe: some myths about the management of industrial safety. Cognition, Technology and Work, Springer Verlag, 2014, 16 (1), pp.13-23.
<10.1007/s10111-012-0237-4>.


See also:

Is It Enough to Use Best Practices?
Is It a Best Practice to Normalize a SQL Database?
Is It a Recommended/Best Practice to Have a Code Freeze?
Is It a Best/Recommended Practice to Use Branching with Version Control Systems?
Is It a Best/Recommended Practice to Check Code in Frequently?

Is It a Best/Recommended Practice to Update a Server when Clients/Apps May Not Be Compatible with the Version?
Is It a Best/Recommended Practice to Use Log4j?
Is It a Recommended Practice to Use Production Data when Testing Software?
Are Zero Trust Networks More Secure than VPN-Protected Networks?
What Are the Recommended Practices of Monitoring?
What Are the Recommended Practices of Logging?
What is difference between "Good Practice" and "Best Practice"?
What is I.T. Operational Readiness?

For a definition of best practices outside the I.T. industry, click here.

How Do You Troubleshoot the Hadoop Message “Exception in thread “main” java.nio.file.AccessDeniedException: /home/jdoe/./mapper.py”?

Problem scenario
You are trying to run a hadoop command (to kick off a mapreduce job). But you get this error:
"Exception in thread "main" java.nio.file.AccessDeniedException: /home/jdoe/./mapper.py"

What should you do?

Solution (short version)
Change to a directory where the user can write files to. Retry the command.

Solution (long version)
Create a directory that is owned by the user and the group associated with the user that is running this command. For example, if you are trying to run the hadoop command with hduser, make sure you are in a subdirectory that is owned by the hduser. If you do not know the group this user is associated with, run the "groups" command.

Here is an example of the command that could help you:
chmod hduser:hadoop /path/to/foobar

Then move into foobar and run your hadoop command. Make sure the files in the foobar subdirectory are owned/associated by the user/group of the user that will run the command.

What is a Hacker?

Question
You have seen the word "hacker" appear in different contexts. You have seen it used to describe a cybercriminal. Moreover some companies proudly claim that they employ no hackers while only hiring trained software developers. In some articles the term hacker has a positive connotation. What is a hacker?

Answer
"Hacking tends to be a misunderstood topic, and the media likes to sensationalize, which only exacerbates this condition. Changes in terminology have been mostly ineffective--what's needed is a change in mind-set. Hackers are just people with innovative spirits and an in-depth knowledge of technology. Hackers aren't necessarily criminals…" This quote was taken from page 451 of Hacking The Art of Exploitation 2nd Edition.

To paraphrase Eric S. Raymond, a hacker is someone who has technical knowledge, enjoys solving problems, and overcomes his/her own personal limitations. (The source was http://www.catb.org/esr/faqs/hacker-howto.html#what_is.)

Page xv of Linux Hardening in Hostile Networks by Kyle Rankin says that hackers were originally motivated by curiosity (many years ago) but now they are motivated by profit.

According to Hacking for Dummies (page 8), a hacker is someone who enjoys tinkering with technology, but "[i]n recent years, hacker has taken on a new meaning: someone who maliciously breaks into systems for personal gain. Technically, these criminals are crackers (criminal hackers)."

There are two definitions in an older CompTIA Linux+ book: "1. An individual who is skilled at using or programming computers and who enjoys using these skills in constructive ways. Many Linux programmers consider themselves hackers in the sense of the term. 2. A cracker…This use of the term is more prevalent in the mass media, but it's frowned upon in the Linux community." This quote was taken from page 532 of CompTIA Linux+ Complete Study Guide by Roderick W. Smith published in 2010 by Sybex.

What Are Some Ways to Prevent XSS Attacks with a Web Page That Uses JavaScript?

Problem scenario
You know that cross-site scripting (aka XSS) attacks are a big concern in today's world. OWASP places XSS security risks as the seventh biggest web application risk as of June 2020. What are some techniques to stop such attacks from happening when designing a website that uses JavaScript?

Possible Solution #1
Have the HTML and JavaScript validate and escape regularly throughout the code. (The source of this is page 52 of JavaScript Security by Y.E. Liang.)

Possible Solution #2
Design cookies to allow the user from the IP address of issuance to interact with the web page and disallow any different IP address from interacting with the web page. (The source of this is page 52 of JavaScript Security by Y.E. Liang.)

Possible Solution #3
Minimize the utilization of JavaScript code to the least possible. (The source of this is page 52 of JavaScript Security by Y.E. Liang.)

Possible Solution #4
Use Content-Security-Policy (according to page 9 of Node.js Security by Liran Tal) to enable filtration of content; this is particularly useful if your website application will interact with an untrusted source.

Possible Solution #5
Use X-XSS-Protection, an HTTP header, can block XSS attacks (according to page 14 of Node.js Security by Liran Tal).

Possible Solution #6
Implement xss-filters library (according page 47 of Node.js Security by Liran Tal); this is particularly useful if your website application will interact with an untrusted source.

How Do You Find the Owner of an AWS Resource?

Problem scenario
In AWS you find some components, such as a Security Group, has an owner. You cannot find the owner in IAM. How do you learn more about this user?

Solution
The owner is not an IAM user. In the AWS Console, go to Support -> Support Center. Search for the owner number there. You may have different account aliases. If you log into each one, eventually you will find the owner number matches with what you see in Support -> Support Center.

How Do You Change the Version of Python that Ansible Uses?

Problem scenario
You are using Ansible with -vvv to see what Python version it is using. (Or you use ansible --version.) You see an incorrect version of Python being used.

You tried ansible_python_interpreter=/usr/bin/python3 in your playbook and in the ansible.cfg file. Neither worked.

What should you do?

Solution
In the playbook, find the hosts stanza. Underneath it use this (where python3 is the version you want and "/usr/bin" is the path to it):

vars:
ansible_python_interpreter: /usr/bin/python3

How Do You Use a NACL in AWS?

Problem scenario
You have a VPC. You want a NACL to protect your VPC from hackers and other networks. What do you do?

Solution
Network ACLs are created by default when you create a VPC. They can be configured the same way Security Groups are configured in the AWS Console. Go to the VPC Dashboard -> Security -> Network ACLs. Click on the relevant Network ACL. Click either the "Inbound Rules" or the "Outbound Rules" and modify them accordingly to harden your VPC.