How Do You Upload Files to Nexus 3 in a Docker Container on Linux CentOS?

Problem scenario
You want to upload files into Nexus 3 running in a Docker container.  You installed Nexus 3 with Docker 1.9 using these directions. The OS is CentOS 7.2.  The web UI for Nexus 3 works via Docker, but you cannot upload files into any repository. When you go to Repositories, but you do not see the "Artifact tab."  How do you upload files?

Solution
Prerequisite
Install Docker; if you need assistance, see this link.

Procedures
It does not matter if Maven is installed or not.  The only prerequisite to Nexus 3 is JRE (the Java Runtime Environment).  If you want directions installing the Java Development Kit (which includes the JRE), see this posting.  While this solution can be done interactively, this solution is a programmatic way of uploading artifacts.  It can enable scripting and getting around using the web UI manually.

First, find the correct URL for the repository.  Open a web browser and go to the URL. Log into Nexus 3's web UI.  The default username is admin with a password of "admin123" (with no quotes).  Go to Repository -> Repositories.  Look at the "Type:" column and find one that is not "group" nor "proxy."  A repository should have the Type of "hosted."  Obtain the value in the "URL" column for the repository you want.

From a Linux server with access to the URL for Nexus 3 (usually the IP address of the Linux server that houses Docker with ":8081" at the end of it), run this command:

curl -v -u jdoe:P@$$w0rd --upload-file /tmp/artifactfile http://123.123.123.123:8081/artifactfile

# where...

1.  http://123.123.123.123:8081/ is the URL that you copied in the first part of this solution.   
2.  /tmp/ is the path to the file you want to upload
3.  artifactfile is the name of the file you want to upload
4.  jdoe is the username that can log into the Nexus 3 web URL
5.  P@$$w0rd it the password for jdoe

The above command will allow you to upload files to Nexus 3.

How do you install Hadoop on Linux SUSE?

Updated on 1/1/18
You may want to see this different posting that works for Linux SUSE as well as other non-SUSE distributions of Linux.

Problem scenario

You want to install the open source version of Hadoop 3.0.0 on Linux SUSE 12 SP3.  What do you do?

Solution
These directions would work on a non-SUSE distribution if you install Java a different way from the first command in the script.  This script requires that your AWS SUSE Linux server is in a security group that has access to the Internet.  But this script should work if your server is not in AWS.  The script takes approximately three minutes to run.  But bandwidth and resources on your instance may vary.

1.  Create a file called installer.sh in /tmp/ with the following content:

zypper -n install java
curl http://www.apache.org/dist/hadoop/core/current/hadoop-3.0.0.tar.gz > /usr/bin/hadoop-3.0.0.tar.gz
cd /usr/bin
tar -xzvf hadoop-3.0.0.tar.gz

var1=$(find / -name hadoop-env.sh)
var2=$(cat $var1 | grep -ic JAVA_HOME)
echo "******************************"
echo "You will have to call the full directory path to hadoop or hdfs to run the commands."
echo "******************************"
echo "There are "$var2" instances of JAVA_HOME in "$var1
echo "You need to comment out uncommented JAVA_HOME stanza(s), if any, in "$var1
echo "Then you need to add this stanza"
echo 'export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")'

2.  Run the installation script:  sudo bash /tmp/installer.sh

3.  Modify hadoop-env.sh in two ways.  First, make sure "JAVA_HOME" stanzas (lines with "JAVA_HOME") are commented out.  Second, make sure you have one that is uncommented like this:

export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")

4.  Test it with these two commands:

/usr/bin/hadoop-3.0.0/bin/hadoop version
/usr/bin/hadoop-3.0.0/bin/hdfs version

How Do You Download Big Files with PowerShell?

Problem scenario
You want to download a very large file using PowerShell (from a file share on your network or URL on the Internet).  But the file is larger than the available RAM on your server.  How do you download a large file using Powershell?

Solution
Invoke-restmethod, Invoke-webrequest, curl and wget (even with the redirect ">" command) all put the file being downloaded into memory.  If RAM is scarce on your system or the file is very large, you will want to avoid bringing the file into memory.  To bypass memory consumption when downloading a file from a URL and write directly to disk using PowerShell, try these lines of code:

$webClient = New-Object System.Net.WebClient
$webURL = "https://www.continualintegration.com"
$filePath = "c:\temp\foo.html"
$Webclient.DownloadFile($webURL, $filePath)

On some systems, the above may not work.  In that case you could try to use start-bitstransfer.  The drawback to start-bitstransfer is that remotely executed scripts will not work.  This command, start-bitstransfer, only works interactively from local logon sessions or in a script that was launched by a local user.  To get around this shortcoming of start-bitstransfer you can create a Scheduled Task to run start-bitstransfer.  The Scheduled Task can be launched remotely with PowerShell.

To explain how to use start-bitstransfer in a script, follow these steps.  First write a PowerShell script that uses start-bitstransfer.  Next create a Scheduled Task to execute this PowerShell script, we will call it "foobar" for this example.  Finally run this as a PowerShell or batch command to invoke the Scheduled Task, use a command like this:

SCHTASKS /RUN /TN foobar

where "foobar" is the name of the Scheduled Task that you want to run.  Now the start-bitstransfer will run even when it is in a script and the script has been executed remotely.


To buy a book on PowerShell, see this page.

How do you run Windows Malicious Software Removal Tool?

Problem scenario:  You go to the Start menu and type in the box "malicious" or "Windows Malicious Software Removal Tool" and no program shows up.  You go to "Programs and Features" in the "Control Panel" but you cannot find anything with the word "Malicious" in it.  How do you run Windows Malicious Software Removal Tool?

Solution from a command prompt:  Open a command prompt (hold the Windows button and tap "r").  Enter this text in the quotes without the quotes "mrt.exe" and press enter.

Solution from PowerShell:  Open PowerShell as Administrator.  Run this command with no quotes (change path if needed): "C:\Windows\System32\mrt.exe"

For more details you may want to view this link.

How Do I Create a Manifest with Puppet Enterprise to be run on a server other than the Puppet Master server itself?

Question: How Do I Create a Manifest with Puppet Enterprise to be run on a server other than the Puppet Master server itself?

Assumption:  Puppet Master is installed on a Linux server.

Answer:  Manifests are .pp files.  The Puppet syntax (with its domain specific language) is beyond the scope of this posting.  This post is to get you started. Go to the directory where the modules are on the Puppet Master server.  Use this command if you are unsure: find / -name modules

Throughout these directions substitute "continualintegration" with the module name of your choice.

Run this command: puppet module generate jdoe-continualintegration

# The username "jdoe" and hyphen are optional.  Naturally you would use your own username or your first name. Puppet recommends this convention

Accept the defaults to the subsequent prompts.

Then issue these two Linux commands:

cd continualintegration
cd manifests

Modify init.pp to be the new manifest. Save the changes.

Go to the Puppet Enterprise Console (sometimes referred to as the Puppet Management GUI or Puppet Dashboard) by opening a web browser and going to https://x.x.x.x where x.x.x.x is the IP address of the Puppet Master server.

Log in.

Go to Classification -> Facter -> Add New Class

In this field, type "continualintegration".  As you type you should see a suggestion appear below.  Click on it.  Click "Add Class."  In the lower right hand corner, find the "Commit 1 change" button.  Click this button.

Now when Puppet Agent runs on the client, the configuration should be pushed down.  For Windows Servers, Puppet Agent may need to be ran as administrator.  To clarify you may need to open PowerShell as administrator and run with this command: puppet -t puppet.continualintegration.com -d # where puppet.continualintegration.com is the FQDN of the Puppet Master server in your network.

How do I get a forward slash (“/”) to be passed from a Puppet manifest on a Linux server with Puppet Master to a Windows server with Puppet Agent?

Question:  How do I get a forward slash ("/") to be passed from a Puppet manifest on a Linux server with Puppet Master to a Windows Server with Puppet Agent?

Answer # 1 (for creating Scheduled Tasks with Puppet manifests): For a Scheduled Task, the forward slash may be crucial.  A forward slash (that Puppet between Linux and Windows will automatically substitute for a back slash) with a shutdown.exe server is the only way for a reboot to work properly with a Scheduled Task.  In a Puppet manifest (.pp file), declare a Scheduled_Task resource (where the link used to be here https://docs.puppet.com/puppet/latest/reference/types/scheduled_task.html).  This is a built-in resource type for Puppet.

While this solution does not give you a forward slash in the end result, it will give you a solution by having an "argument."  The "arguments =>" property will allow you to place an "\r" in the "Add arguments" field in the "Edit Action" window found by clicking "Edit" in the "Actions" tab of the Scheduled Task.  When you manually create a Scheduled Task, the "...shutdown.exe /r ..." can all appear the "Program/script" field together.  A backslash before the "r" in the "Program/script" field will not allow you to reboot the server. But a backslash before the "r" in the "Add arguments" field will allow you to reboot the server.

Answer #2 (for passing a "/" to in a PowerShell command):  Use the [char]47 command in PowerShell to get a variable with a forward slash "/".   PowerShell commands can be compound with semi colons ";" separating distinct lines.  Therefore something like this will work (with assigning [char]47 to a variable and concatenating strings to ultimately call everything with the invoke-expression command:

command => '$fslash = [char]47; $construct = "schtasks " + $fslash + "create " + $fslash + "sc monthly " + $fslash + "tn continualIntegration " + $fslash + "tr c:\windows\system32\fun.exe "  + $fslash + "sd 09/08/2016"; invoke-expression $construct'

Answer #3 (transfer a file via a manifest):  A manifest can have a resource definition like this:
file { "c:\temp\special.ps1":
          source => puppet:/// /special.ps1
   }

# special.ps1 can have forward slashes in it. 
# This file would have to be in /etc/puppetlabs/code/environments/development/modules/foobar/files/
# Note that "files" (the subdirectory holding the file) is omitted from the source declaration in the manifest. 

This method could involve an exec resource definition with a command call.   This way you could run a PowerShell command with a forward slash.  Other files (with forward slashes) besides those with .ps1 extensions could similarly be transferred to a Windows server.

Answer #4  (use the provider key word):  A manifest with a resource definition of command can have a provider declaration too.  A manifest written like this will actually work:

command => '$construct = "schtasks /create /sc monthly /tn continualIntegration /tr c:\windows\system32\fun.exe /sd 09/08/2016"; invoke-expression $construct',
provider => 'powershell',

How Do You Find if MongoDB is Installed on Linux?

Problem scenario:  You are not sure if MongoDB is installed on Linux.  How do you find out for sure?

Solution:  Use this command to see if it is actively running:  ps -ef | grep mongo
However, MongoDB could be installed but not running.  So this command will not definitively answer the question.  On Debian (e.g., Ubuntu) or RedHat (e.g., CentOS or Fedora) distributions of Linux, use this command: mongod --version
The results will indicate if MongoDB is installed or not.  You'll see the version if it is installed.  The command will not work if MongoDB is not installed (e.g., with command not found).

Cloud Computing Is Changing the Global Economy

The Economist, specifically the August 27th - September 2nd 2016 issue, on page 47 and 48, in the article "Linux and AWS," reports four fascinating facts:

1) The world's I.T. budget is approximately $3,400,000,000,000 (almost 3.5 trillion) dollars.   
2) As of mid-2016, Google's market share for cloud services increased 162% from the previous year!  
3) Roughly 65% of the over one million AWS customers are startups.
4) AWS has computing capacity that is ten times greater than its closest 14 competitors combined! (As of early September 2016.)

Update as of February 2021: If you combine IaaS and PaaS, AWS accounts for 32% of the market according to this website.

Update in early 2022: Worldwide spending on I.T. is predicted to be $4.5 trillion dollars for 2022 (according to Gartner). To put that in perspective, the GDP of the U.S. was estimated to be roughly $23 trillion dollars in 2021 (according to the Bureau of Economic Analysis).

How Do You Unzip Files Using Ubuntu (Using the Command Line Interface)?

Goal:  You want to use unzip with Ubuntu Linux without using the GUI desktop.
Background:  The unzip character-based utility is useful for many reasons.  At the command line you can access compressed files that normally occupy little space on the disk.  Transferring .zip files is great for keeping network bandwidth utilization low.

Try this command to see if unzip is installed: man unzip

Part 1:  Installing Unzip
If the command cannot be found, use this command:  sudo apt-get install unzip

If the above command does not work, (for example, you do not have access to the Internet, and your network has no Debian package repositories configured), get the file from here:  https://packages.debian.org/wheezy/unzip

To find the correct installation media (and choose the correct link from the website above), you need to determine the architecture.  To do this, use this command: dpkg --print-architecture

Once you get the correct file on to Ubuntu, install it with this command: dpkg -i unzip.*.deb

Part 2:  Using Unzip
Now that unzip is installed, you can use it with a command like this (assuming you have a file named foobar.zip):

unzip foobar.zip

The above command will decompress the contents while retaining a copy.  The extracted contents will be decompressed in the same directory you run the command in.  The extracted files will not go to the directory where foobar.zip is if you use the path of foobar.zip in the invocation of the unzip command (e.g., if you are in /root/ and run unzip /tmp/foobar.zip the extracted files will go into the /root/ directory).

How Do You Zip Files Using Ubuntu Linux From The Command Line Interface?

Goal:  You want to use zip with Ubuntu Linux without using the GUI.  You want to use the character prompt exclusively.

Background:  The zip character-based utility is useful for many reasons.  At the command line you can free up space while retaining files for future usage.  Try this command to see if zip is installed: man zip

Try this command to see if zip is installed: man zip

Part 1: Installing Zip
If the command cannot be found, use this command:

sudo apt-get install zip

If you do not have access to the Internet, and your network has no Debian package repositories configured, get the file from here:  https://packages.debian.org/wheezy/zip

To find the correct installation media (and choose the correct link from the website above), you need to determine the architecture.  To do this, use this command: dpkg --print-architecture

Once you get the file over, install it with this command: dpkg -i zip.*.deb

Part 2  How to Use Zip
Now that zip is installed, you can use it with a command like this (assuming you have three regular files named pub.txt, pub1.txt, pub2.txt):

zip target.zip pub.txt pub1.txt pub2.txt

The above command will create a zip file called target.zip.  The files will be compressed (that is, the .zip file will be smaller than the sum of the three .txt files' sizes).  This can help reduce disk I/O when fetching the file, moving it around, and reduce network congestion with fewer packets over the network when it is in transit.