How Do You Control, Manage, List, Upload and Download Files to and from S3 without Using the GUI?

Problem scenario
You are using a Debian distribution of Linux.  You want to upload and download files to S3 without using the GUI.  You want to be able to automate processes with scripts that interact with S3 from your Ubuntu Linux server.  How do you do this?

The procedures have two parts after you have the prerequisites met.

i. You need to have python-setuptools to be installed. To test if it is installed, run these two commands:

from setuptools import setup

If you get a message like "ImportError" or it is not installed, see this posting.

ii. You need the clock on your server to be configured to be relatively accurate. For servers that deviate significantly, s3cmd will not work. To configure ntp, see this external posting.

Part 1 Set up s3cmd
1.  Download the installation media.  Get the latest .tar.gz file from here (at least 2.0).
2.  Save it to the /tmp/ directory initially because the write permissions are usually relaxed.  Then copy it to /opt/.
3.  Run this command (but substitute s3cmd-* if there is more than one file with this file name prefix to identify the file you just downloaded):

sudo tar -xvf s3cmd-*

4.  Run these three commands:

cd s3cmd*
sudo python build
sudo python install

5.  Run this command and read the steps below to help answer the prompts you will see:

sudo s3cmd --configure

# Respond to the prompts of the above command.  If you do not know how to get the Access Key ID or the Secret Access Key, go to this URL and look at "Part 2."

For the encryption password, make up any value you want.

You can accept the default for this prompt:

"Path to GPG program [/usr/bin/gpg]:"

For use "HTTPS Protocol" you can enter "No" unless you want a different option.

For this prompt, just hit enter:

"HTTP Proxy server name:"

For test access, choose "Y."

Then save the settings if it was successful.

Part 2  Use s3cmd to test
Run this command:
s3cmd ls

To learn more about the s3cmd commands, go to this site.

Leave a comment

Your email address will not be published. Required fields are marked *