How I'm Leveraging Amazon S3

Amazon S3 is cloud storage for the Internet. It is designed to make web-scale computing easier for developers. Amazon S3 provides a simple web-services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web.

Amazon S3 allows me to securely (and inexpensively) store and archive my data.  With the number of devices that I own, and the number of times a year I reload Windows, having my critical documents, pictures and files securely stored via Amazon gives me piece of mind.

How do I use it today?

The Tool

In order to get the most control of what I want to do inside of Amazon S3, I use a command line tool called  S3 Sync by SprightlySoft.  You can leverage a 60 day free license to see if it suits your needs, then (as I willingly did), you can pay $29.99 for a permanent license that provides support for a year, as well as all in-version upgrades of the application.

You also need to have an active account for Amazon Web Services, so that you have your security credentials available.

The Script

First I start out by setting command line variables that I use in the event I don’t run this batch file interactively (backup.bat documents, for example):

:: Command Line Actions
if "%1" == "documents" set backuptype=documents
if "%1" == "documents" goto execute
i "%1" == "desktop" set backuptype=desktop
if "%1" == "desktop" goto execute
if "%1" == "utilities" set backuptype=utilities
if "%1" == "utilities" goto execute
if "%1" == "music" set backuptype=music
if "%1" == "music" goto execute

Next, I use the old DOS choice.exe to provide me with an interactive menu system:

echo What do you want to backup?
echo [A] Exit
echo [B] Desktop
echo [C] Documents
echo [D] Music
echo [E] Videos
echo [F] Utilities
echo [G] Installation Files
"c:Google~1Utilit~1choicechoice" /c:abcdefg /n Please make a selection...
if errorlevel 7 set backuptype=installations
if errorlevel 7 goto execute
if errorlevel 6 set backuptype=utilities
if errorlevel 6 goto execute
if errorlevel 5 set backuptype=videos
if errorlevel 6 goto execute
if errorlevel 4 set backuptype=music
if errorlevel 4 goto execute
if errorlevel 3 set backuptype=documents
if errorlevel 3 goto execute
if errorlevel 2 set backuptype=desktop
if errorlevel 2 goto execute
if errorlevel 1 goto SkipAll

Based on the selected option above, I set a variable called %backuptype% that is the key to the rest of the script. Next, I set a whole bunch of variables that are critical to the execution:

set "GoogleDrive=C:Google Drive"
set executable="%GoogleDrive%UtilitiesS3syncS3Sync.exe"
set ssexecutable="%GoogleDrive%UtilitiesS3SyncShadowSpawn-x64.exe"
set awsaccesskeyid=???
set awssecretaccesskey=???
set s3synclicense=???
set syncdirection=upload
set showtransferprogress=True
set s3folderkeyname=""
set outputlevel=1
set overwriteexistingexistinglogfile=True
set deletes3itemswherenotinlocallist=False
set s3folderkeyname=""
set localfoldrpath="%GoogleDrive%%backuptype%"
set bucketname=sanz-andre-backup-%backuptype%
set logfilepath="%GoogleDrive%S3_backup_%backuptype%.log"

if "%backuptype%" == "music" set ExcludeLocalFoldersRegularExpression=".*iTunes.*"
if "%backuptype%" == "documents" set ssmappeddrive=I:
if "%backuptype%" == "desktop" set ssmappeddrive=Z:
if "%backuptype%" == "utilities" set ssmappeddrive=L:
if "%backuptype%" == "music" set ssmappeddrive=J:

With the use of Shadow Spawn, I’m able to backup everything I want without the concern of files being open and not backed up.  I set different mapped drives based on %backuptype%, because this script is run as an automated backup solution for many things, and can be run multiple times with different command line options, so this prevents errors from trying to shadow the same drive letter.

Next we execute:

Title Backing up %backuptype% in other window...
%ssexecutable% %localfoldrpath% %ssmappeddrive% %executable% ^
-AWSAccessKeyId %awsaccesskeyid% ^
-AWSSecretAccessKey %awssecretaccesskey% ^
-LicenseKey %s3synclicense% ^
-RequestEndpoint %requestendpoint% ^
-SyncDirection %syncdirection% ^
-LocalFolderPath %ssmappeddrive% ^
-BucketName %bucketname% ^
-S3FolderKeyName %s3folderkeyname% ^
-OutputLevel %outputlevel% ^
-LogFilePath %logfilepath% ^
-OverwriteExistingLogFile %overwriteexistingexistinglogfile% ^
-ShowTransferProgress %showtransferprogress% ^
-ExcludeLocalFilesRegularExpression %ExcludeLocalFilesRegularExpression% ^
-ExcludeLocalFoldersRegularExpression %ExcludeLocalFoldersRegularExpression% ^
-DeleteS3ItemsWhereNotInLocalList %deletes3itemswherenotinlocallist%

Finally, I add a variable that will popup the created log file in the event that the batch file is run interactively:

:: Command Line Actions
if "%1" == "" "%GoogleDrive%UtilitiesNotepad++notepad++.exe" "%GoogleDrive%S3_%backuptype%.log"

A complete copy of this batch file can be downloaded here.

Leave a Reply

Your email address will not be published. Required fields are marked *