Ran into a brick wall when trying to upload some of these large video files to the Amazon S3 buckets. Tried converting the 2Gb QuickTime file to an .mp4
which was a quarter of that size, but still no luck. Even tried uploading a zip-compressed version of that file, not yet knowing if I’d be able to unzip it.
Soon learned that AWS offers Amazon S3 Multi-part uploads, which can be achieved on OSX with an App like MCSTools-S3, which is fairly inexpensive at like twenty-something bucks, or – my choice: a command line library developed in Python by Michal Ludvig called s3cmd. Free and open source.
What a nice alternative to the web portal, too.
I created a user for myself via the Amazon IAM
service and for permissions attached a policy that grants AmazonS3FullAccess
. Then simply followed the s3cmd docs to s3cmd --configure
(Configuration saved to '/Users/username/.s3cfg'
).
Awesomely it automatically uses Multi-part upload for files larger than 15mb – but it can be s3cmd multi-part upload disabled or configured.
In order for items to be publicly accessible via the web, permissions need to be configured accordingly, which can also be done with s3cmd. There’s a relative full list of commands here in the docs.
UPDATE: With the above policy, I was unable to s3cmd cflis
so under IAM policies I added a policy, AdministratorAccess
to the user I’m using s3cmd
with.