21 Dec 2017, 21:40

Upload to S3 from URL

I recently had the need to transfer a large file from a remote host to S3 but had insufficient local storage to make a temporary local copy. Fortunately AWS command line tools allows for this by reading the piped output of curl as follows:

curl https://remote-server/file | aws s3 cp - s3://mybucket/file