Download multiple files from s3

Log in Sign up. Best strategy for downloading multiple files from S3. Report spam. Justin Miller asked December 29, pm in General. Here are the ideas I've come up with so far: Strategy 1: Rails creates a zip file and streams the zip to the user. Pros: Eliminates the need to create the zip file on the fly. Users can pull directly from S3. Cons: Any change to files means the zips need to be deleted and recreated. Need to still be able to serve up files if the zip copy doesnt exist on S3.

Strategy 3: Post processing on the AWS side to create a zip whenever a bucket changes. Pros: Seems like a clean way to do it. Cons: No idea how to actually implement this and the AWS documentation is pretty poor.

Would need a way to have AWS inform my app after processing has been completed. Pros: Simplest to implement Cons: Worst user experience. I don't feel like it would make sense to automatically create zip files whenever a bunch changes.

I know that Dropbox will initialize a download as a background job and email you a link to it when it's finished. This could be a good approach, and you could dynamically spin up an EC2 server to run that download and zip, then kill the server afterwards. This is obviously useful if you're doing larger downloads that might need a large amount of disk space or ram for compressing. Do you have an idea of what usage you're expecting?

Chris, thanks for the very detailed response. Thanks for the advice! Jose - Nov SnowBit - Nov Ashutosh Dubey - Oct Nilanth - Nov 6. DEV Community is a community of , amazing developers We're a place where coders share, stay up-to-date and grow their careers.

Who owns this outage? Building intelligent escalation chains for modern SRE. Podcast Who is building clouds for the independent developer? Featured on Meta. Now live: A fully responsive profile.

Reducing the weight of our footer. Related Hot Network Questions. Question feed. Stack Overflow works best with JavaScript enabled. In this post, I will give you a more personalized review of each of the courses on my list. These co You ca We will create a serverless app together. And I will explain to you each and every step which will h So, in this blog post, I will share with you some of the best courses that I found.

These courses wi About Me. Close Menu. Posts Open Menu. Share this post. Post written by Abhishek Sharma. But wait Open the S3 console Click on the bucket from which you want to download the file Select all the files which you want to download and click on Open.

Look at the picture below.