About the S3 uploads, they cannot go direct to AWS servers. It has to go through the Instagram servers since the upload needs to be authenticated AND the S3 connection secrets can/should not be kept on the client end.
Hey Ravi! The diagram simplifies this a bit so that there aren’t too many arrows going everywhere. But yes, you are absolutely correct! The application server is involved in image uploads for the reasons you mentioned.
We don't want clients to upload images to the application servers, so the servers can then put them into your object store. This will just burn resources and slow things down. Instead, the application server will be used to create signed URL requests so that the client can upload directly to S3.
This was a really nice read that helps really solidify some system design theoretical concepts seeing how it's applied in the real-world.
I also appreciated the diagram at the end too to help piece it all together.
Thanks for the article!
I always appreciate your input Jordan. Thanks so much!
Thank you for this convenient post!
About the S3 uploads, they cannot go direct to AWS servers. It has to go through the Instagram servers since the upload needs to be authenticated AND the S3 connection secrets can/should not be kept on the client end.
Hey Ravi! The diagram simplifies this a bit so that there aren’t too many arrows going everywhere. But yes, you are absolutely correct! The application server is involved in image uploads for the reasons you mentioned.
We don't want clients to upload images to the application servers, so the servers can then put them into your object store. This will just burn resources and slow things down. Instead, the application server will be used to create signed URL requests so that the client can upload directly to S3.
Thanks for the article!
Keep things simple is all your need! In my daily work, I also asked our team to do things as simple as we can.
If we get to the situation that all engineers have to fix bugs. No innvocation can happen.
Maybe to identify the shard for the stored user data