[AWS]AWS Certified Solutions Architect - Day3

S3 - Security & Encryption

Securing your buckets

• By default, all newly created buckets are PRIVATE.

• You can setup access control to your buckets using;

  • Bucket Policies

  • Access Control Lists

• S3 buckets can be configured to create access logs which log all requests made to the S3 bucket. This can be done to another bucket.

Encryption

• In Transit;

  • SSL/TLS

• At Rest

  • Server Side Encryption

    • S3 Managed Key - SSE-S3

    • AWS Key Management Service, Managed Keys - SSE-KMS

    • Server Side Encryption With Customer Provided Keys - SSE-C

• Client Side Encryption

Storage Gateway

• AWS Storage Gateway is a sevice that connects an on-premises software appliance with cloud-based storage to provide seamless and secure integration between an organization's on-premises IT enviornment and AWS's storage infrastructure. The service enables you to securely store data to the AWS cloud for scalable and cost-effective storage.

• AWS Storage Gateway's software appliance is available for download as a virtual machine (VM) image that you install on a host in your datacenter. Storage Gateway supports either VMware ESXi or Microsoft Hyper-V. Once you've installed your gateway and associated it with you AWS account through the activation process, you can use the AWS Management Console to create the storage gateway option that is right for you.

Four Types of Storage Gateways

• File Gateway (NFS)

• Volumes Gateway (iSCSI)

  • Stored Volumes

  • Cached Volumes

• Tape Gateway (VTL)

File Gateway

• Files are stored as objects in your S3 buckets, accessed through a Network File System (NFS) mount point. Ownership, permissions, and timestamps are durably stored in S3 in the user-metadata of the object associated with the file. Once objects are transferred to S3, they can be managed as native S3 objects, and bucket policies such as versioning, lifecycle management, and cross-origin replication apply directly to objects stored in your bucket.

Volume Gateway

• The volume inteface presents your applications with disk volumes using the iSCSI block protocol.

• Data written to these volumes can be asynchronously backed up as point-in-time snapshots of your volumes, and stored in the cloud as Amazon EBS snapshots.

• Snapshots are incremental backups that capture only changed block. All snapshot storage is also compressed to minimize your storage charges.

Volume Gateway - Stored Volumes

• Stored volumes let you store your primary data locally, while asynchronously backing up that data to AWS. Stored volumes provide your on-premises applications with low-latency access to their entire datasets, while providing durable, off-site backups. You can create storage volumes and mount them as iSCSI devices from your on-premises application servers. Data written to your stored volumes is stored on your on-premises storage hardware. This data is asynchronously backed up to Amazon SImple Storage Service (Amazon S3) in the form of Amazon Elastic Block Store (Amazon EBS) snapshots. 1 GB - 16TB in size of Stored Volumes.

Volume Gateway - Cached Volume

• Cached volumes let you use Amazon Simple Storage Service (Amazon S3) as your primary data storage while retaining frequently accessed data locally in your storage gateway. Cached volumes minimize the need to scale your on-premises storage infrastructure, while still providing your applications with low-latency access to their frequently accessed data. You can create storage volumes up to 32 TB in size and attach to them an iSCSI devices from your on-premises application servers. Your gateway stores data that you write to these volumes in Amazon S3 and retains recently read data in your on-premises storage gateway's cache and upload buffer storage. 1 GB - 32TB in size for Cached Volumes.

Volume Gateway - Tape Gateway

• Tape Gateway offers a durable, cost-effective solution to archive your data in the AWS Cloud. The VTL interface it provides lets you leverage your existing tape-based backup application infrastructure to store data on virtual tape cartridges that you create on your tape gateway. Each tape gateway is preconfigured with a media changer and tape drives, which are available to your existing client backup applications as iSCSI devices. You add tape cartridges as you need to archive your data. Supported by NetBackup, Backup Exec, Veeam etc.

Exam Tips

• File Gateway - For flat files, stored directly on S3.

• Volume Gateway:

  • Stored Volumes - Entire Dataset is stored on site an is asynchronouly backed up to S3.

  • Cached Volumes - Entire Dataset is stored on S3 and the most frequently accessed data is cached on site.

• Gateway Virtual Tape Library (VTL)

  • Used for backup and uses popular backup applications like NetBackup, Backup Exec, Veeam etc.

Snowball

Import/Export Disk

• AWS Import/Export Disk accelerates moving large amounts of data into and out of the AWS cloud using portable storage devices for transport. AWS Import/Export Disk transfer your data directly onto and off of storage devices using Amazon's high-speed internal network and bypassing the Internet.

Types of Snowballs

• Snowball

• Snowball Edge

• Snowmobile

Snowball

• AWS Snowball is a service that accelerates transferring large amounts of data into and out of AWS using physical storage devices, bypassing the Internet. Each AWS Snowball device type can transport data at faster-than internet speeds. This transport is done by shipping the data in the devices through a regional carrier. The devices are rugged shipping containers, complete with E Ink shipping labels.

• With a Snowball, you can transfer hundreds of terabytes or petabytes of data between your on-premises data centers and Amazon Simple Storage Service (Amazon S3). AWS Snowball uses Snowball appliances and provides powerful interfaces that you can use to create jobs, transfer data, and track the status of your jobs through to completion. By shipping your data in Snowballs, you can transfer large amounts of data at a significantly faster rate than if you were transferring that data over the Internet, saving you time and money.

Exam Tips

• Understand what Snowball is

• Understand what Import Export is

• Snowball Can

  • Import to S3

  • Export from S3

S3 Transfer Acceleration

What is S3 Transfer Acceleration?

• S3 Transfer Acceleration utilizes the CloudFront Edge Network to accelerate your uploads to S3. Instead of uploading directly to your S3 bucket, you can use a distinct URL to upload directly to an edge location which will then transfer that file to S3. You will get a distinct URL to upload too;

Create A Static Website Using S3! - Lab

Exam Tips

• You can use bucket policies to make entire S3 buckets public.

• You can use S3 to host STATIC websites (such as .html). Websites that require database connections such as Wordpress etc cannot be hosted on S3.

• S3 Scales automatically to meet your demand. Many enterprises will put static websites on S3 when they think there is going to be a large number of requests (such as for a movie preview for example).

Exam Tips for S3 101

• Remeber that S3 is Object based i.e. allows you to upload files.

• Files can be from 0 Bytes to 5TB.

• There is unlimited storage.

• File are stored in Buckets.

• S3 is a universal namespace, that is, names must be unique globally.

• https://s3-eu-west-1.amazonaws.com/acloudguru

---

• Read after Write consistency for PUTS of new Objects.

• Eventual Consistency for overwrite PUTS and DELETES (can take some time to propagate)

---

Storage Tiers/Classes

• S3 Standard : 99% availability, 99% durability, stored redundantly across multiple devices in multiple facilities, and is designed to sustain the loss of 2 facilities concurrently.

• S3 - IA : (Infrequently Accessed) : For data that is accessed less frequently, but requires rapid access when needed. Lower fee than S3, but you are charged a retrieval fee.

• S3 One Zone - IA : want a lower-cost option for infrequently accessed data, but do not require the multiple Availability Zone data resilience.

• Glacier : Very cheap, but used for archival only. Expedited, Standard or Bulk. A Standard retrieval time takes 3 - 5 hours.

---

Exam Tips for S3 101

• Remember the core fundatmentals of S3;

  • Key(name)

  • Value(data)

  • Version id

  • Metadata

  • Access control lists

• Object based storage only (for files)

• Not suitable to install an OS on.

---

Versioning

• Stores all versions of object (including all writes and even if you delete an object)

• Great backup tool.

• Once enabled, Versioning cannot be disabled, only suspended.

• Integrates with Lifecycle rules.

• Versioning's MFA Delete capability, which uses multi-factor authentication, can be used to provide an additional layer of security.

• Cross Region Replication, requires versioning enabled on the source bucket.

---

S3 - Lifecycle Management

• Can be used in conjunction with versioning.

• Can be applied to current versions and previous versions.

• Following actions can now be done;

  • Transition to the Standard - Infrequent Access Storage Class (128Kb and 30 days after the creation date).

  • Archive to the Glacier Storage Class (30 days after IA, if relevant)

  • Permanently Delete

---

CloudFront - Exam Tips

• Edge location - This is the location where content will be cached. This is separate to an AWS Region/AZ.

• Origin - This is the origin of all the files that the CDN will distribute. This can be either an S3 Bucket, an EC2 Instance, and Elastic Load Balancer or Route53.

• Distribution - This is the name given the CDN which consists of a collection of Edge Locations.

  • Web Distribution - Typically used for Websites.

  • RTMP - Used for Media Streaming.

• Edge locations are not just READ only, you can write to them too. (ie put an object on to them).

• Objects are cached for the life of the TTL (Time To Live)

• You can clear cached objects, but you will be charged.

---

Securing your buckets

• By default, all newly created buckets are PRIVATE

• You can setup access control to your buckets using;

  • Bucket Policies

  • Access Control Lists

• S3 buckets can be configured to create access logs which log all requests made to the S3 bucket. This can be done to another bucket.

---

Encryption

• In Transit;

  • SSL/TLS

• At Rest

  • Server Side Encryption

    • S3 Managed Keys - SSE-S3

    • AWS Key Management Service, Managed Keys - SS-KMS

    • Server Side Encryption With Customer Provided Keys - SSE-C

• Client Side Encryption

---

Exam Tips

• File Gateway - For flat files, stored directly on S3.

• Volume Gateway

  • Stored Volumes - Entire Dataset is stored on site and is asynchronously backed up to S3.

  • Cached Volumes - Entire Dataset is stored on S3 and the most frequently accessed data is cached on site.

• Gateway Virtual Tape Library (VTL)

  • Used for backup and uses popular backup applications like NetBackup, Backup Exec, Veeam etc.

---

Snowball Exam Tips

• Snowball

• Snowball Edge

• Snowmobile

• Understand what Snowball is

• Understand what Import Export is

• Snowball Can

  • Import to S3

  • Export from S3

---

Exam Tips - S3 Transfer Acceleration

• You can speed up transfers to S3 using S3 transfer acceleration. This costs extra, and has the greatest impact on people who are in far away location.

---

Exam Tips - S3 Static Websites

• You can use S3 to host static websites

• Serverless

• Very cheap, scales automatically

• STATIC only, cannot host dynamic sites.

---

Last Few Tips

• Write to S3 - HTTP 200 code for a successful write.

• You can load files to S3 much faster by enabling multipart upload.

• Read the S3 FAQ before taking the exam. It comes up A LOT!