NOTES TO SELF:
Cloud Evolution: Data center -> IAAS –> PAAS –> Containers –> Serverless
Lambda is the ultimate abstraction layer:
Using lambda:
Lambda functions can trigger other lambda functions
API gateway does the load balancing. A million requests will launch million lambda functions.
serverless architecture:
user -> api gateway -> lambda -> RDS
Languages:
Pricing:
EXAM TIPS:
API gateway is the trigger.
distributed queue system.
decouple components of an application.
SQS eases message management between components
256kB of Text in any format.
Messages can be kept in the queue from 1 minute to 14 days; default retention period is 4 days.
Types:
Visibility Timeout default is 12 hours.
SQS Long Polling saves Money.
SWS is a web service that makes it easy to coordinate work across distributed application components. SWS enables application for a range of use casess to be designed as a coordination of tasks.
Tasks represent invocations of various processing steps in a application which can be performed by executable code, web service calls, human actions, an scripts. Eg: the entire workflow of ordering a book on amazon.com and getting it delivered. Human Interactions
SWF Actors:
Flexible message delivery over multiple Transport protocols.
TOPIC iss an accesspoint for subscribers
SNSS is stored across multiple AZs.
NO PULL only push
REVIEW: Topics
fully managed service that makes it eassy for developers to publish, maintain, monitor and secure APIs at any scale.
“Front door”. Distribute load (somewhat like a load-balancer).
Throttle gateway to prevent attacks
Log results to CLoudwatch
Configuring an API Gateway:
Deploying an API gateway:
API gateway caching:
Same origin policy
CORS - Cross origin resource shaging
Streaming data. Examples - purchases, stocks, game data, social network data, geospatial data, IoT sensor data.
Kinesis is a platform on AWS to ssend your streaming data to.
Kinesis Streams:
Shards – 5 transactions/ for reads. Maximum total data read rate of 2MBPS and upto 1,000 records per second for write, up to max total data write rate of 1MBPS, including partition keys.
Data capacity of steram = Sum(capacity of shard)
Kinesis Firehose:
Kinesis Analytics: [TODO]
REVIEW: clear distinction between Kinesis and SQS.
web id provider – amazon, facebook, google … following succesful authentication, user receives an authn code from the web id provider, which they can trade for temp AWS security credentials.
cognito is a BROKER.
Cognito synchronization – uses SNS to push updates and sync user data across multiple devices.
Could be:
Typically web facing.
you can have private LBs (inside VPC)
AUto scaling group needs “launch configuration”.
Auto scaling group:
Advanced details:
Scaling policy:
x
and y
instancesYou can add a notification when ACG kicks off. (SMS).
TARGET GROUP
index.html
health checks. ++ success code.
add targets (EC2 instances)
THEN create load balancer.
You can add RULES (intelligent) to ALBs (compared to Classic LB).
REVIEW: ELB FAQ for CLASSIC Load Balancers
Classic LBs routes each request independently to the registeed EC2 instance with smallest load.
Sticky Sessions allow you to bind a user’s session to a specific EC2 instance.
Sticky session can be enabled for ALBs, but the traffic will be sent at the TargetGroup level.
Useful if you are storing information locally to that instance
Create listners with rules to forward requests based on URL path.
eg: route /api/foo
to TargetGroupA and /api/bar
to TargetGroupB.
REVIEW: IA vs One-zone, etc., S3 storage classes
REVIEW: Pricing of diff types of S3.
REVIEW: What is the max size of objects stored in S3. Is that a soft limit?
“Your proposed upload exceeds the maximum allowed object size.”. –> Design your application to use the Multipart Upload API for all objects.
How many S3 buckets can I have per account by default? == 100
Until 2018 there was a hard limit on S3 puts of 100 PUTs per second. To achieve this care needed to be taken with the structure of the name Key to ensure parallel processing. As of July 2018 the limit was raised to 3500 and the need for the Key design was basically eliminated. Disk IOPS is not the issue with the problem. The account limit is not the issue with the problem.
The following options allows users to have secure access to private files located in S3:
CloudFront Signed URLs
Serving Private Content with Signed URLs and Signed Cookies - Amazon CloudFront
S3 has READ AFTER WRITE consistency for PUTs
S3 has overwrite PUTS not UPDATEs
S3 has Eventual consistenty for overwrite PUTS and DELETES.
Remember: Organization units are not part of IAM.
New users start with no permissions
Integrates with existing active directory account allowing single sign-on.
Configure Users and Policy Documents only once, as these are applied globally.
AWS CLI command to create a snapshot of an EBS volume? aws ec2 create-snapshot
encryption at rest. Using AWS managed keys to provide EBS encryption at rest is a relatively painless and reliable way to protect assets
EBS Root volume – the DeleteOnTermination attribute for root volumes is set to ‘true.’ this attribute may be changed at launch by using either the AWS Console or the command line. For an instance that is already running, the DeleteOnTermination attribute must be changed using the CLI. Terminate Your Instance - Amazon Elastic Compute Cloud
aws ec2 modify-instance-attribute --instance-id i-1234567890abcdef0 --block-device-mappings file://mapping.json
[
{
"DeviceName": "/dev/sda1",
"Ebs": {
"DeleteOnTermination": false
}
}
]
To know both the private IP address and public IP address of your EC2 instance, retrieve the instance Metadata from http://169.254.169.254/latest/meta-data/
curl http://169.254.169.254/latest/user-data
#prints boostrap info
curl http://169.254.169.254/latest/meta-data/
#prints all available metadata keys
curl http://169.254.169.254/latest/meta-data/public-ipv4
curl http://169.254.169.254/latest/meta-data/local-ipv4
#print public and local ipv4 addresses
Spread placement groups have a specific limitation that you can only have a maximum of 7 running instances per Availability Zone.
I can use the AWS Console to add a role to an EC2 instance after that instance has been created and powered-up.
One of the easiest options is to drive more I/O throughput than you can provision for a single EBS volume, by striping using RAID 0. You can join multiple gp2, io1, st1, or sc1 volumes together in a RAID 0 configuration to use the available bandwidth for these instances. You can also choose an EC2 instance type that supports EBS optimisation. This ensures that network traffic cannot contend with traffic between your instance and your EBS volumes. The final option is to manage your snapshot times, and this only applies to HDD based EBS volumes. When you create a snapshot of a Throughput Optimized HDD (st1) or Cold HDD (sc1) volume, performance may drop as far as the volume’s baseline value while the snapshot is in progress. This behaviour is specific to these volume types. Therefore you should ensure that scheduled snapshots are carried at times of low usage.
Underlying hypervisors for EC2 – Xen (old), Nitro (New). EC2 FAQ.
EBS, EFS, and FSx are storage services based on Block storage
Lowest cost EBS options – Throughput optimized (st1
) & Cold (sc1
).
You cannot delete a snapshot of an EBS Volume that is used as the root device of a registered AMI.
You cannot attach one EBS volume to more than one EC2 instance.
FSx = Fully managed third-party file systems. provides you with the native compatibility of third-party file systems with feature sets for workloads such as Windows-based storage, high-performance computing (HPC), machine learning, and electronic design automation (EDA).
Standard Reserved Instances cannot be moved between regions. You can choose if a Reserved Instance applies to either a specific Availability Zone, or an Entire Region, but you cannot change the region
VPC.= virtual data center in the cloud. logically isolated section of the AWS. completely control over virtual network.
/16
. /8
is not allowed.Private address ranges:
Tools:
What can we do with a VPC?
Default VPC vs custom VPC