PoshJosh's Blog

Notes on Amazon Web Services 2 - Storages databases compute and content delivery

March 02, 2020

AWS Storages

  • S3 - Store and access any type of data over the internet. Create a bucket and

upload objects to the bucket. The bucket grows as objects are added.. theoritically unlimited size.

  • Glacier - Cheapest storage, designed for archiving of data (so not readily accessible). One could set up lifecycle rule to auto migrate old data from S3 to glacier for archiving.

  • Elastic Block Store (EBS) - Highly available, low latency storage for attaching

to servers launched via EC2 service. Like attaching a hard drive to computer system.

  • Elastic File System (EFS) - Network attached storage for EC2 service. Allows

multiple servers access same data source.

  • Storage Gateway - Enables hybrid storage between on premises local and AWS

cloud. More frequently accessed data stored on premises while less frequently accessed data is stored on the AWS cloud.

  • Snowball - Portable peta byte scale data storage. Download your data to a Snow

ball device and send to AWS which will upload the data to you S3 bucket.

AWS Clould Sample Usage

Create own VPC in AWS. Our VPC is a secure fortress. Launch servers in the VPC Attach EBS device, one for each server. Each server then has access to own EBS. For both servers to share storage (i.e network), we use EFS with mount target to each server.

Automated solution which automaticaly migrates old data for archiving.

  • Use S3 with lifecycle rule to auto migrate old data from S3 to glacier for archiving.
  • S3 bucket is located in the AWS cloud not our VPC. So create a VPC endpoint to allow traffic flow in and out of our VPC.
  • AWS storage gateway ensures data between local data centre and AWS cloud is synced. It will store copies of frequently accessed data in on site storage, whereas all data in amazon S3 buket.

AWS Database Services

  • Relational Database Service (RDS) - For relational database like MySQL, Oracle, PostgreSQL, Amazon Aurora (Amazons MySQL compatible database)
  • Amazon Dynamo Db - serverless NoSQL database service.
  • Amazon Redshit - Fast, fully managed petabyte database good for big data storage.
  • ElasticCache - In memory datastore or cache, in the cloud. Faster than disk based database services like the one listed above.
  • Database Migration Service (DMS) - for migrating from one amazon or other db to another.
  • Amazon Neptune - Fast fully managed graph database service.

Create VPC in AWS

  • Migrate with DMS from local database to own VPC.
  • Use ElasticCache for frequently accessed data. Check cache first then actual db.
  • Goto: Management Console -> Services -> Database Services -> RDS and follow steps to setup a MySQL database.
  • Under Backups: Set backup retention period to zero (0) to prevent backups which cost money.

AWS Compute Services

  • EC2 - Provide virtual servers in the AWS cloud.
  • EC2 Autoscaling - Allows dynamic scaling of EC2 capacity based on conditions specified.
  • Lightsail - Easiest way to launch virtual servers running applications.. provides dns management etc
  • Elastic Container Service (ECS) - For docker images
  • AWS Lambda -

AWS Cloud

  • Own VPC instance
  • EC2 instances
  • Elastic Load balancing receives traffic from end users and distribute the traffic to an available EC2 instance. It will balance the load accross multiple EC2 instances.

If an EC2 instance is unhealthy.. the EC2 instance will file a report with the Elastic load balance which will no-longer send traffic its way

  • Autoscaling Service handles creating, terminating and performing health checks for EC2 instances.

Networking & Content Delivery)

  • CloudFront - CDN securely delivers content to various endpoints providing - protection against DDoS Attacks
  • Virtual Private Cloud (VPC)
  • AWS Direct Connect - High speed direct connection to AWS cloud.
  • Elastic Load Balancing (ELB)
  • Route 53 - Domain Name System (DNS)
  • API Gateway - Fully managed serverless service which helps developers develop and deploy APIs.

Example Usage CDN with Multiple Availability Zones

  • ELS can distribute traffic across multiple availability zones
  • Use CDN for content which does not change often which needs to be delivery with high speed and low latency e.g images, videos.
  • Use CloudFront distribution to handle the above.
  • For other requests, CloudFront forwards to ELB which forwards to EC2 instance.
  • The CloudFront distribution will have its own DNS name by which we can browse to the content. DNS name will be quite user unfriendly (long unweidly URI).
  • Route 53 will is a DNS which provide user friendly names and translates requests to that name to the actual unfriendly URI which will be used to connect to CloudFront.

Quick Deployment of Wordpress

  • Management Console -> Services -> Compute Services -> EC2
  • Click on Instances -> Launch and Instance
  • Choose Amazon Machine Image (AMI)
  • Go to AWS Market place to select Wordpress AMI
  • Enable public IP
  • To connect to the linux OS of the instance use a key. So create a key pair.
  • Follow instructions to Review and Launch the instance
  • Wordpress password is in the logs.
  • To find your password: Management console -> Actions - Settings - Logs

Acronyms

  • VPC - Virtual Private Cloud
  • S3 - Simple Storage Service
  • EBS - Elastic Block Store
  • RDS - Relational Database Service
  • ELB - Elastic Beanstalk
  • ECS - Elastic Container Service
  • DNS - Domain Name Service

Notes

S3 Bucket names must be unique across the whole AWS buckets.

Links

References


Written byChinomso IkwuagwuExcélsior

Limited conversations with distributed systems.

Modifying legacy applications using domain driven design (DDD)

Gherkin Best Practices

Code Review Best Practices

Hacking Cypress in 9 minutes

Some common mistakes when developing java web applications

How to make a Spring Boot application production ready

SQL JOINS - A Refresher

Add Elasticsearch to Spring Boot Application

Add entities/tables to an existing Jhipster based project

CSS 3 Media Queries - All over again

Maven Dependency Convergence - quick reference

Amazon SNS Quick Reference

AWS API Gateway Quick Reference

Amazon SQS Quick Reference

AWS API Gateway Quick Reference

AWS Lambda Quick Reference

Amazon DynamoDB - Quick Reference

Amazon Aurora

Amazon Relational Database Service

AWS Database Services

AWS Security Essentials

Amazon Virtual Private Cloud Connectivity Options

Summary of AWS Services

AWS Certified Solutions Architect - Quick Reference

AWS CloudFront FAQs - Curated

AWS VPC FAQs - Curated

AWS EC2 FAQs - Curated

AWS Achritect 5 - Architecting for Cost Optimization

AWS Achritect 4 - Architecting for Performance Efficiency

AWS Achritect - 6 - Passing the Certification Exam

AWS Achitect 3 - Architecting for Operational Excellence

AWS Achitect 2 - Architecting for Security

AWS Achitect 1 - Architecting for Reliability

Amazon DynamoDB Accelerator (DAX)

Questions and Answers - AWS Certified Cloud Architect Associate

Questions and Answers - AWS Certified Cloud Architect Associate

AWS Connectivity - PrivateLink, VPC-Peering, Transit-gateway and Direct-connect

AWS - VPC peering vs PrivateLink

Designing Low Latency Systems

AWS EFS vs FSx

AWS Regions, Availability Zones and Local Zones

AWS VPC Endpoints and VPC Endpoint Services (AWS Private Link)

AWS - IP Addresses

AWS Elastic Network Interfaces

AWS Titbits

Jenkins on AWS - Automation

Jenkins on AWS - Setup

Jenkins on AWS - Best practices

Introduction to CIDR Blocks

AWS Lamda - Limitations and Use Cases

AWS Certified Solutions Architect Associate - Part 10 - Services and design scenarios

AWS Certified Solutions Architect Associate - Part 9 - Databases

AWS Certified Solutions Architect Associate - Part - 8 Application deployment

AWS Certified Solutions Architect Associate - Part 7 - Autoscaling and virtual network services

AWS Certified Solutions Architect Associate - Part 6 - Identity and access management

AWS Certified Solutions Architect Associate - Part 5 - Compute services design

AWS Certified Solutions Architect Associate - Part 4 - Virtual Private Cloud

AWS Certified Solutions Architect Associate - Part 3 - Storage services

AWS Certified Solutions Architect Associate - Part 2 - Introduction to Security

AWS Certified Solutions Architect Associate - Part 1 - Key services relating to the Exam

AWS Certifications - Part 1 - Certified solutions architect associate

AWS Virtual Private Cloud (VPC) Examples

Curated info on AWS Virtual Private Cloud (VPC)

Notes on Amazon Web Services 8 - Command Line Interface (CLI)

Notes on Amazon Web Services 7 - Elastic Beanstalk

Notes on Amazon Web Services 6 - Developer, Media, Migration, Productivity, IoT and Gaming

Notes on Amazon Web Services 5 - Security, Identity and Compliance

Notes on Amazon Web Services 4 - Analytics and Machine Learning

Notes on Amazon Web Services 3 - Managment Tools, App Integration and Customer Engagement

Notes on Amazon Web Services 2 - Storages databases compute and content delivery

Notes on Amazon Web Services 1 - Introduction

AWS Auto Scaling - All you need to know

AWS Load Balancers - How they work and differences between them

AWS EC2 Instance Types - Curated

Amazon Web Services - Identity and Access Management Primer

Amazon Web Services - Create IAM User

Preparing Jenkins after Installation

Jenkins titbits, and then some

Docker Titbits

How to Add Chat Functionality to a Maven Java Web App

Packer - an introduction

Terraform - an introduction

Versioning REST Resources with Spring Data REST

Installing and running Jenkins in Docker

Automate deployment of Jenkins to AWS - Part 2 - Full automation - Single EC2 instance

Automate deployment of Jenkins to AWS - Part 1 - Semi automation - Single EC2 instance

Introduction to Jenkins

Software Engineers Reference - Dictionary, Encyclopedia or Wiki - For Software Engineers