Topics covered: RDS,DynamoDB
What RDS is used for?
Baseline I/O performance for General Purpose SSD storage is 3 IOPS for each GiB. This relationship means that larger volumes have better performance. For example, baseline performance for a 100-GiB volume is 300 IOPS. Baseline performance for a 1-TiB volume is 3,000 IOPS. And baseline performance for a 5.34-TiB volume is 16,000 IOPS.
Volumes below 1 TiB in size also have ability to burst to 3,000 IOPS for extended periods of time. Burst is not relevant for volumes above 1 TiB. Instance I/O credit balance determines burst performance. For more information about instance I/O credits see, I/O Credits and Burst Performance.
Many workloads never deplete the burst balance, making General Purpose SSD an ideal storage choice for many workloads. However, some workloads can exhaust the 3000 IOPS burst storage credit balance, so you should plan your storage capacity to meet the needs of your workloads.
What happens when we modify the storage
Auto Minor Version Upgrade is a feature that you can enable to have your database automatically upgraded when a new minor database engine version is available.
The version numbering sequence is specific to each database engine. For example, Amazon RDS MySQL 5.7 and 8.0 are major engine versions and upgrading from any 5.7 version to any 8.0 version is a major version upgrade. Amazon RDS MySQL version 5.7.22 and 5.7.23 are minor versions and upgrading from 5.7.22 to 5.7.23 is a minor version upgrade.
For major version upgrades, you must manually modify the DB engine version through the AWS Management Console, AWS CLI, or RDS API. For minor version upgrades, you can manually modify the engine version, or you can choose to enable auto minor version upgrades.
A major version upgrade can take longer than a minor version upgrade. You can also run into compatibility issues during or after a major version upgrade, which are very unlikely after a minor version upgrade.
For both minor and major version upgrades, the system takes the following steps during the upgrade process for MySQL and MariaDB for Amazon RDS:
- A snapshot is taken (if backups are enabled) while the instance is still running the previous version.
- The instance shuts down in a special mode called a slow shutdown to ensure data consistency.
- The instance is restarted with mysqld running the new engine version with networking disabled to prevent remote connections.
- The mysql_upgrade program runs on the database to convert system and user tables to the new version.
- An RDS-specific upgrade script runs to upgrade RDS-managed tables and stored procedures.
- The instance is restarted with networking enabled to allow for remote connections.
The main difference between the process for a minor version upgrade and a major version upgrade is the use of the mysql_upgrade program. This program checks all tables (both system and user) on the database. Depending on the state of a table, the program might need to physically recreate the table to conform to the new version. In most cases, no physical changes are needed and mysql_upgrade merely updates metadata in the table to indicate that the table has been upgraded.
In the case of a minor version upgrade, these table checks are usually trivial. The entire process usually takes only a few seconds to a few minutes depending on the number of tables. However, a major version upgrade can require more extensive changes to tables, as discussed later.
MultiAZ fallacy about upgrades
One other caveat about upgrade downtime is how Multi-AZ fits into the picture. One common fallacy is that Multi-AZ configurations prevents downtime during an upgrade. We do recommend that you use Multi-AZ for high availability, because it can prevent extended downtime due to hardware failure or a network outage. However, in the case of a MySQL or MariaDB engine upgrade, Multi-AZ doesn’t eliminate downtime. The slow shutdown and the physical changes made on the active server by the mysql_upgrade program require this downtime.
RDS Read Replica
Backups and MultiAZ
IO suspension doesn’t happen for Multi-AZ. From AWS RDS user guide “During the automatic backup window, storage I/O might be briefly suspended while the backup process initializes (typically under a few seconds) and you might experience a brief period of elevated latency. No I/O suspension occurs for Multi-AZ DB deployments because the backup is taken from the standby.”
Max Connections in SQL SERVER
Amazon RDS encrypted DB instances provide an additional layer of data protection by securing your data from unauthorized access to the underlying storage. You can use Amazon RDS encryption to increase data protection of your applications deployed in the cloud, and to fulfill compliance requirements for data-at-rest encryption.
You can encrypt your Amazon RDS DB instances and snapshots at rest by enabling the encryption option for your Amazon RDS DB instances. Data that is encrypted at rest includes the underlying storage for DB instances, its automated backups, Read Replicas, and snapshots.
Amazon RDS encrypted DB instances use the industry standard AES-256 encryption algorithm to encrypt your data on the server that hosts your Amazon RDS DB instances. After your data is encrypted, Amazon RDS handles authentication of access and decryption of your data transparently with a minimal impact on performance. You don’t need to modify your database client applications to use encryption.
Amazon RDS also supports encrypting an Oracle or SQL Server DB instance with Transparent Data Encryption (TDE).
TDE automatically encrypts data before it is written to storage, and automatically decrypts data when the data is read from storage.
TDE can be used with encryption at rest, although using TDE and encryption at rest simultaneously might slightly affect the performance of your database. You must manage different keys for each encryption method.
Amazon RDS supports TDE for the following SQL Server versions and editions:
– SQL Server 2017 Enterprise Edition
– SQL Server 2016 Enterprise Edition
– SQL Server 2014 Enterprise Edition
– SQL Server 2012 Enterprise Edition
– SQL Server 2008 R2 Enterprise Edition
- You can only enable encryption for an Amazon RDS DB instance when you create it, not after the DB instance is created.However, because you can encrypt a copy of an unencrypted DB snapshot, you can effectively add encryption to an unencrypted DB instance. That is, you can create a snapshot of your DB instance, and then create an encrypted copy of that snapshot. You can then restore a DB instance from the encrypted snapshot, and thus you have an encrypted copy of your original DB instance. For more information, see Copying a Snapshot.
- DB instances that are encrypted can’t be modified to disable encryption.
- You can’t have an encrypted Read Replica of an unencrypted DB instance or an unencrypted Read Replica of an encrypted DB instance.
- Encrypted Read Replicas must be encrypted with the same key as the source DB instance.
- You can’t restore an unencrypted backup or snapshot to an encrypted DB instance.
- To copy an encrypted snapshot from one AWS Region to another, you must specify the KMS key identifier of the destination AWS Region. This is because KMS encryption keys are specific to the AWS Region that they are created in.The source snapshot remains encrypted throughout the copy process. AWS Key Management Service uses envelope encryption to protect data during the copy process. For more information about envelope encryption, see Envelope Encryption.
Point in time recovery
- New Script migration tool from Windows to Linux. You can use it in any virtual machine hosted in AWS or located in your datacenter as long as SQL SERVER is 2008 and above, and import to an EC2 instance running SQL SERVER 2016.
- Import S3 Data into PostgreSQL: csv,text or binary supported from copy command of PostgreSQL
- Storage AutoScaling:
- automatically scales storage capacity with 0 downtimes. Applies to Mysql, PostgreSQL, Oracle, MariaDB. All you have to do is set up maximum storage limit and AutoScaling will monitor storage consumption and scale when required.
- Can be enabled to existing or new RDS with no additional cost.
- RDS Now Available on VMWARE
- bringing many of the benefits of Amazon Relational Database Service (RDS) to your on-premises virtualized environments. These benefits include the ability to provision new on-premises databases in minutes, make backups, and restore to a point in time. You get automated management of your on-premises databases, without having to provision and manage the database engine.
- Microsoft SQL Server, PostgreSQL, and MySQL.
- Deletion protection, for Aurora, RDS.
- Publish RDS PostgreSQL logs to CloudWatch
- Data Warehouse
- OLAP transaction
- Single node (160GB)
- Leader node, manages client connections
- Computer node, up to 128 compute nodes
- Columnar Data Storage: instead using rows, AWS Redshift organizes the data by column.
- Advanced Compressión: can be compressed much more than row-based because similar data is stored sequentially on disk. Dont require indexes or views, uses less space than traditional relational database systems. When loading data in a table, AWS Redshift automatically samples your data and selects most appropriate compressions scheme.
- Massively Parallel Processing: distributes data and query loads across all nodes.
- Encrypted in transit using SSL
- Encrypted at rest using AES-256
- automatic workload management for Redshift. Now makes it easy to maximize query throughput and get consistent performance for your most demanding analytics workloads. Automatic workload management (WLM) uses machine learning to dynamically manage memory and concurrency helping maximize query throughput. In addition, you can now easily set the priority of your most important queries, even when hundreds of queries are being submitted.
- 5x better performance than mysql
- 10 GB of allocated space at start, 10 GB increments up to 64 TB (autoscaling storage!!!!!!!)
- can scale up to 32vCPUS 244GB of RAM, if you scale there is a maintenance window, couple of minutes
- 2 copies of your db in each of availability zone, with a minimum of a 3 availability zone. 6 copies of your data.
- designed to handle the loss of up to two copies of data without affecting database write availability and up to three copies of without affecting read availability.
- storage is self-healing.
- Account metrics: you can set up to send an alert if you are close to your account limits, you find the metrics in CloudWatch like the maximum number of reads, etc.