Context
From time to time, colleagues ask me about logs from our production RDS instances to triage and troubleshoot issues with customers. Instead of going through the AWS Console and clicking my way to a solution, I prefer to use awslogs.
awslogs, as mentioned in the GitHub Readme, is a command line tool for querying groups, streams and events from AWS CloudWatch
Solution
For this, I assumed that the RDS instance(s) of interest have CloudWatch integration already configured. Otherwise, you need to set it up first. See here for details.
In order to download, you need to find out the which log group and stream to access for the particular RDS instance. The following steps will help you achieve that:
- Listing all CloudWatch groups for RDS instances. This will usually match with the type of query log enabled for each RDS instances. For MySQL logs in particular this would be general, error and slowquery. And the corresponding CloudWatch groups would be of the form
/aws/rds/instance/<rds_name>/<log_type>. Other RDS instances may support different set of logs, see here for more information.
awslogs groups | grep /aws/rds/instance
You can also filter it further by the RDS instance name
awslogs groups | grep /aws/rds/instance | grep <rds_name>
- You may have multiple streams within a log group in CloudWatch. Thus, you need to determine what stream to get the logs from:
awslogs streams <group_name>
- And finally getting a particular set of logs:
awslogs get <group_name> <stream> --start='2020-04-10 15:00:20' --end='2020-04-10 15:00:30' > stream.log
Notes
One recommendation is to try to reduce the interval size as much as possible, otherwise the amount of logs that you have to download can grow large. In my experience, downloading the logs for a 5 mins interval can easily be 100+ Mbs.