Provisioned Lambda timeout on Loading Secret Manager
Need help to make lambda able to read secrets. Get this error when provisioning lambda from docker Log: ``` [run-lambda-entrypoint] time="2023-02-07T02:24:55Z" level=debug msg="Loading Secret Manager entry arn:aws:secretsmanager:ap-southeast-3:*****:secret:***-MpQE8U as environment variables." [run-lambda-entrypoint] time="2023-02-07T02:25:06Z" level=debug msg="Loading Secret Manager entry arn:aws:secretsmanager:ap-southeast-3:*****:secret:***-MpQE8U as environment variables." START RequestId: 90f7d618-8fdf-4d96-b8c8-b31cd8a4e348 Version: $LATEST 2023-02-07T02:25:36.085Z 90f7d618-8fdf-4d96-b8c8-b31cd8a4e348 Task timed out after 30.03 seconds ``` ``` [run-lambda-entrypoint] time="2023-02-07T03:02:21Z" level=error msg="FAIL Loading Secret Manager entry arn:aws:secretsmanager:ap-southeast-3:***:secret:***-MpQE8U fail operation error Secrets Manager: GetSecretValue, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , request send failed, Post \"https://secretsmanager.ap-southeast-3.amazonaws.com/\": dial tcp 108.136.159.12:443: i/o timeout." ``` my configuration: ``` name = "edo-daily-installments-cron" image_uri = local.image run_in_vpc = true vpc_id = dependency.vpc.outputs.vpc_id subnet_ids = dependency.vpc.outputs.private_app_subnet_ids environment_variables = merge( include.envcommon.locals.environment_variables, { IMAGE_URI = local.image, SECRETS_MANAGER_ARN = local.edo_cron_workers_secrets_manager_arn, } ) iam_policy = { SecretsAccess = { actions = [ "secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret", "secretsmanager:ListSecretVersionIds", "secretsmanager:PutSecretValue", "secretsmanager:UpdateSecret", "secretsmanager:TagResource", "secretsmanager:UntagResource" ], resources = ["${local.edo_cron_workers_secrets_manager_arn}"] effect = "Allow" } } cloudwatch_log_group_retention_in_days = 14 ``` seems like related to https://aws.amazon.com/blogs/security/how-to-connect-to-aws-secrets-manager-service-within-a-virtual-private-cloud/ Thank you --- <ins datetime="2023-02-07T02:34:19Z"> <p><a href="https://support.gruntwork.io/hc/requests/109873">Tracked in ticket #109873</a></p> </ins>
Solved: steps: 1. create VPC endpoint ``` terraform { source = "git::git@github.com:gruntwork-io/terraform-aws-vpc.git//modules/vpc-interface-endpoint?ref=v0.22.4" } inputs = { vpc_id = dependency.vpc.outputs.vpc_id subnet_ids = dependency.vpc.outputs.private_app_subnet_ids create_https_security_group = true enable_secretsmanager_endpoint = true } ``` 2. make sure enable "should_create_outbound_rule" on lambda provisioning ``` run_in_vpc = true vpc_id = dependency.vpc.outputs.vpc_id subnet_ids = dependency.vpc.outputs.private_app_subnet_ids should_create_outbound_rule = true ```