How can I detect/deal with SES emails that are too large for S3?

Question

I have a RuleSet that places incoming messages into S3, then calls a lambda. However, if the message is larger than 30MB, the message is not placed into S3, and the lambda is never called.

How can I detect when this happens? Ideally I'd like to notify cloudwatch, or call another lambda. Currently the sender is not notified either, so these messages fail silently.

It seems quite unexpected that SES just silently discards the message. Have you confirmed that the message was really received by SES from the perspective of the sending side? Or is it possible that the sender is still retrying the mail, which means the sending system is still retrying and the message will eventually bounce? — Michael - sqlbot, Apr 30 '18 at 11:25
@Michael-sqlbot I have verified - when sending large emails they are dropped. I get no bounce message, the emails never hit the lambda at all. — user31415629, Apr 30 '18 at 14:55

score 1 · Accepted Answer · answered May 09 '18 at 14:38

You can configure receipt rules to send you notifications using Amazon SNS. Since you're placing objects within an S3 bucket in your RuleSet, an example of the notification sent to Amazon SNS could look like this:

{
"notificationType": "Received",
"receipt": {
"timestamp": "2015-09-11T20:32:33.936Z",
"processingTimeMillis": 406,
"recipients": [
    "recipient@example.com"
],
"spamVerdict": {
    "status": "PASS"
},
"virusVerdict": {
    "status": "PASS"
},
"spfVerdict": {
    "status": "PASS"
},
"dkimVerdict": {
    "status": "PASS"
},
"action": {
    "type": "S3",
    "topicArn": "arn:aws:sns:us-east-1:012345678912:example-topic",
    "bucketName": "my-S3-bucket",
    "objectKey": "\email"
}
},
"mail": {
"timestamp": "2015-09-11T20:32:33.936Z",
"source": "0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@amazonses.com",
"messageId": "d6iitobk75ur44p8kdnnp7g2n800",
"destination": [
    "recipient@example.com"
],
"headersTruncated": false,
"headers": [
    {
        "name": "Return-Path",
        "value": "<0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@amazonses.com>"
    },
    {
        "name": "Received",
        "value": "from a9-183.smtp-out.amazonses.com (a9-183.smtp-out.amazonses.com [54.240.9.183]) by inbound-smtp.us-east-1.amazonaws.com with SMTP id d6iitobk75ur44p8kdnnp7g2n800 for recipient@example.com; Fri, 11 Sep 2015 20:32:33 +0000 (UTC)"
    },
    {
        "name": "DKIM-Signature",
        "value": "v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug; d=amazonses.com; t=1442003552; h=From:To:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Date:Message-ID:Feedback-ID; bh=DWr3IOmYWoXCA9ARqGC/UaODfghffiwFNRIb2Mckyt4=; b=p4ukUDSFqhqiub+zPR0DW1kp7oJZakrzupr6LBe6sUuvqpBkig56UzUwc29rFbJF hlX3Ov7DeYVNoN38stqwsF8ivcajXpQsXRC1cW9z8x875J041rClAjV7EGbLmudVpPX 4hHst1XPyX5wmgdHIhmUuh8oZKpVqGi6bHGzzf7g="
    },
    {
        "name": "From",
        "value": "sender@example.com"
    },
    {
        "name": "To",
        "value": "recipient@example.com"
    },
    {
        "name": "Subject",
        "value": "Example subject"
    },
    {
        "name": "MIME-Version",
        "value": "1.0"
    },
    {
        "name": "Content-Type",
        "value": "text/plain; charset=UTF-8"
    },
    {
        "name": "Content-Transfer-Encoding",
        "value": "7bit"
    },
    {
        "name": "Date",
        "value": "Fri, 11 Sep 2015 20:32:32 +0000"
    },
    {
        "name": "Message-ID",
        "value": "<61967230-7A45-4A9D-BEC9-87CBCF2211C9@example.com>"
    },
    {
        "name": "X-SES-Outgoing",
        "value": "2015.09.11-54.240.9.183"
    },
    {
        "name": "Feedback-ID",
        "value": "1.us-east-1.Krv2FKpFdWV+KUYw3Qd6wcpPJ4Sv/pOPpEPSHn2u2o4=:AmazonSES"
    }
],
"commonHeaders": {
    "returnPath": "0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@amazonses.com",
    "from": [
        "sender@example.com"
    ],
    "date": "Fri, 11 Sep 2015 20:32:32 +0000",
    "to": [
        "recipient@example.com"
    ],
    "messageId": "<61967230-7A45-4A9D-BEC9-87CBCF2211C9@example.com>",
    "subject": "Example subject"
}
}
}

Now that you're pushing events to an SNS topic, you could have a Lambda function subscribed to your SNS topic that does the following:

Reads the notificationType key in the message and stops execution if the value is Received (helps save Lambda execution costs)
If the value for notificationType is anything else, the function retrieves the value for the from key (object value for the commonHeaders key and uses the SES API to send out an e-mail stating that no action was taken since their e-mail size exceeded your application's permissible limits [30 mb]

How can I detect/deal with SES emails that are too large for S3?

1 Answers1