Gmail to S3 Attachment Workflow

System Architecture Overview

Gmail Trigger

→

Email Processing

→

Attachment Extraction

→

S3 Upload

→

Notification/Logging

Key Benefits

Fully automated attachment processing
Organized S3 storage structure
Built-in error handling and retries
No-code dashboard interface
Scalable and secure architecture

Phase 1: Environment Setup & Prerequisites

1.1 Gmail Integration Setup

Step 1.1.1: Configure Google Cloud Console

Go to Google Cloud Console
Create new project or select existing project
Navigate to "APIs & Services" → "Library"
Search and enable "Gmail API"

Step 1.1.2: Create OAuth Credentials

Go to "APIs & Services" → "Credentials"
Click "+ Create Credentials" → "OAuth client ID"
Select "Web application"
Add authorized redirect URI: https://passport.useparagon.com/oauth
Save Client ID and Client Secret

Step 1.1.3: Configure in Paragon Dashboard

Open Paragon Dashboard → "Integrations" → "Gmail"
Under "App Configuration", click "Configure"
Enter Client ID and Client Secret from Step 1.1.2
Select required scopes:
- https://www.googleapis.com/auth/gmail.readonly
- https://www.googleapis.com/auth/gmail.compose
Save changes

1.2 Amazon S3 Integration Setup

Step 1.2.1: AWS IAM Configuration

Create IAM user with programmatic access
Attach policy with S3 permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        }
    ]
}
            

Step 1.2.2: Configure S3 in Paragon

Paragon Dashboard → "Integrations" → "Amazon S3"
Enter AWS Access Key ID and Secret Access Key
Test connection and save

1.3 Recommended S3 Bucket Structure

your-bucket/
├── gmail-attachments/
│   ├── 2025/
│   │   ├── 01/
│   │   │   ├── 01/
│   │   │   │   ├── email-{timestamp}-{sender}/
│   │   │   │   │   ├── attachment1.pdf
│   │   │   │   │   └── attachment2.docx
            

Phase 2: Workflow Design & Architecture

2.1 Workflow Components Architecture

Detailed Flow:

Gmail Trigger

→

Email Validation

Attachment Check

→

Email Metadata Extraction

Attachment Processing Loop

→

File Type Validation

S3 Path Generation

→

S3 Upload

Upload Verification

→

Logging/Notification

2.2 Data Flow Design

Input Data Structure:

{
  "email": {
    "id": "email_id",
    "threadId": "thread_id",
    "sender": "sender@example.com",
    "subject": "Email Subject",
    "date": "2025-01-01T10:00:00Z",
    "attachments": [
      {
        "filename": "document.pdf",
        "mimeType": "application/pdf",
        "size": 1024000,
        "attachmentId": "attachment_id"
      }
    ]
  }
}
                

Phase 3: Detailed Workflow Implementation

3.1 Create New Workflow

Step 3.1.1: Initialize Workflow

Paragon Dashboard → "Workflows" → "Create New Workflow"
Name: "Gmail Attachments to S3 Processor"
Description: "Automatically processes Gmail attachments and uploads to S3"

3.2 Configure Gmail Trigger

Step 3.2.1: Add Trigger Step

Click "Add Trigger" → Select "Gmail"
Choose trigger type: "New Email" or "New Thread"
Configure trigger settings:
- Label Filter: (Optional) Specific Gmail label
- Query Filter: has:attachment to only trigger on emails with attachments
- Sender Filter: (Optional) Specific sender domains

Step 3.2.2: Test Trigger

Click "Test Trigger" button
Send test email with attachment to connected Gmail account
Verify trigger captures email data correctly

3.3 Email Processing Steps

Step 3.3.1: Add Email Validation Step

Add "Function" step after trigger
Name: "Validate Email Data"
JavaScript code:

function main(trigger) {
  const email = trigger.email;
  
  // Validate required fields
  if (!email.id || !email.attachments || email.attachments.length === 0) {
    throw new Error('Invalid email data or no attachments found');
  }
  
  // Filter out inline images and unwanted file types
  const validAttachments = email.attachments.filter(att => {
    const isInline = att.disposition === 'inline';
    const isImage = att.mimeType.startsWith('image/');
    const isValidSize = att.size > 0 && att.size < 26214400; // 25MB limit
    
    return !isInline && isValidSize;
  });
  
  return {
    email: email,
    processableAttachments: validAttachments,
    metadata: {
      sender: email.from,
      subject: email.subject.replace(/[^\w\s-]/g, ''), // Clean subject for folder naming
      timestamp: new Date().toISOString(),
      totalAttachments: validAttachments.length
    }
  };
}
            

Step 3.3.2: Add Attachment Extraction Step

Add "Gmail" integration step
Action: "Get Attachment"
Configure:
- Message ID: {{trigger.email.id}}
- Attachment ID: {{validation.processableAttachments[0].attachmentId}}
Enable "Continue workflow if step fails"

3.4 S3 Upload Processing

Step 3.4.1: Add Path Generation Function

Add "Function" step
Name: "Generate S3 Path"
JavaScript code:

function main(validation, attachment) {
  const metadata = validation.metadata;
  const date = new Date(metadata.timestamp);
  
  // Generate organized path structure
  const year = date.getFullYear();
  const month = String(date.getMonth() + 1).padStart(2, '0');
  const day = String(date.getDate()).padStart(2, '0');
  const hour = String(date.getHours()).padStart(2, '0');
  const minute = String(date.getMinutes()).padStart(2, '0');
  
  // Clean sender email for folder name
  const senderClean = metadata.sender.replace(/[^\w.-]/g, '_');
  const subjectClean = metadata.subject.substring(0, 50).replace(/\s+/g, '_');
  
  const basePath = `gmail-attachments/${year}/${month}/${day}`;
  const folderName = `${hour}${minute}_${senderClean}_${subjectClean}`;
  
  return {
    bucketName: 'your-s3-bucket-name',
    fullPath: `${basePath}/${folderName}/${attachment.filename}`,
    metadata: {
      'Content-Type': attachment.mimeType,
      'X-Gmail-Sender': metadata.sender,
      'X-Gmail-Subject': metadata.subject,
      'X-Gmail-Date': metadata.timestamp,
      'X-Gmail-MessageId': validation.email.id
    }
  };
}
            

Step 3.4.2: Add S3 Upload Step

Add "Amazon S3" integration step
Action: "Put Object"
Configure:
- Bucket: {{pathGeneration.bucketName}}
- Key: {{pathGeneration.fullPath}}
- Body: {{attachment.data}}
- Content-Type: {{pathGeneration.metadata.Content-Type}}
- Metadata: {{pathGeneration.metadata}}
Enable auto-retries (3 attempts)

3.5 Multiple Attachments Handling

Step 3.5.1: Add Fan-Out Step

Add "Fan Out" step after validation
Fan Out On: {{validation.processableAttachments}}
Variable Name: currentAttachment

Step 3.5.2: Restructure Attachment Processing

Move attachment extraction, path generation, and S3 upload steps inside Fan-Out
Update references to use {{currentAttachment}} instead of array indexing

3.6 Error Handling & Logging

Step 3.6.1: Add Error Handling Function

function main(s3Upload, pathGeneration, validation) {
  const result = {
    success: true,
    uploadedFile: pathGeneration.fullPath,
    fileSize: s3Upload.ContentLength || 'unknown',
    timestamp: new Date().toISOString(),
    emailId: validation.email.id
  };
  
  // Log successful upload
  console.log('File uploaded successfully:', result);
  
  return result;
}
            

Step 3.6.2: Add Notification Step (Optional)

Add "Request" step for webhook notification
Or add "Slack" step to notify team channel
Configure with upload summary

3.7 Workflow Optimization Settings

Step 3.7.1: Configure Workflow Settings

Timeout: 300 seconds (5 minutes)
Retry Policy: Exponential backoff
Concurrency: Limit to 5 concurrent executions
Rate Limiting: Respect Gmail API limits (250 quota units per user per second)

Step 3.7.2: Add Conditional Logic

Add "Conditional" step before S3 upload
Condition: {{validation.processableAttachments.length > 0}}
Skip upload if no valid attachments

Phase 4: Testing & Deployment

4.1 Unit Testing Each Step

Step 4.1.1: Test Individual Components

Test Gmail trigger with various email types
Test validation function with edge cases
Test S3 upload with different file types
Verify error handling scenarios

Step 4.1.2: Integration Testing

Send test emails with:

Single attachment
Multiple attachments
Large files (near 25MB limit)
Various file types (PDF, DOCX, images)
Emails without attachments

4.2 Performance Testing

Step 4.2.1: Load Testing

Send batch of emails simultaneously
Monitor workflow execution times
Check S3 upload success rates
Verify no data loss during high volume

4.3 Deployment

Step 4.3.1: Deploy Workflow

Click "Deploy" button in workflow editor
Verify deployment status
Test with live Gmail account

Step 4.3.2: Monitor Initial Operations

Watch workflow executions in dashboard
Check S3 bucket for uploaded files
Monitor error logs and execution times

Phase 5: Monitoring & Maintenance

5.1 Setup Monitoring

Step 5.1.1: Configure Alerts

Set up alerts for failed executions
Monitor quota usage for Gmail API
Track S3 storage costs

Step 5.1.2: Regular Maintenance

Review and clean up old attachments
Monitor and optimize workflow performance
Update file type filters as needed

5.2 Security Considerations

Regular review of OAuth scopes
Monitor S3 bucket access logs
Implement data retention policies
Regular security audits of stored attachments

Important Security Notes

Never store sensitive credentials in workflow code
Use environment variables for API keys
Implement proper IAM policies with least privilege
Regular rotation of access keys
Monitor for unusual access patterns

Conclusion

System Benefits

This comprehensive system design provides:

Automation: Complete end-to-end attachment processing
Scalability: Handles multiple emails and attachments simultaneously
Organization: Structured S3 storage with metadata
Reliability: Built-in error handling and retry mechanisms
Maintainability: Dashboard-based configuration and monitoring
Security: OAuth authentication and proper access controls

Next Steps

Set up your Paragon account and integrations
Follow the phase-by-phase implementation guide
Test thoroughly with your specific use cases
Deploy to production with monitoring enabled
Iterate based on usage patterns and feedback

Table of Contents