Introduction: Rethinking API Gateway

If you ask many developers what Amazon API Gateway is, they’ll often say,

“It’s a way to trigger Lambda functions with HTTP requests.”

While that’s a common use case, this view undersells the service dramatically.

In reality, API Gateway is a complete API management platform. It’s your first and most critical line of defense, capable of handling validation, transformation, authentication, throttling, and observability before your Lambda function even runs.

It shifts the burden of common API concerns from your code to a managed, scalable layer.

To prove this, we’ll build TrailLog — a serverless API for logging hiking trips.

We won’t just connect endpoints; we’ll use API Gateway’s full potential to create a robust, secure, and observable API.

By the end of this guide, you will learn how to:

• Validate & Transform: Enforce data contracts at the gateway, keeping Lambda code clean.
• Secure Endpoints: Implement a custom Lambda authorizer for token-based access control.
• Achieve Observability: Gain deep insights into API performance with structured CloudWatch logs.
• Orchestrate a Governed API: Deploy a production-ready API with full visibility and control.

---

The Blueprint: TrailLog System Architecture

Our fully serverless architecture is designed for scalability and simplicity.

Each AWS service plays a clear role in this ecosystem.

Component	Responsibility
Amazon API Gateway (REST API)	The unified entry point — handles routing, validation, auth, and logging
AWS Lambda	Executes backend logic for creating and retrieving trips
Amazon DynamoDB	Stores all hiking data in a serverless NoSQL table
Amazon CloudWatch	Centralizes logs and metrics for monitoring and debugging

This stack scales automatically with traffic, without managing or patching servers.

Architecture Diagram

Client application sends incoming HTTPS REST requests to Amazon API Gateway, which triggers AWS Lambda functions for GET, POST, and DELETE operations. Lambda interacts with Amazon DynamoDB for data storage and sends logs to Amazon CloudWatch.

---

Foundation: Setting Up Permissions with IAM

All Lambda functions in TrailLog share a single IAM role named lambda-traillog-role,

configured via an inline policy that grants access to DynamoDB and CloudWatch logs.

IAM Role: lambda-traillog-role

All Lambda functions in TrailLog share a single IAM role named lambda-traillog-role.

This role is granted access to the DynamoDB table TrailLog through the following inline policy.

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DDBWriteTrailLog",
"Effect": "Allow",
"Action": [
"dynamodb:PutItem"
],
"Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
},
{
"Sid": "DDBReadTrailLog",
"Effect": "Allow",
"Action": [
"dynamodb:GetItem",
"dynamodb:Scan",
"dynamodb:Query"
],
"Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
},
{
"Sid": "DDBDescribeForChecks",
"Effect": "Allow",
"Action": [
"dynamodb:DescribeTable"
],
"Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
}
]
}

This role is attached to all Lambda functions in the project.

---

Deep Dive 1: The POST /trips Endpoint — Validation & Transformation

The first endpoint demonstrates API Gateway’s ability to offload validation and data transformation

before invoking the backend logic.

DynamoDB Table: TrailLog

Property	Value
Partition Key	tripId (String)
Billing Mode	Pay-per-request

---

Lambda Function: create-trip (Python 3.12)

This Lambda function remains simple because API Gateway enforces all validation upfront.

import json, os, uuid, boto3
from datetime import datetime
from decimal import Decimal, InvalidOperation

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def lambda_handler(event, context):
# API Gateway has already validated input; focus on core logic.
trip_id = str(uuid.uuid4())
item = {
"tripId": trip_id,
"userId": event["userId"],
"trailName": event["trailName"],
"date": event["date"],
"distanceKm": event["distanceKm"],
"durationMin": event["durationMin"],
"elevationGainM": event["elevationGainM"],
"notes": event.get("notes", ""),
"createdAt": datetime.utcnow().isoformat() + "Z"
}
table.put_item(Item=item)
return {
"statusCode": 201,
"headers": {"Content-Type": "application/json"},
"body": json.dumps({"ok": True, "tripId": trip_id})
}

---

The API Gateway Configuration: Non-Proxy Integration

Disabling Lambda Proxy Integration gives us control over request handling and validation.

Setting	Value
Integration Type	Lambda Function
Proxy Integration	Disabled
Validator	Body & Params
Model	TripCreate

Request Model: TripCreate

{
"type": "object",
"required": ["userId", "trailName", "date", "distanceKm", "durationMin", "elevationGainM"],
"properties": {
"userId": {"type": "string"},
"trailName": {"type": "string"},
"date": {"type": "string", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"},
"distanceKm": {"type": "number"},
"durationMin": {"type": "integer"},
"elevationGainM": {"type": "integer"},
"notes": {"type": "string"}
}
}

Now, any malformed input (for example, an invalid date format or incorrect type)

is rejected at the gateway, saving Lambda invocations and improving reliability.

Successfully created a new hiking trip via the POST /trips endpoint, returning a 201 status and generated tripId.

---

Deep Dive 2: The GET /trips/{tripId} Endpoint — Simple Proxy Integration

For simple reads, we use Lambda Proxy Integration to pass the entire request context directly to Lambda.

Lambda Function: get-trip

This function extracts tripId from the path and fetches the corresponding item from DynamoDB.

import json, os, boto3, decimal

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def _to_native(x):
if isinstance(x, decimal.Decimal):
return int(x) if x % 1 == 0 else float(x)
return x

def lambda_handler(event, context):
trip_id = (event.get("pathParameters") or {}).get("tripId")
if not trip_id:
return {"statusCode": 400, "body": "Missing tripId"}

res = table.get_item(Key={"tripId": trip_id})
item = res.get("Item")
if not item:
return {"statusCode": 404, "body": "Not found"}

return {
"statusCode": 200,
"headers": {"Content-Type": "application/json"},
"body": json.dumps(_to_native(item))
}

Test Request

curl https://<api-id>.execute-api.us-east-1.amazonaws.com/dev/trips/<tripId>

Fetched detailed hiking trip data successfully using the GET /trips/{tripId} endpoint.

---

Deep Dive 3: Security and Observability — /secure/admin/stats

This administrative endpoint combines two advanced API Gateway features:

custom token authorization and centralized CloudWatch observability.

1. Securing the Route with a Lambda Authorizer

We use a custom Lambda authorizer to protect the /secure/admin/stats route.

Authorizer Function: lambda-authorizer

import json

def lambda_handler(event, context):
DEMO_TOKEN = "Bearer trail123"
token = event.get("authorizationToken", "").strip()

if token == DEMO_TOKEN:
effect, role = "Allow", "admin"
else:
effect, role = "Deny", "guest"

return {
"principalId": "trail-admin",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": effect,
"Resource": event["methodArn"]
}]
},
"context": {"role": role}
}

Any request without X-Auth-Token: Bearer trail123 will receive a 403 Forbidden response.

---

2. Stats Aggregation Function: admin-stats

This function aggregates trip statistics across all records.

It’s invoked only if the authorizer grants access.

import json, os, boto3, decimal

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def _n(x):
if isinstance(x, decimal.Decimal):
return int(x) if x % 1 == 0 else float(x)
return x

def lambda_handler(event, context):
scan = table.scan(ProjectionExpression="tripId, distanceKm, durationMin, elevationGainM")
items = scan.get("Items", [])
total = len(items)
km = sum(_n(i.get("distanceKm", 0)) for i in items)
mins = sum(_n(i.get("durationMin", 0)) for i in items)
gain = sum(_n(i.get("elevationGainM", 0)) for i in items)

stats = {
"totalTrips": total,
"totalKm": round(float(km), 2),
"totalHours": round(float(mins) / 60.0, 1),
"totalElevationM": int(gain)
}

return {
"statusCode": 200,
"headers": {"Content-Type": "application/json"},
"body": json.dumps(stats)
}

Test Request

curl -H "Authorization: Bearer trail123" "$BASE/secure/admin/stats"

Access denied without a valid token — API Gateway blocking unauthorized requests using the custom Lambda authorizer.

Authorized admin access to /secure/admin/stats returning aggregated trail statistics.

---

The Crown Jewel: Deep Observability with CloudWatch

To gain complete visibility into API behavior, enable both Execution Logs and Access Logs in Amazon API Gateway.

These logs provide the foundation for debugging, performance monitoring, and request analytics.

---

Create a Log Group for Access Logs

Before enabling access logging, create a dedicated CloudWatch Log Group where API Gateway will push structured log entries.

This ensures all API requests are tracked consistently and stored centrally.

Steps:

1. Open CloudWatch Console → Log groups → Create log group.
2. Enter the name:

/apigw/traillog/dev

1. (Optional) Set the retention period to 7, 30, or 90 days depending on your monitoring needs.
2. Note down the log group ARN — you’ll use it in your API Gateway stage configuration:

arn:aws:logs:us-east-1:579273601939:log-group:/apigw/traillog/dev

1. Ensure API Gateway has permission to write logs by attaching this managed policy to its execution role:

arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs

CloudWatch log group /apigw/traillog/dev created to store structured access logs from API Gateway.

---

1. Execution Logs

Execution logs trace the internal request flow through API Gateway —

from input validation and mapping templates to Lambda invocation and response transformation.

They’re your primary tool for debugging and tracing how API Gateway processes a request.

Property	Description
Log Group	API-Gateway-Execution-Logs_{restApiId}/dev
Purpose	Traces internal request flow and mapping templates
Best For	Debugging authorization, validation, or mapping issues

Ensure your API stage has CloudWatch execution logging enabled and that API Gateway can push logs.

Attach the following managed policy to grant permissions:

arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs

---

2. Access Logs

Access logs summarize each API request and response in a structured JSON format.

These are invaluable for analytics, performance tracking, and auditing user behavior.

Property	Description
Log Group ARN	arn:aws:logs:us-east-1:579273601939:log-group:/apigw/traillog/dev
Purpose	Capture per-request analytics and response metadata
Best For	Monitoring latency, traffic, and client behavior

Access Log Format Template

To enable structured access logs, open your API Gateway stage and set the log format using the following JSON template.

{
"requestId": "$context.requestId",
"ip": "$context.identity.sourceIp",
"time": "$context.requestTime",
"method": "$context.httpMethod",
"path": "$context.resourcePath",
"status": $context.status,
"integrationStatus": $context.integration.status,
"userAgent": "$context.identity.userAgent",
"authorizer": "$context.authorizer.role",
"error": "$context.error.message"
}

Once redeployed, every request will produce:

• Execution Logs: Detailed traces for debugging.
• Access Logs: High-level JSON entries for analytics.

API Gateway Execution Logs showing Lambda invocation flow, request mapping, and response transformation details.

Structured JSON access logs capturing requestId, method, path, status, and authorizer context for analytics.

---

Conclusion: You’ve Built More Than an API

TrailLog isn’t just a CRUD backend. It’s a governed, secure, and observable API platform built with managed AWS services.

Key Takeaways

• Pre-Lambda Validation: Offload schema checks to API Gateway for cleaner, cheaper code.
• Custom Authorization: Implement fine-grained security without burdening business logic.
• Structured Logging: Use CloudWatch for comprehensive visibility and debugging. By embracing these patterns, you can use Amazon API Gateway not just as a trigger mechanism, but as a complete API management engine — the foundation of scalable, production-ready serverless systems.