AWS Managed Services

Deploy the Observability Stack to AWS using managed services. This creates the same observability platform as the local Docker Compose stack, backed by:

• Amazon OpenSearch Service or Amazon OpenSearch Serverless — logs, traces, and service map storage
• Amazon Managed Service for Prometheus — time-series metrics
• Amazon OpenSearch Ingestion (OSIS) — OTLP ingestion pipeline (replaces OTel Collector + Data Prepper)
• OpenSearch Dashboards — visualization and exploration

Choose an OpenSearch backend

	Managed (AOS)	Serverless (AOSS)
Scaling	Fixed instance count	Auto-scales with OCU
Authentication	FGAC + basic auth	IAM only
Index management	Built-in index types	Explicit index templates
Master password	Yes (Secrets Manager)	No
Best for	Predictable workloads, fine-grained control	Variable workloads, zero cluster management

Both backends use the same OSIS ingestion pipeline and Prometheus workspace. The choice only affects the OpenSearch storage layer.

Choose a deployment method

CLI installer

Interactive single-command deploy. Creates all resources, configures dashboards, and optionally launches an EC2 instance running the OpenTelemetry Demo for sample telemetry.

Managed (default):

npx @opensearch-project/observability-stack

Serverless:

npx @opensearch-project/observability-stack --serverless

Takes ~15 minutes. See aws/cli-installer/README.md for full options.

Deploy into a VPC

Pass --vpc-id with --subnet-ids and --security-group-ids to place the OpenSearch domain, the OSIS ingestion pipeline, and the EC2 demo instance inside a VPC on private endpoints. Omit --vpc-id and you get the default public-endpoint deployment. VPC mode requires the managed backend.

npx @opensearch-project/observability-stack --advanced --managed \
  --pipeline-name obs-stack \
  --region us-east-1 \
  --os-domain-name obs-stack \
  --vpc-id vpc-xxxxxxxx \
  --subnet-ids subnet-aaaa,subnet-bbbb \
  --security-group-ids sg-xxxxxxxx

Use this when your telemetry sources (EKS, ECS, EC2) run inside a VPC and you want the OpenSearch stack in the same network boundary.

• Pass 1-3 subnets, each in a distinct AZ. With more than one subnet the domain is created zone-aware and the pipeline attaches to the first two.
• Security groups must allow the intra-VPC traffic between the pipeline, domain, and demo instance. A single group that allows traffic from itself works.
• The installer validates the VPC, subnets, and security groups against the EC2 Describe APIs before creating anything, so wrong network inputs fail in seconds rather than partway through domain or pipeline creation.

In interactive mode, advanced mode adds a Network topology step where you pick public endpoints or a VPC, then select subnets and security groups from your account.

You can run this from anywhere

You do not need to be inside the VPC. A VPC-private domain’s own endpoint isn’t reachable from your machine, but the managed OpenSearch UI (Application) endpoint is public and proxies to the domain over the AWS-internal network. For VPC domains the installer sets your IAM principal as the domain’s master user, authorizes the OpenSearch UI service (application.opensearchservice.amazonaws.com) to reach the domain through its VPC endpoint, and runs all FGAC role mapping and UI setup through that reachable Application endpoint. No bastion or VPN is required.

If you deploy a VPC domain by other means, run aws opensearch authorize-vpc-endpoint-access --domain-name <name> --service application.opensearchservice.amazonaws.com yourself, or the UI returns No Living connections.

Demo

Some loading time is cut from the video.

CDK

Infrastructure as code using AWS CDK. Deploys two stacks — one for the OpenSearch domain (or serverless collection) and Prometheus workspace (~17 min), one for the OSIS pipeline, dashboards, and optional demo workload (~6 min).

Managed (default):

cd aws/cdk
npm install
cdk deploy --all

Serverless:

cd aws/cdk
npm install
cdk deploy --all --context opensearchType=serverless

See aws/cdk/README.md for configuration, architecture details, and SigV4 telemetry setup.

Sending telemetry

Both methods create an OSIS ingestion endpoint that accepts OTLP data. Configure your OpenTelemetry Collector to export using SigV4 authentication:

extensions:
  sigv4auth:
    region: us-west-2
    service: osis

exporters:
  otlphttp/logs:
    logs_endpoint: ${OSIS_ENDPOINT}/v1/logs
    auth: { authenticator: sigv4auth }
    compression: none
  otlphttp/traces:
    traces_endpoint: ${OSIS_ENDPOINT}/v1/traces
    auth: { authenticator: sigv4auth }
    compression: none
  otlphttp/metrics:
    metrics_endpoint: ${OSIS_ENDPOINT}/v1/metrics
    auth: { authenticator: sigv4auth }
    compression: none

The IAM principal sending data needs osis:Ingest permission on the pipeline ARN.

For a VPC deployment the ingest endpoint is VPC-private, so your collector must run inside the VPC (or reach it over peering/Transit Gateway). A VPC OSIS endpoint also takes ~10-15 minutes after the pipeline reports ACTIVE before it serves requests; until then connections close with no HTTP response. This is AWS-side ingestion warmup, not a configuration problem.

Cleanup

CLI installer:

node bin/cli-installer.mjs destroy --pipeline-name obs-stack --region us-west-2

The destroy command automatically detects and cleans up both managed domains and serverless collections associated with the pipeline name.

CDK:

cd aws/cdk
cdk destroy --all