Common fields: one taxonomy across apps, hosts and ClickStack

A fleet is searched by the same few fields every time: which service domain and group, which host, which environment. This lab is the adoption guide for those fields — where each one is set, how the collector normalises the names, and the ClickStack tables that keep them fast — with the configs as files and every piece checked.

Verified on: schema/schema.sql in clickhouse local 26.9.6.6 (tests/test_schema.py: every table created, the columns fill, a filter on them uses the primary key; the negative checks and the injected fault in Tests), 2026-11-07, and on ClickHouse Cloud 26.8.1.608 with Managed ClickStack (managed/demo.py, the screens below), 2026-11-07. collector/standard-fields.yaml in otelcol-contrib 0.155.0 (tests/test_collector.sh: the rename, its precedence, the negative check and the injected fault), 2026-11-07. The read-volume numbers come from labs/hostmetrics-volume (logs and gauge metrics, settled run, 2026-11-06). Traces and the other metric types follow the same rule and were not measured.

The fields

Field	What	Set by
svc.domain, svc.group	your service taxonomy, two levels (rename to your own keys)	apps: OTEL_RESOURCE_ATTRIBUTES; hosts: EC2 tags svc.domain / svc.group
service.namespace, service.name	project and service	apps: OTEL_RESOURCE_ATTRIBUTES, OTEL_SERVICE_NAME
host.name	the machine	the collector's resourcedetection (system, ec2)
deployment.environment.name	qa / live	apps: OTEL_RESOURCE_ATTRIBUTES; hosts: the agent's resource processor
service.version	the deployed version	apps: OTEL_RESOURCE_ATTRIBUTES (from the build)

All are resource attributes: they describe who sent the data, so they are the same on every span, log line and metric point of that process, and they land in the ResourceAttributes column. Keep the standard to these few; everything else stays free-form in the attribute maps — slower to filter, still searchable.

1. Apps (development teams)

No code change: the OpenTelemetry SDKs and agents read two environment variables.

OTEL_SERVICE_NAME=checkout-api
OTEL_RESOURCE_ATTRIBUTES=service.namespace=shop,svc.domain=commerce,svc.group=payments,deployment.environment.name=qa,service.version=1.4.2
OTEL_EXPORTER_OTLP_ENDPOINT=http://<gateway NLB>:4318

• Java agent (-javaagent:opentelemetry-javaagent.jar): the same variables, or the -Dotel.service.name / -Dotel.resource.attributes system properties. Logs from Logback and Log4j are exported with the trace and span id of the request that wrote them, so a trace in ClickStack shows its log lines.
• Kubernetes: set the two variables in the Deployment's env; take service.version from the image tag at build time.
• Values must not contain commas or =; URL-encode them if they do.

2. Hosts (infrastructure)

Tag each instance where it is launched (launch template, Auto Scaling group, Terraform) with svc.domain and svc.group. The host agent reads the tags; nothing is typed into a config file per host.

processors:
  resourcedetection:
    detectors: [env, system, ec2]
    ec2:
      tags: ['^svc\.']        # -> ec2.tag.svc.domain, ec2.tag.svc.group

The instance role needs ec2:DescribeTags. The Windows profile otel-profiles/profiles/windows-host already does this (WINDOWS_HOST_EC2_TAGS).

3. Gateway: one place for the names

collector/standard-fields.yaml merges into the gateway collector's config. It renames ec2.tag.svc.domain / ec2.tag.svc.group to svc.domain / svc.group, keeps a value an app set itself, removes the ec2.tag.svc.* keys, and sets create_schema: false on the ClickHouse exporter with the table names of step 4. Doing it at the gateway means the key names are fixed in one file, not on 4,000 hosts.

otelcol-contrib --config gateway.yaml --config standard-fields.yaml

4. ClickHouse: the fields as columns, in the sort key

schema/schema.sql is the ClickHouse exporter's own DDL (opentelemetry-collector-contrib v0.155.0) with three changes:

1. svc_domain, svc_group, host are MATERIALIZED columns from ResourceAttributes, each with a skip index.
2. The sort key leads with the field fleet queries filter on most: logs and traces (svc_group, ServiceName, …), metrics (ServiceName, MetricName, svc_group, host, …).
3. A 30-day TTL — set each table to your retention.

clickhouse client --database otel --multiquery < schema/schema.sql

Apply it into an empty database before the first export. ClickStack needs no change: a search on the map key (svc.group:"payments") is rewritten to the column.

What it buys, measured on ClickHouse Cloud 26.8.1.608 with synthetic data the size of 4,000 hosts (labs/hostmetrics-volume, 2026-11-06, settled tables):

Query	Exporter default	This schema
One host, one metric	254 K rows · 5.7 MiB	8 K rows · 0.12 MiB
One host, every metric	16.8 M rows · 188 MiB · 572 ms	1.1 M rows · 3.3 MiB · 16 ms
One group's ERROR logs (10 M rows)	1.78 M rows · 290 ms	0.53 M rows · 17 ms
Storage, metrics	41.3 MiB	41.6 MiB

5. In ClickStack: one source per table, exact match in searches

Point a ClickStack source at each table, as for the defaults. Nothing else changes: ClickStack finds the materialized column and rewrites a search on the map key to it.

On the demo (ClickHouse Cloud 26.8.1.608, 2026-11-07; 2 M log rows an hour, 200 services in 20 groups), ResourceAttributes.svc.group:"g7" over 15 minutes became svc_group = 'g7' and read 49,152 rows — six granules — for 24,836 results:

Without quotes, svc.group:g7 is a substring match ("contains"), sent as svc_group ILIKE '%g7%'. On this schema it read the same 49,152 rows, because the column's set index can test each granule's values; on the bench's tables, which had the column without that index, a substring search read all 10 M rows. Write saved searches, alerts and dashboard filters on the common fields as exact matches anyway: a substring also matches every value that contains it (pay matches payments and paypal), and it is the index, not the search, that kept it cheap.

The same fields tie the signals together. Searching the trace source for ResourceAttributes.svc.group:"g7" StatusCode:"Error" and opening a span shows its log lines on the same timeline (the trace source is linked to the log source; the logs carry the span's trace id) and the common fields under Resource Attributes:

Screens: Managed ClickStack on ClickHouse Cloud 26.8.1.608, Seoul, 2026-11-07, managed/demo.py, the 15 minutes 12:39–12:54 KST. system.query_log shows both searches' queries (svc_group = 'g7' and svc_group ILIKE '%g7%') reading 49,152 rows, and a direct count() on the window gives the 24,836 results.

Run the demo on Managed ClickStack

python3 managed/demo.py      # database common_fields_demo, schema.sql, synthetic data, 3 linked sources
python3 managed/demo.py --cleanup

Reads ~/.config/khai-harness/secrets/clickhouse-seoul.env (CH_HOST, CH_USER, CH_PASSWORD) and chc-api.env (CHC_KEY, CHC_SECRET) by default; --ch-env / --api-env point elsewhere.

Tests

python3 -m unittest discover -s tests     # schema.sql in clickhouse local (skipped without the binary)
tests/test_collector.sh                    # standard-fields.yaml in otelcol-contrib 0.155.0 (Docker)

Each test has a positive check, a negative check and a fault it injects into a copy of the file and must catch (2026-11-07, clickhouse local 26.9.6.6 · otelcol-contrib 0.155.0: 5/5 tests, 6/6 checks, both faults caught):

Test	Positive	Negative	Injected fault
test_schema.py	svc_group = 'g7' (logs) and svc_group + host (gauge) prune primary-key granules	ResourceAttributes['svc.group'] = 'g7' does not use the primary key (ClickHouse does not rewrite the map lookup; ClickStack's search does); a row without svc.* gets ''	svc_group dropped from the logs sort key: the pruning check fails
test_collector.sh	EC2 tags become svc.domain / svc.group; an app's own svc.group is kept; ec2.tag.svc.* removed	a log with neither tags nor svc.* gets no svc.* key	the svc.group rule without its "app value first" guard: "app: its own svc.group kept" fails

OTEL_SERVICE_NAME=checkout-api
OTEL_RESOURCE_ATTRIBUTES=service.namespace=shop,svc.domain=commerce,svc.group=payments,deployment.environment.name=qa,service.version=1.4.2
OTEL_EXPORTER_OTLP_ENDPOINT=http://<gateway NLB>:4318

processors:
  resourcedetection:
    detectors: [env, system, ec2]
    ec2:
      tags: ['^svc\.']        # -> ec2.tag.svc.domain, ec2.tag.svc.group

otelcol-contrib --config gateway.yaml --config standard-fields.yaml

clickhouse client --database otel --multiquery < schema/schema.sql

python3 managed/demo.py      # 데이터베이스 common_fields_demo, schema.sql, 합성 데이터, 서로 연결된 소스 3개
python3 managed/demo.py --cleanup

python3 -m unittest discover -s tests     # clickhouse local의 schema.sql(바이너리가 없으면 건너뜀)
tests/test_collector.sh                    # otelcol-contrib 0.155.0의 standard-fields.yaml(Docker)