Clickhouse Solutions Architect

The Role

We are looking for a ClickHouse Solutions Architect to join our team supporting the design and implementation of a greenfield, enterprise-scale ClickHouse observability platform for a global banking client.

This is a genuine greenfield build at significant scale — there is no incumbent platform to inherit or work around. You will be shaping the architecture from first principles, in an environment where the performance, multi-tenancy and retention requirements are demanding enough that the design decisions made early will determine whether the platform holds up in production.

About the project

We are architecting a central repository for three classes of time-series telemetry — metrics, traces and logs — serving the bank's application estate. The system requirements are:

Ingest path

• 3 million events per second sustained, each event carrying a timestamp and full transaction payload

Read path

• Approximately 1,000 concurrent users
• Several hundred real-time dashboards

Multi-tenancy and workload isolation

• Hundreds of distinct application codes as tenants
• Throttling and quota control required on both read and write paths, so that no single tenant can degrade another

Standards and tooling constraints

• OpenTelemetry-native throughout: native OTel agent and native OTel collector
• No proprietary components anywhere in the agent or collector pipeline

Retention and storage tiering

Getting this right means solving for ingest throughput, query concurrency, tenant isolation and storage cost simultaneously — the interesting part of the work is that these pull against each other, and the architecture has to reconcile them rather than optimise for any one in isolation

Key Activities

Discovery and architecture

• Run technical discovery with client stakeholders to establish workload characteristics — query patterns, latency targets, concurrency, ingest volume and velocity, retention and cost envelope
• Translate those into a target-state architecture: deployment model (ClickHouse Cloud vs self-managed on Kubernetes/VMs), cluster topology, shard and replica strategy, Keeper configuration, storage tiering
• Design the data model — table engine selection across the MergeTree family, primary/sort key design, partitioning, projections, materialized views, dictionaries, TTL and object-storage tiering
• Design the ingestion architecture — streaming (Kafka/Kinesis, Kafka table engine or ClickPipes), batch and CDC paths, idempotency and deduplication strategy, schema evolution
• Produce sizing and cost modelling, with options and trade-offs rather than a single answer
• Define the semantic and consumption layer — BI tool integration, API access patterns, downstream contracts

Build and validation

• Stand up the reference environment and prove the architecture against representative data volumes and query shapes
• Build and tune the ingestion pipelines to agreed throughput and freshness SLOs
• Benchmark and tune query performance against the stated latency targets; iterate on the data model where benchmarks disprove the design
• Establish infrastructure-as-code, CI/CD and environment promotion for schema and configuration changes

Productionisation

• Define and implement observability — system table monitoring, metrics, alerting thresholds, capacity headroom tracking
• Establish backup, restore and disaster recovery, and validate them by test
• Implement security and governance — RBAC, row/column-level access, encryption, network isolation, audit, and any sector-specific compliance requirements
• Define upgrade, patching and capacity-management procedures

Client enablement and handover

• Set engineering standards and guardrails the client's teams work within — naming, modelling patterns, query review criteria, anti-patterns
• Coach and upskill the client's data engineers through pairing, review and structured sessions
• Run the handover to the client's run team against an agreed readiness checklist
• Act as escalation point for complex performance and design questions during transition

Advisory and practice contribution (weighted more heavily at principal level)

• Support pre-sales, bid and scoping work — estimating, technical qualification, solution shaping
• Build reusable assets, reference architectures and accelerators from delivered work
• Represent the practice credibly in client and community forums

#J-18808-Ljbffr