Software Engineer: Data Pipelines and ML (Mid-career / Senior)

About the role

Theia Insights builds foundational financial intelligence products, including industry classification, knowledge graphs and factor risk models, for institutional investors. We serve some of the largest asset managers, hedge funds, index providers and sell-side banks.

What you’ll do

• Build and maintain pipelines that classify global public equities across our five-level taxonomy, sector, industry, sub-industry, major theme and micro theme, by extracting information from filings and web content, and assigning thematic exposures based on this information.
• Run large-scale NLP and LLM inference (entity extraction, classification, knowledge graph construction) over company documents, with cost- and throughput-aware batch execution.
• Ingest market data and publish datasets to external distributors.
• Own schema and contract evolution for datasets with real downstream consumers.
• Work with economists and engineers to turn modelling decisions into reliable production data.

Essential

• Strong production Python.
• Datasets in pandas and Parquet/Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake.
• Orchestrated batch pipelines you’ve operated, not just written: Dagster or Airflow, S3-based data flows, and a habit of testing outputs for correctness rather than only for exceptions.
• Applied ML in production: embeddings and semantic similarity, clustering, or operationalising models (not necessarily training from scratch).
• AWS fluency and CI/CD discipline.

Nice to have

• Practical LLM engineering: prompting, batch inference, and cost and throughput trade-offs across providers.
• SageMaker, Bedrock, or comparable managed ML tooling.
• Financial and equities domain knowledge: classification taxonomies, factor models, index construction. Valuable but learnable.
• Infrastructure as code (AWS CDK or Terraform) and Docker.

We care more about what you’ve owned than years on a CV. If you’ve built a pipeline that runs on a schedule against real volume, and you were the person who got paged when it broke, you’re in scope. More senior candidates will typically have made the cost and throughput trade-offs, and set the standard for how a team tests data quality.

Experience we’re looking for

• Competitive salary plus EMI share options
• 25 days holiday + bank holidays
• Private health insurance, pension
• Hybrid working in UK from London or Cambridge

#J-18808-Ljbffr