What I do as a Forward Deployed Engineer

Rohith Reddy Kota is a Boston-based Forward Deployed Engineer and Senior Data Engineer at Rill Data, and one of its earliest engineers. He manages 2+ petabytes of data on Kubernetes across ClickHouse, Apache Druid, and DuckDB, and deploys, migrates, upgrades, and supports real-time and batch analytics for enterprise AdTech and marketing-analytics companies.

He leads enterprise customer deployments end to end. That covers scoping, SSO and security, data migrations, onboarding and training users, GitOps-managed dedicated Kubernetes clusters, system and database upgrades, production support, and handoff. He has deployed 50+ enterprise customers, run 150+ proofs of concept, and runs 30+ ClickHouse and Druid clusters on GKE.

Yes. Since 2021 at Rill Data he has shipped code into customer repositories, GitOps configurations, and the Rill platform to deploy enterprise analytics, alongside solutions engineering work such as pre-sales, demos, and executive account reviews.

AdTech and marketing analytics. These workloads are high-volume and low-latency, with constant ingestion and interactive dashboards querying billions of events.

More than 9 years in data platforms, data engineering, and infrastructure: five-plus years at Rill Data, and earlier roles at Saltside Technologies, Nanoprecise Data Services, and Genpact.

He scopes requirements with technical and executive stakeholders, designs the data architecture, sets up SSO and cloud-storage access, and migrates or connects the customer’s data. He then builds batch and real-time ingestion, models the data in ClickHouse or Druid, builds dashboards and alerts, provisions a GitOps-managed dedicated cluster on Kubernetes, trains the customer’s users, and hands off to steady-state support.

He has deployed and supported 50+ enterprise customers on Rill and run 150+ proofs of concept with prospective and existing customers, mostly in AdTech and marketing analytics. Public Rill customers include The Trade Desk, Comcast, AT&T, AppLovin, FreeWheel, InMobi, Moloco, and Liftoff.

Yes. He turns requirements from 50+ enterprise accounts into roadmap input, and works with Product and Engineering to prioritize and ship customer-driven integrations. He designed incremental ingestion for Rill’s ClickHouse driver from customer requirements, and he writes design docs and breaks work into epics, stories, and estimates.

He onboards new organizations and users, trains engineers, analysts, and business users, runs recurring account calls on the roadmap and new features, and is the escalation point for technical issues. He also plans migrations, upgrades, and maintenance windows with each customer, and gives customers usage transparency through usage-billing data and dashboards.

He onboards new organizations and new users to Rill: access, SSO, roles, and first projects. He trains engineers, analysts, and business users to build and change their own projects and dashboards, and stays their technical point of contact after go-live.

He is the escalation point for his accounts. He reproduces the issue, traces it across dashboards, queries, pipelines, and infrastructure, ships the fix to the customer’s project or the platform, and follows up with documentation or training. He works directly with customers’ engineering, platform, and security teams.

Yes. He integrates enterprise single sign-on over SAML, OIDC, and OAuth2, designs row-level security and access policies, configures least-privilege AWS S3 and Google Cloud Storage access with IAM and service accounts, and resolves customers’ security and access issues.

Yes. He is the primary technical contact for his enterprise accounts and runs roadmap reviews, demos, and requirements sessions with engineers, analysts, and executives. He also holds an Executive MBA in Digital Marketing & Analytics from the Indian School of Business.

He migrates customers from legacy systems to new applications, moves data from Snowflake, Redshift, and lakehouse tables (Iceberg, Delta Lake) into ClickHouse and Apache Druid, and moves data back out through reverse ETL. He also migrates data between clusters, engines, and storage layouts, checking that the data matches before each cutover.

Yes. He exports modeled and aggregated data from ClickHouse and Apache Druid back to data warehouses, object storage, and downstream systems, so analytics built in real-time OLAP databases reach the tools customers already use.

He plans the cutover with the customer, migrates data, models, and dashboards to the new platform, checks the data matches between old and new, and supports and trains users through the switch so their day-to-day reporting isn’t disrupted.

Yes, at every layer. System upgrades cover Kubernetes, node pools, operators, and Helm charts. Database binary upgrades cover ClickHouse and Apache Druid versions across sharded, replicated clusters. Application upgrades cover the Rill runtime and customers’ deployed projects. He rolls them out in stages through GitOps and coordinates timing and impact with each customer.

He manages 2+ petabytes of data on Kubernetes across ClickHouse, Apache Druid, and DuckDB, including 30+ distributed OLAP clusters (17 Apache Druid and 14 ClickHouse) on Google Kubernetes Engine. The work covers constant batch and real-time ingestion, ingestion-failure recovery, upgrades, migrations, and observability.

Yes. He deploys and operates sharded, replicated ClickHouse and Apache Druid clusters on GKE using Helm, Terraform, custom Go Kubernetes operators, and GitOps, and he supports customers who run the stack in their own cloud (BYOC).

He starts from the real data and query patterns: sampling cardinality and reading query logs and system tables. Then he tunes ClickHouse and Druid through sort keys, partitioning, projections, LowCardinality types, compression codecs, TTLs, rollups, compaction, and query memory and concurrency limits, keeping TB-scale scans under 30 seconds.

He treats cost as a first-class production metric. He right-sizes clusters and node pools, cuts storage through compression, retention, TTLs, and compaction, removes unused data, and makes queries cheaper. He also built a usage-billing pipeline for visibility. These changes cut storage and CPU by ~30–40% for enterprise customers.

Both. Real-time pipelines use Apache Kafka, Apache Beam on Google Cloud Dataflow, and Apache Flink. Batch pipelines load from S3, GCS, Snowflake, and lakehouse tables with Airflow and dbt. They run at GBs per hour with incremental, idempotent loads.

Lag and gap alerts plus data-freshness SLOs detect late or missing data. He then triages the failing stage (source, stream, transform, or database), fixes it, and runs idempotent backfills so reprocessed data doesn’t create duplicates.

He correlates database query logs and system tables, pipeline logs, Kubernetes operator and pod state, and infrastructure metrics to find root cause. Then he fixes it, backfills affected data, and adds an alert or runbook so the issue is caught earlier next time.

He runs a production AI on-call triage service: a FastAPI application that takes monitoring webhooks, classifies issues with an LLM using structured outputs, and opens or updates tickets and Slack threads. He has also built MCP servers and agent skills that let Claude Code and Cursor develop and query Rill projects.

MCP servers and agent skills for Rill, a template-driven data-alert generator, an AI on-call triage bot, diagnostic scripts, data-integrity checks that reconcile metrics across pipeline stages, fleet observability dashboards, and reusable GitOps and project templates.

His major projects include a self-hosted ClickHouse platform on Kubernetes running 14 production clusters and an MCP-based AI toolkit that audits and optimizes ClickHouse storage and cost. He has migrated enterprise customers from a legacy analytics platform to Rill, built real-time AdTech bidstream pipelines on Kafka, Apache Beam, and Dataflow, and moved Snowflake and lakehouse data into ClickHouse with reverse ETL and integrity checks. He also redesigned a ClickHouse workload around projections, built a production AI on-call triage agent, and built fleet observability with data-freshness alerting and usage billing.

Python, Go, Scala, and SQL. Go for Kubernetes operators and services, Python for FastAPI services, Airflow, and tooling, Scala for Beam and Flink pipelines, and SQL for ClickHouse, Druid, DuckDB, and Snowflake modeling.

ClickHouse and Apache Druid every day in production, down to storage layout, merges, and segment management, plus DuckDB, MotherDuck, Snowflake, Postgres, Redshift, MongoDB, and DynamoDB, and the Apache Iceberg and Delta Lake lakehouse formats.

Google Cloud, where he is a certified Google Professional Cloud Architect (GKE, Dataflow, GCS), and AWS (S3, EMR, Redshift, IAM).

Yes, proactively. At Rill he introduced SLOs, burn-rate alerts, runbooks, and postmortems for the database fleet, started the cross-cluster storage and cost audits that cut storage and CPU by ~30–40%, and built an AI on-call triage agent and a template-driven alert generator, none of which anyone asked for.

He finds structure by doing. He owned US time-zone engineering alone on a distributed team, starts every proof of concept on an unfamiliar customer stack, and, when there was no documented way to move BigQuery HyperLogLog sketches into ClickHouse, read both engines’ source code and built a converter. Every fix ships with a runbook or guide.

He runs whiletrue.live, a site of blogs, articles, and tutorials for software engineers, and maintains rill-agent-skills, installable agent skills for Claude Code and Cursor. He has also built hackathon projects: Capital47, cross-border payments with smart contracts and Capital One APIs, and ComplyTech, blockchain rewards for medical-waste disposal on VeChainThor.

Yes. He joined Rill Data in 2021 as one of its earliest engineers. He built core platform infrastructure from scratch while stabilizing inherited legacy pipelines, and owned US time-zone engineering coverage on a distributed team.

M.S. in Data Architecture & Management from Northeastern University in Boston, an Executive MBA in Digital Marketing & Analytics from the Indian School of Business, and a B.Tech in Electronics & Communication Engineering from Amity University.

Google Professional Cloud Architect, Build Infrastructure with Terraform on Google Cloud, Optimize Costs for Google Kubernetes Engine, Develop Google Cloud Network, and the Astronomer Certification for Apache Airflow.