Job Opportunity Posted yesterday Updated 19 Nov 2026

Sr Software Engineer - Data Engineering

Job Description

What wearelookingfor:

Experienceandeducation:

  • Bachelor's degree in Computer Science, InformationTechnology, Electronics or a related engineering discipline.Amaster's (ME, MTech, MS or MCA) in a computing or quantitative discipline is preferred.
  • Threetofiveyearsofprofessionalsoftwareengineering,withademonstrablerecordofshipping andsupportingproductionsystems.
  • Production pipelines built and operated at scale, not only prototypes or reports, with a significant component owned end to end including its production behavior.
  • Data modellingforanalyticalconsumption,withrealaccountabilityforcorrectness.
  • Ingestionbuiltforstreaming orchangedatacapture,andmentoringoflessexperiencedengineers,formallyorinformally
  • Experience with data governance, privacy or regulatory reporting, or supplying data to machine learning and generativeAI workloads, is a strong advantage.

Coretechnicalskills:

  • Databricks:Delta Lake, Unity Catalog, Lakeflow Declarative Pipelines or an equivalent, Workflows, and cluster and SQLwarehouse configuration.
  • PySpark andSparkSQL:partitioning,shuffle and skewbehaviour,caching and queryoptimisationon Databricks.
  • AdvancedSQL:windowfunctions, complexjoins,incrementalandmergepatterns, andqueryperformancetuning.
  • Python:toaproductionstandard,withattention topackaging,testingandcodestructureratherthannotebooks alone.
  • DeltaLake:opentableformatswithDeltaLakeasthedefault,including schemaevolution,timetravelandcompaction.
  • Unity Catalog:governance,lineageandaccesscontrol,including classification,maskingandfine-grainedgrants.
  • Power BIorDatabricksSQL:dashboardsontopofwhatyoudeliver,withasemantic layerthebusiness readsthenumbers from.
  • dbtandAirflow:transformation,testingandorchestrationwithlineageanddependencymanagement,onAirfloworDatabricksWorkflows.
  • MySQL and DynamoDB:the operational sources the platform ingests from, including change data capture against MySQLand export and streaming patterns from DynamoDB.

Nice to Have:

  • Streaming with Kafka or Kinesis, and change data capture with Debezium against MySQL sources.
  • MLflow, feature stores and machine learning operations practice.
  • Privacy engineering at scale: tokenisation, pseudonymisation and deletion.
  • TerraformandGitHubActions fordata pipelines.
  • Snowflake orRedshift,formigration andinteroperabilitywork.
  • Databricks Data EngineerAssociate or Professional, or a generative AI credential.

How you work:

  • Owns outcomes, not tickets.Turns a half-formed requirement into something running, observable and supportable.
  • Raises the bar around them. Reviews carefully and kindly, and mentors the engineers nearby.
  • Engineering judgement. Picks the simplest design that meets the requirement, and can say what was left out.
  • Clear in writing. Design notes, data dictionaries and handover documentation others can follow.
  • Rigorous about correctness. Would delay a dataset rather than publish a wrong number.

Whatyouwillown:

Platformandpipelineengineering:

  • Ingestion:batchandstreaming,changedatacapture, schemaevolution,lateandduplicate data,andreconciliationagainstsourcesystems.
  • Thelakehouse layers: bronze,silverandgoldDeltaLakelayerswithsensiblepartitioning,clustering,compactionandretention.
  • Data products:dimensional and event models, slowly changing dimensions, and semantic definitions that mean the same thing to every consumer.
  • Datacontracts:agreedwiththeproducingteamssoupstream changedoesnotbreakdownstream silently.
  • Dataqualityasengineering:tests,freshnessandcompletenessmonitoring,anomalydetection,andalertsroutedtoanamedowner.
  • Orchestration and recovery:Databricks Workflows orAirflow, with dependency management and backfill designed as a standing capability rather than a rescue operation.
  • Platformcost:cluster andwarehousesizing,efficientjobs,andconsumptionmadevisibleforthedomains youown.

Governance,privacyandcompliance:

  • Unity Catalog:classification,ownership,lineageandfine-grainedaccesscontrolfortheassetsyouown.
  • Privacy controls:personal-datadiscoveryandclassification,maskingandtokenisation,purposelimitation,retention anddeletion.
  • Data residency:pipelines and storage designed so data stays in the jurisdiction that requires it, including India's DPDP obligations and the regimes of the markets served.
  • Audit evidence:ISO27001andpartnerobligationsforthepipelines youown.

EnablingAIandanalytics consumers:

  • Retrieval corpora and embedding pipelines:a first-class deliverable, including chunking strategy, refresh cadence and permission-aware access.
  • Evaluation and ground-truth datasets:curated, versioned and reproducible, so retrieval quality is a measured property rather than an impression.
  • Machine learningsupport:featureengineering,MLflowexperimentandmodeltracking, andreproducibletraining data.
  • Analyticalself-service:documented,trustworthydatasetssoroutinequestions donotqueuebehindthedatateam.

Craft,reviewandproductionownership:

  • Design beforebuilding:short,cleardesignnotesfornon-trivialwork,takentoreview,withthetrade-offswrittendown.
  • Review asrealwork:giveitpropertimein your week.Itisnotoverhead here.
  • Operate what you build:production support for the data platform and its critical pipelines, incident response, and the follow-up actions seen through.
  • Observabilityforyourpipelines: dashboards,alertsthatleadtoaction,andlogsacolleague candiagnosefromunaided.
  • Mentoring:pairwithlessexperiencedengineers,givereviewfeedbackthatteaches,andhandoverworkthatstretchesthem.

AI-accelerateddevelopmentiscoretohowwework:

  • Every engineer here uses an approved assistant such as Claude Code, Cursor, GitHub Copilot or Databricks Assistant as part of normal work, and the expectation runs past autocomplete: a scoped multi-file task, a diff you review properly, and tests and static analysis as the gate.
  • Treat a prompt as an egress path. No customer data, no credentials, and nothing outside theAsurion acceptable-use and data-classification standards.
About this job listing
This job opportunity is provided through our external job listing network. MyJobAlerts helps you discover job opportunities and redirects you to the original listing to apply.