Senior Data Platform Engineer
Anuncio original
ATRC's Data & AI Office is a product and platform execution team - not a traditional enterprise data function. We build and operate AI-enabled systems that create measurable operational impact across the organization. Our primary output is working software in production.
What we are building - now:
An executive reporting portal serving senior leadership, integrating live data from SAP, Microsoft 365, SuccessFactors, and internal systems across multiple operational views (finance & budget, KPIs, technology portfolio, news, wellbeing, and a personalized executive home)
A sovereign data and AI platform: data warehouse, governed pipelines, metadata catalog, and classification enforcement across all entities
Data mesh architecture with federated ownership - each entity owns its data products; the platform provides governed access
Data readiness for the organization's agentic AI platform - clean, structured, governed data so AI agents can operate on trusted sources
Operational AI agents delivering real value on top of the agentic platform (document triage, anomaly detection, automated briefings, classification automation)
AI-enabled situational awareness and crisis management systems
Data governance and classification - managing an active external vendor engagement, bridging framework deliverables into platform-enforced rules, and building the ongoing
governance operations the organization requires
How we work:
Small team, high ownership - every engineer owns a module end-to-end
Two-week sprints with demonstrated output at every review
Ship and improve - working systems beat perfect designs
Core team owns all architecture and product decisions; external capacity executes within defined scope
Decisions take hours, not weeks. Daily technical choices are made by the engineer.
Feature scope decisions take 24 hours. Architecture pivots take 48 hours. Nothing sits in limbo.
Oversee 5 other workstreams powered by 3rd party vendors within the ecosystem but whom we are ultimately accountable for
Anchors the data platform architecture and owns enterprise data extraction end-to-end.
You are the senior technical owner of ATRC's data platform - the pipelines, staging layer, and infrastructure that every downstream system depends on. You define how data moves from enterprise source systems (SAP, M365, SuccessFactors) into a governed, queryable layer. You work directly with the Principal Data Architect on schema design and the Principal AI Engineer on data contracts for the agentic platform.
In the first 30 days, your primary job is getting data out of SAP FI/CO and into a staging database so the executive portal can show real numbers. Everything else follows from that.
What This Role Owns:
Platform architecture: pipeline design patterns, orchestration strategy, infrastructure-as-code, monitoring stack
Enterprise data extraction: SAP FI/CO (budget, actuals, commitments via OData/RFC), Microsoft Graph API (SharePoint, M365), SuccessFactors OData API - you own the extraction approach and build the first pipelines yourself
Staging layer design: PostgreSQL staging schema, raw-to-clean transformation patterns, refresh strategies
Ingestion observability: logging standards, alerting on failures, monitoring dashboards for pipeline health
Data quality framework: defining quality checks, deciding what pauses a pipeline vs. what triggers a warning
Classification enforcement: working with the Governance Lead to embed automated classification tagging into every pipeline
Technical mentorship: setting standards and reviewing work for the Senior Data Platform Engineer
Key Decisions
SAP extraction method (OData, RFC, SAP Data Services, CDS views) based on available configuration
Pipeline orchestration tool selection (Airflow, Azure Data Factory)
Data quality failure handling - warning vs. pipeline stop
Schema migration strategy
Incremental vs. full load per source
Deduplication key design across sources
Does Not Do
Define the data model or classification schema (Principal Data Architect)
Define governance policies or classification levels (Governance Lead)
Build application API endpoints (Backend Engineer)
Write AI prompts or classification logic (AI Engineer)
Build frontend components (Full Stack Engineer)
Ideal Candidate
Has designed and built a data platform from scratch in an enterprise environment - not inherited a running system, built one. Has pulled data from SAP or comparable ERP into a warehouse and survived upstream failures without silent data loss. Sets standards that other engineers can follow without three clarification meetings. Can make architecture calls quickly and correct them later. Has strong opinions about pipeline observability and data quality.
SAP data extraction: OData services, RFC/BAPI, SAP Data Services, or CDS views - hands-on experience pulling data from SAP FI/CO modules
Python - async data pipelines, background jobs, scheduled tasks
PostgreSQL - schema design, migrations (Alembic), query optimization, partitioning
Azure Data Lake Storage + Synapse Analytics
Microsoft Graph API - SharePoint, M365, organizational data
Apache Airflow or Azure Data Factory
dbt or equivalent for transformation and quality testing
Redis - queue management, TTL, cache invalidation
Observability - structured logging, Azure Monitor or Prometheus/Grafana
Docker - containerized pipeline jobs
Infrastructure-as-code (Terraform or equivalent)
Candidatura gestionada por ZooLATECH