Lead Data Engineer
Role Overview
• Own end-to-end architecture for ACDP: data warehouses, tenant data stores, lakes, and feature stores supporting analytics, ML, and AI agent use cases.
• Design and build the tenant registry and tenant data warehouse layer that underpins multi-tenant AI Agent workloads.
• Build and govern the Data Agent layer that supplies on-prem/customer-controlled data to cloud LLM reasoning in the Hybrid AI zone, with staged, explainable data outputs and human review checkpoints.
• Define data classification and routing rules that determine which data can flow to Cloud AI, which requires the Hybrid AI path, and which must remain fully on-prem/deterministic — protecting customer IP and production-critical logic.
• Set engineering standards for ETL/ELT design, orchestration, and data quality across the team.
• Lead design and code reviews; mentor data engineers and act as technical escalation point.
• Drive automation and reliability of pipelines, including SLAs, observability, and incident response.
• Own data security, access control, lineage, and governance compliance at the platform level, across cloud, hybrid, and on-prem tiers.
• Partner with data scientists and analysts to ensure platform data is ML-ready and supports predictive modeling.
• Build and maintain the data/event pipelines connecting ERP source systems (e.g., EnCore) to the EnCore Agent Hub, supporting secure read/write and event-and-trigger flows.
• Guide GenAI/LLM data enablement, including embeddings, vector database design, and RAG pipeline architecture, in support of Amtech AI Cloud agent orchestration.
• Represent data engineering in cross-functional architecture (ARB) and platform roadmap discussions.
• Optimize pipelines for low-latency access; establish partitioning, caching, and indexing strategies as team standards.
Key Skills for This Role
Full Job Posting
Job Description
Own end-to-end architecture for ACDP: data warehouses, tenant data stores, lakes, and feature stores supporting analytics, ML, and AI agent use cases.
Design and build the tenant registry and tenant data warehouse layer that underpins multi-tenant AI Agent workloads.
Build and govern the Data Agent layer that supplies on-prem/customer-controlled data to cloud LLM reasoning in the Hybrid AI zone, with staged, explainable data outputs and human review checkpoints.
Define data classification and routing rules that determine which data can flow to Cloud AI, which requires the Hybrid AI path, and which must remain fully on-prem/deterministic — protecting customer IP and production-critical logic.
Set engineering standards for ETL/ELT design, orchestration, and data quality across the team.
Lead design and code reviews; mentor data engineers and act as technical escalation point.
Drive automation and reliability of pipelines, including SLAs, observability, and incident response.
Own data security, access control, lineage, and governance compliance at the platform level, across cloud, hybrid, and on-prem tiers.
Partner with data scientists and analysts to ensure platform data is ML-ready and supports predictive modeling.
Build and maintain the data/event pipelines connecting ERP source systems (e.g., EnCore) to the EnCore Agent Hub, supporting secure read/write and event-and-trigger flows.
Guide GenAI/LLM data enablement, including embeddings, vector database design, and RAG pipeline architecture, in support of Amtech AI Cloud agent orchestration.
Represent data engineering in cross-functional architecture (ARB) and platform roadmap discussions.
Optimize pipelines for low-latency access; establish partitioning, caching, and indexing strategies as team standards.
Qualifications
- Bachelor's degree in Computer Science, Software Engineering, or related discipline.
- 10+ years of experience in data engineering or software development, including 3+ years in a technical lead or architect capacity.
- Deep expertise in data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
- Strong programming skills in Python, Java, or Scala.
- Proven experience with orchestration tools (e.g., Airflow, dbt, AWS Glue).
- Hands-on ownership of cloud platforms at scale (AWS, Azure, or GCP), plus familiarity with on-prem/private deployment models.
- Proficiency with big data frameworks (e.g., Spark, Flink) and streaming tools (Kafka, Kinesis).
- Expert-level SQL and NoSQL databases; familiarity with time-series or graph databases is a plus.
- Experience setting data validation, governance, and observability standards (e.g., Great Expectations, Prometheus, Grafana).
- Demonstrated experience mentoring engineers or leading a technical team.
- Experience designing multi-tenant data architectures (tenant registries, tenant-scoped data stores/warehouses) is a strong plus.
- Understanding of data sensitivity/risk classification and hybrid cloud/on-prem routing for AI workloads is a strong plus.
Preferred Qualifications
- Advanced training in Data Engineering, Cloud Architecture, or Industrial Data Systems.
- Experience with ERP, MES, or manufacturing data environments.
- Familiarity with IoT/OT data pipelines, including machine telemetry and SCADA signals.
- Applied experience with GenAI tools such as LangChain, vector databases, and RAG pipelines in production.
- Exposure to agent orchestration platforms, MCP-style tool routing, or event-driven agent architectures.
- Experience building or governing hybrid AI architectures balancing cloud acceleration with on-prem/private control layers. Why Join Amtech At Amtech, you will drive meaningful financial impact in a growing enterprise software organization while benefiting from Vista’s world-class ecosystem. You’ll collaborate with talented peers, leverage cross-portfolio learning programs, and help shape the future of Amtech’s financial operations and systems. Build your career with Amtech — backed by the strength, scale, and innovation culture of Vista.
About Amtech Software
Private enterprise software company providing ERP, MES, CRM, and automation software to packaging manufacturers.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Amtech Software
Senior Software Engineer
, IND
About Vista Equity Partners Vista Equity Partners is a leading global investment firm focused exclusively on enterprise software, data, and technology-enabled businesses. With over $100B in assets under management and a
Senior Software Engineer
, IND
AI Engineer III – Agentic Platform
Bengaluru, IND
Site Reliability Engineer III
Bengaluru, IND
Services Engagement Lead
, USA
Documentation Specialist
Bengaluru, IND
Software Engineer III
Bengaluru, IND
Manager, PMO
Bengaluru, IND
IT Support Engineer III
Bengaluru, IND