Principal Architect - Reinforcement Learning (Physical AI)
Role Overview
Your Work Shapes the World at Caterpillar Inc.
When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other.
We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities.
We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live.
Together, we are building a better world, so we can all enjoy living in it.
Help Build the Future of Caterpillar –
At Caterpillar, technology always has a purpose, which is to solve our customers’ toughest challenges.
Through Cat Technology, we are solving problems by building the intelligence layer that connects machines, data, and people to make jobsites safer, more productive, and more sustainable.
By combining deep domain expertise in physical systems with software, connectivity, autonomy, and AI, we deliver solutions that work in the real world—on real jobsites, at global scale.
You’ll build and deploy against one of the most unique data foundations—over 1.6 million connected assets generating real-world data daily.
These data and platform capabilities are enabling the development of AI models, edge computing architectures, and software systems that scale across fleets, products, and industries.
The result will be a new generation of machines that continuously learn, improve, and deliver performance at scale.
Be Part of What’s Next in Autonomous Construction Sites
Construction autonomy is one of the most complex challenges in applied AI, and at Caterpillar, advancements in physical AI, simulation, sensing, and edge computing are turning things that once felt impossible—intelligent machines operating in dynamic jobsites—into reality.
Our connected ecosystem brings together massive volumes of high-quality data to create a foundation where engineers like you can build and deploy against.
If this work motivates you, we invite you to join our team.
In these roles, you’ll work at the intersection of the physical and digital worlds.
You’ll help design and deliver intelligent systems that enable machines to perceive their environment, make informed decisions, and support safer, more productive operations.
Apply today to build the new era of construction autonomy at Caterpillar.
Full Job Posting
What You Will Do:
- Developing detailed architecture deliverables to solve business problems.
- Designing an application's technical infrastructure, such as specific databases, programming languages, utilities, and testing approaches.
- Leading the evaluation and deployment of new technologies to add or enhance existing digital technical capabilities.
- Participating in addressing business requirements for applications and collaborating with cross-functional teams to deliver digital solutions that meets business results.
- What You Will Have:
- Requirements Analysis: Knowledge of tools, methods, and techniques of requirement analysis; ability to elicit, analyze and record required business functionality and non-functionality requirements to ensure the success of a system or software development project.
- Analytical Thinking: Knowledge of techniques and tools that promote effective analysis; ability to determine the root cause of organizational problems and create alternative solutions that resolve these problems.
- Platform Architecture: Knowledge of technologies and methods to design processing mechanisms and roadmaps to execute business application systems; ability to design these roadmaps and deploy supportive interfaces for end-users to access related systems, in accordance with standards and processes.
- Target Architecture: Knowledge of target architecture; ability to develop the IT blueprint and roadmap while aligning the architecture and processes with business strategies and objectives.
- Effective Communications: Understanding of effective communication concepts, tools and techniques; ability to effectively transmit, receive, and accurately interpret ideas, information, and needs through the application of appropriate communication behaviors.
- Top Candidates Will Have:
- Reinforcement Learning for Physical Systems: Typically 8+ years (or PhD plus 5+) in robot learning with RL taken from algorithm to deployment on real hardware: PPO, SAC, GRPO, model-based / world-model RL, reward shaping, curriculum, and compounding-error handling in closed-loop training.
- Imitation Learning and Foundation-Model Post-Training – Behavior cloning, offline RL (IQL, GAIL), distillation, diffusion policies; vision-language-action or robot foundation models with RL-based post-training (RLHF / DPO / GRPO).
- Large-Scale Closed-Loop Training in Simulation – Architected RL frameworks on GPU-parallel simulators (NVIDIA Isaac Lab / Isaac Sim, MuJoCo, Brax) with Ray, Kubernetes, or Slurm; used neural / hybrid simulation (NVIDIA Cosmos, neural reconstruction) for training or augmentation.
- Sim-to-Real Transfer – Closed sim-to-real gaps via domain randomization, system identification, actuator and latency modeling, and real-to-sim reconstruction, measured by field task success; hydraulic or heavy-machinery dynamics highly valuable.
- Policy Evaluation and Safety – Metrics for learned policies (safety, progress, productivity, comfort), statistically rigorous policy comparison, continuous evaluation and regression, adversarial / long-tail scenario testing, runtime safeguards; safety-case thinking (ISO 26262 / SOTIF / ISO 13849).
- Controls and Robotics Foundations – MPC, impedance control, trajectory optimization; judgment on learned versus classical approaches; perception-to-action pipelines under real-time constraints.
- Data Flywheel Literacy – Shapes demonstration and fleet data collection, auto-labeling, and synthetic data strategy for imitation and offline RL.
- Software Engineering – Strong Python and PyTorch (JAX a plus), working C++ for real-time integration, CUDA familiarity, experiment tracking, Bazel or monorepo tooling, CI/CD.
- Architecture Leadership – Turns research into repeatable engineering; sets reproducibility and benchmark standards; mentors ML engineers and researchers; led a shipped learning-based capability.
- Domain Exposure – Off-highway, construction, or mining autonomy (Bedrock Robotics, Built Robotics, Applied Intuition), or robot learning at NVIDIA (Isaac / GEAR), Waabi, Wayve, Zoox, Figure, 1X, Agility, Boston Dynamics, or Tesla Optimus.
- Master’s or doctoral degree in Computer Science, Robotics, Machine Learning, Controls, or related field preferred; research or publication record a plus.
- Additional Details:
- This position requires the candidate to work full-time at the Irving, TX office (Dallas)
- Domestic relocation assistance is available for this position.
- Visa sponsorship is available with this position.
- Why Caterpillar
- You'll work on some of the most challenging applications of AI in the physical world, helping build systems that transform vast amounts of machine and sensor data into the intelligence that powers autonomous and AI-enabled equipment. If you're excited about applying AI to real-world machines operating in complex environments, we'd like to hear from you.
- Summary Pay Range:
- $147,760.00 - $240,110.00
- Compensation and benefits offered may vary depending on multiple individualized factors, job level, market location, job-related knowledge, skills, individual performance and experience. Please note that salary is only one component of total compensation at Caterpillar.
Benefits:
- Subject to plan eligibility, terms, and guidelines. This is a summary list of benefits.
- Medical, dental, and vision benefits
- Paid time off plan (Vacation, Holidays, Volunteer, etc.)
- 401(k) savings plans
- Health Savings Account (HSA)
- Flexible Spending Accounts (FSAs)
- Health Lifestyle Programs
- Employee Assistance Program
- Voluntary Benefits and Employee Discounts
- Career Development
- Incentive bonus
- Disability benefits
- Life Insurance
- Parental leave
- Adoption benefits
- Tuition Reimbursement
- These benefits also apply to part-time employees
- This position requires working onsite five days a week.
- Relocation is available for this position.
- Visa sponsorship is available for eligible applicants.
- Posting Dates:
- Any offer of employment is conditioned upon the successful completion of a drug screen.
- Caterpillar is an Equal Opportunity Employer, Including Veterans and Individuals with Disabilities. Qualified applicants of any age are encouraged to apply.
- Not ready to apply? Join our Talent Community.
About Caterpillar Inc.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Caterpillar Inc.
Engineering Team Leader - Turbomachinery
Bengaluru, IND
Career Area: Engineering Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are
Inventory Analyst
Bengaluru, IND
Career Area: Supply Chain and Logistics Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about eac
Engineering Team Leader - Turbomachinery
Bengaluru, IND
Career Area: Engineering Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are
Regional Certification Liaison 1
Bengaluru, IND
Career Area: Engineering Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are
IT Techno Functional Analyst
Bengaluru, IND
Career Area: Technology, Digital and Data Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about e
Service information Specialist-2
Chennai, IND
Career Area: Product Support Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We
Digital Tech Support Analyst
Bengaluru, IND
Career Area: Technology, Digital and Data Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about e
Service Information Developer-1
Chennai, IND
Career Area: Product Support Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We
Engineering Team Leader - Turbomachinery
Bengaluru, IND
Inventory Analyst
Bengaluru, IND
Engineering Team Leader - Turbomachinery
Bengaluru, IND
Regional Certification Liaison 1
Bengaluru, IND
IT Techno Functional Analyst
Bengaluru, IND
Service information Specialist-2
Chennai, IND
Digital Tech Support Analyst
Bengaluru, IND
Service Information Developer-1
Chennai, IND