oracle
Data Engineer (Software Developer 2)
Stanford University
Stanford, USA
Full-time
Mid
Onsite
Discovered 3 weeks ago
airflowdockerkubernetespythonrustsql
Key Skills for This Role
airflowdockerkubernetespythonrustsql
Full Job Posting
Responsibilities:
- Design, build, and operate ETL/ELT and high-throughput ingestion pipelines connecting acquisition software, processing code, metadata registries, object storage, and databases
- Own the data layer end-to-end: schema design, indexing, query optimization, migrations, backups, and durable data models that hold up over years of scientific work
- Improve reliability, scalability, observability, and performance across the data stack, and lead troubleshooting/incident response when things break
- Build tools with and to help researchers and our in-house AI agents discover datasets, inspect processing state, and safely diagnose or rerun failed jobs
- Productionize research code — testing, packaging, deployment, monitoring, documentation
- Partner with data acquisition, infrastructure engineering, scientific analysis and AI modeling teams.
- Conceptualize design, implement, and develop solutions for complex systems/programs.
- Work with a variety of users to gain information, and develop intra-system tradeoffs between different users, as necessary; interact with a diverse client base and outside vendor contacts.
- Document system builds and application configurations; maintain and update documentation as needed.
- Provide technical analysis, design, development, conversion, and implementation work.
- Work as a project leader, as needed, for projects of moderate complexity.
- Serve as a technical resource for applications.
- Compare, evaluate, and implement new features and technologies, and integrate them into the computing environment.
- Follow team software development methodology.
- Mentor junior software developers.
- Expertise in designing, developing, testing, and deploying applications.
- Proficiency with application design and data modeling.
- Ability to define and solve logical problems for highly technical applications.
- Effective communication skills with both technical and non-technical clients.
- Ability to lead activities on structured team development projects.
- Ability to select, adapt, and effectively use a variety of programming methods.
- Knowledge of application domain.
Preferred Qualifications:
- Strong proficiency in Python and SQL
- Experience with relational and NoSQL databases, as well as object storage (e.g., S3, MinIO, Ceph)
- Experience designing, building, and operating production ETL/ELT pipelines
- Experience designing durable data models and schemas for complex, evolving datasets
- Experience building high-throughput ingestion systems and optimizing data movement across storage, compute, and network boundaries
- Experience with Docker and container orchestration platforms such as Kubernetes
- Strong communication skills and experience working in cross-functional teams
- Experience designing APIs or service interfaces used by multiple teams
- Experience with scientific computing and large-scale neuroscience or other scientific data
- Workflow orchestration experience with Airflow, Dagster, Prefect, or similar tools
- Experience in building pipelines with and for autonomous, agentic AI systems
- Experience with performance-critical compiled or systems languages (C, C++, or Rust)
- Familiarity with data provenance and lineage tracking, metadata catalogs, or dataset versioning
- As part of your application, please submit the following two separate attachments:
- Cover Letter - Please provide a cover letter describing your interest in joining our lab and the position.
- Technical Experience Responses - In a separate document, please answer each of the questions below in a few sentences. Please upload your responses as a single, separate attachment to your application.
- Please describe your experience working with SQL and NoSQL databases.
- Please describe you experience operating ETL/ELT data pipelines.
- Please describe your experience with orchestration tooling such as Kubernetes and Ray.
- You’ll Thrive in This Role If You:
- Are excited by a fast-paced, production-focused research environment that often requires switching between many hats
- Can move quickly without sacrificing rigor
- Care about making complex workflows reliable and easy to use
- Enjoy translating ambiguous research needs into pragmatic technical systems
- Take ownership across design, implementation, deployment, and operations
- Enjoy collaborative design and working closely across team boundaries
- Are energized by working as part of a cohesive team pursuing ambitious, long-term neuroscientific and AI goals
- What We Offer:
- A highly collaborative environment across neuroscience, engineering, and AI
- Opportunity to contribute to a next-generation neurotechnology platform
- Competitive salary and benefits
- Strong mentoring and career development support
Responsibilities
- Core Duties:
- Minimum Education and Experience:
- Bachelor's degree and five years of relevant experience, or a combination of education and relevant experience.
- Knowledge, Skills and Abilities:
Qualifications
- Strong proficiency in Python and SQL
- Experience with relational and NoSQL databases, as well as object storage (e.g., S3, MinIO, Ceph)
- Experience designing, building, and operating production ETL/ELT pipelines
- Experience designing durable data models and schemas for complex, evolving datasets
- Experience building high-throughput ingestion systems and optimizing data movement across storage, compute, and network boundaries
- Experience with Docker and container orchestration platforms such as Kubernetes
- Strong communication skills and experience working in cross-functional teams
- Experience designing APIs or service interfaces used by multiple teams
- Experience with scientific computing and large-scale neuroscience or other scientific data
- Workflow orchestration experience with Airflow, Dagster, Prefect, or similar tools
- Experience in building pipelines with and for autonomous, agentic AI systems
- Experience with performance-critical compiled or systems languages (C, C++, or Rust)
- Familiarity with data provenance and lineage tracking, metadata catalogs, or dataset versioning
- Cover Letter - Please provide a cover letter describing your interest in joining our lab and the position.
- Technical Experience Responses - In a separate document, please answer each of the questions below in a few sentences. Please upload your responses as a single, separate attachment to your application.
- Please describe your experience working with SQL and NoSQL databases.
- Please describe you experience operating ETL/ELT data pipelines.
- Please describe your experience with orchestration tooling such as Kubernetes and Ray.
- Are excited by a fast-paced, production-focused research environment that often requires switching between many hats
- Can move quickly without sacrificing rigor
- Care about making complex workflows reliable and easy to use
- Enjoy translating ambiguous research needs into pragmatic technical systems
- Take ownership across design, implementation, deployment, and operations
- Enjoy collaborative design and working closely across team boundaries
- Are energized by working as part of a cohesive team pursuing ambitious, long-term neuroscientific and AI goals
About Stanford University
Higher Education20692 employeesFounded 1885
Private research university in Stanford, California.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Stanford University
Groundskeeper
Stanford, USA
MidFull-time
This position is represented by SEIU Local 2007 and the collective bargaining agreement between the university and SEIU Local 2007 governs the terms and conditions of employment. Welcome to Land, Buildings & Real Estate
Discovered TodayView →
Senior Graphic Designer
, USA
SeniorFull-time
Main Work Location: Historic Stanford Campus Schedule: Open to Remote Budgeted Salary Range: The expected pay range for this position is $72,500 to $142,700 per annum. This role is open to candidates anywhere in the Unit
Discovered TodayView →
Staff Scientist
Menlo Park, USA
SeniorFull-time
Position Overview The Accelerator Research and Technology Department at SLAC National Accelerator Laboratory seeks a talented Low-Level RF and Control Systems Specialist with demonstrated expertise in next-generation LLR
Discovered TodayView →
Tree Specialist II
Stanford, USA
MidFull-time
This position is represented by SEIU Local 2007 and the collective bargaining agreement between the university and SEIU Local 2007 governs the terms and conditions of employment. Welcome to Land, Buildings & Real Estate
Discovered TodayView →
Administrative Services Manager
Stanford, USA
ManagerFull-time
This position has been deemed critical by the School of Engineering Dean’s Office and is exempt from the hiring freeze. For Stanford University employees, please apply through the Opportunity Marketplace within Sequoia.
Discovered TodayView →
Pediatric Research Program Manager
Stanford, USA
ManagerFull-time
Stanford University seeks an Academic Program Professional 2 to serve as Pediatric Research Program Manager in the Department of Orthopaedic Surgery. This role will work under the direction of the Chief of Pediatric Orth
Discovered TodayView →
Cancer Clinical Research Coordinator Associate – GI (OnSite)
Stanford, USA
MidFull-time
Discovered TodayView →
Cancer Clinical Research Coordinator Associate – GI (OnSite)
Stanford, USA
MidFull-time
Discovered TodayView →
Groundskeeper
Stanford, USA
Discovered TodayFull-time
Senior Graphic Designer
, USA
Discovered TodayFull-time
Staff Scientist
Menlo Park, USA
Discovered TodayFull-time
Tree Specialist II
Stanford, USA
Discovered TodayFull-time
Administrative Services Manager
Stanford, USA
Discovered TodayFull-time
Pediatric Research Program Manager
Stanford, USA
Discovered TodayFull-time
Cancer Clinical Research Coordinator Associate – GI (OnSite)
Stanford, USA
Discovered TodayFull-time
Cancer Clinical Research Coordinator Associate – GI (OnSite)
Stanford, USA
Discovered TodayFull-time