Astronomer QA Lead - Remote

About This Role

Your Profile

• Bachelor’s, Master’s, or PhD degree in Astronomy, Astrophysics, Physics, Space Science, Planetary Science, Cosmology, or a closely related field.
• Strong grasp of the English language to follow project guidelines, communicate with teams, and provide clear written feedback.
• 3+ years of experience in astronomy/astrophysics research, teaching, science communication, academic review, data analysis, observatory work, or related scientific workflows.
• Strong understanding of celestial mechanics, stellar evolution, galaxies, cosmology, electromagnetic radiation, observational methods, spectroscopy, planetary systems, black holes, and scientific uncertainty.
• Ability to evaluate astronomy/astrophysics content against detailed rubrics and identify issues such as incorrect physical assumptions, wrong units, flawed calculations, hallucinated facts, misleading explanations, or oversimplified conclusions.
• Familiarity with tools or methods such as Python, astronomical datasets, telescope/observatory data, spectroscopy, photometry, simulations, LaTeX, Jupyter notebooks, or scientific visualization is preferred.
• Experience leading or supporting remote teams of researchers, educators, reviewers, annotators, science writers, or QAs is strongly preferred.
• Comfortable working in fast-moving remote environments using tools such as Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• Highly detail-oriented and organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, calibration tasks, and documentation.
• Experience with AI training, data annotation, LLM evaluation, scientific QA, academic review, or rubric-based review is a strong plus.

Key Responsibilities

• Quality monitoring: Spot-check astronomy/astrophysics items, identify quality issues, provide ongoing feedback through DMs, and escalate recurring or critical issues.
• Scientific review: Evaluate AI-generated astronomy/astrophysics explanations, calculations, diagrams, observational interpretations, comparisons, and step-by-step reasoning for accuracy and clarity.
• Trainer and QA communication: Update trainers and QAs on Discord about new item guidelines, project changes, workflow updates, quality expectations, and astronomy/astrophysics-specific review standards.
• Question handling: Respond to trainer/QA questions clearly and promptly, especially around physical assumptions, units, astronomical terminology, observational methods, formulas, and rubric interpretation.
• Trainer/QA activation management: DM contributors who are inactive or not working, encourage activation, track follow-ups, and flag availability issues when needed.
• Documentation: Create and maintain astronomy/astrophysics project documentation, including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Onboarding and training: Schedule and run onboarding/training calls with trainers and QAs to explain project expectations, workflows, rubrics, quality standards, and astronomy/astrophysics-specific review requirements.
• Quality alignment: Ensure all trainers and QAs apply astronomy/astrophysics review guidelines consistently and understand updates as projects evolve.
• Risk review: Flag misleading, overconfident, physically impossible, numerically incorrect, or poorly sourced astronomy/astrophysics claims.
• Process improvement: Identify recurring quality gaps, propose workflow improvements, and help build scalable QA processes for astronomy/astrophysics AI training projects.