Engineer - OpenAI & Chatbot Support

Role overview

Responsibilities

• Monitor service availability, latency, errors, usage, and operational health.
• Triage incidents involving conversation flows, model endpoints, prompts, retrieval, APIs, authentication, and dependent services.
• Investigate inaccurate, incomplete, unsafe, or irrelevant responses and document corrective actions.
• Maintain approved prompts, system instructions, chatbot topics, knowledge sources, configuration records, and version history.
• Support RAG content ingestion, indexing, access controls, and knowledge refresh activities.
• Validate releases and configuration changes across development, testing, and production.
• Analyze logs, traces, user feedback, and usage metrics to identify recurring issues and improvement opportunities.
• Apply responsible-AI, privacy, security, content-safety, and human-escalation controls.
• Coordinate L2/L3 escalation and problem resolution with application, Data & AI, security, cloud, and business teams.
• Maintain SOPs, known-error records, support knowledge articles, test evidence, and service reports; use prompt engineering.

Technical skills

• Enterprise LLM services such as Azure OpenAI or comparable platforms; deployment, configuration, endpoints, and operational support.
• Chatbot platforms such as Microsoft Copilot Studio, Azure Bot Service, Bot Framework, or comparable platforms.
• RAG concepts, document ingestion, chunking, vector and semantic search, grounding, and knowledge-base maintenance.
• Prompt configuration, conversation flows, fallback handling, escalation paths, response-quality testing, and version control.
• REST APIs, JSON, authentication, connectors, webhooks, and dependent application integration troubleshooting.
• Logging, tracing, analytics, content filtering, privacy controls, access controls, and responsible-AI operational practices.
• Incident, problem, change, release, SLA, runbook, and knowledge-management practices.

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• Strong analytical, problem-solving, and troubleshooting skills.
• Excellent communication and documentation abilities.
• Ability to work independently and collaborate across technical and service-management teams.
• Customer-focused approach and disciplined ownership of incidents, requests, and operational commitments.

Good to have

• Experience with Azure AI Search and enterprise knowledge bases.
• Exposure to Semantic Kernel or agent orchestration frameworks.
• Experience with LLM observability, evaluation, or prompt-testing tools.
• Relevant Microsoft Azure, AI, or ITIL certifications.
• Exposure to front-end stacks such as Next.js or React.js; Python experience.

About Malomatia