Senior AI Engineer - Trust & Explainability (AI Platform)
Job Summary
The AI Platform team is building the shared runtime that every AI agent at MeridianLink will run on as part of the MeridianLink One platform: model gateway, orchestration, memory, and observability, delivered as reusable services so product teams build agents once and run them safely for hundreds of credit union and lender customers. This role leads the trust and explainability layer of that runtime: the tracing, evaluation, and explanation capabilities that let engineers understand exactly what a multi-agent workflow did and why, and that let product teams show end customers the right level of explanation to trust an agent's output.
This role has a strong dotted-line relationship to MeridianLink's Security Operations team. You are the trust layer's primary technical counterpart to Security, so the trust layer's tests and Security's threat models are built from the same picture.
About the Opportunity
This is a senior, hands-on engineering role. You will own the design and delivery of the trust and explainability layer, from the trace schema through the explanation primitives product teams ship to lenders, credit unions, and their borrowers. You will lead framework evaluations for the trust layer, bring your designs to architecture review, and contribute to the platform's standards for how agents are instrumented and evaluated. You bring deep experience in at least one of LLM tracing, LLM evaluation, or AI security, and enough breadth across the others to make sound trade-offs. You work alongside a Staff engineer team lead and architects who set the platform's overall direction, and you mentor the L1 and L2 engineers on the team.
What it means to be a Senior AI Engineer at MeridianLink
Senior individual contributors own their work end-to-end, identify problems before they're surfaced, and make the engineers around them better. Senior engineers at MeridianLink are active, daily users of AI-assisted development tools.
Technical Execution & Delivery
•Owns features and infrastructure end-to-end: design through production release, limited guidance required
•Identifies edge cases and failure modes independently within assigned scope
•Participates actively in code review with constructive, specific feedback
•Surfaces blockers early rather than waiting for check-ins
Craft & Professionalism
•Writes tests that catch regressions without over-engineering the suite
•Monitors shipped work, responds to issues, and follows incidents to resolution
•Puts institutional knowledge into shared systems rather than individual heads
Agent Tracing & Explainability
•Designs the trace data model for the trust layer: how a single conversation is followed from the first user message through every model call, tool call, retrieval, agent handoff, and decision to the final response
•Solves correlation across complex multi-agent workflows, including handoffs, parallel branches, retries, and long-running human-in-the-loop steps
•Separates debugging telemetry from audit-grade records, and designs both with clear retention, access, and PII handling rules
Customer-Facing Trust & Explanation
•Designs the explanation tiers the trust layer supports: what an engineer, a loan officer, a compliance reviewer, and a borrower each need to see
•Works with Product and Compliance on what an agent must disclose at the product surface: what it did, what data it used, how confident it was, and what a human should verify
•Makes and defends judgment calls on explanation depth in a regulated lending and account-opening context
Trust & Explainability Frameworks
•Knows the open-source landscape for LLM observability, tracing, and evaluation in depth and leads framework evaluations for the trust layer with written trade-offs
•Presents framework and design choices in architecture reviews and adjusts the design based on what review surfaces
•Contributes to the platform's instrumentation standard, aligned to OpenTelemetry GenAI conventions, so traces stay portable across tools
Agent Evaluation & Quality
•Designs evaluation approaches for non-deterministic output: golden datasets, rubric and LLM-as-judge scoring, regression baselines, and statistical thresholds for pass and fail
•Builds and owns the trust layer's evaluation checks that run in CI and on a schedule, so a model swap, prompt change, or tool change is a measured event before it reaches customers
•Works with agent-owning teams on quality baselines and treats evaluation data as versioned, reviewed code
Safety & Tenant Isolation
•Knows the common attack patterns against LLM applications (prompt injection, jailbreaks, tool misuse, data exfiltration) and the frameworks that catalog them, and designs test coverage against them
•Builds the tests that verify the platform's tenant isolation guarantee and treats a cross-tenant leak as a Sev 1 class defect, in test or in production
•Contributes to the shared guardrail layer so every agent inherits protection without re-implementing it
What Success Looks Like
In the first few months, a successful hire owns the trace schema for the trust layer and has shipped tracing that follows a full multi-agent workflow end to end, has taken the explanation primitive design through architecture review, and has the evaluation framework running regression checks against real agents. Over the first year, success looks like this: evaluation, red-team, and isolation checks run on every agent release, product teams instrument their agents with the trust layer's tooling, and the L1 and L2 engineers on the team are measurably stronger for working with this person. When an engineer asks why an agent did something, or a customer or auditor asks whether to trust it, the trust layer answers. Engineers who thrive here want to become the team's go-to on how MeridianLink builds AI people can see into and trust.
Key Responsibilities
Multi-Agent Tracing & Explainability
•Design and lead delivery of tracing for the trust layer across the platform's gateway, orchestration, memory, and tool layers, and own the trace schema for the trust layer
•Own the correlation model that links actions across agents in a single multi-agent workflow
•Lead the developer-facing trace view so a full agent conversation is readable to someone who did not write the agent
•Own the explanation layer that turns trace data into a human-readable account of why an agent did what it did
Customer-Facing Trust
•Own the design and contract of the trust layer's explanation primitives: explanation records, confidence and provenance metadata, and "what the agent relied on" summaries
•Lead adoption of those primitives in the Document Request Agent and the MLM agents, working directly with product engineers
•Run the feedback loop with product teams, Compliance, and customer-facing staff, and drive the explanation format forward from what they learn
Trust & Explainability Frameworks
•Lead evaluation of open-source observability, tracing, and evaluation frameworks for the trust layer, write the recommendation, and take it through architecture review
•Extend chosen frameworks where our agent workloads need more, and contribute fixes and extensions upstream where it makes sense
•Scope and lead the build of trust and explainability tooling the ecosystem does not provide yet
Evaluation Tooling
•Own the trust layer's evaluation framework: golden dataset management, test runners, scoring pipelines, and regression reporting
•Build the tooling agent-owning teams use to create and version golden datasets from de-identified real traffic and synthetic cases
•Work with agent-owning teams on quality baselines and alert when an agent drifts below them
•Run model and prompt comparisons for the platform's shared components and present results and recommendations to the team lead and architects
Safety & Isolation Testing
•Own the red-team and adversarial test suites covering prompt injection, jailbreak attempts, tool misuse, and data exfiltration, and feed their results into the platform's release process
•Own the automated test suite that verifies agents on the shared runtime cannot cross tenant boundaries through memory, retrieval, tool calls, or model context
•Maintain the trust and isolation checklist new agents complete before onboarding to the platform
•Act as the trust layer's standing point of contact for Security Operations: join threat modeling sessions and AI-related incident response, bring their findings into the trust layer's tests, and feed new attack patterns back to them
Engineering Practices
•Contribute to the platform's standards for tracing, evaluation, and explanation, and write the trust layer's guides and examples
•Lead design reviews for trust and explainability work within the AI Platform team
•Help product teams instrument and evaluate their agents well through working sessions and code review
Collaboration & Growing Others
•Participate in design discussions and code reviews; give and receive feedback constructively
•Mentor other engineers on the team
•Contribute to documentation and onboarding materials that reduce tribal knowledge
Qualifications
Required Experience
•5+ years of professional software engineering experience, delivering features and infrastructure independently in production
•Proficiency with standard development tooling: Git, Docker, automated testing, and modern scripting languages
•Active daily use of AI-assisted development tools
•Bachelor's degree in Computer Science, Software Engineering, or equivalent experience
•2+ years of hands-on experience building software that integrates large language models into production systems (LLM APIs, agent frameworks, RAG pipelines, or similar)
•Deep proficiency in Python with production experience owning and delivering major features; TypeScript a plus
•Deep production experience in at least one of the following, and working experience in at least one more: LLM observability and tracing, LLM evaluation and testing, agent frameworks and multi-agent orchestration, or AI and application security testing
•Track record of designing and leading delivery of a tracing, observability, or test infrastructure component that other engineers relied on
•Strong grasp of system design, distributed systems, and architectural trade-offs, with experience presenting designs in architecture or technical design reviews
•Working understanding of the typical technical stack: front end, back end, authentication, and APIs
•Experience running workloads on Azure or AWS, including identity and access management, networking, and secrets management
•Experience mentoring other engineers
Preferred Qualifications
•Experience with OpenTelemetry, including the GenAI semantic conventions, or OpenLLMetry
•Experience with LLM observability and evaluation tools (Langfuse, Arize Phoenix, LangSmith, Braintrust, promptfoo, DeepEval, or equivalent)
•Familiarity with LLM security frameworks and threat models (OWASP Top 10 for LLM Applications, MITRE ATLAS, NIST AI RMF)
•Contributions to open-source AI observability, evaluation, or agent framework projects
•Experience with cloud-managed model services such as AWS Bedrock or Azure OpenAI, including their guardrail and logging features
•Experience building or operating multi-tenant SaaS systems where tenant isolation was a hard requirement
•Prior experience in financial services, fintech, or another regulated industry where explainability and audit trails shaped technical decisions
•Exposure to model risk management, fair lending, or responsible AI practices from an engineering perspective