Senior Applied AI Engineer -- Evaluation & Reliability

They make consequential decisions and answer for them. They don't need analysis that sounds right. They need analysis they can defend and drives expected outcomes.\n\nOur ambition is larger than producing good reports.

We want to be the analytical infrastructure institutionals relies on, and getting there means solving problems no one has fully solved yet.\n\nAbout the role This role exists to solve the hardest of those problems: making AI-generated analysis trustworthy at an institutional standard.\n\nA serious report is long, dense, and full of numbers that must agree with one another and trace back to their sources. Language models don't do this reliably on their own. They produce work that is fluent and often subtly wrong, and across a long document those errors compound.

You would own the layer that catches them — the evaluation, grounding, and quality systems that decide whether what we produce can be trusted. This is central engineering, not a side function.\n\nWhat you'll do Build the evaluation systems that show, continuously and precisely, where our output is wrong and why.\nDesign the architecture that keeps generated analysis grounded in its sources and consistent across a long document.\nDefine what \"ready to deliver\" means in terms a machine can enforce, and build the gates that hold that line as volume grows.\nWork alongside the engineering team — tech lead, engineers, scrum master — owning the reliability layer in particular.\n\nYou may be a good fit if you Have several years of production engineering experience, strong in Python, on systems others relied on.\nHave built and operated LLM systems in production, including multi-step or agentic ones.\nHave designed evaluation or validation for systems that don't return the same answer twice — not unit tests, but ways to measure how and where a model fails.\nAre bothered by a wrong answer in a way that's hard to switch off.\n\nNice to have A data-engineering background: provenance, normalization, retrieval, grounding.\nTime in a domain where a wrong answer has real consequences — finance, medicine, anything safety-critical.\nJudgment about when a problem needs a fixed rule and when it needs a model in the loop.\n\nWorking here We are a small, senior team, Institutional investor funded, based in Toronto and comfortable with remote work across North American hours. You would own the reliability of the product directly.

The package includes meaningful equity, and we are open to contract-to-hire for the right person.\n\nEvery application is read by someone on the team. If this is the problem you want to spend the next few years on, email with a short note and something that shows how you think about correctness in real systems — a project, a write-up, a repo.