Kiseki Labs — AI that survives contact with production

Evidence first, architecture second.

Establish ground truth before touching architecture

Before any design decision, I build the evaluation suite: the set of real questions the system has to get right. Every argument after that gets settled by measurement rather than seniority. It is the single highest-leverage week in any engagement and it is almost always skipped.

Size the approach to the organisation

A 400-person logistics operator and a regulated financial institution need different things, and neither needs what a Series A startup needs. Governance, delivery pace and architecture get matched to how your organisation actually makes decisions and passes review.

iii

Build for production, not for the demo

Verification layers, self-healing retries, CI, dependency hygiene, a real test suite. None of it is interesting. All of it is the difference between a system your teams depend on and one they route around within a quarter.

Hand over completely

Documentation and knowledge transfer are scheduled deliverables. Your engineers should be able to extend the system without me in the room. Consultants who engineer dependency get renewals. They don't get referrals.

03i

KYC & document verification

An hour of manual checking per applicant became five minutes.

A financial services provider onboarding self-employed drivers. Every applicant submitted a form plus supporting documents. An operations analyst then verified each one by hand — reading the documents, cross-checking the details against what the applicant had typed, confirming declared income against actual bank activity. Thirty to sixty minutes per applicant, and the queue only ever grew.

Engagement

Function: Onboarding / KYC
Sector: Financial services
Pattern: Extract, reconcile, escalate
Decisions: Stay with the human

Before 30–60min

→

After 5min

Analyst time per applicant. The saving compounds with volume — and it removed the queue as a constraint on how fast the business could sign people up.

Documents ingested

Driving licence
Bank statements
Vehicle registration
Application form responses

Reconciliations performed

Vehicle registration: document vs. declared
Identity details: document vs. declared
Ride-hail earnings: claimed vs. bank activity
Exceptions named, not just counted

Situation

Manual review of every application, document by document. Slow, inconsistent between analysts, and impossible to scale without hiring. The work was necessary — it was a regulated check, not busywork — which is exactly why throwing it away was never an option.

Approach

A pipeline that ingests the submission and its attachments, extracts the fields from each document, and reconciles them against each other and against what the applicant entered. Anything that doesn't match is flagged with the specific discrepancy named and routed to a person.

Outcome

Review time fell from 30–60 minutes to about five. The more useful change was qualitative: the team stopped hunting for problems and started resolving them. The system says what doesn't reconcile, not merely that something doesn't.

Why this should matter to you. This is KYC. The documents differ by industry, but the shape does not: extract from unstructured submissions, reconcile across sources that should agree, escalate the exceptions to a human who now has a specific thing to look at. Any onboarding, claims or compliance function running a manual review queue has this exact problem.

And it never approves anyone. The system narrows the work; the judgment stays with the analyst. That was a deliberate design constraint, and it is the reason the thing was deployable inside a regulated process at all.

Human-in-the-loop by design, not by omission.

03ii

Natural-language analytics

Two queries in five were failing. Then almost none were.

A platform that let users ask questions of their spreadsheet data in plain English. After months of development it was still returning malformed output, schema validation errors and runtime exceptions on roughly 40% of queries, at an unpredictable 20,000–50,000 tokens a time.

Engagement

Duration: 4 weeks
Scope: Diagnose, rebuild, hand over
Team: Me, plus their engineers
Domain: Analytics over spreadsheet data

Query success rate

Before <60%

After 98%

Measured against a fixed 35-case evaluation suite. Zero syntax errors reached execution.

Week 01

Foundations before architecture: dependency management, linting, CI via GitHub Actions, documentation. Nothing clever.

Week 02

Built the evaluation suite — 35 cases, six business domains — then used it to settle the core architectural question on evidence: custom DSL, or direct Python generation.

Week 03

Production pipeline: four specialised agents — planner, code generator, verifier, summariser — with self-healing retry logic.

Week 04

Integration, documentation, and knowledge transfer to the engineers who would own it afterwards.

The 35 cases spanned six domains

Building management
Supply chain
EV sales
Emissions
Formula 1
Social media

79tests handed over

4agents in pipeline

60s → 25stypical response

0syntax errors at execution

Why this should matter to you. The client was small. The problem isn't. Every organisation past a certain size has operational data sitting in spreadsheets that nobody can query without opening a dozen files by hand: finance reporting, supply chain, claims, procurement. The engineering that makes an answer trustworthy is identical at four hundred people as at forty. So is the discipline of proving it.

Read the full write-up

Engagements

Four ways to start.

Fixed scope, fixed fee, defined end date. Straightforward to put through procurement.

Engagement	Duration	What you get	Right when
Opportunity Diagnostic Fixed fee	2–3 weeks	Prioritised, costed use-case shortlist Build vs. buy call on each one A business case that survives the board Risks and governance named up front	The mandate is clear but the target isn't.
Pilot to Production Fixed fee · most requested	4–8 weeks	Evaluation harness and measured baseline Production architecture, built and deployed Test suite, CI, operational documentation Knowledge transfer to your engineers	Something already half-works and needs to ship.
Advisory Retainer Monthly	Ongoing	Architecture and vendor review Roadmap and investment challenge Executive and board briefings Direct access between sessions	The decisions are frequent and expensive.
Team Enablement Per cohort	4 weeks	Live cohort for your product and business teams Built against your stack — Jira, Confluence, your analytics Capstone on a real workflow, not a toy exercise Materials and recordings your organisation keeps	Your teams need to use AI, not be briefed about it.

Risk & governance

The four questions your risk team will ask.

They are the right questions, and they are usually asked late. Here are the answers before you have to ask them.

"How do you know the output is correct?"

Every engagement begins by defining what correct means and building the test set that measures it. Accuracy is stated against a named ground truth and a known test set, or it isn't stated. A system whose quality can't be measured isn't ready to carry a business process.

"Where does our data go?"

Data boundaries are a design input, not a retrofit. Which providers see what, retention terms, regional processing, and what never leaves your perimeter are decided at architecture time and written down for your security review.

"What happens when it's wrong?"

It will be, sometimes. Systems get explicit verification layers, defined failure behaviour, and human review wherever being wrong is expensive. The goal was never a system that never errs. It's one that fails visibly instead of silently.

"What happens when you leave?"

You own all of it: code, tests, evaluation suites, and documentation written for the engineers who inherit it. Knowledge transfer is a scheduled deliverable, not a gesture of goodwill in the final week.

Who you'd be working with

Eddie Forson

I have been building AI systems since 2017. Computer vision first, then machine learning platforms, now LLM agents in production. Much of that was inside large regulated organisations, which means I have spent about as much time in security reviews and change boards as in codebases.

That combination is the whole proposition. The technical half is increasingly commoditised. The half that knows why your deployment will actually be blocked in month three is not.

I take a small number of clients at a time, and I do the work myself.

Nine years building production AI systems
Engineering roles at Morgan Stanley and Commerzbank
Current AI delivery work inside regulated financial services
Teaches Applied AI to product managers and product teams
Host of AI Mavericks, a private dinner series for senior AI leaders
Regular at AI Engineer and the London AI Engineer meetup

AI that survives contact with production.

Nobody's AI programme fails because of the model.

The pilot was never built to ship

There is no number for the board

The approach was calibrated for a different company

Evidence first, architecture second.

Establish ground truth before touching architecture

Size the approach to the organisation

Build for production, not for the demo

Hand over completely

An hour of manual checking per applicant became five minutes.

Situation

Approach

Outcome

Two queries in five were failing. Then almost none were.

Four ways to start.

Opportunity Diagnostic

Pilot to Production

Advisory Retainer

Team Enablement

The four questions your risk team will ask.

"How do you know the output is correct?"

"Where does our data go?"

"What happens when it's wrong?"

"What happens when you leave?"

Written for people who have to decide.

From 60% to 98%: What a 4-Week AI Consulting Engagement Actually Looks Like

Key Principles for Building Effective AI Agents

AI Engineer Summit 2025: 8 Key Trends Shaping the Year of Agents

What Menlo Ventures’ 2024 Report Reveals About Generative AI in the Enterprise

Eddie Forson

Start with a briefing,
not a sales call.

AI that survives contact with production.

Nobody's AI programme fails because of the model.

The pilot was never built to ship

There is no number for the board

The approach was calibrated for a different company

Evidence first, architecture second.

Establish ground truth before touching architecture

Size the approach to the organisation

Build for production, not for the demo

Hand over completely

An hour of manual checking per applicant became five minutes.

Situation

Approach

Outcome

Two queries in five were failing. Then almost none were.

Four ways to start.

Opportunity Diagnostic

Pilot to Production

Advisory Retainer

Team Enablement

The four questions your risk team will ask.

"How do you know the output is correct?"

"Where does our data go?"

"What happens when it's wrong?"

"What happens when you leave?"

Written for people who have to decide.

From 60% to 98%: What a 4-Week AI Consulting Engagement Actually Looks Like

Key Principles for Building Effective AI Agents

AI Engineer Summit 2025: 8 Key Trends Shaping the Year of Agents

What Menlo Ventures’ 2024 Report Reveals About Generative AI in the Enterprise

Eddie Forson

Start with a briefing,not a sales call.

Start with a briefing,
not a sales call.