Expertise
Capability across five domains
AI agents and voice, RAG and multimodal, the software platform underneath, and the model internals when a system has to get cheaper or faster. Skills and tools per domain, with one decision each that shows how the calls get made.
AI Agents & Voice Agents
Systems that use tools, call other systems and hand off to people. The engineering is in the failure modes, not the happy path.
What I build
- Tool-using agents over MCP servers, with checkpointing that survives a process restart
- Multi-step orchestration with retries, timeouts and human review on low-confidence paths
- Streaming voice loops — ASR into dialogue into speech — budgeted per hop
- Confidence gating that routes the cases which should not be automated
Skills & tools
Orchestration
- LangGraph
- LangChain
- MCP
- Durable execution
Voice
- Streaming ASR
- faster-whisper
- Piper TTS
- Barge-in
Reliability
- Bounded concurrency
- Retry classification
- Cost ledgers
Runtime
- Python
- FastAPI
- WebSockets
- anyio
A decision that mattered
Topology is downstream of the failure you fear
A supervisor architecture can cost several times the tokens for no accuracy gain, and is worse on error cascades because fresh contexts discard the error history.
RAG & Multimodal AI
Getting the right evidence in front of the model, including when the input is a scanned page or a photograph rather than clean text.
What I build
- Hybrid dense and lexical retrieval with re-ranking and a filter strategy that does not silently drop results
- Chunking chosen against document shape rather than a default token count
- Vision-language extraction over scanned and photographed documents
- Retrieval quality measured separately from generation quality
Skills & tools
Vector
- pgvector
- FAISS
- Pinecone
- HNSW tuning
Retrieval
- Hybrid search
- BM25
- Re-ranking
- RRF
Embeddings
- sentence-transformers
- ColPali
- Visual embeddings
Multimodal
- VLM extraction
- OCR
- OpenCV
- YOLO
A decision that mattered
One RAG score cannot debug a RAG system
Retrieval and generation fail differently. A blended end-to-end metric is the wrong thing to gate a deploy on — recall can regress measurably while the blend reports no change.
Software Engineering & Platforms
The product around the model. Most AI features fail on ordinary engineering — schema, concurrency, caching, deployment — long before they fail on model quality.
What I build
- Backend services and APIs in Python and FastAPI, with C++ where latency demands it
- Relational and vector data layers designed against their query plans
- Event-driven pipelines with at-least-once delivery and idempotency keys
- Containerised deployments with CI/CD, tracing and per-tenant budgets
Skills & tools
Backend
- Python
- FastAPI
- C++
- REST
- WebSockets
Data
- PostgreSQL
- Redis
- MongoDB
- Query planning
Infrastructure
- Docker
- Kubernetes
- AWS
- Azure
- CI/CD
Operations
- OpenTelemetry
- SLOs
- Load testing
- Migrations
A decision that mattered
The obvious index is often a real improvement and still wrong
An index on the filter column finds the rows without supplying the ordering, so the sort node stays in the plan. Read the plan before and after; a composite covering index removes both.
LLM Engineering & Evals
Adapting open models to a domain, and the measurement that decides whether a change shipped an improvement or a regression.
What I build
- Supervised fine-tuning with correct chat templating and completion-only loss masking
- Preference optimisation chosen against the shape of the feedback actually available
- LLM-as-judge harnesses calibrated against human labels
- Golden datasets and CI regression gates that fail a build on a detectable drop
Skills & tools
Post-training
- LoRA
- QLoRA
- DPO
- GRPO
- TRL
- PEFT
Evals
- LLM-as-judge
- Cohen's kappa
- Bootstrap CIs
- Golden sets
Data
- Synthetic generation
- Decontamination
- MinHash
Frameworks
- PyTorch
- Transformers
- Accelerate
- FSDP
A decision that mattered
An uncalibrated judge is a random number generator with manners
Report agreement as kappa against human labels, not raw accuracy — a coin-flip judge agrees half the time. And state the smallest delta the sample size can actually detect.
AI Internals & Optimisation
What a token costs and why. Useful when a system has to get cheaper or faster and the obvious levers have already been pulled.
What I build
- Self-hosted open-weights serving with quantisation, sized against a latency target and a cost per million tokens
- Capacity models that answer how much hardware a throughput target needs
- Long-context evaluation that separates retrieval from composition
- Attention and KV cache implementations from the arithmetic up
Skills & tools
Serving
- vLLM
- llama.cpp
- Continuous batching
- KV cache
Quantisation
- AWQ
- GPTQ
- INT8
- ONNX Runtime
- TensorRT
Architecture
- GQA
- MoE routing
- RoPE
- YaRN
- Speculative decoding
Profiling
- Roofline analysis
- Arithmetic intensity
- Goodput
A decision that mattered
Prefill and decode are different machines
One is compute-bound, the other bandwidth-bound. Batching helps one and not the other, which is why tokens per second is the wrong headline and goodput under an SLO is the right one.
Contact
Get in touch
Happy to talk about anything here, or about LLM systems, evaluation and serving generally. Mentioning a repository by name gets you a faster and more useful answer.
- Based in
- Abu Dhabi, UAE
- GitHub
- github.com/malcomzww
- linkedin.com/in/malcomzw