From Software Engineer
to Lead GenAI Engineer

Linear algebra: matrix operations, eigenvalues, SVD — understand these as transformations of space, not just arithmetic. Probability: Bayes' theorem, distributions, MLE — this is the language all ML speaks. Calculus: gradients, chain rule, partial derivatives — gradient descent is just "follow the slope downhill," and that metaphor only clicks once you feel what a gradient is geometrically.

Linear regression, logistic regression, decision trees, SVMs, ensemble methods (bagging, boosting). Implement at least two in pure NumPy. You must understand loss functions, regularisation (L1/L2), and overfitting before touching neural networks. The bias-variance tradeoff is not an abstraction — it's a design decision you'll make constantly.

Build a multilayer perceptron in pure NumPy: forward pass, compute cross-entropy loss, backpropagate gradients using the chain rule, update weights with SGD. Then rebuild it in PyTorch and verify your NumPy gradients match PyTorch's autograd. Understanding what autograd is actually computing is what separates engineers from ML engineers.

Convolutional layers: why local connectivity and weight sharing work for images, receptive fields, pooling, architecture intuition (LeNet, VGG, ResNet residual connections). Then RNNs: sequential computation, hidden state, BPTT. LSTMs: gates, cell state, why they solved vanishing gradients. Understand this history — it leads directly and inevitably to the transformer.

Implement every component by hand: byte-pair encoding tokenisation, learned token embeddings, sinusoidal and learned positional encodings, scaled dot-product attention (QKV), multi-head self-attention, layer normalisation, position-wise FFN, residual connections, causal masking for decoders. Read the original "Attention Is All You Need" paper — not just summaries. Understand why this architecture parallelises where RNNs couldn't.

DataLoaders and custom Dataset classes, learning rate schedulers (cosine annealing, warmup), gradient clipping, mixed-precision training (FP16 with torch.cuda.amp), gradient accumulation for large effective batch sizes, model checkpointing and resuming, Weights & Biases experiment tracking. Train something non-trivial on a GPU. Profile GPU memory usage and know how to reduce it.

Decoder-only architecture (GPT family) vs encoder-decoder (T5, BART) vs encoder-only (BERT). Tokenisation deep-dive: BPE algorithm, WordPiece, SentencePiece — implement BPE by hand. Context window mechanics. KV cache: what it stores, why it trades memory for latency, how PagedAttention improves on it. Autoregressive generation: temperature, top-p, top-k, repetition penalty — implement each sampling strategy. Read the GPT-2 and GPT-3 papers.

Zero-shot, few-shot, chain-of-thought (CoT), tree-of-thought, self-consistency, ReAct prompting. Output structuring: JSON mode, XML schemas, function calling for structured extraction. System prompts, role prompting, constitutional AI prompting. The key insight: prompt engineering without evaluation is guessing. Build an automated evaluation harness that scores prompt variants — LLM-as-judge, BLEU, ROUGE, custom rubrics.

Dense embeddings: what they encode (semantic relationships as geometric proximity), cosine similarity mechanics. Embedding model selection: MTEB benchmark, task-specific vs general models, dimensionality trade-offs. Sparse embeddings: TF-IDF, BM25 — when keyword matching beats semantics. Vector databases: Pinecone, Weaviate, pgvector. HNSW indexing internals: the hierarchical graph structure, how it achieves O(log n) approximate nearest neighbour search, recall vs speed trade-off.

Start with naive RAG (chunk → embed → retrieve → generate) and measure it. Then fix its failure modes one by one. Chunking: fixed-size vs semantic chunking vs late chunking — understand what information is lost at chunk boundaries. Hybrid retrieval: dense + sparse with Reciprocal Rank Fusion — measure improvement on your dataset. Re-ranking: cross-encoder models (ColBERT, BGE-reranker) and why they outperform bi-encoders for precision. Query expansion, HyDE (hypothetical document embeddings), contextual compression. RAGAS evaluation framework: faithfulness, answer relevancy, context precision, context recall.

ReAct loop: Reason + Act, the observation feedback cycle. OpenAI function calling and Anthropic tool use — implement both. LangGraph: stateful agents as directed graphs with cycles, conditional edges, checkpointing, human-in-the-loop interrupts. Memory architecture: in-context (limited), external vector store (semantic retrieval), episodic (conversation history), procedural (tool selection memory). Multi-agent patterns: supervisor-worker, peer-to-peer (AutoGen), sequential chains. Agent evaluation: where it fails (tool selection errors, infinite loops, off-task drift) and how to diagnose each.

Build production AI APIs: FastAPI async endpoints with streaming responses via Server-Sent Events (SSE), Pydantic v2 request/response validation, middleware (auth, rate limiting, logging), error handling and retry logic, background tasks. Containerise with Docker. Your cloud background is an enormous advantage here — you already understand deployment. The focus is on AI-specific patterns: streaming token generation to the client, async LLM calls without blocking, cost tracking middleware.

First, the decision framework: when does fine-tuning beat prompting? (Domain-specific vocabulary, consistent format requirements, proprietary knowledge too large for context, latency-critical applications). LoRA: low-rank decomposition of weight updates — mathematically, you're constraining ΔW to be a product of two small matrices. QLoRA: 4-bit NF4 quantisation + LoRA — trains a 70B model on one consumer GPU. Supervised fine-tuning (SFT) data preparation: instruction-response pair curation, data quality over quantity. HuggingFace TRL + Unsloth for efficient training. RLHF conceptually: reward model training, PPO optimisation loop. DPO (Direct Preference Optimisation): why it replaces PPO in most practical settings — simpler, more stable, no separate reward model. Push your fine-tuned model to HuggingFace Hub.

CLIP: contrastive pretraining on image-text pairs — how it creates a shared embedding space for both modalities, zero-shot image classification, image-text similarity. LLaVA architecture: visual encoder (CLIP) + projection layer + LLM decoder — the projection layer is what maps visual tokens into the LLM's token space. GPT-4V/Claude's vision capabilities conceptually. Whisper for ASR: encoder-decoder transformer trained on 680K hours — build audio transcription pipeline. Document intelligence: OCR (Tesseract, AWS Textract) + layout understanding + LLM reasoning over structured documents.

Neo4j graph database: nodes, relationships, properties, Cypher query language. Entity extraction: spaCy NLP pipeline, GLiNER (generalised NER), LLM-based extraction. Relationship extraction: RE models, LLM-based relation triples. Entity linking: resolving extracted mentions to canonical entities. KG construction pipeline: extract → normalise → link → store. KG-augmented RAG: when graph traversal beats vector search — multi-hop reasoning ("find all papers by authors who collaborated with researchers who cited paper X"), relationship queries, structured fact retrieval. Microsoft GraphRAG: community detection, hierarchical summarisation, global query capability.

Hallucination taxonomy: intrinsic (contradicts source) vs extrinsic (unverifiable fabrication). Detection methods: SelfCheckGPT (sample multiple times, check consistency), G-Eval (LLM-as-judge with fine-grained criteria), grounding scores (does the answer appear in the retrieved context?). Guardrails frameworks: NVIDIA NeMo Guardrails (dialogue flow control), Guardrails.ai (output validation with validators). Structured output forcing: JSON schema validation, Pydantic model enforcement, retry loops on validation failure. Constitutional AI: principle-based critique and revision. Input/output filtering: topic classifiers, PII detection, toxicity filtering. Basic red-teaming: prompt injection, jailbreaking patterns, indirect injection via retrieved content.

Apache Spark with PySpark: RDDs vs DataFrames, transformations (map, filter, groupBy) vs actions (collect, count, write), lazy evaluation, shuffle operations and why they're expensive, partitioning strategy. Spark for ML data preprocessing at scale: feature engineering pipelines, data cleaning, tokenisation over 100M+ records. MongoDB: document model, aggregation pipeline, Atlas Vector Search for hybrid document + vector queries. Building end-to-end data pipelines that feed ML training and inference systems. Your cloud background means you can deploy these on EMR, Dataproc, or Azure HDInsight natively.

Quantisation methods: GGUF (CPU-optimised, llama.cpp), GPTQ (GPU, post-training), AWQ (activation-aware, better quality at 4-bit), FP8 (emerging standard). Speculative decoding: small draft model proposes tokens, large model verifies — how it achieves latency reduction without quality loss. vLLM: PagedAttention — manages the KV cache as non-contiguous paged memory (like OS virtual memory), eliminating fragmentation waste. TGI (Text Generation Inference) by HuggingFace. Continuous batching vs static batching: why continuous batching achieves 10–23x throughput improvement. Latency vs throughput optimisation trade-offs. Load test everything — measure p50, p95, p99 latencies under load.

ML-specific CI/CD: automated evaluation gates before every deployment (if RAGAS score drops by >5%, block the merge), model versioning with MLflow (experiment tracking, model registry, stage promotion: Staging → Production), DVC for dataset versioning. Production monitoring: latency p95/p99, token usage and cost per request, output quality score sampling, hallucination rate via automated detection. Data drift detection: statistical tests (KS test, Population Stability Index) on embedding distributions. Concept drift: when model performance degrades on production data even when data distribution appears stable. A/B testing model versions: traffic splitting, statistical significance. Feedback loops: human rating collection → retraining pipeline.

Apply your cloud knowledge directly to AI workloads. AWS: SageMaker endpoints (real-time, async, batch), Bedrock for managed LLMs, ECS/EKS for custom model serving, Cost Explorer for GPU cost optimisation. Azure: Azure ML managed endpoints, Azure OpenAI Service, AKS with GPU node pools. GCP: Vertex AI Model Garden, Cloud Run for serverless inference, GKE Autopilot. GPU instance selection: A10G vs A100 vs H100 — when each makes sense economically. Autoscaling for variable AI workloads: CPU-based vs custom metrics (queue depth, token throughput). Spot/preemptible instances for training; on-demand for inference.

Multi-tenant AI API design: tenant isolation (separate vector namespaces, API key scoping, rate limiting per tenant), audit logging (who queried what, when, with what result — required for regulated industries). Data residency: keeping data in specific regions for GDPR/data sovereignty compliance. Enterprise data integration: SharePoint via MS Graph API, Salesforce SOQL + Bulk API for CRM data, JDBC for legacy databases. PII detection and redaction before sending data to LLMs. Token budget governance: per-user, per-department cost caps with graceful degradation. SLA management: what happens when the LLM provider is down — fallback models, cached responses, circuit breakers.

The Lead role requires designing AI systems, not just implementing them. Practice whiteboard architecture sessions on real constraints: "design a customer support AI for 10M users on $50K/month cloud budget." Make explicit trade-offs — not "RAG is better than fine-tuning" but "given these latency requirements and data update frequency, RAG is better because..." Practice the questions that expose shallow thinking: What breaks at 10x load? What's the rollback plan if this model degrades? How does this scale to 50 languages? How do we handle GDPR deletion requests on vector stores? A Lead who can't say "this approach fails at X scale because Y" is not ready to own an architecture.

One paper per week from arXiv (cs.LG, cs.CL), HuggingFace Daily Papers, or Papers with Code. The goal is not to read papers — it's to assess them: What is the core contribution? Does it beat existing baselines and by how much? What are the failure modes the authors don't discuss? Would this change how I build anything in production? Start an internal technical notes document — one paragraph per paper: contribution, key result, production relevance. This habit, started now and continued forever, is what "staying current" in the job description actually means.

The ability to explain transformer attention to a non-ML engineer, or to translate a business requirement ("we need the chatbot to stop hallucinating product specs") into a precise technical intervention (grounding via RAG + output validation against product database), is 50% of a Lead role. Practice three levels: peer-level explanation (full technical depth), junior explanation (mechanism without mathematical notation), executive explanation (business impact without mechanism). Write internal documentation for everything you've built. Give a talk — at a meetup, internally, or record for YouTube. The act of teaching reveals every gap in your own understanding.

Build the most complex system you've built. It must use everything: multimodal input processing, knowledge graph construction from raw documents, KG-augmented hybrid RAG with reranking, fine-tuned domain model for generation, agentic orchestration via LangGraph, production FastAPI serving with vLLM, MLflow versioning with automated eval gates, Grafana monitoring, and a guardrails layer. Deploy publicly. Document every architecture decision with explicit trade-offs. Present it at a meetup or publish a detailed technical blog post. This is your portfolio centrepiece. It must be live and demoable in interviews.