AI expert Chip Huyen outlines six common pitfalls when building generative AI applications: using GenAI unnecessarily, confusing bad product with bad AI, starting too complex, over-indexing on early success, forgoing human evaluation, and crowdsourcing use cases without strategy. The article provides real-world examples and practical advice to avoid these mistakes.
Many problems do not require generative AI; simpler methods like linear programming can be more effective and reliable.
Poor product experience is often mistaken for poor AI; UX is the key differentiator.
Intelligent agents are considered the ultimate goal of AI. This article from AI Engineering (2025) provides a comprehensive framework for understanding agents, focusing on tools and planning. It covers agent overview, tool categories (knowledge augmentation, capability extension, write actions), planning processes (generation, validation, execution, reflection), and evaluation of agent failures.
An agent is defined by its environment and the actions it can perform, augmented by tools.
Tools are categorized into knowledge augmentation, capability extension, and write actions, significantly boosting model performance.
After studying how companies deploy generative AI applications, this post outlines the common components of a generative AI platform. Starting from a simple query-response architecture, it progressively adds context enhancement (RAG, query rewriting), guardrails (input/output), model routers and gateways, caching (prompt, exact, semantic), complex logic and write actions, and observability/orchestration. Each component's trade-offs and implementation considerations are discussed.
A generative AI platform consists of context construction, guardrails, model router/gateway, cache, complex logic/write actions, and observability/orchestration.
RAG is the most common pattern for context construction, combining term-based and embedding-based retrieval in hybrid search.
The author explores three metrics for personal growth: rate of change, time to solve problems, and number of future options. Inspired by discussions with friends, she proposes heuristics that prioritize novelty and exploration over traditional measures like net worth.
Personal growth can be measured by rate of change, problem-solving speed, and future options.
The author suggests becoming a new person every 3-6 years, solving big problems quickly, and maximizing future options.
Chip Huyen analyzes nearly 900 popular open-source AI projects, revealing explosive growth in applications and AI engineering layers in 2023, while infrastructure remained relatively stable. The Chinese open-source ecosystem is diverging significantly from the West, with many Chinese-focused models and tools emerging.
Searched GitHub for GPT, LLM, and generative AI repos with 500+ stars, yielding 845 software repositories after filtering out tutorials and lists.
AI stack divided into infrastructure, model development, and application development; applications and AI engineering saw the most growth in 2023.
This article explores predicting user preferences for AI model responses to enable model routing and improve efficiency. The author demonstrates that preference prediction is feasible with a small amount of data and shows its performance across different prompts.
Predictive human preference can predict which model users prefer for a given prompt, enabling model routing and budget planning.
Chatbot Arena ranking accuracy is 74.1%, while a preference predictor with prompts achieves 76.2%.
An in-depth exploration of sampling strategies including temperature, top-k, top-p, test time compute, and structured outputs, explaining how they affect the creativity, consistency, and reliability of AI model responses.
Temperature adjusts logits to balance creativity and determinism; higher values increase diversity but may reduce coherence.
Top-k and top-p sampling trade off computational efficiency and output diversity; top-p dynamically selects the most relevant tokens.
This comprehensive article explores multimodal AI systems, particularly Large Multimodal Models (LMMs). It covers the rationale for multimodality, data modalities, multimodal tasks, and dives into the architectures and training of CLIP and Flamingo models. The post also discusses active research directions such as generating multimodal outputs, instruction-following, and efficient adapters.
Multimodal systems integrate text, images, audio, and more to enhance AI capabilities in real-world scenarios.
CLIP uses contrastive learning to create a shared embedding space, enabling zero-shot image classification.
This article summarizes ten major research directions in large language models, covering hallucinations, context learning, multimodality, speed and cost, new architectures, GPU alternatives, agents, human preference learning, chat interface efficiency, and non-English language models. Based on discussions with industry and academia, the author analyzes the current state and challenges of each direction.
Hallucination is a major barrier to LLM adoption, requiring better measurement and mitigation.
Context length and construction efficiency are critical for RAG and other applications.
Chip Huyen's talk at Fully Connected provides a simple framework for teams to navigate generative AI strategy, born from conversations with friends struggling to define their approach. She plans to expand the talk into a full article while welcoming community feedback.
Talk offers a practical framework for exploring generative AI
Inspired by conversations with friends about strategic direction