Exploring the utility of large language models in autonomous driving: Can they be trusted for self-driving cars, and what are the key challenges?

The article introduces vec2text, a method that can perfectly reconstruct original text from embeddings, revealing significant security vulnerabilities in current RAG systems and vector databases.

This article explores predicting user preferences for AI model responses to enable model routing and improve efficiency. The author demonstrates that preference prediction is feasible with a small amount of data and shows its performance across different prompts.

High-quality data is the fuel for modern deep learning model training. This article explores how to collect high-quality data through human annotation, including task design, rater selection and training, data aggregation, and quality assurance. It covers the wisdom of the crowd, methods for measuring rater agreement (e.g., Cohen's Kappa, MACE), and two annotation paradigms (descriptive vs. prescriptive). Additionally, it discusses techniques to identify mislabeled data using influence functions, training dynamics (e.g., data maps, forgetting events, AUM), and noisy cross-validation.

An in-depth exploration of sampling strategies including temperature, top-k, top-p, test time compute, and structured outputs, explaining how they affect the creativity, consistency, and reliability of AI model responses.

Qdrant 1.7.0 introduces native support for sparse vectors, enabling keyword-based search and hybrid search; a new Discovery API for more precise vector search, including discovery search and context search; user-defined sharding for flexible data distribution; snapshot-based shard transfer for efficient cluster scaling; and various performance improvements.

A comprehensive survey of adversarial attacks on large language models, covering threat models, attack types including token manipulation, gradient-based attacks, jailbreak prompting, and red-teaming techniques. The article discusses the challenges and methods for both black-box and white-box settings.

This comprehensive article explores multimodal AI systems, particularly Large Multimodal Models (LMMs). It covers the rationale for multimodality, data modalities, multimodal tasks, and dives into the architectures and training of CLIP and Flamingo models. The post also discusses active research directions such as generating multimodal outputs, instruction-following, and efficient adapters.

This article summarizes ten major research directions in large language models, covering hallucinations, context learning, multimodality, speed and cost, new architectures, GPU alternatives, agents, human preference learning, chat interface efficiency, and non-English language models. Based on discussions with industry and academia, the author analyzes the current state and challenges of each direction.

Since ChatGPT's launch in late 2022, generative AI has sparked massive interest and hype. Organizations are racing to adopt GenAI, but there's a gap between desire and readiness. A recent IDC survey shows nearly 80% of executives are confident their companies will use GenAI, yet only 30% are well prepared. The article explores build vs. buy decisions, use cases (productivity, business functions, industry-specific), and key prerequisites like governance, privacy, and data architecture.

The digital skills gap persists, with AI/ML shortages particularly acute. A SAS survey found 63% of decision-makers cite AI/ML as the biggest skills gap. O'Reilly's report shows surging interest in NLP and deep learning, while data engineering and cloud skills remain in high demand. Organizations prioritize training and upskilling to address the gap.

Databricks acquires MosaicML for $1.3 billion to boost generative AI capabilities. MosaicML offers open-source models like MPT-7B and MPT-30B. The deal aims to democratize AI and reduce training costs.

This article explores autonomous agents powered by large language models (LLMs) as their core controller. The system comprises three main components: planning (task decomposition and self-reflection), memory (short-term via in-context learning, long-term via external vector stores), and tool use (calling external APIs). It covers case studies like ChemCrow and Generative Agents, proof-of-concepts such as AutoGPT, GPT-Engineer, and BabyAGI, and discusses challenges like finite context windows.

The article explores the impact of AI on application developers. While AI tools like ChatGPT and GitHub Copilot can boost productivity, they lack creativity and can hallucinate, so human developers remain essential. A survey shows only 33% of organizations use such tools, but actual usage is likely higher. Training is the biggest challenge, not ease of use.

European lawmakers overwhelmingly approved the draft EU AI Act, which categorizes AI by risk levels, requiring impact assessments for high-risk systems and banning unacceptable risk uses. The act may become a global standard but faces criticism for lack of flexibility.

Cohere, the provider of an AI development platform, announced it has raised $270 million in a Series C round led by Inovia Capital, with participation from Nvidia, Oracle, Salesforce Ventures, and others. The funding will accelerate its enterprise AI platform and customization offerings.

Weights & Biases unveiled two new products at its inaugural in-person conference: W&B Weave, an application building toolkit, and W&B Production Monitoring, an observability tool for ML models.

Chip Huyen's talk at Fully Connected provides a simple framework for teams to navigate generative AI strategy, born from conversations with friends struggling to define their approach. She plans to expand the talk into a full article while welcoming community feedback.

An Altair survey reveals high adoption of data and AI strategies, but projects stall due to organizational, technical, and financial friction. Talent shortages, low AI literacy, legacy systems, and upfront costs are key barriers. Despite high failure rates, organizations continue AI investments for long-term gains.

Qdrant announces $7.5M seed funding led by Unusual Ventures. The article discusses the importance of vector databases in the AI age, the explosive growth of unstructured data, and Qdrant's progress as an open-source vector similarity search solution.

This article provides a comprehensive overview of prompt engineering methods for large language models, covering basic prompting, instruction prompting, self-consistency sampling, chain-of-thought prompting, automatic prompt design, and augmented language models.

This article is a major update to Lilian Weng's 2020 post on the Transformer family, doubling its length. It systematically reviews numerous recent improvements to the Transformer architecture, covering attention mechanisms, positional encoding, long-context support, adaptive modeling, and efficient attention, including the latest advances such as Transformer-XL, Rotary position embedding, ALiBi, and the Universal Transformer.

A comprehensive overview of techniques to optimize inference for large transformer models, including distillation, quantization, pruning, sparsity, mixture-of-experts, and architectural improvements. The article discusses challenges such as memory footprint and low parallelizability, and presents methods to reduce memory usage, computation, and latency.

Microsoft CTO Kevin Scott discusses the rapid advancements in AI, including large models like GitHub Copilot and DALL·E 2, their impact on productivity and creativity, the role of AI in scientific breakthroughs, and Microsoft's commitment to responsible AI.

Microsoft integrates DALL·E 2 into Azure OpenAI Service, enabling brands like Mattel and RTL Deutschland to generate custom images for design and personalization. Other AI tools like Microsoft Syntex and Power Automate automate workflows and content management, boosting productivity.

Microsoft is open-sourcing Project FarmVibes, including FarmVibes.AI, to help farmers leverage data and AI for better yields and lower costs. The toolkit runs on Azure and provides algorithms for fertilizer optimization, weed detection, microclimate forecasting, and carbon sequestration estimation.

The application of data and AI is set to revolutionize contact centres in financial services, improving customer experience, reducing costs, and enhancing compliance.

MAUI63 uses AI-powered drones to study the critically endangered Māui dolphins, with only 54 remaining. The technology helps identify individuals and understand their behavior, aiding conservation efforts.

14-year-old Eithne from the UK was struggling with math after COVID disruptions. The online tutoring service Eedi used AI developed by Microsoft to dynamically quiz students, identify misconceptions, and assign personalized learning paths. The platform resolves 95% of student misconceptions and boosts confidence. Next-generation causal ML models are being developed for even more personalized recommendations.

SI-com and its parent company ISCEC Japan have developed AI-Mimi, a hybrid captioning system combining human input with Microsoft Azure Cognitive Services, to provide cost-effective real-time captions for local TV stations in Japan, serving Deaf and Hard of Hearing communities. The system was tested successfully and received positive feedback.