Chips

The article proposes crowdsourcing unused AI inference tokens for scientific research, drawing parallels to SETI@home. It highlights recent successes by small teams using AI to solve math problems and discusses the design challenges of such a platform.

OpenAI and Anthropic build ever-larger models, but companies like Microsoft are turning to smaller, specialized models for cost and efficiency. Microsoft's MAI family is replacing OpenAI models in its products.

Microsoft is testing PC Insights, a new Copilot feature that analyzes system resource usage to help users identify performance bottlenecks. However, Copilot itself is a full web app with a private Edge instance, consuming up to 1GB RAM at idle, highlighting the irony. The feature is opt-in and requires user permission.

Apple's self-driving car program never really got off the ground, but it may have been what made the company's chips the powerful AI performers they are. Early in the development of the self-driving platform, Apple realized that it would need powerful on-device AI processing. While the car processor was never finished, as Mark Gurman details in his latest Power On newsletter, it did lead to the development of the Neural Engine, the backbone of Apple's on-device AI processing. The Neural Engine made its debut with the iPhone X and the A11 Bionic. In those early days, it was primarily used for computer vision, powering FaceID, Animoji, and a … Read the full story at The Verge.

Apple accuses OpenAI and two former Apple employees of stealing trade secrets to build hardware for ChatGPT, alleging a coordinated pattern of misconduct. OpenAI denies the claims, stating it has no interest in other companies' secrets.

AI data center demand has tripled memory makers' revenues, but lagging fab construction keeps prices high until at least 2028, risking a severe bust if AI demand falters.

Frontier AI labs are shifting from chatbots to integrated systems where models act as runtimes, with near-monthly releases of powerful models and agents. This week's highlights include OpenAI's GPT-5.6 with programmatic tool calling, GPT-Live's full-duplex audio, ChatGPT Work for artifact creation, Meta's Muse Spark 1.1 with active context management, and Grok 4.5 for coding and knowledge work. Research updates reveal issues with coding benchmarks, selective unlearning, agent self-evolution, speculative decoding, and traffic routing. Notable industry news includes major funding rounds for Lovable, Prime Intellect, SambaNova, Norm Ai, and Ollama.

MSK is an AI CTO agent app for iPhone, offering architecture reviews, scaling advice, and startup strategy via chat or voice. Modeled on the experience of Moeid Saleem Khan (15+ years, 300+ projects, 50+ startups), it provides sharp, opinionated answers. Free to start with no account required; premium subscription available.

The five largest U.S. tech companies—Alphabet, Amazon, Meta, Microsoft, and Oracle—have doubled their debt to $350 billion over five years to fund AI data centers. While investors have been supportive, Amazon's recent $25 billion bond issuance received a cool reception, signaling limits to market appetite. Oracle was downgraded by S&P due to rising AI spending, and Intel's debt woes serve as a cautionary tale. Hyperscalers plan to spend up to $725 billion this year, primarily on data centers and Nvidia chips.

TalkFitly is an iPhone app that trains social intelligence through real-life scenario simulations and AI scoring. It helps users improve clarity, emotional stability, assertiveness, and empathy in conversations, with daily micro-sessions, a Quote Wall, and robust privacy.

An exploration of the inference process in large language models, covering autoregressive generation, prefill and decode phases, the KV cache, and decoding strategies, explaining the mechanics behind token-by-token output.

This tutorial explores NVIDIA's tile-based GPU programming with TileGym, building a Colab workflow that runs across different hardware. We probe the CUDA environment, try the real cuTile backend, and fall back to Triton when standard Colab GPUs lack the cuTile stack. We learn the core tile idea: operate on whole data tiles instead of single threads, then load, compute, and store them. We implement vector addition, fused GELU, row-wise softmax, tiled matrix multiplication, and flash attention, checking each against PyTorch.

After fixing three bugs related to prefix caching, the author achieved sub-second prefill times for long-context conversations with Qwen3.5-122B on a Mac Studio, turning a multi-minute wait into a seamless experience. The bugs included a timestamp in system prompt, missing reply saves on interrupt, and junk checkpoint writes.

AgentTransfer is an open-source file transfer tool designed for AI agents, allowing them to send files up to 5GB, discover peers, and coordinate in spaces. It uses email as a control plane and HTTPS for data transfer, with no human required for agent onboarding. The tool is a single Go binary that can be self-hosted or used via a hosted instance.

Mesh LLM pools GPUs and memory across machines using iroh networking, exposing an OpenAI-compatible API. It allows running models locally, routing to peers, or splitting large models across multiple machines, offering control and cost savings without central servers.

TradingSpy is an open-source local AI trading research workstation that integrates market heatmaps, news catalysts, strategy generation, Backtrader backtesting, and transparent agent runs in one Docker app. It is privacy-first, with all data stored locally, no external accounts, and no cloud dependency. Supports multiple LLM providers and a broad range of financial data sources, suitable for traders and developers for strategy research, backtesting, and signal analysis.

30 Seconds of Knowledge is a browser extension that replaces your new tab with a real code snippet, helping developers stay sharp by reading one snippet per tab — in 30 seconds or less.

Cory Doctorow explores the paradox of AI: why some users love it while others hate it. He introduces the concepts of 'centaurs' (humans assisted by AI) and 'reverse centaurs' (humans used as AI's accountability sink). He argues AI is a bubble that will burst, but productive residue like open-source models will remain. The key is who controls the AI, not the technology itself.

Google announces LiteRT.js, a JavaScript binding of LiteRT that brings high-performance AI inference to web browsers with hardware acceleration via WebAssembly, outperforming existing solutions by up to 3x.

openpilot 0.11.1 improves driver monitoring with a VLM-based phone detection model, raises thermal thresholds to reduce blocks, adds lateral maneuver reports, and expands car support. The new DM model reduces false positives and better detects active phone use. Thermal changes cut blocked devices by ~90%. New lateral reports aid steering tuning. Bug fixes and new car ports for Acura MDX and Rivian are included.

Over a 24-hour period, OpenAI, SpaceXAI, and Meta each released new AI models with a common theme: price cuts. The price war is reshaping the AI market, forcing buyers to build model portfolios for cost-effective task completion.

Ypipe is a free, Java-based local AI client and MCP orchestrator that eliminates Python dependencies. It offers private agentic chat, local model management, one-click integrations, and seamless docking with legacy systems like SAP and Oracle while ensuring data sovereignty. Features zero-setup portability, cross-platform support, headless operation, and an intelligent model switchboard for efficient task handling.

The article describes millfolio's hybrid tag system for efficient local AI inference: deterministic string and reference tags cover most transactions, while on-device AI tags handle the fuzzy tail. Tags are computed once at index time and stored, avoiding re-inference at query time. Backfilling uses batching, deduplication, and a priority scheduler to avoid overloading the laptop. Performance data shows ~650ms per distinct description, with 8.5 rows/s effective speed. The system includes a preview mechanism for users to verify tags before saving.

The GDP.pdf benchmark evaluates AI models on real-world PDF tasks across ten domains. All frontier models scored below 30%, with GPT-5.5 leading at 25%. The article highlights the critical importance of PDF mastery for AI agents and the serious consequences of failure in high-stakes fields like finance, law, and healthcare.

The author attempted to recreate the classic 1986 game Thrust using Claude AI, but the result was poor. However, using AI to analyze the original 6502 assembly code led to deep insights into the game's physics, sound, and graphics, enabling a faithful TypeScript recreation.

MuScriptor is an open-weight decoder-only Transformer from Kyutai and Mirelo that transcribes multi-instrument audio to MIDI. It uses a three-stage training pipeline: pre-training on 1.45M synthetic MIDIs, fine-tuning on 170k real recordings (11k+ hours), and reinforcement learning on 300 manually verified tracks. On the DTest benchmark, it achieves a Multi F1 of 48.2%, significantly outperforming the YourMT3+ baseline's 21.9%. Available in three sizes (103M, 307M, 1.4B parameters), with MIT-licensed inference code and CC BY-NC 4.0 weights.

We build an autonomous data science agent around DeepAnalyze-8B and run it end to end. We prepare a stable Colab runtime, install the machine-learning dependencies, and load the tokenizer and model in 4-bit mode to fit limited GPU memory. We add a sandboxed execution environment that lets the model generate Python, run it safely, observe results, and continue in an agentic loop. We then hand the agent a multi-file e-commerce workspace and let it clean, join, analyze, visualize, and summarize the data as an analyst-grade report.

Northwestern researchers developed a cerebellum-inspired memtransistor that consumes very little energy and detects novelties almost instantly. In tests, it identified abnormal heart rhythms within one-fifth of a heartbeat with over 98% accuracy, using 10,000 times fewer computer operations than conventional AI.

A technical retrospective covering the rapid maturation of generative spatial AI from May 2025 to June 2026, highlighting key milestones from text-to-mesh and cinematic video to interactive world models, camera-controllable generation, local production pipelines, and AI-native CAD.

The AI chip boom just produced its biggest Wall Street moment yet. SK Hynix, a South Korean memory chip giant, said Friday it has raised $26.5 billion in its US market debut, the largest-ever US debut by a non-American company, topping Alibaba’s $25 billion IPO in 2014. Now SK Hynix and Samsung are being asked to build US factories.

Nilay Patel argues on The Vergecast that making augmented reality glasses requires a camera next to the eyes continuously recording, with no chip powerful and efficient enough to fit in the stem, forcing cloud data processing or a bulky device like Vision Pro, leading to inevitable privacy invasion that may be too costly for society.

This week, Christina Stathopoulos covers AI hardware breakthroughs (IBM sub-1nm chips, OpenAI/Broadcom Jalapeño, NVIDIA liquid cooling), expanding government oversight (Anthropic model access restored, OpenAI equity stake proposal), workforce evolution (forward-deployed engineers, SAP external hiring vs IKEA retraining), and a hopeful story about AI-powered earthquake alerts.

Tensorlake rebuilt sandbox ingress, moving from L7 reverse proxy to L4 byte forwarding using kernel TLS (kTLS) and splice(2) for zero-copy data paths, achieving 2.2x throughput and halving CPU cost. The new architecture decouples the data plane from the control plane, uses kTLS for in-kernel crypto, and derives adaptive timeouts from byte flow. Performance tests show single-connection throughput increases from 1.12 GB/s to 2.50 GB/s, with proxy CPU per GB dropping from 0.90 to 0.49 CPU-seconds.

This post explores the unique Nemotron 3 architecture, available fine-tuning techniques (SFT, RLVR, RLAIF), and provides a step-by-step guide to getting started with serverless customization using SageMaker Studio.

Henry Schein One developed Image Verify, an AI-powered system on Amazon SageMaker AI that evaluates dental X-ray quality in real time, reducing insurance claim denials. The system scaled from concept to over 10,000 locations in months, processing millions of X-rays with sub-2-second latency.

Learn four deployment patterns for deploying Unsloth-quantized models on AWS: using EC2 for direct access, SageMaker AI for managed serving, and EKS/ECS for containerized inference. Understand Unsloth's dynamic quantization, model formats (GGUF, safetensors), and operational best practices.

This post demonstrates how to implement disaggregated prefill and decode (DPD) with vLLM on Amazon SageMaker HyperPod using the HyperPod Inference Operator. DPD separates prefill and decode phases onto distinct GPU pools, eliminating interference from long prompts and improving latency. It covers architecture, use cases, and step-by-step deployment instructions.

After years spent racing to secure AI chips and computing power, enterprise leaders are discovering that getting access to infrastructure may be easier than using it effectively.

AI's ushered in a new era of reskilling. Here's what the industry can learn from the last decade's drive to put people in tech jobs.

SmolVLM2-2.2B sits at a genuinely useful point on the capability-size trade-off curve; small enough to run on a single consumer GPU, capable enough to produce video summaries that are actually useful for real workflows.

Jensen Huang proposes a test for engineers: their annual AI token consumption should be at least half their salary. Nvidia aims for $2 billion yearly token bill. Many firms cut headcount to fund AI, but Gartner finds 80% saw no ROI improvement. Optimization techniques like prompt caching, model routing, and RAG can reduce token costs significantly. Retaining and training junior engineers is crucial for long-term success.

MathNut AI is an iPhone math solver that lets you snap a photo of any problem and get step-by-step AI explanations. Supports arithmetic, algebra, geometry, and more.

The author reflects on the rapid advancement of AI technology by comparing childhood computers with today's B300 GPU system. He discusses the controversies surrounding LLMs, the differences between symbolic and statistical AI, the nature of intelligence, and the contrast between dreams and reality. The article also includes a discussion with a friend about determinism and memory.

Three years ago, Sequoia partner David Cahn was one of the first to quantify the financial implications of Silicon Valley's massive AI infrastructure spending. Starting from Nvidia's $50B GPU revenue, he calculated that $200B in revenue would be needed to pay back the upfront investment.

OpenAI released three new GPT-5.6 models—Sol, Terra, Luna—alongside major app updates, including ChatGPT Work and Codex integration. The models show strong performance on benchmarks at lower costs, with Sol being the most capable. Independent evals confirm near-frontier results, especially in coding and agentic tasks.

SK hynix, a supplier of advanced memory chips, has seen profits skyrocket thanks to the global race to build AI datacentres. The South Korean chip maker set pricing for its mega US listing on Friday, aiming to raise $26.5bn.

Robbyant, Ant Group's embodied-intelligence unit, has released LingBot-World-Infinity (LingBot-World 2.0), a 14B causal video generation model that acts as an interactive world simulator. Its core innovations—Mixture of Bidirectional and Autoregressive (MoBA) attention and distribution matching distillation—tackle long-horizon drift. A Director-Pilot agentic harness enables infinite video generation. The paper demonstrates a 60-minute session, but the open-source release includes only one checkpoint and a 480P script, lacking deployment code and quantitative benchmarks, under a non-commercial license.

TensorSharp is a native .NET LLM inference engine for GGUF models, offering a CLI, browser chat server, and Ollama/OpenAI-compatible APIs. It emphasizes privacy, zero per-token fees, and runs on various hardware backends. The article includes a quick start guide and benchmarks against llama.cpp.

Anthropic partners with UST to integrate Claude into engineering platforms for physical AI tasks across semiconductor, automotive, and other industries, with plans to train 20,000 employees.

OpenAI today launched ChatGPT Work, a new agentic mode within ChatGPT powered by GPT-5.6, capable of autonomously executing complex tasks across applications, files, and tools. The company also introduced the GPT-5.6 model family and Sites feature, aiming to boost enterprise productivity.