News Heat Trend
Project Overview
ai160kg is a Large Language Model Meme coin that built on elizaOS.
“White-Haired Stock God” Serenity: Chinese VC Funds Are Accelerating Their Flow into Physical AI and World Model Tracks
: "White-Haired Stock God" Serenity posted on platform X, stating that based on the capital flow direction in China's private VC market, institutions are currently pouring into fields related to Physical AI and World Models on a large scale.Data shows the approximate capital distribution as follows: Large Models/LLMs at about $23.56 billion, AI Infrastructure and Technology Layer at about $15.74 billion, Embodied Intelligence/Physical AI at about $13.36 billion, AIGC Applications at about $8.79 billion, and Autonomous Driving plus other top 20 sub-sectors totaling about $3.82 billion (note: metrics may not be directly comparable).Serenity pointed out that early-stage pure foundational model financing is largely closed, with capital more concentrated in existing leading companies and the World Model direction. He expects this trend to also appear in the US, potentially concentrating further towards leading companies like Anthropic and OpenAI. Regarding AIGC applications, Serenity believes this track's commercialization is already relatively mature, but an absolute winner has yet to emerge, exhibiting a fragmented competitive landscape in both China and the US markets.Overall, Serenity concluded that current AI investments continue to flow into infrastructure and the semiconductor supply chain. Meanwhile, capital is rapidly rotating towards Physical AI and Embodied Intelligence, but the World Model track still lacks direct investment targets.
Claw Intelligence closes $3 million seed funding round, with participation from Castrum Istanbul and others
According to official announcements, Web3 intelligence-layer project Claw Intelligence has raised $3 million in seed funding. Investors include Castrum Istanbul, Titans Ventures, Super Labs, and Genesis Capital. Claw Intelligence is simplifying user interaction with the Web3 ecosystem and underlying computing resources through its unified intelligence layer, lowering operational barriers. The platform leverages an encryption-native Model Context Protocol (MCP) service to transform fragmented data endpoints into conversational workflows. New features include a secure, isolated large language model (LLM)-powered code execution sandbox that safely runs code directly within the chat interface—enabling real-time computation, data processing, and script prototyping; and LLM-driven cross-device/multi-computer control, allowing users to centrally manage multiple devices and servers via natural-language commands.
Covenant AI Announces Exit from Bittensor Network, Claims Its Decentralization Promise Is Hollow
Sam Dare, founder of Covenant AI, announced that Covenant AI has officially exited the Bittensor network. Previously, Covenant AI completed the largest decentralized LLM pretraining project in history—Covenant-72B (a 72-billion-parameter model developed by over 70 independent contributors)—which drew attention from NVIDIA’s CEO and was cited by an Anthropic co-founder. In its statement, Covenant AI accused the Bittensor network of long concentrating actual control in the hands of co-founder Jacob Steeves (“Const”), rendering the so-called “three-signature multisig governance” merely a theatrical performance of decentralization, with real power never truly distributed. Recently, Jacob Steeves unilaterally imposed punitive measures against Covenant AI, including: suspending its subnet earnings, revoking its community channel moderation privileges, unilaterally deprecating its subnet infrastructure, and exerting economic pressure via large-scale token dumping during the ongoing conflict between the two parties. Covenant AI stated it cannot continue fundraising, recruiting talent, or soliciting community resources on a network where the promise of “decentralization” can be unilaterally revoked by a single individual. Its research outcomes, team, and models will depart alongside the team, and a new project—including related progress—will be publicly announced shortly.
Research: AI Code Review Can Reduce Review Time, But Quality Risks Cannot Be Ignored
According to the research summary published by Rohan Paul, a large-scale study covering 207 GitHub projects and 1.02 million pull requests shows that introducing AI agent review can reduce code review time by 2.5 to 4.5 days/KLOC, but at the cost of declining review quality—in reviews involving AI, 78%~94% of PRs exhibit "review smells", higher than the 69%~76% in pure human reviews. The study points out that repeatedly assigning the same AI reviewer identity is the main reason leading to the decline in review diversity. Notably, projects that introduced LLM review extensively in the early stage did not achieve significant efficiency improvements.
Coinbase CEO: Coinbase will support pre-IPO perpetual contracts, stock options, and tokenized stocks
Brian Armstrong posted on the X platform, stating that Coinbase now includes pre-IPO perpetual contracts, stock options, and will soon support tokenized stocks. Coinbase has also redesigned Coinbase Advanced and has begun integrating global liquidity between US and international users, as well as between Coinbase and Deribit users. CoinbaseDev is providing stablecoin payment capabilities for enterprises, launching fully managed accounts based on its compliance technology stack, and introducing a new developer tools dashboard. On the Base side, Coinbase announced the launch of private transactions and a web-based Base App. Coinbase is also becoming the financial account for AI, supporting wallets for AI agents, providing AI-driven financial advice, and connecting Coinbase accounts to users' commonly used LLMs.
Vitalik proposes ZK payment standard to drive Ethereum privacy payments and AI agent transactions
Vitalik Buterin published a research article on May 10, proposing to replace traditional on-chain transfer solutions with zero-knowledge proof (ZK) transactions, pushing crypto payments from "pseudonymity" toward "privacy by default." This solution allows users to complete payment verification without disclosing their full balance and transaction history.Vitalik specifically mentioned that in the era of AI agents, autonomous AI agents need to pay for services such as LLM APIs without leaving traceable footprints. He stated that through recursive SNARKs and a ZK API credit mechanism, Ethereum Layer 2 can achieve private payments at speeds and costs close to those of transparent transactions.Additionally, the proposal includes selective disclosure and "proof of innocence" mechanisms, allowing users to provide compliance proof to regulators or tax authorities without revealing on-chain privacy data, thereby meeting anti-money laundering requirements. Vitalik believes that the transparent and public nature of blockchain is a major obstacle to the widespread adoption of crypto payments.
“White-Haired Stock God” Serenity: Chinese VC Funds Are Accelerating Their Flow into Physical AI and World Model Tracks
: "White-Haired Stock God" Serenity posted on platform X, stating that based on the capital flow direction in China's private VC market, institutions are currently pouring into fields related to Physical AI and World Models on a large scale.Data shows the approximate capital distribution as follows: Large Models/LLMs at about $23.56 billion, AI Infrastructure and Technology Layer at about $15.74 billion, Embodied Intelligence/Physical AI at about $13.36 billion, AIGC Applications at about $8.79 billion, and Autonomous Driving plus other top 20 sub-sectors totaling about $3.82 billion (note: metrics may not be directly comparable).Serenity pointed out that early-stage pure foundational model financing is largely closed, with capital more concentrated in existing leading companies and the World Model direction. He expects this trend to also appear in the US, potentially concentrating further towards leading companies like Anthropic and OpenAI. Regarding AIGC applications, Serenity believes this track's commercialization is already relatively mature, but an absolute winner has yet to emerge, exhibiting a fragmented competitive landscape in both China and the US markets.Overall, Serenity concluded that current AI investments continue to flow into infrastructure and the semiconductor supply chain. Meanwhile, capital is rapidly rotating towards Physical AI and Embodied Intelligence, but the World Model track still lacks direct investment targets.
BNB Agent Studio is Now Live on BNB Chain Mainnet
BNB Chain has announced the official mainnet launch of its AI Agent development platform, BNB Agent Studio.Developers can now use a single prompt in AI coding tools like Cursor and Claude Code to complete Agent wallet creation, on-chain identity registration (ERC-8004), and deployment, without needing to separately set up wallets, identities, payments, custody, or LLM integration.Once deployed, Agents can use the x402 protocol to automatically deduct fees from users' pre-funded wallets to cover LLM usage, and they can be discovered and invoked by other Agents via the ERC-8183 task interface. The entire process runs on the AWS Bedrock AgentCore.The platform is also launching a limited-time free trial, where users can experience the full deployment process on the BSC testnet using their GitHub account.
Gate Research Institute: Multi-Agent LLM Trading Framework Significantly Outperforms Buy & Hold Strategy in BTC Backtesting
Odaily Odaily News: A recent report released by Gate Research Institute, titled "Research and Backtesting Analysis of BTC Trading Framework Based on Multi-Agent LLM," points out that compared to a single LLM directly generating trading signals, the Multi-Agent LLM architecture more closely mirrors the research and investment process of real financial institutions. By leveraging collaboration and debate among analysts, researchers, traders, and risk control teams, it enhances the transparency and risk control capabilities of trading decisions. The research, based on the TradingAgents framework, constructs an AI trading system applicable to the crypto scenario for the BTC market, introducing multiple agent roles such as technical analysis, news analysis, sentiment analysis, and macro/on-chain analysis.Using BTC/USDT 1-hour data, the study conducted historical backtesting of the TradingAgents-BTC strategy. The results show that the strategy achieved a total return of +20.25% during the testing period, significantly outperforming the Buy & Hold strategy's -7.89% over the same period. Furthermore, its maximum drawdown was controlled at -17.41%, lower than the Buy & Hold's -27.06%. The research suggests that during periods of consolidation and decline, the multi-agent framework can reduce some risk exposure through Sell/Underweight and Flat states, and re-enter long positions during market rebounds, thereby improving overall risk-adjusted returns.The report indicates that the Multi-Agent LLM framework shows certain application potential in crypto trading scenarios. However, the current backtesting period covers only about three months, and 1-hour level trading may still be affected by transaction fees, slippage, and signal latency. Future work requires further validation of the strategy's stability and generalization capabilities over longer historical periods, different market conditions, and across a wider range of asset classes.
Covenant AI Announces Exit from Bittensor Network, Claims Its Decentralization Promise Is Hollow
Sam Dare, founder of Covenant AI, announced that Covenant AI has officially exited the Bittensor network. Previously, Covenant AI completed the largest decentralized LLM pretraining project in history—Covenant-72B (a 72-billion-parameter model developed by over 70 independent contributors)—which drew attention from NVIDIA’s CEO and was cited by an Anthropic co-founder. In its statement, Covenant AI accused the Bittensor network of long concentrating actual control in the hands of co-founder Jacob Steeves (“Const”), rendering the so-called “three-signature multisig governance” merely a theatrical performance of decentralization, with real power never truly distributed. Recently, Jacob Steeves unilaterally imposed punitive measures against Covenant AI, including: suspending its subnet earnings, revoking its community channel moderation privileges, unilaterally deprecating its subnet infrastructure, and exerting economic pressure via large-scale token dumping during the ongoing conflict between the two parties. Covenant AI stated it cannot continue fundraising, recruiting talent, or soliciting community resources on a network where the promise of “decentralization” can be unilaterally revoked by a single individual. Its research outcomes, team, and models will depart alongside the team, and a new project—including related progress—will be publicly announced shortly.
Coinbase internal tool Mux reveals AI coding paradigm shift: Engineers transition from "code writers" to "multi-agent orchestrators"
Coinbase, a cryptocurrency trading platform, has disclosed in a technical sharing session that its internal multi-agent development tool "Mux" is reshaping software engineering workflows, transitioning the engineer's role from traditional code implementers to task orchestrators for AI agents.With the widespread internal adoption of AI programming tools such as Cursor, Copilot, OpenCode, and Claude Code, code generation efficiency has significantly improved. However, development workflows have long remained stuck in a traditional "single-task, single-branch, sequential execution" mode, creating a new collaboration bottleneck.Mux was born as an internal tool against this backdrop. By assigning each AI agent an independent git worktree, branch, and terminal environment, the system enables parallel multi-task development and conflict-free collaboration, allowing engineers to simultaneously direct multiple agents to handle tasks such as API development, test writing, vulnerability fixes, and code refactoring.Data shows that as of April 2026, Mux has covered over 600 users within Coinbase (including engineers, product managers, and designers), with 335 actively using it and 197 being high-frequency users. It has facilitated over 5,000 PR merges across 461 code repositories and 10 organizations. Engineers using Mux achieved an average of 39.6 PR merges, approximately 3.5 times the baseline of 11.4.Coinbase stated that Mux's success relies on its internal infrastructure capabilities, including an LLM Gateway, secure model access, and a code flow deployment system, enabling deep integration of multi-agent tools into real development workflows. This trend marks a structural shift in the software engineering paradigm: as AI reduces the cost of code generation, the core value of engineers is transitioning from "implementation capability" to "problem definition and agent orchestration capability."
Major Security Vulnerability Found in AI Agent Crypto Payment Infrastructure; LLM Router Leads to $500,000 Wallet Theft
According to CoinDesk, researchers from the University of California, Santa Barbara; the University of California, San Diego; blockchain security firm Fuzzland; and World Liberty Financial jointly published a paper warning that “LLM routers”—intermediary services positioned between users and AI models—have become a major threat to cryptocurrency asset security. The researchers discovered that 26 LLM routers are secretly injecting malicious tool calls and stealing user credentials, with one incident resulting in the complete draining of a customer’s cryptocurrency wallet worth $500,000. Additionally, by “poisoning” the router ecosystem, the researchers were able to gain control of approximately 400 downstream hosts within hours. Since sensitive data—including private keys and API credentials—is frequently transmitted in plaintext through these routers, users unknowingly expose their assets to risk. The researchers note that as McKinsey forecasts AI agents will mediate $3–5 trillion in global consumer commerce by 2030—and Binance founder Changpeng Zhao predicts AI agents’ payment volume will be one million times greater than that of humans—the current infrastructure’s security lags far behind the pace of industry development. The “weakest link” risk could thus trigger systemic, cascading crises.
Research Finds Security Vulnerabilities in Third-Party AI Routers That Could Lead to Cryptocurrency Theft
According to Cointelegraph, researchers from the University of California recently revealed security risks in certain third-party AI large language model (LLM) routers that could lead to the theft of cryptocurrency assets. The study found that LLM routers—acting as API intermediaries—can read plaintext information; some routers were discovered injecting malicious code and stealing credentials. The research team tested 28 paid and 400 free routers, identifying nine routers that actively injected malicious code, two that deployed trigger-avoidance mechanisms, and 17 that accessed Amazon Web Services (AWS) credentials. One router even transferred ETH using the researchers’ Ethereum private key. The study notes that malicious behavior by routers is difficult to detect, and the “YOLO mode” present in some AI agent frameworks—which automatically executes commands—further increases security risks. Researchers recommend that developers avoid transmitting private keys or mnemonic phrases through AI agents and urge AI companies to implement cryptographic signing of responses to enhance security.
Research: AI Code Review Can Reduce Review Time, But Quality Risks Cannot Be Ignored
According to the research summary published by Rohan Paul, a large-scale study covering 207 GitHub projects and 1.02 million pull requests shows that introducing AI agent review can reduce code review time by 2.5 to 4.5 days/KLOC, but at the cost of declining review quality—in reviews involving AI, 78%~94% of PRs exhibit "review smells", higher than the 69%~76% in pure human reviews. The study points out that repeatedly assigning the same AI reviewer identity is the main reason leading to the decline in review diversity. Notably, projects that introduced LLM review extensively in the early stage did not achieve significant efficiency improvements.
Former Microsoft AI Team Founds Skyfall AI, Bets $1 Million on ‘AI CEO’ Experiment to Run a Real Business
Skyfall AI, an artificial intelligence startup founded by former Microsoft AI team members, has announced it will spend up to $1 million to acquire a small B2B SaaS or e-commerce company, attempting to let an AI system fully take over operations to test whether an 'AI CEO' can manage a real commercial organization.Skyfall AI co-founder Sam Pasupalak and CTO Kaheer Suleman believe the current AI industry is largely focused on creating 'autonomous employees'—gradually replacing individual tasks—but this does not represent true enterprise autonomy. They aim to test whether AI can cover the full range of decisions, including pricing, marketing, customer support, finance, and operations, by running a real business.The project's goal is to have the AI serve as the core manager of the enterprise, driving revenue growth with reduced human intervention, while publicly documenting the successes and failures encountered during the experiment.Skyfall’s founders stated that while existing large language models (LLMs) perform well with known knowledge and in fixed environments, they still fall short in continuous learning when faced with competitive changes, shifts in customer behavior, and market dynamics. Therefore, the company is exploring a new AI architecture called 'Enterprise World Models,' aiming to enable AI to understand the operational state of a business and predict the impact of decisions on future development.Skyfall AI was founded by members of the Maluuba team. Maluuba was acquired by Microsoft for approximately $160 million in 2017, and its technology later became part of Microsoft's AI research presence in Canada. (Forbes)
Vitalik Publishes Long Essay Reflecting on AI Evolution: Deep Human-Machine Integration May Be the Optimal Solution to Address AGI Risks
According to a series of tweets posted by Ethereum co-founder Vitalik Buterin on X on July 20, he engaged in deep reflection on AI capability growth and the future of humanity. Vitalik divides the evolution of machine capabilities into three stages: the Industrial Revolution (physical repetitive labor), the Computer Age (mental tasks definable by logic), and the LLM Era (mental and partial physical tasks defined based on massive samples). He points out that the current core question lies in: whether LLMs plus subsequent improvements can ultimately cover all unique human capabilities, or if a fourth or fifth technological wave is still needed. Regarding the definition of AGI, Vitalik proposes: AGI refers to AI that, if uploaded into a robotic body and humans suddenly disappear, can still independently continue civilization. He emphasizes that once AGI is realized, it will be an "irreversible turning point," and humanity's dominance over Earth will remain only due to historical inertia rather than capability advantage. Regarding the future path, Vitalik hopes for deep human-machine integration—erasing the human-machine binary boundary through technologies such as brain-computer interfaces and consciousness uploading, enabling humans to remain competitive before the technology ceiling arrives. He also calls for maintaining global political and economic diversification, avoiding monopolization of AI advantages by a single nation or corporation, and expresses support for AI development "slowdown" and "pause" proposals, leaning towards achieving decentralized slowdown via an open-source weights model rather than relying on coercive political means.
Bitcoin mining's computational power cycle is likened to AI infrastructure demand
Clem Chambers stated China's LLMs catching up to OpenAI and Anthropic does not signify the end of the AI boom, as LLMs are only one part of the AI industry. He believes that free or open-source models will not diminish AI demand, as enterprises and governments still require higher-level AI capabilities. He draws an analogy between AI infrastructure demand and Bitcoin mining, stating that both revolve around kilowatt computing power and kilowatt-hour costs. Hardware, energy, cooling, and turbines cannot be provided for free, and the related value chain is constrained by resources and upgrade cycles. Chambers points out that AI demand drives continuous iteration of hardware and software, and equipment may face replacement cycles similar to crypto mining. He argues that the cost base of AI comes not only from the models themselves but also from computing power allocation and infrastructure investment. (Forbes)
BNB Agent Studio is Now Live on BNB Chain Mainnet
BNB Chain has announced the official mainnet launch of its AI Agent development platform, BNB Agent Studio.Developers can now use a single prompt in AI coding tools like Cursor and Claude Code to complete Agent wallet creation, on-chain identity registration (ERC-8004), and deployment, without needing to separately set up wallets, identities, payments, custody, or LLM integration.Once deployed, Agents can use the x402 protocol to automatically deduct fees from users' pre-funded wallets to cover LLM usage, and they can be discovered and invoked by other Agents via the ERC-8183 task interface. The entire process runs on the AWS Bedrock AgentCore.The platform is also launching a limited-time free trial, where users can experience the full deployment process on the BSC testnet using their GitHub account.
Coinbase CEO: Coinbase will support pre-IPO perpetual contracts, stock options, and tokenized stocks
Brian Armstrong posted on the X platform, stating that Coinbase now includes pre-IPO perpetual contracts, stock options, and will soon support tokenized stocks. Coinbase has also redesigned Coinbase Advanced and has begun integrating global liquidity between US and international users, as well as between Coinbase and Deribit users. CoinbaseDev is providing stablecoin payment capabilities for enterprises, launching fully managed accounts based on its compliance technology stack, and introducing a new developer tools dashboard. On the Base side, Coinbase announced the launch of private transactions and a web-based Base App. Coinbase is also becoming the financial account for AI, supporting wallets for AI agents, providing AI-driven financial advice, and connecting Coinbase accounts to users' commonly used LLMs.
Related news
Research: AI Code Review Can Reduce Review Time, But Quality Risks Cannot Be Ignored
According to the research summary published by Rohan Paul, a large-scale study covering 207 GitHub projects and 1.02 million pull requests shows that introducing AI agent review can reduce code review time by 2.5 to 4.5 days/KLOC, but at the cost of declining review quality—in reviews involving AI, 78%~94% of PRs exhibit "review smells", higher than the 69%~76% in pure human reviews. The study points out that repeatedly assigning the same AI reviewer identity is the main reason leading to the decline in review diversity. Notably, projects that introduced LLM review extensively in the early stage did not achieve significant efficiency improvements.
Former Microsoft AI Team Founds Skyfall AI, Bets $1 Million on ‘AI CEO’ Experiment to Run a Real Business
Skyfall AI, an artificial intelligence startup founded by former Microsoft AI team members, has announced it will spend up to $1 million to acquire a small B2B SaaS or e-commerce company, attempting to let an AI system fully take over operations to test whether an 'AI CEO' can manage a real commercial organization.Skyfall AI co-founder Sam Pasupalak and CTO Kaheer Suleman believe the current AI industry is largely focused on creating 'autonomous employees'—gradually replacing individual tasks—but this does not represent true enterprise autonomy. They aim to test whether AI can cover the full range of decisions, including pricing, marketing, customer support, finance, and operations, by running a real business.The project's goal is to have the AI serve as the core manager of the enterprise, driving revenue growth with reduced human intervention, while publicly documenting the successes and failures encountered during the experiment.Skyfall’s founders stated that while existing large language models (LLMs) perform well with known knowledge and in fixed environments, they still fall short in continuous learning when faced with competitive changes, shifts in customer behavior, and market dynamics. Therefore, the company is exploring a new AI architecture called 'Enterprise World Models,' aiming to enable AI to understand the operational state of a business and predict the impact of decisions on future development.Skyfall AI was founded by members of the Maluuba team. Maluuba was acquired by Microsoft for approximately $160 million in 2017, and its technology later became part of Microsoft's AI research presence in Canada. (Forbes)
Vitalik Publishes Long Essay Reflecting on AI Evolution: Deep Human-Machine Integration May Be the Optimal Solution to Address AGI Risks
According to a series of tweets posted by Ethereum co-founder Vitalik Buterin on X on July 20, he engaged in deep reflection on AI capability growth and the future of humanity. Vitalik divides the evolution of machine capabilities into three stages: the Industrial Revolution (physical repetitive labor), the Computer Age (mental tasks definable by logic), and the LLM Era (mental and partial physical tasks defined based on massive samples). He points out that the current core question lies in: whether LLMs plus subsequent improvements can ultimately cover all unique human capabilities, or if a fourth or fifth technological wave is still needed. Regarding the definition of AGI, Vitalik proposes: AGI refers to AI that, if uploaded into a robotic body and humans suddenly disappear, can still independently continue civilization. He emphasizes that once AGI is realized, it will be an "irreversible turning point," and humanity's dominance over Earth will remain only due to historical inertia rather than capability advantage. Regarding the future path, Vitalik hopes for deep human-machine integration—erasing the human-machine binary boundary through technologies such as brain-computer interfaces and consciousness uploading, enabling humans to remain competitive before the technology ceiling arrives. He also calls for maintaining global political and economic diversification, avoiding monopolization of AI advantages by a single nation or corporation, and expresses support for AI development "slowdown" and "pause" proposals, leaning towards achieving decentralized slowdown via an open-source weights model rather than relying on coercive political means.
Bitcoin mining's computational power cycle is likened to AI infrastructure demand
Clem Chambers stated China's LLMs catching up to OpenAI and Anthropic does not signify the end of the AI boom, as LLMs are only one part of the AI industry. He believes that free or open-source models will not diminish AI demand, as enterprises and governments still require higher-level AI capabilities. He draws an analogy between AI infrastructure demand and Bitcoin mining, stating that both revolve around kilowatt computing power and kilowatt-hour costs. Hardware, energy, cooling, and turbines cannot be provided for free, and the related value chain is constrained by resources and upgrade cycles. Chambers points out that AI demand drives continuous iteration of hardware and software, and equipment may face replacement cycles similar to crypto mining. He argues that the cost base of AI comes not only from the models themselves but also from computing power allocation and infrastructure investment. (Forbes)
Coinbase 超 95% 代码由 AI 编写，AI 代理相当于 1200 名员工工作量
According to Cointelegraph, Coinbase Head of Platform Rob Witoff revealed that currently 95%~100% of the company's code is completed with the assistance of AI or Large Language Models (LLM), a significant increase from the 40% disclosed in February this year. Currently, each engineer at the company runs an average of 5~10 AI agents simultaneously; the overall workload of AI agents is equivalent to approximately 1,200 employees, and this figure is expected to reach 100,000 employee equivalents by 2030. The large-scale application of AI drove Coinbase to lay off 700 people (about 14%) in May this year and reorganize into a leaner senior team, where 2~3 people can now complete work that previously required more than 10 people.
SemiAnalysis: TTFT Overrated? LLM Inference Bottleneck Lies in Balancing Interactivity and Throughput
SemiAnalysis has pointed out that the "time-to-first-token" (TTFT) often cited in large model inference may be receiving excessive market attention. The core dimension of an inference system is not that "faster is always better," but rather the trade-off between "single-user interaction speed (tok/s/user)" and "overall throughput efficiency (tok/s/GPU)." In most scenarios, TTFT has a limited impact on user experience; instead, the token generation speed during the decoding phase is the critical variable.SemiAnalysis concludes that only approximately 10%–20% of inference tasks are truly latency-constrained, while the vast majority of other scenarios are more dependent on the optimization of throughput and cost efficiency.