7 hours Ago
The fast-paced world of AI is never short on headlines. Last November, for instance, Cerebras claimed to have built the fastest AI inference chip in the world . This April, OpenAI launched what it called a new “giant” and “expensive” model that “feels like you’re talking to a thoughtful person.
” Yet, beneath the noise, something is unfolding inside enterprise teams. While many CEOs are talking up their fine-tuned copilots, multimodal marvels and big money investments, engineers — often deep in data trenches where AI actually lives or dies — are watching their dashboards light up: query times spike, pipelines clog and models stall. The generative AI boom has sparked a race to scale larger models, with OpenAI CEO Sam Altman as its most vocal champion.
But amid the hype, there’s a hidden analytics bottleneck undermining many enterprise AI ambitions today. Most organizations are still trying to make sense of massive, messy datasets. And while GPUs have radically accelerated model training, data preparation — the unglamorous task of getting the inputs right — remains stuck in CPU-bound architectures that were never designed for this kind of scale.
Put simply, data volumes are growing faster than our ability to process them. As Elad Sity, CEO and cofounder of NeuroBlade , noted in an interview, “while the industry has long relied on CPUs for data preparation, they’ve become a bottleneck — consuming well over 30 percent of the AI pipeline.” The result? Sluggish pipelines, ballooning costs and a widening gap between AI aspirations and real-world ROI.
That pressure, explained Sity, has triggered a major shift in mindset, as the industry now actively pursues solutions. “It’s powering a silent revolution that’s reshaping data analytics — shifting from human-driven insights to AI models consuming and acting on ever-growing datasets, fueling a cycle of accelerated data collection and processing,” he said. NeuroBlade, the Israeli semiconductor startup Sity cofounded, believes the answer lies in a new category of hardware specifically designed to accelerate data analytics.
Their Analytics Accelerator isn’t just a faster CPU — it’s fundamentally different architecture purpose-built to handle modern database workloads. But can it meaningfully reshape the economics of enterprise AI? Enterprises are discovering that the bottleneck in AI isn’t always in the model, but upstream in the data itself. As one report by Pragmatic Institute revealed, “data practitioners spend 80% of their valuable time finding, cleaning, and organizing the data.
” While estimates vary on the actual time that data scientists spend cleaning and preparing data, industry consensus is clear: data scientists are spending a lot more time on data preparation than on analysis and modeling. Data preparation includes extracting, transforming and joining massive volumes of structured and semi-structured data, which often reside in sprawling lakehouse environments. That’s a problem.
Because most of this work still runs on general-purpose CPUs. According to AMD, the market currently supports roughly 2 million CPU sockets for analytics workloads — expected to grow to 4-5 million by 2027. That’s a lot of general-purpose hardware powering queries that weren’t designed to be run at petabyte-scale.
“Companies are hitting a wall with traditional CPU scaling,” Sity explained. “We're seeing hyperscalers and cloud providers spending billions annually just to maintain their analytics infrastructure. But the returns are flattening.
” Adding more CPUs only gets you so far, as scaling the cluster exponentially increases node-to-node communication overhead — creating performance, power, and cost barriers. Beyond a certain point, the costs — including hardware, energy, data center real estate — begin to outweigh the performance gains. This dynamic is especially problematic in AI environments where latency matters and stale data can degrade model accuracy.
NeuroBlade’s Accelerator unlocks the full potential of data analytics platforms by dramatically boosting performance and reducing query times. By offloading operations from the CPU to purpose-built hardware — a process known as pushdown—it increases the compute power of each server, enabling faster processing of large datasets with smaller clusters compared to CPU-only deployments. “General-purpose CPUs can’t scale fast enough to meet rising performance demands and data volumes, pushing organizations to grow cluster sizes,” noted Sity.
“Purpose-built hardware that boosts each server’s compute power for analytics reduces the need for massive clusters and helps avoid bottlenecks like network overhead, power constraints, and operation complexity.” In TPC-H benchmarks — a standard for evaluating decision support systems — Sity noted that the NeuroBlade Accelerator delivers about 4x faster performance than leading vectorized CPU implementations such as Presto-Velox. NeuroBlade’s pitch is that by offloading analytics from CPUs and handing them to dedicated silicon, enterprises can achieve better performance with a fraction of the infrastructure — lowering costs, energy draw and complexity in one move.
This shift isn’t happening in isolation. NeuroBlade has worked closely with Amazon Web Services to integrate its Analytics Accelerator into AWS EC2 F2 instances, making it accessible to cloud-based customers — especially in domains like financial analytics or AI model refreshes. “This shift toward custom analytics processing is similar to what we saw with the rise of GPUs in AI,” Sity told me.
“First the hyperscalers get onboard, then the broader market follows.” Major semiconductor players are also circling the space. As Nvidia’s AI accelerator dominance grows, companies like Intel and AMD are looking for footholds in adjacent compute categories.
What that means is that specialized analytics hardware could become the next big AI battleground. Not long ago, AI and analytics ran on separate systems. But today, thanks to platforms called data lakehouses , companies can use a single data source to power both dashboards and machine learning models.
While this setup can improve efficiency, it also creates new risks. If the data inside the lakehouse is messy or out of date, it can slow down everything — from business reports to AI performance. Traditional business intelligence tools are built for people and often run on a schedule.
AI systems, on the other hand, are always on. They need data constantly — in real time, at a massive scale. That’s why speeding up how companies prepare and process their data is no longer optional.
It’s essential to keeping models fresh, insights relevant and decision-making fast. “By reducing the time it takes to prepare and query data, companies can refresh models more frequently, shorten feedback loops and drive better real-time decision-making, especially in high-stakes industries like finance, cybersecurity, healthcare and more,” Sity explained. To be clear, the analytics acceleration market is still young.
So, adoption will likely take time. Enterprises move slowly when it comes to core infrastructure, and switching out silicon is never simple. But the signs are already there: big cloud integrations, promising benchmarks and a growing awareness that AI’s real value hinges not just on model size , but on how efficiently you can move and make sense of your data.
“We’re at the beginning of a new era in specialized compute,” Sity noted. “Just as GPUs transformed AI by providing purpose-built hardware for neural networks, analytics processors will transform how we process data by addressing the specific bottlenecks in query execution.” For organizations struggling to turn AI investment into tangible ROI , solving the analytics bottleneck may be the first real unlock.
The future of AI won’t be won by bigger models alone — but by how efficiently you can get the right data to them..
