How AI Is Giving the Edge the Application It’s Been Waiting For

This Industry Viewpoint was authored by Kevin Sheehan, CTO of the Americas, Ciena

It’s nearly impossible to avoid artificial intelligence (AI) in our day-to-day anymore. What started as a bit of a gimmick, or centered on creating humor, is now beginning to show up all around us. Many initially feared that AI would be centered on cost cutting and eliminating jobs, but it is turning out to be squarely aimed at making things better and easier. Whether you are watching the recent Netflix documentary Churchill at War or creating some custom imagery for your Zoom background, AI is improving the quality of the experience.

Enterprises are looking to develop or deploy AI capabilities of some kind. Coca-Cola has leveraged generative AI to create new image-rich television commercials with the goal of pulling in more viewers. And AI has already had a huge impact on inter- and intra-data center connectivity: AI applications, such as large language model (LLM) training leveraging deep learning (DL) and artificial neural networks, involve moving massive amounts of data within a data center over short, reliable, high-bandwidth networks operating at 400Gb/s and 800Gb/s, and are likely to require 1.6Tb/s and higher in the very near future.

For now, having some latency with the likes of ChatGPT is fine, as these types of applications are not yet truly mission critical and quite forgiving in nature. But as cities and organizations get smarter and rely on AI to make rapid decisions by analyzing data in real-time—known in AI parlance as inference—it will need to move from major data center hubs to more localized capabilities closer to the users. Those applications simply can’t wait multiple seconds to deliver an outcome.

Additionally, enterprises are developing sophisticated generative AI applications that must leverage more GPU than they can realistically host in-house. Therefore, they are turning to ‘local’ dedicated data centers that are offering GPU-as-a-Service to enterprise locations.  High bandwidth, reliable, low-latency connections are required between the generative AI applications and the GPU farms.

Edge computing has existed for many years, albeit often out of the limelight, lacking the technological, economic, and operational drivers to justify large-scale adoption in service provider networks. Now, with the Fourth Industrial Revolution well underway, AI is anticipated to play a significant role in addressing many of these challenges, providing a compelling reason to revisit edge investments in both fixed and mobile applications.

The Growing Need for AI Inference at the Edge

It’s important to understand the difference between training and inference when it comes to AI. Training in the AI sense involves creating a model that can identify patterns and make decisions based on clearly defined data; the model can then learn from its mistakes and evolve its offering. Inference, meanwhile, is the next stage when the trained AI model is able to make quick, reliable decisions and predictions from new data sets.

Once an LLM is properly trained, it will be optimized and ‘pruned’ to provide an acceptable inferencing (i.e., using AI in the real world) accuracy within a much smaller footprint in terms of compute, storage, and energy requirements.

The optimized AI algorithms are then pushed out to the edge to reduce the strain on core data centers hosting LLM training, reduce latency, and abide by regulations related to data privacy concerns by hosting data locally. At the enterprise level, the digital co-worker is becoming a logical next step in agentic AI applications. But each digital co-worker is the front face of hundreds of digital agents, each of which is performing highly specialized tasks with real-time inter-workings required to work on synchrony to empower a single digital coworker.

These applications all require low-latency responses—we simply can’t afford for inference to hinge on the data traversing back to a centralized data center or clouds quickly and reliably, with responses needing to be measured as close to real time as possible. There is also a need to minimize backhaul traffic to centralized data centers or clouds through optimizing bandwidth efficiency.

Thus placing AI storage and compute assets in geographically distributed data centers closer to where AI is created and consumed, whether by humans or machines, allows for faster data processing for near real-time AI inferencing to be achieved. This means more edge data centers to interconnect.

This sounds like an easy solution, yet challenges must be addressed and the network optimized to make these applications work as intended.

Cloud-to-Edge Communication Is Key

It’s important to note that edge networks should not operate as a silo—there is still a need for a centralized hub, likely cloud, to make it all work seamlessly and enable edge clouds to communicate.

Therefore, seamless connectivity between cloud services and edge devices is critical. While inference at the edge can be done within a smaller physical footprint, such as a node or even in some cases a device such as a traffic light, the consistent evolution of the AI application requires centralized compute and storage to continue training the algorithms and also to allow for communication across vast distances.

Accordingly, training in the cloud will require enterprises to move huge amounts of training data securely between their premises and the cloud, as well as across different cloud instances. This will require edge-to-cloud networks to operate with dynamic and higher speed bandwidth interconnections.

Power Consumption: The Challenge of AI Workloads

AI models are notoriously power-hungry in their training phase, consuming immense amounts of electricity. Recent research suggests that data center power demand will grow 160% by 2030. That same research found that data centers worldwide will consume 3-4% of the world’s overall power by the end of the decade, up from estimates of 1-2% at present.

Power needs will rise as models become more complex, requiring constantly increasing amounts of compute, storage, and networking capabilities, and as we roll out edge networks to enable rapid inference.

This is already having an impact on the grid, requiring data centers to be rolled out at new, often regional or rural, locations—or even abroad—to reduce the grid impact and to power them with more sustainable energy sources. Historically, data centers were built around existing fiber connectivity. In our AI world, availability of electricity is often the primary driver. 

Optimizing Edge Network Architecture for AI

Ideally, and when possible, much of the processing can be done at the edge. This will require the edge devices and edge networks to be efficient and built with AI optimization in mind.

Technologies such as edge AI chips, edge servers, and specialized processing units for energy-efficient inference will be vital to managing as much of the load as possible at the edge, both to reduce the load on edge-to-cloud infrastructure and to reduce energy consumption.

Yet ultimately, to maintain efficiency and manage workloads across the entire ecosystem, decisions will need to be made as to when to process data at the edge or send it to the cloud. Thankfully, the answer is more AI.

AI-driven network management will be key, requiring machine-learning algorithms for dynamic network optimization and traffic management. And predictive analytics for capacity planning, congestion management, and fault detection will help make the decisions for the network, making the ecosystem as smart as the devices it is empowering.

The Network to Be Driven By—and to Drive—AI’s Evolution

In short: AI will reshape the edge network by driving demand for localized inference, beyond the embedded capabilities of end-user devices such as phones and surveillance cameras, and will likely reshape the architecture of connectivity, data flow, and traffic management.

But as the cloud-edge relationship evolves, and power consumption inevitably increases, it will be incumbent on service providers and stakeholders to invest in AI-optimized edge infrastructure, power-efficient technologies, and secure, scalable solutions.

The world has moved quickly from basic AI applications to the likes of ChatGPT and Chatbots, and this year digital co-workers are likely to become increasingly present. At this rate of innovation, if the network is optimized to cater to AI, the only limit on AI’s continued evolution will be the imagination.

If you haven't already, please take our Reader Survey! Just 3 questions to help us better understand who is reading Telecom Ramblings so we can serve you better!

Categories: Artificial Intelligence · Datacenter · Industry Viewpoint · Metro fiber