Celestial AI raises $100M to expand Photonic Fabric technology platform

Celestial AI, a developer of optical interconnect technology, has introduced a profitable sequence B funding spherical, elevating $100 million for its Photonic Fabric technology platform. IAG Capital Partners, Koch Disruptive Technologies (KDT) and Temasek’s Xora Innovation fund led the funding. 

Other members included Samsung Catalyst, Smart Global Holdings (SGH), Porsche Automobil Holding SE, The Engine Fund, ImecXpand, M Ventures and Tyche Partners.

According to Celestial AI, their Photonic Fabric platform represents a big development in optical connectivity efficiency, surpassing current applied sciences. The firm has raised $165 million in whole from seed funding by sequence B.

Tackling the “memory wall” problem

Advanced synthetic intelligence (AI) fashions — such because the extensively used GPT-4 for ChatGPT and advice engines — require exponentially growing reminiscence capability and bandwidth. However, cloud service suppliers (CSPs) and hyperscale knowledge facilities face challenges due to the interdependence of reminiscence scaling and computing, generally referred to as the “memory-wall” problem.

The limitations {of electrical} interconnect, similar to restricted bandwidth, excessive latency and excessive energy consumption hinder the expansion of AI enterprise fashions and developments in AI. 

To handle these challenges, Celestial AI has collaborated with hyper scalers, AI computing and reminiscence suppliers to develop Photonic Fabric. The optical interconnect is designed for disaggregated, exascale computing and reminiscence clusters.

The firm asserts that its proprietary Optical Compute Interconnect (OCI) technology permits the disaggregation of scalable knowledge heart reminiscence and permits accelerated computing.

Memory capability a key drawback

Celestial AI CEO Dave Lazovsky informed VentureBeat: “The key problem going forward is memory capacity, bandwidth and data movement (chip-to-chip interconnectivity) for large language models (LLMs) and recommendation engine workloads. Our Photonic Fabric technology allows you to integrate photonics directly into your silicon die. A key advantage is that our solution allows you to deliver data at any point on the silicon die to the point of computing. Competitive solutions such as Co-Packaged Optics (CPO) cannot do this as they only deliver data to the edge of the die.”

Lazovsky claims that Photonic Fabric has efficiently addressed the difficult beachfront drawback by offering considerably elevated bandwidth (1.8 Tbps/mm²) with nanosecond latencies. As a outcome, the platform gives absolutely photonic compute-to-compute and compute-to-memory hyperlinks.

The latest funding spherical has additionally garnered the eye of Broadcom, who’s collaborating on the event of Photonic Fabric prototypes based mostly on Celestial AI’s designs. The firm expects these prototypes to be prepared for cargo to prospects inside the subsequent 18 months.

Enabling accelerated computing by optical interconnect

Lazovsky acknowledged that the info charges should additionally rise with the growing quantity of information being transferred inside knowledge facilities. He defined that as these charges enhance, electrical interconnects encounter points like sign constancy loss and restricted bandwidth that fails to scale with knowledge development, thereby proscribing the general system throughput.

According to Celestial AI, Photonic Fabric’s low latency knowledge transmission facilitates the connection and disaggregation of a considerably larger variety of servers than conventional electrical interconnects. This low latency additionally permits latency-sensitive functions to make the most of distant reminiscence, a chance that was beforehand unattainable with conventional electrical interconnects.

“We enable hyperscalers and data centers to disaggregate their memory and compute resources without compromising power, latency and performance,” Lazovsky informed VentureBeat. “Inefficient usage of server DRAM memory translates to $100s millions (if not billions) of waste across hyperscalers and enterprises. By enabling memory disaggregation and memory pooling, we not only help reduce the amount of memory spend but also prove memory utilization.”

Storing and processing bigger units of information

The firm asserts that its new providing can ship knowledge from any level on the silicon straight to the purpose of computing. Celestial AI says that Photonic Fabric surpasses the constraints of silicon edge connectivity, offering a bundle bandwidth of 1.8 Tbps/mm², which is 25 occasions higher than that provided by CPO. Furthermore, by delivering knowledge straight to the purpose of computing as a substitute of on the edge, the corporate claims that Photonic Fabric achieves a latency that’s 10 occasions decrease.

Celestial AI goals to simplify enterprise computation for LLMs similar to GPT-4, PaLM and deep studying advice fashions (DLRMs) that may vary in measurement from 100 billion to 1 trillion-plus parameters.

Lazovsky defined that since AI processors (GPU, ASIC) have a restricted quantity of excessive bandwidth reminiscence (32GB to 128GB), enterprises immediately want to join tons of to 1000’s of those processors to deal with these fashions. However, this method diminishes system effectivity and drives up prices.

“By increasing the addressable memory capacity of each processor at high bandwidth, Photonic Fabric allows each processor to store and process larger chunks of data, reducing the number of processors needed,” he added. “Providing fast chip-to-chip links allows the connected processor to process the model faster, increasing the throughput while reducing costs.”

What’s subsequent for Celestial AI? 

Lazovsky stated that the cash raised on this spherical can be used to speed up the productization and commercialization of the Photonic Fabric technology platform by increasing Celestial AI’s engineering, gross sales and technical advertising groups. 

“Given the growth in generative AI workloads due to LLMs and the pressures it puts on current data center architectures, demand is increasing rapidly for optical connectivity to support the transition from general computing data center infrastructure to accelerating computing,” Lazovsky informed VentureBeat. “We expect to grow headcount by about 30% by the end of 2023 to 130 employees.”

He stated that because the utilization of LLMs expands throughout varied functions, infrastructure prices may also enhance proportionally, main to unfavourable margins for a lot of internet-scale software program functions. Moreover, knowledge facilities are reaching energy limitations, proscribing the quantity of computing that may be added.

To handle these challenges, Lazovsky goals to reduce the reliance on costly processors by offering excessive bandwidth and low latency chip-to-chip and chip-to-memory interconnect options. He stated this method is meant to scale back enterprises’ capital expenditures and improve their current infrastructures’ effectivity.

“By shattering the memory wall and helping improve systems efficiencies, we aim to help shape the future direction of AI model progress and adoption through our new offerings,” he stated. “If memory capacity and bandwidth are no longer a limiting factor, it will enable data scientists to experiment with larger or different model architectures to unlock new applications and use cases. We believe that by lowering the cost of adopting large models, more businesses and applications would be able to adopt LLMs faster.”

VentureBeat’s mission is to be a digital city sq. for technical decision-makers to acquire data about transformative enterprise technology and transact. Discover our Briefings.