Introduction to io.net: Can AI computing power be decentralized?

io.net is a network based on the Solana blockchain, allowing users with free computing capacity to provide computational power to resource-intensive artificial intelligence companies.Previously, io.net completed its Series A financing with a valuation of $1 billion, raising $30 million, led by Hack VC, with participation from Multicoin Capital, Animoca Brands, Solana Ventures, Aptos, OKX Ventures, and others. This podcast episode features a discussion with io.net’s CSO&CMO Garrison Yang (0xHushey) on the technological advantages of io.net, future development plans, and how to address the pain points in the AI and cloud computing industries.

The views expressed by the interviewee are personal opinions and do not represent the views of WuBlockchain or constitute any financial advice. Readers are advised to strictly comply with local laws and regulations.

Audio-to-text conversion uses GPT, and there may be errors. Please listen to Youtube for the full podcast：https://youtu.be/HSBBGT5Vqvg

What is io.net?

io.net is in the process of building the largest AI compute network, and our goal is essentially to establish a decentralized competitor to AWS (Amazon Web Services). The project commenced in 2020 when our founder and CEO was developing quantitative models for algorithmic trading and sought more cost-effective compute resources. Even in 2020, purchasing computing power from AWS and Azure proved to be prohibitively expensive. Consequently, he explored compute sources worldwide, including in countries like Saudi Arabia and the APAC region, at independent data centers, and among crypto miners in use. He interconnected these sources, acquiring GPUs from various geographically distributed locations to form a massive network for computing. This initiative was successful. Initially, he utilized Ray, which at the time was not as widely recognized because most individuals preferred Kubernetes, popularized by Amazon Web Services. He independently operated this network to support his trading models for two and a half years.

In 2023, OpenAI launched ChatGPT, and suddenly, the entire world turned its attention to AI, particularly the significant computing power it requires. ChatGPT announced in 2023 that it was spending $700,000 daily to train their ChatGPT model. People also questioned how it could offer real-time inferences — responses to queries without charging users. Ahmad recognized that OpenAI employed the Ray architecture to expand their computing network, which led to the inception of our business. Ahmad realized he possessed a resource highly sought after in the AI industry: the capability to scale up decentralized and distributed nodes, thus providing computing power at a lower cost to AI companies. Hence, io.net was established. Remarkably, we were among the winners of the Solana hackathon in April 2023. The platform was officially released to the public in November 2023, and in just over four months, we have seen rapid progress.

Another reason for developing io.net, besides aspiring to create the largest AI compute network, is our desire to ensure the existence of a decentralized, community-driven, and community-owned compute network globally. This goal aligns with the way blockchain technology enables uncensorable money, and Ethereum serves as an uncensorable, accessible, global computer. By providing a decentralized source of computing power, we aim to guarantee that applications, data sources, AI models, and AI model inferences remain globally accessible, uncensorable, and usable across borders in the future.

How does io.net address pain points in the AI and cloud computing industries?

That’s a fantastic question. AI has been part of our lives for quite some time. It’s really consumer AI and consumer-visible enterprise AI that are newer developments. ChatGPT, for instance, has opened a lot of doors, as have general AI image and video platforms, which have become tools we use in our everyday lives.

What most people don’t realize is that we’ve been using AI for a long time. Facebook uses AI to scan images that you post, Google uses AI to help power their search engine, and Siri uses AI to understand and respond to our queries. Now, as AI becomes more popular and as more companies want to develop using AI, their costs increase. Compute has become one of the most important and increasingly scarce resources in the world. Today, AI relies on GPU compute rather than CPU compute, making GPUs in very high demand.

The issue is that we have significantly less compute available than we need — the market needs 2.5 times more computing power than AWS, Azure, and Google Cloud combined. They don’t have enough GPU compute to serve everyone, despite these big companies hoarding many of the graphics cards. Then there’s a lot of compute in independent data centers, crypto miners, and our consumer devices like MacBooks and gaming PCs. The problem is that these resources are hard to aggregate and utilize.

What io.net does is aggregate the compute capacity across GPUs from various sources. Some sources are higher quality, more reliable, and more valuable than others, but we aggregate it all and provide that compute capacity in a cluster for AI companies to access. The result is that AI companies can access this compute for up to 90% cheaper than on-demand prices from centralized cloud providers. And it’s faster. With io.net, there are no contracts to sign, no KYC, no long-term commitments, and you get to choose where your nodes are from, what speed they are, etc. This approach helps us address the major pain points in the AI and cloud computing industries by providing a decentralized, cost-effective, and accessible platform for compute power.

Can io.net match enterprise GPU power for AI model training?

The short answer is yes, clustering together multiple, slightly less enterprise-grade cards can give you the same amount of compute for a lower cost. The challenge is always scaling up that physical infrastructure, which is what io.net specializes in doing.

Furthermore, we can cluster together enterprise cards as well. Independent data centers might have a handful of cards here and there, and typically, these enterprise data centers are not going to coordinate with each other to serve a single customer. But if they pool their capacity on io.net, we can cluster their resources together. This not only maximizes the utilization of these enterprise-grade GPUs but also enhances their monetization. Essentially, io.net acts as a platform that facilitates the aggregation of computing power from diverse sources, creating a more efficient and cost-effective compute environment for AI model training.

We’re tapping into a resource that would have been underutilized anyways, so we can pass through that underutilized capacity to an end customer. Because that capacity was never going to be utilized optimally, it’s cheaper economically for the end customer.

But I want to be very clear. While we aim to build a decentralized alternative to centralized cloud providers, I don’t expect it to fully replace the existing infrastructure. The best comparison is to think about the power grid and renewable energy, like solar and batteries, as a way to manage the grid. We’re not going to replace base level power generation, such as nuclear power, which generates a lot of energy over time, is stable, cheap, and efficient. But nuclear power can’t spike up and down, which is why it’s called baseload. It’s the same with AWS. If an enterprise company signs a five-year contract with AWS for a certain amount of compute, they pay for it whether they use it or not. So it can be very cost-efficient to have some base level amount of compute with a centralized cloud provider.

However, when you have demand spikes, that’s where something like io.net comes in for on-demand use. For instance, if you want to run an experiment today or you know your load is going to spike for seven days or a month, you can buy that extra capacity from io.net much more cost-effectively. A decentralized network doesn’t require contracts or KYC, and you can purchase varying sizes of customizable capacity from all sorts of different sources. Then, when you’re done, you’re done. This gives enterprises a lot of different options and provides people a lot of choice and pricing power.

What are the challenges and strategies for io.net in balancing between individual and enterprise contributors?

Our ideal supplier is two types of people. One is enterprise data centers that have leftover capacity, which is high-grade stock with fast internet speeds, ideal for enterprise customers first. The second type fits into the bucket of a crypto miner — someone who buys a dedicated rig to provide capacity or run a GPU for crypto mining or similar activities, and typically has above-average connection speeds and dedicated hardware that isn’t used for everyday consumer tasks.

We liken GPUs to houses as investment assets. You invest in a house to rent it out and make money, and GPUs are the same. Despite expectations of depreciation due to newer chips, we’ve seen prices appreciating because of the current shortage. An A100 GPU can now pay back its investment in about nine months due to high demand, which is a very attractive ROI compared to the traditional expectations of crypto mining economics.

Adding to the financial aspects, tokens are not included in this calculation — that’s just stable coins or real revenue. When you introduce crypto economics and tokens on top, the incentives are significant. It’s akin to validator economics in blockchain, where you’re earning tokens with variable value while also earning revenue from the customers.

Who do you think will use io.net?

As for the demand side, currently, io.net is utilized for end-to-end AI/ML workloads. While model training does require colocated GPUs, it doesn’t have stringent latency requirements. Inferencing, however, can scale quickly and unpredictably, which io.net is well-suited to handle. We provide enterprises the flexibility to manage demand spikes cost-effectively, supplementing base load compute capacity with decentralized, on-demand resources.

Regarding the market share and costs, training models are currently more expensive in absolute value, but inferencing is likely to take over as the industry shifts more towards model tuning and utilizing models as a service, which is less costly. We anticipate the industry will move towards inferencing as the major cost factor as more developers enter the space and utilize base models developed by the big tech companies for their own specific needs.

In the broader scope, while we’re starting with AI/ML because of its strong unit economics, we aspire for io.net to become the largest decentralized, generalized compute layer, gradually expanding our customer base. For example, we actually built a platform product called BC8.AI, which originally was just a gen AI demo, to show you how inferencing would work on the network. But now it’s really popular. I think we’re doing like under 25,000 transactions today where people are using BC8.AI on the network. And all of that is inferences.

Which parts of the io.net stack are blockchain related?

The management of permissionless transactions between demand and supply is on-chain. We basically allow you to join the network in a permissionless fashion. There’s no human intermediary in the matching of supply and demand. You cluster, and you pay, and you know that the people receive the money and that they will provide you the compute queue. There’s actually a proof where the supply is committing capacity for a certain amount of time. If you rent for an hour, I commit to providing it for an hour. If I don’t provide that capacity for the full hour, I get slashed. The economics and the mechanisms are very similar to validators in that sense.

We also store proof of compute on-chain. So if you look at a given inference, you can always track down exactly which node provided that compute, how much compute was provided, how much it was paid, and when it provided that inference. It provides a running log for where that inference came from. Today it’s more of a ledger, right? It’s functional.

But in the future, this kind of transparency and data storage will be really helpful to track down where some of these computations are being produced. You might look at an image and always be able to track down who influenced it, who provided the compute, and which model it was inferred from. And I think as the AI industry develops, this type of transparency is going to be really important.

Lastly, we chose Solana for our blockchain needs because it’s fast and cheap. Despite some recent scalability challenges, it’s proven to handle the transaction load we’ve seen. We aim to maintain as much of our core infrastructure on Solana as possible, taking advantage of its capacity and the forthcoming updates like Firedancer for even greater scalability. Our operational and scaling challenges on io.net, especially when adding thousands of nodes, reflect similar challenges to what blockchain infrastructures like Solana face, and so far, Solana has been resilient to our use case.

How does the matching process work for buyers selecting computing clusters on io.net?

Currently, buyers don’t often delve into excessive detail; they simply state the type of device they want, the connection speed, and the locations they prefer. This provides buyers with all the information they need to determine the amount of GPUs necessary based on the capacity they’re looking for. As the industry evolves, and factors such as latency and capacity are not the sole determinants, I anticipate buyers will make more nuanced choices. They may opt for certain locations for specific reasons or choose between lower and higher-end devices based on the economic value. Currently, due to the scarcity, if buyers need capacity, they will take whatever is available that can serve their needs. They’ll look at our network, and if there’s a significant capacity they can rent, they’ll just rent it. Or, if they need a certain amount of capacity, they’ll check if there are enough 4090s that meet their speed requirements and then just cluster them.

The demand side of the industry is still maturing, and it’s one of the aspects that I believe people will continue to learn about as a decentralized cloud provider grows.

Every node on io.net has a reputation score, which allows you to see how often a node remains available, its uptime, and other performance metrics, aiding in decision-making.

There is concern about low-quality nodes, especially once token airdrops or farming programs start. That’s where time scores and reputation scores become crucial. We constantly ping every node, and if a node doesn’t respond, it’s not considered available. If it’s not available, it doesn’t earn rewards. The crypto economic incentives are straightforward: if a node is available, it provides better service to the demand side and gets hired more often, earning more rewards. As long as nodes maintain availability and performance when hired, the demand side receives the desired compute power, so it’s a win-win.

It requires more than just having computing power ready; it’s akin to managing a miner, with a similar level of attention and sophistication. That’s why those with dedicated rigs, like crypto miners, find it to be a sweet spot for a network like io.net.

How do you see io.net development over the next five years?

You know, we’re launching on April 28th, right? We’re still pre-launch, about five weeks away. And I’m sure the world will change after that depending on how it goes. But what we envision is building an ecosystem of products and services on top of compute as an accessible commodity. If you create a tokenized way to allow people to get exposure to compute, then they can build a lot of things. Today, we know compute is very valuable, but outside of buying GPUs, you as a consumer can’t really get exposure to it or use that as an asset in the same way you could with other commodities.

Our goal is to turn $IO into the compute currency. If compute is digital oil, then it needs a petrodollar, and we’re trying to build that petrodollar. This allows a bunch of different things to happen. We’ve already created a marketplace for it, so people can transact, and now that people can derive value from that asset, you can imagine DeFi products built on top of it. As we go into generalized compute, even more things can happen because more industries can get involved.

The way that we see it, we need to 10x the supply side, we need to improve the demand side self-service customer experience, and we need to expand the use cases. We’ll move into more synchronous use cases like cloud gaming, look at image rendering, look at video rendering — things that maybe don’t sit under traditional AI workloads. That allows the market to expand on both sides because then you can support lower-end cards for lower-end use cases and support all these different customer types to improve your market share.

Right now, we’re doing about seven or eight thousand dollars a day in top-line revenue, which is annualized about 2.3 million dollars. And again, we’re only four months in. So we just need to continue to grow both sides of the marketplace and let those network effects spin. And I hope that, in the next four or five years, the platform gets really big, moving a lot of compute capacity to the network. Our goal is to progressively transition the DePIN to the community. The DePIN should always be a decentralized asset. The different apps built on top of it, whether it’s Io.net or other protocols and services that are powered by the DePIN, those are totally separate. But the progressive decentralization of the DePIN is super important as a resource.

What is the relationship between io.net, Render, and Filecoin?

io.net plays two roles: Firstly, there is a user interface (UI) that simplifies things, which is helpful.But more critically, io.net brings two things to other Decentralized Compute Networks (DePINs): network architecture and orchestration software, which operates the client that runs on the device. Network architecture is vital because not all DePINs can cluster thousands of geographically distributed GPUs into a single cluster. By putting GPUs on our DePIN, other networks can access use cases requiring that scale, mainly AI.

Then there’s the hardware, which is just that — hardware. To serve AI workloads, your client on that hardware needs the capacity to serve those workloads, similar to how a validator node operates. Render is outstanding for rendering images — it’s what the network was built and tooled to do. Filecoin, similarly, was built for storage. These companies excel at their respective functions. We excel at serving AI workloads. Our network isn’t built for providing storage, but we started with AI workloads first because the economics are strong. Other DePINs can access our network and our tooling to monetize AI customers.

As for the future, io.net doesn’t aim to replace platforms like Render or Filecoin, but it offers the possibility to do something similar based on what they’re doing. AI is exclusively on io.net, and if someone wants to leverage their computing power for Filecoin or other projects like Render, they’re free to do so using io.net as a platform.

Lastly, the question of modes within io.net and how we deal with marketplace dynamics: Being early, big, and having a head start is advantageous. Our network architecture — like deploying a cluster with 6000+ nodes that are completely geo-distributed — is something others can’t easily do.

We’re also considering the larger picture, such as AI as a service and marketplace layers on top of compute. It’s easier to build the compute base layer first and then add marketplaces on top rather than the other way around. This allows us to pivot and adapt in the future, and the possibility of decentralized infrastructure being critical in an increasingly segmented global economy could make something like io.net fundamentally important.

What are io.net’s unique technological advantages?

When considering DePINs, akin to layer1s in blockchain, it’s important to recognize that while much of the code is open source and forking is common, most of the innovations are incremental. Many layer1s are very similar but may have differences in consensus mechanisms, data layers, or how they handle storage.

For DePIN architecture, the general strategy isn’t drastically different across the board — it’s about aggregating GPUs, and a significant leap is managing to do so without the necessity for physical proximity. Another crucial factor is clustering GPUs; how many you can cluster together, whether they must be the same model or can be different, the permissible distance between them, and how the latency increases with the size of the cluster.

These points might seem simple, but the ability to achieve these network capabilities and the trade-offs made are crucial. Copying our approach, like forking Ray and providing Ray clusters, doesn’t guarantee the same latency mitigation. In the world of computing as a commodity, competition hinges on minor differences in latency, availability, and connection speed.

Yes, there’s a technical moat. It encompasses network architecture as well as orchestration layer construction. It includes the ease of deployment for developers, connection simplicity for workers, and a multitude of UI/UX decisions. There’s also a market strategy moat: deciding whether to start with image rendering, video rendering, storage, AI workloads, synchronous cloud gaming, or pixel streaming. These aren’t traditional technical moats in the sense of possessing unique skills unknown to others, but they’re significant strategic choices that require one to forgo certain areas due to resource constraints.

In summary, there are indeed many technical moats, each shaped by tangible decisions and strategies that need to be made. If it weren’t for these differences, the existing networks we see today would be much more alike.

How can users who like io.net get involved?

If you’re interested in io.net, there are several ways to get involved. For those with consumer devices at home, like a MacBook or a 4090 gaming PC, I encourage you to connect to the network and provide capacity. It’s an excellent way to understand how decentralized physical infrastructure networks operate, as they exist to optimize inefficiencies in the world.

For miners, io.net offers an exciting opportunity to contribute and benefit. If you’re intrigued by the idea of investing in GPUs as assets to earn yield, it’s worth considering. GPUs are yield-bearing assets, often offering quicker payback than traditional investments like houses or cars. This isn’t investment advice, but it’s an interesting angle to consider — the financialization of compute power.

We’re essentially turning $IO into the digital petrol dollar. If compute is the digital oil, io.net aims to support an entire ecosystem of financialized products, tools, and services, building a whole DeFi ecosystem on top of it.

Developers can experiment by deploying clusters on io.net. It’s not every day you get the chance to deploy a cluster of thousands of nodes and run advanced models on such a scale, even if just for a short period. It’s a novel and fascinating experience that highlights the power and potential of decentralized computing.

Even if you don’t wish to provide compute power directly, there are still plenty of opportunities to engage. Joining discussions in our community, particularly on topics like data availability, preservation of censorship-resistant AI models, and inferencing, can be incredibly enriching.

And if you’re interested, I invite you to follow us on Twitter or join our Discord to stay updated and participate in our growing community.We have some really smart, really great thinkers in our discord who are sitting around talking about this stuff. And I encourage people to jump in and have a conversation.We’ll be around.

Follow us
Twitter: https://twitter.com/WuBlockchain
Telegram: https://t.me/wublockchainenglish