AI DePIN
Last updated
StarAI Technical Whitepaper
Last updated
Currently, StarAI DePIN supports terminals with various operating systems, including Linux, Windows, and macOS. Therefore, your devices can easily connect to our network without requiring extensive adaptation work. Additionally, whether your device is equipped with NVIDIA or AMD GPU cards, its computing power can be effectively utilized. Of course, the computing power of devices may vary depending on factors such as the GPU model, CPU model, and memory size. Each device may contribute different amounts of computing power.
Similarly, our network supports access from mobile devices. If you have an iPhone or an Android smartphone, you can also connect to our platform. However, due to the limited computing power of mobile devices, we can only support running smaller models on them.
The capacity for hosting different model sizes varies across devices. Servers equipped with professional-grade GPUs naturally possess stronger computing power, allowing them to support larger model sizes. Personal PCs with gaming- grade GPUs may have slightly lower computing power but can still support some medium-sized models. Smart terminals equipped only with CPUs, without GPUs, may support smaller model sizes.
StarAI DePIN selects suitable models for deployment on your connected devices based on the results of computing power evaluations. The models we choose to deploy will utilize a portion of your device’s computing power, leaving some resources available for other activities. For example, if your device is equipped with a professional-grade GPU like the A100, we may deploy larger models like llama3 70b, which have more model parameters and often result in better and more precise inference. Conversely, if your device has a consumer-grade GPU like the GTX 3060, we may deploy smaller models like phi3 3.8b, which require fewer resources. This approach ensures that device perfor- mance is effectively utilized without wasting computing resources and that device computing resources are not overly consumed, thereby ensuring the normal operation of daily tasks.
StarAI DePIN connects devices from around the world, and it also handles inference tasks from various global sources. Some devices have high computing power, while others have lower computing power. Additionally, some connected devices may have high idle capacity, while others may be heavily loaded. StarAI DePIN comprehensively considers these factors when allocating inference tasks.
Model Matching for Inference Tasks
As previously mentioned, we deploy different models based on the varying computing power of devices. Different inference tasks received by the platform have different requirements for models. Some tasks require larger models,relying on higher computing power, while others can be accomplished with smaller models without significant computing power demands. The platform matches the models deployed on all connected devices according to the model requirements of different tasks. It selects devices with matched models to complete the corresponding inference tasks, ensuring the high-quality completion of inference tasks.
Load Matching
Even if the same model is deployed and can handle similar inference task requirements, the load on different devices may vary. The platform considers the load situation of devices that can complete inference tasks and assigns newly received tasks to relatively idle devices for inference. For example, between a device with 60% load and another with 30% load, we would prioritize the latter for inference.
Geographic Matching
We know that cross-geographical network communication incurs significant overhead and higher latency. To efficiently complete inference tasks, we prioritize assigning received inference tasks to devices in the same geographical region for inference. For instance, if two devices with similar loads and deployed with the same model are available, one in Asia and the other in the Americas, when the platform receives an inference task matched to this model from Asia, it will be prioritized for assignment to the device in Asia.
By considering model requirements, load situations, and geographical locations, StarAI DePIN intelligently distributes inference tasks to connected devices.
StarAI DePIN conducts regular health checks on all connected devices to ensure that tasks are assigned to healthy devices. If a device fails the health check, the platform marks it as inactive for the current check cycle and records the event. Devices that frequently experience failures have their task priority lowered. During the next health check, if a previously faulty device is found to be functioning properly, the platform reactivates it for task assignment.
For inference tasks running on faulty devices that experience timeouts or failures to complete, the platform reallocates these tasks to healthy devices based on the scheduling rules described above.The health check process follows the flow outlined below:
Verify the device’s identity to determine its current owner.
Check the device’s network status, including connec- tivity and bandwidth.
Monitor the device’s CPU status.
Examine the device’s GPU status and driver infor- mation.
Assess the current workload on the device.
Devices connected to the platform by computing power providers can choose to go offline voluntarily. When a computing power provider decides not to take on inference tasks, they can terminate their service by issuing an exit command through the computing power node daemon. Additionally, the platform maintains a record of faulty computing devices and may blacklist devices that experience frequent failures, enforcing mandatory offline status for them.
