The previous article talks about why I want to launch the PPIO project and choose the blockchain direction. PPIO is a decentralized data storage and delivery platform for developers that values speed, affordability, and privacy.
I wrote this article to explain why PPIO’s decentralized storage makes data transfer faster.
First, I want to explain the underlying architecture of the Internet.
Factors affecting network transmission speed:
- The number of routes passing through. Generally speaking, the more routes through the middle, the slower the transmission speed.
- The bandwidth at both ends, and the rest, especially on the server side. If the server side is full，the speed slows down.
- The bandwidth of the intermediate route between the two ends, the bottleneck is often the slowest route in the middle, especially when crossing the ISP, or when crossing the border, it is straightforward to encounter bottlenecks.
- Local hotspots can also easily cause network congestion, because when one user is consuming more bandwidth, other users are affected in terms of speed.
In the actual operation of the Internet, it is similar to a city’s traffic, which is very complicated. There is enormous uncertainty and it is difficult to predict. We cannot simply determine the network speed based on distance. The closer the distance, it doesn’t necessarily mean faster However, it is always certain that if the geographical distance is far away, the network speed is indeed impacted. The following figure is a schematic diagram of the network operation in the United States.
Let’s look at a group of survey data on Southeast Asian countries. The data comes from:https://www.terabitconsulting.com/mt-content/uploads/2018/01/20150602-broadband-infrastructrure-in-the-asean-region.pdf
This is an overview of the bandwidth of southeast Asian countries, and it seems that many countries have very little total access bandwidth.
Besides, the survey form of international traffic is as follows:
From the table it indicates that the International bandwidth per capita of most countries in Southeast Asia is very low, the International connectivity of many countries are also very weak. Base on these findings, I came up with a question, is the world’s top cloud service offering like Amazon is still able to guarantee high-speed access everywhere?
Let’s look at another diagram, this is the global computer room deployment map of Amazon’s cloud service from https://en.wikipedia.org/wiki/Amazon_Web_Services
It seems that AWS cloud service only focuses on the two nodes of Singapore in Southeast Asia. I guess Singapore is safe and robust, presumably, with the lowest management costs. However, according to the previous report, the International connectivity in many countries in the southeast is weak. Imagine if a user is in Thailand and he will have to store files on an AWS server node in Singapore, he has to connect through a multinational connection, the International connectivity across the entire country of Cambodia is very weak, then his transmission speed must be deficient.
So why does the PPIO decentralized storage that I designed is able to solve this problem? Because once my main-net has started, soon our services will be all over the world. Why do we have services all over the world? It is because everyone will have idle hard disk space or bandwidth. If these idle spaces can be rented out to generate money, I believe no one will refuse. PPIO is to rent out the hard disk space and bandwidth of the user, and then use blockchain technology to ensure that the incentive is fair and transparent. As long as the miner connects the storage and bandwidth to the PPIO to provide services, they can get incentives.
When there are service providers all over the world, PPIO is equivalent to providing storage services all over the world. With the closest distance to the nearest person, the transmission speed can naturally be improved. Imagine that there are more people on the Internet and more people on the service side. In short, the more people getting involved, the faster the transmission speed becomes.
When I designed PPIO, I introduced a traditional overlay network of P2P. Simply put, an overlapping network is a layer of logical Internet based on the equality between nodes and nodes on the bottom of the underlying physical Internet.
PPIO’s overlay network is a self-organizing, unstructured overlay network, that encourages connection among local peers. Nearby storage miners are more likely to be selected to store the data. Similarly, data distribution and retrieval are mostly handled by nearby peers as well. As a result, data transfer can be maintained at much higher speed by fully utilizing local network bandwidth.
With more nodes joining PPIO’s storage network, it becomes easier for each node to find nearby peers with a faster connection, for both storage and retrieval. Server bandwidth is no longer a bottleneck, and user growth leads to better speed and experience.
Establishing a Self-Organizing P2P Network
Every node in PPIO’s network maintains a list of neighboring peers to stay connected with. The list gets continuously updated by dropping disappeared nodes and adding newly joined neighbors. As a result, the list should mostly contain active and fast neighbors. With all the nodes following the same process as described below, the self-organizing P2P overlay network can be established and maintained.
Each node needs to maintain two real-time lists to support the above process, a list of current neighbors, and a list of neighbor candidates. Besides, each node also maintains a list of past neighbors. These are essential to node selection in the overlay network.
Preferred storage miners with a closer network distance
When user uploads data to PPIO’s storage network, its neighbors with a closer network distance are prioritized when selecting storage miners to store the data. A large percentage of the copies of the data are stored on the neighboring miners with a faster connection to the user, to achieve higher transfer speed during storage and retrieval. When enough miner nodes join the network in different regions of the world, high data transfer speed can be maintained around the globe.
Ordinary users generally do not change location, but there are special circumstances. I adapted to two special situations when designing PPIO.
When a user node repeatedly travels between two different regions, PPIO’s design can provide a great user experience in both places. When the user node is in location A, it finds neighbors in location A. When the user node is in a second location B, it finds neighbors in location B. As a result, its neighbor list or history list contains peers from both locations. If the user node stays in location A for an extended period, its neighbor list will contain more peers from location A, and vice versa. If the user node repeatedly travels back and forth, the list will contain the similar number of peers from location A and B. In this way, when user node uploads data, neighbor miners from both nodes will store copies of the data, so that fast data retrieval can be achieved in both places.
Beside, when a user node permanently relocates to a different region, PPIO provides a reset feature to help with the reschedule. When the reset is initiated, its neighbor list is rebuilt from scratch. New copies are created for existing user data in the network, and these copies are stored by prioritizing the new neighbors. In this way, fast data retrieval can be achieved for previously stored data, as well as new data to be stored to the network.
Through these efforts, we will achieve faster transmission speeds than centralized storage solutions.
Here comes the question？
The figure below shows the user storage data location map.
On the left is AWS S3 centralized storage service, on the right is PPIO’s decentralized storage network Which solution do you think will be faster?