Why We’re Streaming Media? Part II: The Secrets of PPIO’s Data Delivery Technology

Our follow up article further explores the technology that we’re creating which gives developers the ability to stream media .

There is no question that audio and video applications reign supreme. From the latest Netflix drama to streaming our favorite songs, just about every one of us is entertaining ourselves on a daily basis. Such is the popularity that the perfect time for blockchain technology to make a commercial landing is now. Our previous article about the secrets of our media streaming technologyprovided an in-depth and technical understanding of how PPIO uses PCDN architecture for content distribution. In the second part of our series, we will discuss the download logic of P2SP, the traffic count of the IaaS layer, and how both can be employed to help your future decentralized applications.

Download Logic

The figure above shows the transmission architecture of PPIO’s CDN and P2SP. The P2SP download logic can be divided into three distinct parts: Buffer Management, Download State Machine, and the download algorithm.

Buffer Management

Buffer management means a local device manages the resource situation and thus determines the priority of the Piece that needs to be downloaded. Buffer management is also related to real-life application scenarios. An example in terms of streaming media is preferential downloads of content at and near the video playback location. When we want to watch that part of the video, Buffer Management ensures we have access to that part.

1. Buffer Management Logic of File Download

The technical aspect of Buffer Management for file downloads is relatively simple. When there is no one specific playback location, the urgency level of downloadable content has no priority; therefore Buffer Management will download the scarcest Piece in the network. The scarcer the Piece is, the higher the priority of its download. As shown below:

As depicted above, we assume that all Peers are the same, but because of CDN nodes, the actual situation is much more complicated. CDN can be considered as being a high-quality node but is better suited for downloading continuous data rather than requesting a scattered data fragment. The reason why scarcity is used as a Buffer Management index for ordinary files is to increase the resource health of P2P networks and to allow data to spread in P2P networks as soon as possible.

2. Buffer Management Logic of Streaming Media

Different from file downloads, streaming media downloads have a specific playback location. In order to ensure the quality of the video viewing experience, the data which is closest to the playback location will be preferentially downloaded.

When designing PPIO, we divide streaming media into multiple intervals according to the video playback location during the download process. The closer to the playback location, the higher the download priority.

• Interval Before Playback: The data, which is located before the video playback location, does not need to be downloaded.
• Urgent Interval: Data that needs to be played immediately will use an extreme download strategy to acquire data as quickly as possible. The Piece which is closer to the playback location has the higher download priority. This strategy is more dependent on the CDN and the SuperNode. Despite the risk of repeated downloads, the interval will simultaneously conduct P2P downloads, which means immediately downloading data from multiple peers.
• Normal Download Interval: Data that needs to be played later. The Piece which is closer to the playback location has the higher download priority. The download rule is almost the same as the Urgent Interval level despite the data not being needed as urgently. The only difference is that this strategy does not rely too much on CDN nodes and will try to avoid repeated downloads when conducting P2P downloads.
• Long-term Interval: The data whose playing time is later than the Normal Download Interval. This part is similar to file downloads which follow the logic of the scarcest Piece being downloaded first. The data download for this interval does not depend on the CDN node as much as it could.

Download State Machine

What is the state machine? It is a way to download data based on the Buffer state. There are two ways of downloading data: one is P2S, which is downloading data from the CDN; the other one is P2P, which is downloading data from other peers. So how can the two methods be switched during the download process? This is the problem that can be solved by the download state machine.

1. Download State Machine Design for File Downloads

The state machine for ordinary file downloading evaluates at anytime the download quality and resource status of CDN and P2P, and switches between the three download states until the file is completely downloaded.

2. Download State Machine Design for Media Streaming Downloads

Although the streaming media state machine is similar to the file download state machine, it is slightly improved. When it is determined that the P2P resources are insufficient to support smooth playback, the P2P download mode is directly abandoned, and no further state switching is attempted to avoid unnecessary costs.

Download Algorithm

The download algorithm is an algorithm that assigns a different Piece, which is waiting to be downloaded to multiple different peers. This algorithm is different when using file downloads and streaming media downloads.

1. File Downloads

The scheduling algorithm of PPIO is roughly as follows:

1. For more efficient data transmission, the download node establishes multiple connections with each Peer that owns the resource. The connection here is not limited to the TCP protocol, but also has the session logic implemented by the UDP protocol. PPIO is compatible with multiple protocols at the same time; switching different protocols in different network environments to cope with the heterogeneity of the network.
2. For each Peer, we perform speed predictions based on previous situations. If there is no history recorded, we will use a default value based on historical experience.
3. According to Buffer Management, it confirms the priority of each Piece downloading, then randomly initiates a Piece data request to other peers through the connection, before sending the Piece’s data immediately after receiving the request.
4. Once a connection receives a Piece, our algorithm will update the connection’s speed prediction immediately and will request the next Piece that has not been downloaded according to Buffer Management. This repeats until the download is complete.
5. If the Piece data request does not get a response for a long time, it will request data from other Peers. The Peer that has not responded for a long time will be punished. This penalty may reduce the number of Peer connections, thereby decreasing the Peer download weight.

According to our experience of designing high-speed P2P networks in the past, PPIO should adopt multiple virtual connection downloads for each Peer. This virtual connection supports switching between various protocols. The underlying protocol will try UDP first. If UDP fails it will try TCP. This method significantly improves the transmission efficiency. We have implemented it this way because the TCP protocol was previously designed many years ago for an earlier version of the Internet. Naturally, TCP has a weakness, which is that the congestion control algorithm uses the logic of active retreat. This results in inefficient transmissions and is not entirely suitable for today’s network transmissions.

2. Streaming Media Downloads

PPIO provides an optimized P2P transmission system for streaming media downloads. To guarantee the best watching experience, any data downloaded must be performed in real time and must cope with the instability of a P2P network. Hence, we have used our data-driven P2P download technology, and based on this concept, we made a significant improvement and optimization, including designing a P2P multi-point scheduling system based on pre-allocation. This system can achieve maximum download rates and minimize spending due to duplicate transmissions and requests. The PPIO streaming media scheduling algorithm is designed as follows:

1. Just like the file downloader, the algorithm creates multiple connections for each peer and records the historical download speed of each virtual connection.
2. As covered earlier in our Buffer Management section, the Piece is downloaded according to urgency. The algorithm then pre-allocates them to each connection and predicts the arrival time of each Piece. In general, it will prioritize the most urgent Piece.
3. For the Piece that has not completed the download, the algorithm will pre-allocate them again in a timely manner in order to adapt to the change of the transmission speed of each connection.
4. When each connection receives a Piece, the speed of the channel is updated, the next Piece is then found from the pre-allocated list of the channel, and finally, the algorithm sends a request.
5. For a very urgent Piece (one that requires immediate playback), the algorithm is allowed to request data from multiple peers at the same time.

PPIO’s P2P transmission network is entirely dynamic and can continue to be efficient.

What we have mentioned so far is only about traditional P2SP technology. For those more interested in the blockchain side of things, how does all this relate to PPIO’s blockchain? The content introduced above is built around PPIO’s PaaS layer and AS layer (Application Services layer). The P2SP download engine of the PCDN is supported at the PaaS layer. Streaming media formats and protocols, as well as data splitting and data assembly, is at the AS layer. It is on the IaaS layer that the entire decentralized storage and blockchain infrastructure can be found.

Traffic Count and IaaS Layer

The design of PPIO is different from a traditional P2P transmission project. A traditional P2P project advocates a shared network based on the premise that “everyone is for me, I am for everyone.” The download is “free” on the transmission, and the upload is unpaid. In the case of PPIO, we have established an incentive network mechanism. Users need to pay for downloadable resources. The resource uploader will charge a certain amount of money, i.e., the downloader must pay Tokens to the uploader.

In the IaaS layer of PPIO, to ensure the efficiency of the P2P network, it is not necessary for each Peer to process the payment and settlement of the Token. To address this, we have designed the Owner mechanism. The Owner is not a P2P transmission role, but a payment and settlement role. In the PCDN architecture, each Peer needs to specify an Owner. The cost of this Peer is borne by his Owner, and the income earned by the Peer is also received by his Owner.

If it is on the demand side, it can be understood as a “CPool.” We have previously covered our proxy payment gateway, in our article “Why PPIO Designed Coin Pool Nodes”). If it is on the supply side, it can be understood as a “mine pool.” For PCDN services, the cost of using bandwidth is generally borne by the content publisher.

The most straightforward process when a user wants to download data from a miner is:

1. Owner 1 puts a deposit into a smart contract.
2. The User requests a check (like a bank check) from Owner 1. The check will contain the address of the smart contract, the max amount the voucher will pay, the CheckID, the expiration time, and so on.
3. The Owner reviews whether the check request meets his requirements or not. If it meets the requirements, the check is returned to the User; if it does not match, for example, if it applies for too many tokens, it can be rejected.
4. The User requests data from the Miner (via the P2P data transfer protocol) and successfully receives data.
5. The User will expand the check to a voucher (similar to credentials) and will send it to a Miner. The process in steps 4 and 5 should be repeated several times, and the amount involved in the voucher will continue to increase.
6. No matter if the download finishes or is aborted, the voucher with the max recorded amount (which is also the latest recorded amount) will be submitted to the Miner’s Owner — Owner 2.
7. Owner 2 receives the voucher and initiates a withdrawal to the blockchain smart contract.
8. If the Voucher is correct and has not expired., then the withdrawal will be successful.

For the developer of a PCDN service, you can either deploy the CPool (the Owner) or access other third-party CPools (also the Owner in this situation). In addition, if the downloader and the uploader’s Owner are the same, then the situation will become simpler. This transaction does not need to be conducted via the chain and can be solved by the Owner without the need for external parties. No matter the situation, the amount of money that the Owner will receive will not change. In this way, PPIO effectively combines P2P transmission and blockchain token economics.

Of course, this is just one of the more straightforward billing processes in the IaaS layer in PPIO, and the actual situation is much more complicated. In the future, we will publish more articles that discuss PPIO’s PCDN-related smart contracts.

It is hard to deny that the streaming media field has great potential for many blockchain applications. PPIO will combine its rich P2P experience and cutting-edge blockchain technology to bring a better media streaming experience for everyone. Although we are only in the beginning stages, the potential for great things with our technology is huge. Watch this space as we’re about to do some great things. We’d love for you to be part of it!

Want more? Join the PPIO community on Discord or follow us on Twitter.