Affiliations: [a] School of Modern Posts and Institute of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing, China
| [b]
University of Amsterdam, Amsterdam School of Economics, Amsterdam, Netherlands
Correspondence:
[*]
Corresponding author. Jinsong Xu, School of Modern Posts and Institute of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing 210003, China. E-mail:[email protected].
Abstract: The current massive data storage and sharing algorithm based on file system and NIOT have not considered the heterogeneous and real-time characteristics of data in distributed and heterogeneous environment, which has the problems of high storage capacity, high time delay of data sharing, and poor security of data. To address this problem, massive data storage and sharing algorithm in distributed heterogeneous environment is proposed in this paper. Hierarchical analysis of distributed heterogeneous environment is carried out. By using the file system, database technology, and hybrid technology, with the heterogeneous and real-time characteristics of massive data in distributed heterogeneous architecture, a three-level storage framework is obtained. Distributed storage algorithm is applied to save massive data. Through HBase-based massive data sharing algorithm in distributed heterogeneous environment, based on data integration mode of data warehouse, massive data sharing is achieved based on blackboard system to reduce data sharing delay. The node encryption unit is used to improve data security. Experimental results show that the proposed algorithm can reduce the occupancy rate of storage capacity, with stable storage speed and low sharing time delay. It can play a good role for data security.
