基于Hadoop的云存储系统的设计与研究
Design and research of cloud storage system based on Hadoop
-
摘要: 针对海量数据的存储和处理,设计了一个基于Hadoop的云存储系统.该系统在分布式文件系统和MapReduce编程模型2个核心技术的基础上建立基于Hadoop的云存储模型,优化了存储方式,提高了集群中网络带宽和磁盘的利用率,同时MapReduce编程框架的设计使系统拥有更强的计算能力.该系统可通过Linux集群技术搭建Hadoop平台,进行测试和分析.应用实践表明,该系统具有低成本、高效率、易扩展和安全可靠等特点,能稳定高效地满足海量数据的处理要求.Abstract: Aiming at the problem of mass data storage and processing, a cloud storage system based on Hadoop was designed.The system established a cloud storage model on the basis of studying the two core technologies of Hadoop including Hadoop distributed file system and programming model MapReduce.The system optimized the computer storage mode,and increased the efficiency of the network bandwidth and disk in the cluster.At the same time,the MapReduce programming framework design made the system have higher performance of computing ability.Through testing and analysis of Hadoop platform using Linux cluster technology, the results showed that the system had the characteristics of low cost, high efficiency, easy extension, safty and reliability, and could meet the requirement of the mass data processing stably and efficiently.
-
Key words:
- cloud storage /
- Hadoop /
- distributed parallel computing /
- HDFS /
- MapReduce
-
-
[1]
Boutaba R,Cheng L,Zhang Q.On cloud computational models and the heterogeneity challenge[J].Journal of Internet Services and Applications,2012,3(1):77.
-
[2]
周品.Hadoop云计算实战[M].北京:清华大学出版社,2012.
-
[3]
翟岩龙,罗壮,杨凯,等.基于Hadoop的高性能海量数据处理平台研究[J].计算机科学,2013,40(3):100.
-
[4]
马林山,赵庆峰,肖新国.基于Hadoop的云移动信息服务模型研究[J].情报科学,2013,31(4):28.
-
[5]
Bass L,Kazman R,Ozkaya I.Open Source Systems:Grounding Research[M].Berlin:Springer,2011:50-61.
-
[6]
亢丽芸,王效岳,白如江.MapReduce原理及其主要实现平台分析[J].现代图书情报技术,2012(2):60.
-
[7]
李寒,唐兴兴.基于参数优化的Hadoop云计算平台[J].计算机系统应用,2013,22(3):21.
-
[1]
-
点击查看大图
计量
- PDF下载量: 31
- 文章访问数: 1026
- 引证文献数: 0
下载:
