MULTI-CORE MAP-REDUCE IMPLEMENTATION FOR EFFICIENT RESOURCE MANAGEMENT

Bathina Narendra Kumar, Banda S N V Ramanamurthy, Rayavarapu Sri Divya

Abstract


Big data analysis is one of the prominent field and its requirement is also increasing due to internet wide spread e-commerce and e-governance. Most of the cases data is unstructured in nature. For example web logs, transaction logs, images, time series data etc. Analyzing such type of data with single system is not possible in general. So Map-Reduce based process is required to work with such big data streams. This paper proposed a solution which is an extension to the existing work proposed in [1] by utilizing modern multi-core functionality for Map-Reduce tasks. We simulate the project using Remote Method Invocation (RMI) using Java. Optimal way of placement of available jobs by reducing the number of jobs in the waiting queue is the major objective of this paper. This can be achieved if the number of jobs requested as well as number of resources they can be utilized are known prior to execution.  But, this model will be failed on dynamic demand. For this purpose a novel framework is required to cater dynamic demands. Proposed system considers this objective also.

Keywords


Map-Reduce; RMI; Big Data; Multi-core functionality; Optimizing Data Grouping and Placement;

References


Xiao, Q. ; Yin, J. and Shang, P.DRAW: A New Data-gRouping-AWare Data Placement Scheme for Data Intensive Applications with Interest Locality. Magnetics, IEEE Transactions on (Volume: 49, Issue: 6 ), Page(s): 2514 - 2520

Narasimhaiah Gorla and Kang Zhang. Deriving program physical structures using bond energy algorithm. In Proceedings of the Sixth Asia Pacific Software Engineering Conference, APSEC '99, pages 359- , Washington, DC, USA, 1999. IEEE Computer Society.

Michael C. Schatz. Cloudburst. Bioinformatics, 25: 1363-1369, June 2009.

Saba Sehrish, Grant Mackey, Jun Wang, and John Bent. Mrap: a novel mapreduce-based framework to support hpc analytics applications with access patterns. In Proceedings of the 19th ACM International Symposium on High Peiformance Distributed Computing, HPDC '10, pages 107-118, New York, NY, USA, 2010. ACM.

Matthias Specht, Renaud Lebrun, and Christoph P. E. Zollikofer. Visualizing shape transformation between chimpanzee and human braincases. Vis. Comput., 23:743-751, August 2007.

Shivam Tripathi and Rao S. Govindaraju. Change detection in rainfall and temperature patterns over india. In Proceedings of the Third International Workshop on Knowledge Discovery from Sensor Data, SensorKDD '09, pages 133-141, New York, NY, USA, 2009. ACM.

Dong Yuan, Yun Yang, Xiao Liu, and Jinjun Chen. A data placement strategy in scientific cloud workftows. Future Gener. Comput. Syst., 26:1200-1214, October 2010.


Full Text: PDF

Refbacks

  • There are currently no refbacks.




Copyright © 2012 - 2020, All rights reserved.| ijitr.com

Creative Commons License
International Journal of Innovative Technology and Research is licensed under a Creative Commons Attribution 3.0 Unported License.Based on a work at IJITR , Permissions beyond the scope of this license may be available at http://creativecommons.org/licenses/by/3.0/deed.en_GB.