On the usability of Hadoop MapReduce, Apache Spark & Apache Flink for data science / Bilal Akil, Ying Zhou, Uwe Röhm
- Bib ID:
- 7729597
- Format:
- Book
- Author:
- Akil, Bilal, author
- Online Access:
- Access Conditions:
- National edeposit: Available online.
- Description:
-
- Sydney, NSW : School of IT, University of Sydney, March 2018
- 1 online resource (31 pages)
- ISBN:
- 9781742104324 (PDF)
- Series:
- Technical report (University of Sydney. School of Information Technologies) ; 709.
- Summary:
-
Distributed data processing platforms for cloud computing are important tools for large-scale data analytics. Apache Hadoop MapReduce has become the de facto standard in this space, though its programming interface is relatively low-level, requiring many implementation steps even for simple analysis tasks. This has led to the development of advanced dataflow oriented platforms, most prominently Apache Spark and Apache Flink. Those platforms not only aim to improve performance through improved in-memory processing, but in particular provide built-in high-level data processing functionality, such as filtering and join operators, which should make data analysis tasks easier to develop than with plain Hadoop MapReduce. But is this indeed the case? This paper compares three prominent distributed data processing platforms: Apache Hadoop MapReduce; Apache Spark; and Apache Flink, from a usability perspective. We report on the design, execution and results of a usability study with a cohort of masters students, who were learning and working with all three platforms in order to solve different use cases set in a data science context. Our findings show that Spark and Flink are preferred platforms over MapReduce. Among participants, there was no significant difference in perceived preference or development time between both Spark and Flink as platforms for batch-oriented big data analysis. This study starts an exploration of the factors that make big data platforms more - or less - effective for users in data science.
- Notes:
- Includes bibliographical references.
- Subject:
- Other authors/contributors:
- Also Titled:
- On the usability of Hadoop MapReduce, Apache Spark and Apache Flink for data science
- Copyright:
-
In Copyright
You may copy under some circumstances, for example you may copy a portion for research or study. Order a copy through Copies Direct to the extent allowed under fair dealing. Contact us for further information about copying.
Copyright status was determined using the following information:
- Material type:
- Literary Dramatic Musical
- Published status:
- Published
- Publication date:
- 2018
Copyright status may not be correct if data in the record is incomplete or inaccurate. Other access conditions may also apply. For more information please see: Copyright in library collections.