Crowdsourcing allows organizations and individuals to tap into a large pool of people to accomplish tasks. Recent growth in crowdsourcing has triggered Malaysia to venture into crowdsourcing initiatives.

This study intends to assess the performance of crowdsourcing platforms in Malaysia using the Web of System Performance (WOSP) model.The results give insights on the current state of crowdsourcing platforms in Malaysia and highlight the gaps that exist in these platforms. This eventually help to enhance the design of current platforms and promote higher uptake of crowdsourcing among Malaysians

Ahmad, Rahayu

Chit, Suwannit Chareen

Habbal, Adib M. Monzer

Mahmod, Musyrifah

UUM Repository

Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) Paper No.  231  495  ASSESSING MALAYSIAN CROWDSOURCING PLATFORMS USING WEB OF SYSTEM PERFORMANCE (WOSP) MODEL Adib Habbal, Suwannit Chareen Chit, Rahayu Ahmad and Musyrifah Mahmod School of Computing, Universiti Utara Malaysia, Malaysia,  {adib,chareen,rahayu,musyrifah}@uum.edu.my ABSTRACT. Crowdsourcing allows organizations and individuals to tap into a large pool of people to accomplish tasks. Recent growth in crowdsourcing has triggered Malaysia to venture  into crowdsourcing initia-tives. This study intends to assess the performance of crowdsourcing plat-forms in Malaysia using the Web of System Performance (WOSP) model. The results give insights on the current state of crowdsourcing platforms in Malaysia and highlight the gaps that exist in these platforms. This eventual-ly help to enhance the design of current platforms and promote higher up-take of crowdsourcing among Malaysians. Keywords: crowdsourcing, web of system performance (WOSP) INTRODUCTION The growth of the Internet and modern communication technology has paved ways for the emergence of crowdsourcing. Crowdsourcing allows organizations and individuals to tap into a large pool of people through an open call via the an online platform to perform specific tasks. These tasks include business process operations to increase productivity, as well as menial tasks such as finding the best restaurants in town. Over the years, the concept of crowdsourcing has grown tremendously globally. In Malaysia, efforts have been formulated to help propel the development and use of crowdsourcing platforms.  This is an exploratory study intended to assess the performance of crowdsourcing plat-forms in Malaysia. Four locally developed platforms are assessed using the Web of System Performance (WOSP) model. WOSP is an extension of TAM approach by including system performance based metrics rather than just user perceptions. The findings of this study will highlight the current state of crowdsourcing platforms and suggest the required features to improve the design of future crowdsourcing platforms. BACKGROUND Crowdsourcing is a concept where organizations and individuals outsource tasks common-ly performed by a few experts, to a large crowd through an open call via an online platform. This concept has been reasonably effective and is able to perform tasks at a faster rate (Aris et al., 2013). Crowdsourcing is now widely used to increase productivity and to some extend helps users perform menial tasks such as finding the best restaurants in town (OpenTable, 2015), and seeking alternative routes to avoid traffic jams (Waze, 2015). Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) Paper No.  231  496  Crowdsourcing consists of three components, namely the platform, job providers and workers (Mea et al., 2013). Job providers refer to organizations or individuals who require tasks to be performed. The tasks include translation, transcribing, image tagging, research and report writing, and etc., which must be completed within a specified time ( Aris et al., 2013). The platform (web application), is a medium which facilitates job matching, hiring of workers and other business process activities. The workers (groups or individuals) can search appro-priate jobs that match their qualifications, or through automated job matches.   Malaysia has just started venturing into crowdsourcing, hence there is limited understand-ing on the successful implementation of crowdsourcing. Among the limited studies, Arshad, Salleh, Aris, Janom, & Mastuki, (2013b), for example provided a general analysis on the cur-rent Malaysian crowdsourcing platforms. A more refined analysis is therefore needed to as-sess the system performance of Malaysia’s crowdsoucing platforms.This study will use a performance based model to evaluate local crowdsourcing platforms. Perception based model versus Performance based model Technology Acceptance Model (TAM) has been widely used to gauge user’s acceptance and uptake of technology (Venkatesh, Morris, Davis, & Davis, 2003). Generally, the TAM gauges the acceptance based on a 2-dimensional perspective, namely usefulness (Perception of Usefulness - POU) and ease-of-use (Perception of Ease of Use – POEU). TAM also sug-gests that there exist several factors which influence a user’s decision to uptake and use a specific technology. While studies have validated TAM’s constructs wide range of applica-tions, several studies suggest that the TAM has its shortcomings due to its primary focus sole-ly on user perceptions. As a result, a more system based oriented metric known as Web of System Performance (WOSP) is proposed as enhancement to TAM. WOSP is a theoretical framework used for designing and evaluating advanced information system. The WOSP framework uses four fundamental system elements, namely boundary, internal structure, effectors and receptors to define performance (Whitworth, Fjermestad, & Mahinda, 2006). These four fundamental elements are further linked to eight system design goals. The eight goals can be classified into four “active” properties, which are functionality, flexibility, extendibility, connectivity that could make a system succeed. The remaining four “passive” properties are usability, security, reliability and confidentiality, whose absence can cause detrimental effect to a system.  Details for the fundamental elements of WOSP and its eight design goals are described in the next section.  METHODOLOGY This study evaluates four crowdsourcing platforms in Malaysia industry named YourPart-Time (YourPartTime, 2015), EduScource(EduSource, 2015), MyKerja(MyKerja, 2015), and MyTeksi (MyTeksi, 2015). The first three platforms are proposed by Malaysia Crowdsourc-ing Association as a proof of concepts for Malaysian crowdsourcing. While  MY Teksi, an established platform for taxi booking, is considered a benchmark in this study. Expert review was employed to assess the platforms involving three experts performing different types of tasks for each platform. The assessment is conducted using the following WOSP criteria, as shown in Table 1.    Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) Paper No.  231  497  Table 1. WOSP Criteria Dimension Description Features Functionality To Act Effectively Upon The Envi-ronment. Basic functionality, Job matching, Payment mechanism, Appraisal  Usability To Operate Efficiently Or Easily. Learnability , Efficiency (Task completion time), Naviga-tion (broken links), System Interface Security To Resist Or Avoid Outside Attack Or Take-Over. Password, Random Number, Captcha Image, Verification Code Extendibility To Use Outside Components Or Data. Openness (proprietary), Scalability, Compatibility (run-ning on different platforms), Interoperability (Interaction with other applications) Reliability To Avoid Or Recover From Internal Failure. Internal Failure (Errors), Availability Flexibility To Change To Fit Outer Circum-stances. Adaptability, Customizability, Modifiability Connectivity To Communicate With Other Sys-tems. Communicativeness, Interactivity, Sociability Privacy To Control Internal Information Release. Confidentiality, Secrecy, Social rights, Ownership RESULTS The assessment was based on the eight design goals of WOSP (in Table 2). Basic func-tionality is featured in all platforms. However, a major drawback in these platforms is the lack of job matching facility. Job listings and seeking will be more efficient if web-crawling fea-ture is in place. All platforms do not provide payment facility and left solely to the discretion of both employer and worker.  Table 2. Functionality Assessment of Crowdsourcing Platforms Functionality ( denotes               available) Factors Basic               functionality Job  matching Payment          mechanism Appraisal YourPartTime  None   EduSource  None None NA MyKerja  None None NA MyTeksi   None  As for the usability assessment, all platforms are easy to learn and use (Table 3). Tasks such as job posting and seeking can be accomplished easily, requiring only a few clicks. Most platforms did not comply to the W3C standard by not providing the text resizing module, while some only carry texts in Malay language.  Table 3. Usability of Crowdsourcing Platforms Usability ( denotes available) Factors (ease of use) Learnability Efficiency Navigation Interface YourPartTime    No text resize EduSource    Malay language only MyKerja  None Broken links NA MyTeksi     Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) Paper No.  231  498  In all of the platforms assessed, it was observed that users are required to register and use password for logging in except My Teksi. In general most of the applications lack security measures for preventing unauthorized access as shown in Table 4.  Table 4. Security Assessment of Crowdsourcing Platforms Security ( denotes available) Factors Password Random     Number Captcha Image Verification Code YourPartTime  None None Sent by email EduSource  None None Sent by email MyKerja None  None None None MyTeksi None None None Sent by SMS/voice /email  Table 5 shows that all platforms are proprietary and has low degree of accessibility to view, use, and modify the application. YourPartTime is the only platform that is scalable ena-bled by a widget named Job Ad Network (JANe) that aids job providers to promote their job opportunities. All platforms are compatible with all major browsers and mobile devices. In terms of interoperability, only EduSource allows third party web owners to display their affil-iate program using a script. MyTeksi supports interoperability by using mapquest to automat-ically display routes and locations. Table 5. Extendibility Assessment of Crowdsourcing Platforms Extendibility ( denotes available) Factors Openness Scalability Compatibility Interoperability YourPartTime None   None EduSource None None   MyKerja None None  None MyTeksi None None    Reliability refers to the probability that a required function is carried without failure under stated conditions for a stated period of time. All platforms assessed do not have internal fail-ures which can cause detrimental effects to the system’s performance (Table 6). In terms of availability, all platforms except for MyKerja performed well. Most pages in MyKerja cannot be found and Error 404 is returned. Table 6. Reliability Assessment of Crowdsourcing Platforms Reliability Factors Internal Failure Availability YourPartTime None  EduSource None  MyKerja None None MyTeksi None  Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) Paper No.  231  499  The flexibility of the platforms was assessed and none of the platforms is adaptable, customizable and modifiable (Table 7).  Table 7. Flexibility Assessment in Crowdsourcing Platforms Flexibility Factors Adaptability Customizability Modifiability YourPartTime None None None EduSource None None None MyKerja None None None MyTeksi None None None This study also assessed the platforms for its connectivity (Table 8). This factor refers to the platforms’  ability to interact with users, as well as linking to social media tools. The study shows that only YourPartTime and MyTeksi are linked to social media sites. All platforms do not feature communicativeness and interactivity.  Table 8. Connectivity Assessment in Crowdsourcing Platforms  Connectivity Factors Communicativeness Interactivity Sociability YourPartTime None None  EduSource None None None MyKerja None None None MyTeksi None None  Privacy ensures that personal information cannot be viewed by unauthorized parties. Overall, the privacy concerns are not being addresses by the platforms with the exception of YourPartTime (Table 9).  Table 9. Privacy Assessment of Crowdsourcing Platforms  Connectivity Factors Confidentiality Secrecy Social rights Ownership YourPartTime     EduSource None None None None MyKerja None None None None MyTeksi     DISCUSSION AND CONCLUSION The results of this study suggest that the four crowdsourcing platforms only satisfy the us-ability requirements. This outcome is a typical result if TAM is used as the assessment framework which concentrates only on usability. However, this study based on WOSP gives more insights on the system performance such as security, openness and etc. Our findings indicate that most platforms provide basic functionalities such as job posting and seeking. However, intelligent job matching and secured payment mechanism are still lacking and should be considered. Platforms should also feature strong security mechanisms to address Proceedings of the 5th International Conference on Computing and Informatics, ICOCI 2015 11-13 August, 2015 Istanbul, Turkey. Universiti Utara Malaysia (http://www.uum.edu.my ) Paper No.  231  500  trust issues with payment as well as safeguarding confidential information. Additionally, pri-vacy policies are given less emphasis. Since most of these applications are still in the infancy stages, deliberations on extendibility, connectivity and flexibility have been minimal.  These platforms should also leverage on the power of social media to promote interactions, thus resulting in a higher uptake.  The future work involved in this study covers engagement with developers and users of these platforms to gauge their requirements from such platforms. Coupled with the results of this study, a model to develop a better crowdsourcing platform can be proposed. This would pave ways to a better design of crowdsourcing platforms. REFERENCES Aris, R. S., Mastuki, N., Arshad, N. H., Salleh, S. S., Janom, N., & Aris, S. R. S. (2013). Issues and Challenges in Crowdsourcing Platform Implementation in Malaysia. Applied Computational Science, 95–100. Aris, S. R. S., Janom, N., Arshad, N. H., Salleh, S. S., & Mastuki, N. (2013). A Preliminary Study for Critical Success Factors of Micro Sourcing in Malaysia. 2013 International Conference on Research and Innovation in Information Systems (ICRIIS), 2013, 22–26. doi:10.1109/ICRIIS.2013.6716680 Arshad, N. H., Salleh, S. S., Aris, S. R. S., Janom, N., & Mastuki, N. (2013a). Micro Sourcing Strategic Framework for Low Income Group. International Journal of Advanced Computer Science and Applications, 4(6). Arshad, N. H., Salleh, S. S., Aris, S. R. S., Janom, N., & Mastuki, N. (2013b). Micro Sourcing: The SWOT Analysis on the Demand , Supply and Platforms. Science and Information Conference (SAI), 768–773. EduSource. (2015). Retrieved February 21, 2015, from http://edusource.mmsc.com.my/ Karnouskos, S. (2011). Crowdsourcing information via mobile devices as a migration enabler towards the SmartGrid. In 2011 IEEE International Conference on Smart Grid Communications, 67–72. doi:10.1109/SmartGridComm.2011.6102394 Mea, V. Della, Maddalena, E., & Mizzaro, S. (2013). Crowdsourcing to Mobile Users : A Study of the Role of Platforms and Tasks. In Proceedings of DBCrowd 2013: First VLDB Workshop on Databases and Crowdsourcing, 1,14–19. MyKerja. (2015). Retrieved February 21, 2015, from www.mykerja.com.my MyTeksi. (2015). Retrieved February 21, 2015, from grabtaxi.com/myteksi OpenTable. (2015). Retrieved April 22, 2015, from http://www.opentable.com/start/home Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3). Waze. (2015). Retrieved April 22, 2015, from https://www.waze.com/ Whitworth, B., Fjermestad, J., & Mahinda, E. (2006). The Web of System Performance. Communications of the ACM, 49(5), 93–99. YourPartTime. (2015). Retrieved February 21, 2015, from www.yourparttime.com 

Assessing Malaysian crowdsourcing platforms using web of system performance (WOSP) model

http://repo.uum.edu.my/15604/1/PID231.pdf

Assessing Malaysian crowdsourcing platforms using web of system performance (WOSP) model

Abstract

Similar works

Full text

Available Versions

UUM Repository