Specifications Your second coursework for COM61332 will be based on a group project focussed on Social Media Analytics (SMA) enabled by Text Mining. Intended Learning Outcomes ● to develop SMA tools, i.e., text and data processing pipelines that analyse social media or user-generated content ● to answer socio-politico-economic questions by interpreting quantitative results obtained by SMA tools ● to present and report results in the form of potentially publishable short paper Instructions Given the chosen topic for your group, you will answer the research questions that you set out for that topic by developing a Social Media Analytics (SMA) pipeline in R. Before starting to write any program code, however, your first step should be to identify, together with your group mates, the text mining tasks that will help you answer your research questions. For example: 1) What is the dominant sentiment towards Manchester United/City? [Sentiment analysis] 2) Where are fans of Manchester United/City (i.e., people who feel positively towards a team) located geographically? [Sentiment analysis + Geographic mapping] 3) What are the other interests of Manchester United/City fans (e.g., other sports)? [Sentiment analysis + Topic modelling] A suite of pre-developed R modules and resources will be made available to the class in Week 5. This includes code for the following tasks that you can build upon, such as: A. Working with Twitter a. Extracting tweets with specific hashtags or keywords b. Retrieval of user timelines c. Retrieving the followers of a specific user d. Retrieving a list of accounts (with associated metadata) followed by a user e. Visualising a user’s retweet network B. Data cleaning C. Topic modelling D. Sentiment analysis E. Emotion detection F. Visualisation a. Generation of word clouds b. Plotting word frequency over time c. Geographic mapping As part of your group project, you are expected to adapt (i.e., develop your own enhancements to) any of the R modules provided to you, in order to ensure that you can address your research questions adequately. For example, your data set might need additional, custom steps for cleaning, or you might have to explore the use of a domain- specific sentiment lexicon. You also have the choice to implement any modules from scratch, if you think that this will lead to better results. Your final SMA pipeline should be written in R Markdown, which will allow you to document every step in your pipeline, as well as to more easily visualise results. Note that during the Week 5 Laboratory Session, you will be working on data set acquisition and cleaning (specifications to be given as a Lab Exercise, separately). This should allow you to focus on the text and data processing steps from Week 6 until the deadline. Deliverables There are two deliverables for this coursework: 1. Social media analytics pipeline implemented in R. This should come in the form of an R Markdown file containing well-documented R code for each step in the pipeline, as well as a folder containing all of the resources required by the code (e.g., data sets). 2. Short paper reporting results. This should be in the form of research paper (rather than a technical report/user manual), something that can potentially be published (e.g., on ResearchGate, if not in a workshop). A sample paper2 published in The Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS-2018) can be used as a guide. Avoid plagiarism and discuss ideas in your own words. Template: ACL Conference format3 Recommended length: 2-3 pages (in two-column format), excluding references. Timeline 21 February Submission of group topic proposals 25 February Feedback on topic proposals given back to students Final decision on group project topic Lab exercises exploring SMA modules in R for data set acquisition/cleaning 27 March Submission of deliverables (by 17:00, via Blackboard) 2 https://bit.ly/2tuCUXy 3 http://acl2020.org/downloads/acl2020-templates.zip Marking scheme This coursework accounts for 25% of your final mark for COMP61332, and is worth 100 points. The following rubric will be used in marking your group project, where the first column specifies the various criteria and the second column indicates the maximum number marks your group can be possibly given. Note that everyone in your group will get the same mark, so it is your responsibility to ensure that tasks are delegated fairly and that there are equal contributions. SMA Pipeline Functionality 0 The SMA pipeline is not functional. 5 The SMA pipeline is functional but was not carefully designed to ensure that all of the research questions set out by the group during the proposal stage are answered. 10 The SMA pipeline is functional and was designed to answer all of the research questions set out by the group during the proposal stage. Adaptations 0 There was no attempt to incorporate any enhancements into the R modules provided to the class. 5 Some adaptations and enhancements were added to the R modules provided to the class, although these seem trivial/limited (i.e., not much thought or effort was put into developing these). 10 Sufficient adaptations and enhancements were added to the R modules provided to the class. Documentation 0 The R Markdown code does not contain any form of documentation (e.g., in-line comments, descriptions). 5 The R Markdown code contains some documentation but not enough to clearly explain any adaptations/enhancements incorporated into the pipeline (e.g., how these were implemented or how they improve the pipeline). 10 The R Markdown code contains documentation that clearly explains any adaptations/enhancements to the pipeline (e.g., how these were implemented or how they improve the pipeline). Short paper Academic writing 0 The short paper looks like a technical report/user manual rather than a research paper. 5 The short paper was written for an academic audience. However there are some ideas which were not clearly presented, or it seemed like the discussion lacks originality/argumentation. 10 The short paper was written for an academic audience and can potentially be published in a research workshop or symposium. Ideas were presented in a clear and well-argued manner. Background and introduction 0 The paper does not provide an introduction to the proposed topic (e.g., why it is interesting) and does not clearly present the analytical questions that the SMA project seeks to answer. 5 The paper provides an introduction to the proposed topic and presents the analytical questions that the SMA project seeks to answer. However, the motivation for the choice of topic/questions is not argued well enough nor very convincing. 10 The paper provides an introduction to the proposed topic and presents the analytical questions that the SMA project seeks to answer. The motivation for the choice of topic/questions is well-argued and convincing. Review of related work 0 The paper does not provide any review of related work. 5 The paper provides a review of related work although there are some shortcomings, e.g., it not clear how these relate to the group’s own work, or some of the mentioned work is outdated/more recent work could have been reviewed. 10 The paper provides a good review of related work, showing awareness of recent relevant efforts. How these relate to the group’s own work was clearly presented. Methodology 0 The paper does not provide sufficient details on the group’s methodology (including steps for data collection . 5 The paper provides details on the group’s methodology (including steps for data collection), although some parts need further elaboration. 10 The paper provides sufficient and clear details on the group’s methodology (including steps for data collection). References 0 The references included in the paper are not sufficient to support the group’s arguments. These were cited and added without following the recommend
ed style/format. 5 Sufficient references were included in the paper. However they were cited and provided in a slightly inconsistent style. 10 Sufficient references were included in the paper, cited and provided in a consistent style. Interpretation of results (should be included as a Results or Discussion section in the paper) Analysis and interpretation 0 Quantitative results were obtained by the group’s SMA pipeline but were not analysed and interpreted in order to answer the research questions set out during the topic proposal stage. 5 Quantitative results obtained by the group’s SMA pipeline were analysed in order to answer the research questions set out during the proposal stage, but in some parts the interpretation seems exaggerated (or are not aligned with the results). 10 Quantitative results obtained by the group’s SMA pipeline were adequately interpreted, allowing the group to answer the research questions they set out during the proposal stage. Exemplification/ visualisation 0 The group did not provide any examples nor make use of visualisation to evidence their analysis and interpretation of quantitative results. 5 The group provided examples and made use of visualisation, however some of these do not support or are not aligned with their findings/interpretation. 10 The group explained their findings and interpretation by providing supporting examples or making use of suitable visualisation.
