COMP3419 Graphics and Multimedia Assignment Option 2 (Semester 2, 2019) Option 2: Visual Content Analysis (VCA) 1 Key Information • The mark of “COMP3419 Assignment Option 2: Visual Content Analysis (VCA)” will be given as two parts, namely the canvas submission and the live demonstration: – Canvas Submission [Due Time] before 23:59 Sunday of Week 11 (2019-10-27). – Live Demonstration [Demo Time] timetable-allocated lab session on Week 12 (2019- 10-28). • This individual assignment is worth 18% of your final assessment. • Sample Data (VCA Dataset.zip) can be downloaded from the assessment information page on Canvas. • Submission Deliverables: You are asked to create a zip file of all deliverables, including a project report and the source code. A README txt file (to describe the steps/instructions regarding how to get your source code running to derive the expected output) is suggested if you find it helpful for the marker to get familiar with your submission. • Your assignment will only be marked if all deliverables can be accessed from the Canvas System, and they can be runnable from a lab machine. Once plagiarism detected by the Canvas system, the student will receive no mark immediately, as well as other related penalties from university. 2 General Marking Policy Demonstration Rules: During the demonstration, students may be asked to provide tutor with explanations on their solutions if necessary. If you can not finish all these requirements below, you could provide a live demo for the workable parts to seek for some partial marks awarded. Late Submission & Demonstration Policy: • For the late submission cases, penalties will be assigned according to the university wide late penalties for assignment Clause 7A of the Assessment Procedures. • Late demonstration will not be allowed for this assignment. For students being absent for the on-site live demonstration, tutors would still mark their zip file submitted to Canvas with a 3 Introduction 3 penalty of 5 marks received. Special Consideration and Arrangements: While you are studying, there may be circumstances or essential commitments that impact your academic performance. Our special consideration and special arrangements process is there to support you in these situations. More information on how to lodge the special consideration application, can be found from this webpage. 3 Introduction The aim of this individual assignment (Option 2) is to develop and evaluate methods for image segmentation. The objective is to segment all buildings as indicated in the ground truth masks. The segmentation results are expected to be evaluated using accuracy, dice and IoU metrics. If learning-based approaches are developed, a cross-validation training and testing process should be applied. 4 Dataset Description This dataset (VCA Dataset.zip) contains 5 high-resolution remote sensing images collected from 5 different areas of the earth, respectively, namely, Austin, Chicago, Kitsap Country, Vienna and West Tyrol. Each image is of size 5000 by 5000, capturing 2.25 km2 surface space from these five different areas. Corresponding pixel-wise segmentation ground truths are also provided in this dataset. Figure 1: An input satellite image collected at Vienna for segmentation (left) and the corresponding segmentation ground truth mask (right) for buildings. R Note: This VCA dataset is a selected subset from the Inria Aerial Image Labeling Dataset, which released 36 remote sensing high-resolution tiles for each of these five areas on earth mentioned above (i.e., 180 satellite images and their corresponding annotated ground truths). The full dataset is not compulsory for this assignment. For students interested in fully exploring the performance of their algorithms designed, you are very welcome to send an email to our teaching team to gain this full dataset. 45 Method Design Students are expected to develop an automatic method for this visual content analysis task. Its pipeline might include, but is not limited to morphological processing, feature extraction + classifier training, deep learning, or a combination of these various types of methods. You can use any open-source libraries, such as Pytorch, scikit-learn, Keras and Tensorflow. You are expected to fine-tune your method, complete performance comparison, and discuss the significance of your approach proposed. Some ablation studies should be designed to evaluate the performance gain, resulting from each individual components proposed in your approach and/or other attempts made, such as data augmentation techniques adopted, data preprocessing steps implemented and etc. You can get some ideas about method design from the research papers uploaded on Canvas (Reference_Papers.zip). These papers are well-selected to present a variety of method designs and levels of complexity. You are welcome to talk to your tutor during the “W09-Special Consultation” session, to discuss about your method development plan. Their feedback and guidance provided could ensure that your plan proposed can contain appropriate scope and complexity. Also, you might find “W09-Lecture” helpful for the method design and getting a deeper understanding on the image segmentation problem domain. 6 Project Report The report should contain introduction, methods, experimental setup, results, discussion, and conclusions. It should be 5 – 10 pages, maximum 15 pages (single column, 1.5 line spacing, Word or PDF). A brief guideline of the report sections is as follows: • Introduction: introducing the project aim, methods and findings • Methods: presenting the details of all methods developed, including brief description of method theories and design choices • Experimental setup: describing the dataset and evaluation metrics • Results & Discussion: presenting the evaluation results of each method, including evalua- tion of main design choices, and if applicable presenting results from combining various methods. Based on the experiment results, insightful discussion is expected to demonstrate the corresponding analysis performed. Tables and figures are preferred to be used for result demonstrations. • Conclusions: summarising the study and findings • References: listing literatures and other references (papers and/or online resources) R Demonstration: Students are required to prepare a highlighted oral summary of this image segmentation project to their tutors, during the timetable-allocated lab session on Week 12. You are welcome to follow the style of Three Minute Thesis (3MT) if you like. R Deliverables: The deliverables for Option 2 should include all the files (source code and project report) used to generate the experiment results stated in your report, with a README file to describe how to run these files in order to derive your output. All the corresponding files should be compressed as a zip file for Canvas Submission. 7 Appendix 5 R Three Bonus Marks: 1 bonus mark will be awarded for students who developing their 2 nd approach to tackle this computer vision problem and presenting the corresponding comparison results between these methods developed. An extra 1 bonus mark will be awarded for students who developing more than two approaches. The last 1 bonus mark will be awarded for students who using LATEX(www.overleaf.com) to complete their project reports. 7 Appendix Online Computational Resource:Your can either work locally or use Web-based Ipython notebooks such as Google Colaboratory to gain a better computational resources (CPU, GPU, etc). COMP3419 (Semester 2, 2019)
