5/2/2020 Lab 8 | CSCI 1320 https://cs.brown.edu/courses/csci1320/labs/lab8/lab8.html 1/6 CSCI 1320 LAB 8: TESTING DUE MONDAY, MAY 4 10AM Testing a web application that will be used by the external world is an integral and necessary step towards success. For this lab, you will be put your final project under various types of testing. Testing allows you to answer many questions about your program: Does it work correctly? Does it work correctly under all cicumstances? Is it accessible to a wide range of users? Does it work across devices and browsers? Will it scale? You are not trying to prove that your program has no bugs, but rather vigorously finding them via thorough analysis. In that way, testing cannot prove that your entire program is right, but simply what is not wrong. The difference is subtle, but the more rigorous your testing is, the closer you will be to a flawless program. To be a good tester, you should have a mindset that is concentrated on breaking things and finding edge cases. This will ensure you will find and patch some of the holes that are lurking in your application. This lab provides the resources necessary for you to test the project you have been working on this semester. This lab is your chance to get started and get help. You should work with your final project team to test applicable aspects for your project. INTRODUCTION For the lab, you will be testing your own final project with your fellow group members. Importantly, you should never test production data, data that will be served or used externally. This means that if your program is serving data from a database, you should not run tests that directly modifies that database. The main reason for this is to prevent unwanted changes and manipulations to data used publically. PREPARATION 5/2/2020 Lab 8 | CSCI 1320 https://cs.brown.edu/courses/csci1320/labs/lab8/lab8.html 2/6 Thus, the first task of this lab is to prepare your data from your final project for testing. This will probably mean making a copy of your database (or a subset of data from your database). If your program depends on functionality from integrating with various APIs, such as using Paypal or Stripe for payment purposes, be sure to check the relevant testing documentation provided by each service. Testing is such an important feature that most reliable APIs will provide documentation and testing environments for developers to thoroughly ensure their integrations are running smoothly. We have provided a couple links in next section for some documentation on testing various services that might be used in some final projects, but this list is not complete. Be sure to find the correct documentation for any integrations you may have in your project. If you have trouble with this, reach out to a TA for assistance. The last preparation task you need to complete is to communicate with your group members about dividing the different types of testing detailed below. You must thoroughly test all categories of your program. For each testing category below: determine whether it is applicable to your application If it is not applicable, explain why you think this category is not applicable. If it is applicable, choose at least one of the resources to test your application. You are welcome to use resources not listed to test your application but make sure to explain why you think the tool you used is a good addition. Make sure every team member is involved and work with at least one category of testing. In each category, we use blue background to highlight the tool we recommend. Web WAVE Common Errors: Missing Form Labels : This error happens when a form control does not have a corresponding label. For example, when you have a element by its own. You can fix this problem by supplying a label. If you do not want the label to be actually displayed, you can consider making it screen-reader only . Empty Button : This error happens when a button is empty or has no value text. For example, when you only put a icon inside a button. You can fix this problem by supplying screen-reader only text . CynthiaSays is an alternative website Accessibility scan tool. It requires your application to be online (deployed) and usually takes longer than WAVE to generate a report. However, this site covers some different accessibility categories and can be used as a supplement to WAVE. AChecker . This tool checks single HTML pages for conformance with accessibility standards. FAE : Functional Accessibility Evaluator. toptal : Color blind views of your page. Mobile A11y Ally enables inspecting Android applications’ accessibility. ACCESSIBILITY 5/2/2020 Lab 8 | CSCI 1320 https://cs.brown.edu/courses/csci1320/labs/lab8/lab8.html 3/6 XCode Accessibility Inspector: Getting Started Tutorial Application testing (also known as E2E testing) is about testing the components of the system as a whole, such as how well the web page interfaces with the web server, the server with the database, and the web server with the user server. Web Chrome Devtools Audit (Lighthouse) is an open-source, automated tool for improving the quality of web pages. Selenium : Automates web browser for testing. Selenium is more low-level compared to the following e2e libraries. Cypress Nightwatch.js uses the W3C WebDriver API to drive browsers in order to perform commands and assertions on DOM elements. Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol. TestCafe : Write tests in JS or TypeScript, run them and view results Mobile Detox supports both native apps and React Native projects. Appium is an open source test automation framework for use with native, hybrid and mobile web apps. It drives iOS, Android, and Windows apps using the WebDriver protocol. Appium is more low-level compared to the following e2e libraries. Kobiton supports real device testing on both Android and iOS devices. TestProject is a free end-to-end test automation platform for web, mobile, and API testing APPLICATION TESTING Back end functional testing (or often called unit testing) is about checking if requests are handled correctly and that the proper actions are taken and the proper pages are generated. The specific tools used depends on the technology used in the back end. Feel free to pick one of the tools below, depending on what language your back end uses. Node: Jest , Mocha , Jasmine that also works for Node Python: PyUnit PHP: SimpleTest BACKEND TESTING Compatibility testing is about testing if the web application is compatible across many types of browsers, such as IE, Firefox, Safari, Chrome, Opera, Edge, etc. and across mobile platforms and operating systems. COMPATIBILITY TESTING 5/2/2020 Lab 8 | CSCI 1320 https://cs.brown.edu/courses/csci1320/labs/lab8/lab8.html 4/6 Web With a free trial on browserstack.com , you can test your locally hosted site–just choose a browser to test, click the settings icon, and check the box under Server Testing labeled “Resolve all URLs through my network.” You can use browsershots.org if your site is already hosted! You can also manually test out core functionalities of your application by trying it in different browser (Chrome, Safari, Firefox, Edge), operating system (MacOS, Windows, Linux) combinations. Mobile Kobiton supports real device testing on both Android and iOS devices. TestProject is a free end-to-end test automation platform for web, mobile, and API testing Front end functional testing is about testing the correctness of the HTML, CSS, and JavaScript, which includes aspects like checking if links are correct and active, if forms are validated correctly and have default values, if the site handles cookies correctly (e.g. the application works without cookies, cookies are encrypted correctly, and sessions expire correctly). HTML W3C HTML Validator CSS W3C CSS Validator Links W3C Link Validator JavaScript Jest was built and is constantly maintained by the team at Facebook. It is a zero-configuration javascript testing framework recommended by React. Mocha provides great freedom and flexibility. Jasmine very beneficial for frontend testing. QU
nit originally developed for testing jQuery, jQuery UI and jQuery Mobile, it is a generic framework for testing any JavaScript code. Mobile (Overview) Android: Test apps on Android IOS: Testing with Xcode SauceLabs is a live, automated and continuous testing for web & mobile apps on cloud-based test platform (2 week free trial) Appium is an open source test automation framework for use with native, hybrid and mobile web apps. It drives iOS, Android, and Windows apps using the WebDriver protocol. Appium is more low-level compared to the following e2e libraries. FRONTEND TESTING If your application involves payment, follow the testing documentation of the third-party payment tool you used. PAYMENT TESTING 5/2/2020 Lab 8 | CSCI 1320 https://cs.brown.edu/courses/csci1320/labs/lab8/lab8.html 5/6 Stripe PayPal Square Performance testing is about testing the responsiveness of a web application, the time taken for common actions. It includes load testing and network performance, as well as stress testing the limits of the application. Web Performance Tab of Chrome Devtools. GTmetrix : Gives insight on how well your site loads and provides actionable recommendations on how to optimize it (requires deployment) . JMeter : Designed to load test functional behavior and measure performance. It works with a set of test cases where you can specify the quantity and delays. A fairly complicated and involved system. Mobile IOS: Instruments Tutorial Android: Measure UI Performance Cross Platform: DynTrace (offers free trial), PERFORMANCE TESTING Security testing is about testing the different ways that security can be breached, including URL security, input checking, and other methods. Web Wapiti is a command-line tool that performs “black-box” scans (it does not study the source code) of the web application by crawling the webpages of the deployed webapp, looking for scripts and forms where it can inject data. OWASP Zed Attack Proxy (ZAP) ( Getting Started , Downloads ) is a multi-platform, open source web application security testing tool useful for penetration test. Netsparker is a leading web application security scanner (signup needed). Mobile OWASP Zed Attack Proxy (ZAP) ( Use ZAP to test mobile app , Downloads ) is a multi-platform, open source web application security testing tool useful for penetration test. Mobile Security Framework (MobSF) is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing, malware analysis and security assessment framework capable of performing static and dynamic analysis. Quick Android Review Kit is designed to look for several security related Android application vulnerabilities, either in source code or packaged APKs. SECURITY TESTING USABILITY 5/2/2020 Lab 8 | CSCI 1320 https://cs.brown.edu/courses/csci1320/labs/lab8/lab8.html 6/6 Usability is about testing the effectiveness of the user interface, including accessibility and internationalization testing. It’s about seeing what is liked or disliked about the interface, as well as the speed and ease of use, and whether it is understandable, logical, and easy to follow. This can be done through user studies or log studies. You should use bdognom-v2.cs.brown.edu:5002 to submit a user test and complete at least one user test. Moreover, in addition to complete user testing task for class, you can either ask your friends to test your application (similar as what you did for user feedback report) or use online user testing platform like UserTesting . Here are a few helpful resources for you to get started on using UserTesting: The quickest step-through of using usertesting.com Check out this demo video which will cover what the UserTesting platform does and the basics of setting up a study Fill out this planning worksheet to plan a study Complete guide to user testing Once you have set up an account, further information can be found here: How to launch a desktop study How to launch a mobile website study How to launch an app study How to launch a prototype study Once your team are done testing the various components of your project, one member from the group has to handin a testing report to Gradescope with every team member included in the submission group. The testing report should be a PDF document outlining the following: Basic Information: your final project name, your mentor ta name, your team member name and CS Login Collaboration: Which team member is responsible for which testing categories. Testing Report: for each testing category, describe the following Category Name Which tools are used? Which tests were run as part of the testing? Why you choose these tests? Why do you believe these tests are sufficient? What results you receive and how do you interpret these results? Steps taken to fix the bugs Look at Handin Instructions . HAND IN
