Instructor: L.-P. Chen SS9850B Assignment #2 (Due April 3 2020) Please carefully read the following instructions: • There are 3 questions in this assignment. 4 marks for each subquestion. • Show all your works in details and provide your code for computations. Also, well summarize your numerical results in each question. • This assignment is an INDIVIDUAL work. Plagiarism will earn ZERO mark. • ONLY R functions/packages mentioned in this course are allowed. Using other functions/packages that are not used in this course will lose marks. • Submit your solutions to the Drop Box in the course site. Delayed submission will lose marks. 1 1. (Wine Data Set) These data are the results of a chemical analysis of wines grown in the same region in Italy but derived from three different cultivars. The analysis deter- mined the quantities of 13 constituents (including Alcohol, Malic acid, Ash, Alcalinity of ash, Magnesium, Total phenols, Flavanoids, Nonflavanoid phenols, Proanthocyanins, Color intensity, Hue,OD280/OD315 of diluted wines, and Proline) found in each of the three types of wines. The sample size is 178. The dataset is available in the course site. The main interest of this dataset is to study multiclassification of the three types of wines. Let ŷ denote the predicted class of observations. (a) Instead of estimating Σ (or Σi) by empirical estimates and then take an inverse, here we consider to use glasso to estimate Θ , Σ−1 (or Θi , Σ−1i ) directly. After that, replacing the estimator Θ̂ (or Θ̂i) to the linear discriminant function (or quadratic discriminant function) gives graphical-based linear discriminant analysis (or quadratic discriminant analysis). In this question, use the graphical-based linear discriminant analysis and quadratic discriminant analysis to obtain ŷ. In addition, summarize the confusion table for y and ŷ, use macro averaged metrics to evaluate recall, precision, F-measure, and then conduct performance of classification. (b) Use the support vector machine method to obtain ŷ. In addition, summarize the con- fusion table for y and ŷ, use macro averaged metrics to evaluate recall, precision, F- measure, and then conduct performance of classification. (c) Suppose that we only consider the predictor Proline. Use the nonparametric density estimation method in Section 4.1 to explore the multiclassification. For the choices of kernel function, we examine the Gaussian kernel and the biweight kernel; for the bandwidth selection, we use the cross-validation (CV) method. After obtaining ŷ, summarize the confusion table for y and ŷ, use macro averaged metrics to evaluate recall, precision, F-measure, and then conduct performance of classification. ## Hint : programming code o f CV func t i on : J=function (h){ f ha t=Vecto r i z e ( function ( x ) density (X, from=x , to=x , n=1,bw=h)$y ) f h a t i=Vecto r i z e ( function ( i ) density (X[− i ] , from=X[ i ] , to=X[ i ] , n=1,bw=h)$y ) F=f h a t i ( 1 : length (X) ) return ( i n t e g r a t e ( function ( x ) fhat ( x)ˆ2 ,− In f , I n f )$value−2∗mean(F) ) } (d) Summarize your findings in (a)-(c). 2 2. (Simulation studies) Consider the following linear model: y = X1β1 +X2β2 +X3β3 +X5β5 − 5√ρX6β6 +X7β7 + , (1) where X = (X1, · · · , Xp) is a p-dimensional vector of covariates and each Xk is generated from N(0, 1). The correlations of all Xk except X6 are ρ, while X6 has the correlation √ ρ with all other p− 1 variables. Suppose that the sample size is n = 200. (a) Show that X6 is marginally independent of y. (b) Now, consider p = 1000 and generate the artificial data based on model (1) for 1000 repetitions. Specifically, let βi = 1 for every i = 1, · · · , 7 and set ρ = 0.8. After that, use the SIS and iterated SIS methods to do variable selection and estimate the parameters associated with selected covariates. Finally, summarize the estimator in the following table: Table 1: Simulation result for (b) ‖∆β‖1 ‖∆β‖2 #S #FN SIS Iterated SIS (c) Here we consider the scenario that is different from (b). Let p = 50 and X ∼ N(0,ΣX) with entry (j, k) in ΣX being 0.6 |j−k| for j, k = 1, · · · , p. We generate the artificial data based on (1) for 1000 repetition with βi = 1 for every i = 1, · · · , 7. After that, use the lasso, adaptive lasso, and Elastic net (set α = 0.5) methods to estimate the parameters. Finally, summarize numerical results in the following table. Table 2: Simulation result for (c) ‖∆β‖1 ‖∆β‖2 #S #FN lasso adaptive lasso Elastic net (α = 0.5) (d) Summarize your findings for parts (b) and (c), respectively. Note: Let β̂ be the estimator, then ∆β is defined as ∆β = β̂ − β with the ith component being β̂i − βi. Therefore, ‖∆β‖1 and ‖∆β‖2 are defined as • ‖∆β‖1 = p∑ i=1 ∣∣∣β̂i − βi∣∣∣; • ‖∆β‖2 = √ p∑ i=1 ( β̂i − βi )2 . 3 3. In the class, we have discussed that the estimator of Θ can be obtained by solving Θ−1 − S − λΦ = 0, where S and Ψ are defined in the note. (a) Let W denote the working version of Θ−1 such that WΘ = I, where I is the identity matrix. Show that W11β − s12 − λψ12 = 0, (2) where β = − θ12θ22 , and θ12, θ22, s12, and ψ12 are defined in the note. (b) Suppose that β̂ is obtained by solving (2). Please show that θ̂12 = −β̂ · θ̂22 and θ̂22 = (w22 − w>12W−111 w12)−1. Also, interpret the meaning of θ̂12. 4 Hint: Regarding simulation studies with 1000 repetitions. In Question 2, you are asked to use simulation studies with 1000 repetitions to estimate the parameters. Specifically, based on the kth artificial data that are independently generated, you are able to obtain the estimator, denoted by β̂(k). As a result, with 1000 repetitions, the final estimator is given by β̂ = 11000 1000∑ k=1 β̂(k). 5
