PGEE11136 Applications of Sensor and Imaging Systems
Imaging Systems
项目类别：数学
PGEE11136 Applications of Sensor and Imaging Systems Assignment 3 1. You are given a training set of data points {x(n)}10n=1 where x ∈ R2. Each point has an associated class label y ∈ {0, 1}. These are depicted in Figure 1. (a) Use a k-NN classifier with k = 1 to classify a test point at [ 2 −2]⊤ (1) (b) Use a k-NN classifier with k = 3 to classify a test point at [ 0 2 ]⊤ (1) (c) Use a k-NN classifier with k = 5 to classify a test point at [−2 0]⊤ (1) (d) Use a k-NN classifier with k = 7 to classify a test point at [ 4 −4]⊤ (1) 2. You are given a training set of data points {x(n)}3n=1 where x ∈ R5. Each point has an associated class label y ∈ {1,−1}. x(1) = [−1 2 2 −2 0]⊤ y(1) = −1 x(2) = [ 1 1 0 −1 −1]⊤ y(2) = 1 x(3) = [−2 −4 2 1 0]⊤ y(3) = −1 (a) Use a k-NN classifier with k = 1 to classify a test point at [ 0 1 0 −1 2]⊤ (2) (b) Give an example of something that could be represented with a 5D data point. (1) 4 3 2 1 0 1 2 3 4 x1 4 3 2 1 0 1 2 3 4 x 2 class 0 class 1 Figure 1: A 2D dataset with binary class labels. Page 1 of 4 Continued PGEE11136 Applications of Sensor and Imaging Systems Assignment 3 3. Desmond wants a model that can accurately predict whether a person has a moustache or not from their picture. He assembles a training set and a test set each consisting of images of people with and without moustaches. Desmond trains five different machine learning models using the training set and records the training and test accuracy of each model. He notices that the model with the highest train accuracy and the model with the highest test accuracy are different. He isn’t sure what to do next, so comes to you for advice. What would you tell him? (4) 4. Consider a training set of data points and labels {(x(n), y(n))}4n=1 where x ∈ R and y ∈ {0, 1}: x(1) = −1 y(1) = 0 x(2) = −2 y(2) = 0 x(3) = 2 y(3) = 1 x(4) = 1 y(4) = 1 Now consider a linear classifier which consists of a linear model f(x) = wx+ b and a threshold function that outputs a class prediction yˆ: yˆ = { 1 if f(x) ≥ 0.5 0 if f(x) < 0.5 To learn the parameters of the linear model (w,b) we can minimise the mean squared error loss (MSE) across our training data: LMSE = 1 4 4∑ n=1 (f(x(n))− y(n))2 (a) Plot the training data. Annotate this plot with the classifier’s decision boundary for initial parameters wt=0 = 1 and bt=0 = 1. (2) (b) Determine which side of the decision boundary is allocated to which class, thence classify a test point at x(t) = −0.6. (1) (c) Derive an expression for ∂LMSE ∂w and ∂LMSE ∂b and explain how these may be used in gradient descent to update the classifier’s parameters. (2) (d) Perform a single iteration of gradient descent using your expressions from (c) to compute wt=1 and bt=1. Use a learning rate α = 0.1. (2) (e) Determine where the decision boundary is after this update, and reclassify the test point at x(t) = −0.6. (1) Page 2 of 4 Continued PGEE11136 Applications of Sensor and Imaging Systems Assignment 3 5. Sketch each of the functions below, and determine whether they are convex. (a) f1(x) = |x| (1) (b) f2(x) = e x (1) (c) f3(x) = x 3 (1) (d) f4(x) = { x3 ifx ≥ 0 0 ifx < 0 (1) (e) f5(x) = sinx x (1) 6. Consider the Booth function, which is convex. It takes in a vector θ = [ θ1 θ2 ] and returns a scalar: B(θ) = (θ1 + 2θ2 − 7)2 + (2θ1 + θ2 − 5)2 (a) Derive an expression for ∇θB(θ) (2) (b) Use your answer to (a) to solve minimise θ B(θ) analytically (2) 7. If we have a training set of data point-label pairs {(x(n), y(n))}Nn=1 (x ∈ RD, y ∈ {0, 1}) and a linear model f(x) = [ w⊤ b ] [x 1 ] that outputs the log-odds of some point x belonging to class 1 then the log loss Llog is Llog = − 1 N ∑ n [ y(n) log σ(f(x(n))) + (1− y(n)) log(1− σ(f(x(n)))) ] where σ is the sigmoid function. Let θ = [ w b ] . (a) Show that dσ(a) da = σ(a)(1− σ(a)) for some variable a ∈ R (2) (b) Show that ∂(w ⊤x) ∂w = x (1) (c) Use your answers to (a) and (b) to show that ∇θLlog = − 1 N ∑ n ( y(n) − f(x(n)))[x(n) 1 ] (4) Page 3 of 4 Continued PGEE11136 Applications of Sensor and Imaging Systems Assignment 3 8. Consider a validation set of data points and labels {(x(n), y(n))}4n=1 where x ∈ R2 and y ∈ {c, d}: x(1) = [−1 −1]⊤ y(1) = c x(2) = [ +1 −1]⊤ y(2) = c x(3) = [−1 +1]⊤ y(3) = d x(4) = [ +1 +1 ]⊤ y(4) = d and a classifier consisting of a linear model f(x) = −x1 + x2 + 1 and a threshold function yˆ = { d if f(x) ≥ τ c if f(x) < τ (a) Produce a plot of the validation data in the style of Figure 1 (1) (b) Draw the decision boundary of the classifier when τ = 0 and calculate its vali- dation accuracy (2) (c) Draw the decision boundary of the classifier when τ = 1 and calculate its vali- dation accuracy (1) (d) Is there a value of τ that will give us perfect accuracy on the validation set for this linear model? (1)
