Differences

This shows you the differences between two versions of the page.

--- ece4580:module_recognition [2017/04/04 20:31] – [Module #2: Alternative Classifiers] typos
+++ ece4580:module_recognition [2024/08/20 21:38] (current) – external edit 127.0.0.1
@@ Line 90: / Line 90: @@
   - You have to generate two files, one for training and one for testing.
   - Use the commands you learned last week, report your SVM accuracy.
+__**Week #3: Kernel Trick**__ \\
+  - So far we've applied SVM on our dataset to distinguish car and face. We are going to learn more about SVM to see how it works.
+  - If you played with the nice SVM GUI [[https://www.csie.ntu.edu.tw/~cjlin/libsvm/|here]] by generating some points in different colors, you can observe that the separating hyperplanes are not always a straight line. Why is it?
+  - Actually, one of the good properties of SVM is that you can apply different "kernels" to project your data to higher dimension in order to keep them apart and separate them.
+  - Please read carefully about [[http://www.eric-kim.net/eric-kim-net/posts/1/kernel_trick.html|Kernel trick]]. Figure.1 is an example that you can use a straight line to separate two groups. Figure 3. is an example that a straight line is not able to separate them. Can you plot another example that you need a kernel to help you?
+  - Now, figure out how you can change different kernel in libSVM. Write down your answer.
+  - Here we provide an easier dataset [[https://www.dropbox.com/s/740kqcyc6q3cu4u/mnist_49_3000.mat?dl=0|"mnist_4_9_3000"]]. It contains hand-written digit "4" and "9" for you to do binary classification as you did on car and face. Use first 2000 for training and 1000 for testing.
+  - You can use the following command to visualize your data
+  - >> load mnist_49_3000;
+  - >> [d,n] = size(x);
+  - >> i = 1; % index of image to be visualized
+  - >> imagesc(reshape(x(:,i),[sqrt(d),sqrt(d)])’) % notice the transpose
+  - Train SVM as you did before on this 4_9_3000 dataset with linear kernel. Compare the results with different kernels.
+  - Train SVM on your car and face dataset with linear kernel. Compare the results with different kernels. Which dataset has more improvements with different kernels. Why?
+__**Week #4: Cross Validation**__ \\
+  - Besides kernel, another important thing to apply SVM correctly is to select good hyper-parameters.
+  - If you checked SVM GUI [[https://www.csie.ntu.edu.tw/~cjlin/libsvm/|here]], you might notice that different C might lead to different performance.
+  - Please quickly review the SVM material again, and explain what is C parameter here?
+  - One way to select good hyper-parameter is to apply "cross validation".
+  - Please read carefully about [[https://www.cs.cmu.edu/~schneide/tut5/node42.html|Cross Validation]]. The idea is mainly to leave some data untouched, and use it to test your selected parameter. And repeat this step using different untouched portion in your training data.
+  - Now, apply cross validation by using 10% of your data for cross validation. Try it on mnist_49_3000. Test C = 10^-2, 10^-1, 1, 10^1, 10^2, which one gives you best performance?
+  - What is the difference between K-fold and LOO(leave one out)?
 ---------------
 ;#;
 [[ECE4580:Modules|ECE4580 Learning Modules]]
 ;#;