Differences

This shows you the differences between two versions of the page.

--- ece4580:module_recognition [2017/04/02 14:20] – [Module #2: Alternative Classifiers ===extract libsvm in a directory of your choosing, say C:\libsvm] typos
+++ ece4580:module_recognition [2024/08/20 21:38] (current) – external edit 127.0.0.1
@@ Line 58: / Line 58: @@
 -------------------------
-===== Module #2: Alternative Classifiers ===extract libsvm in a directory of your choosing, say C:\libsvm==
+===== Module #2: Alternative Classifiers =====
 The vanilla Bag-of-Words algorithm utilizes the //k nearest neighbors// (kNN) algorithm to output the final decision.  As the size of the training set increases, so does the kNN algorithm.  Certainly, we humans don't have that problem.  We don't take longer to classify things even though our corpus of known objects increase from age 1 to age 10 and beyond. kNN doesn't try to distill any inherent structure in the data to simplify the decision process.
@@ Line 89: / Line 90: @@
   - You have to generate two files, one for training and one for testing.
   - Use the commands you learned last week, report your SVM accuracy.
+__**Week #3: Kernel Trick**__ \\
+  - So far we've applied SVM on our dataset to distinguish car and face. We are going to learn more about SVM to see how it works.
+  - If you played with the nice SVM GUI [[https://www.csie.ntu.edu.tw/~cjlin/libsvm/|here]] by generating some points in different colors, you can observe that the separating hyperplanes are not always a straight line. Why is it?
+  - Actually, one of the good properties of SVM is that you can apply different "kernels" to project your data to higher dimension in order to keep them apart and separate them.
+  - Please read carefully about [[http://www.eric-kim.net/eric-kim-net/posts/1/kernel_trick.html|Kernel trick]]. Figure.1 is an example that you can use a straight line to separate two groups. Figure 3. is an example that a straight line is not able to separate them. Can you plot another example that you need a kernel to help you?
+  - Now, figure out how you can change different kernel in libSVM. Write down your answer.
+  - Here we provide an easier dataset [[https://www.dropbox.com/s/740kqcyc6q3cu4u/mnist_49_3000.mat?dl=0|"mnist_4_9_3000"]]. It contains hand-written digit "4" and "9" for you to do binary classification as you did on car and face. Use first 2000 for training and 1000 for testing.
+  - You can use the following command to visualize your data
+  - >> load mnist_49_3000;
+  - >> [d,n] = size(x);
+  - >> i = 1; % index of image to be visualized
+  - >> imagesc(reshape(x(:,i),[sqrt(d),sqrt(d)])’) % notice the transpose
+  - Train SVM as you did before on this 4_9_3000 dataset with linear kernel. Compare the results with different kernels.
+  - Train SVM on your car and face dataset with linear kernel. Compare the results with different kernels. Which dataset has more improvements with different kernels. Why?
+__**Week #4: Cross Validation**__ \\
+  - Besides kernel, another important thing to apply SVM correctly is to select good hyper-parameters.
+  - If you checked SVM GUI [[https://www.csie.ntu.edu.tw/~cjlin/libsvm/|here]], you might notice that different C might lead to different performance.
+  - Please quickly review the SVM material again, and explain what is C parameter here?
+  - One way to select good hyper-parameter is to apply "cross validation".
+  - Please read carefully about [[https://www.cs.cmu.edu/~schneide/tut5/node42.html|Cross Validation]]. The idea is mainly to leave some data untouched, and use it to test your selected parameter. And repeat this step using different untouched portion in your training data.
+  - Now, apply cross validation by using 10% of your data for cross validation. Try it on mnist_49_3000. Test C = 10^-2, 10^-1, 1, 10^1, 10^2, which one gives you best performance?
+  - What is the difference between K-fold and LOO(leave one out)?
 ---------------
 ;#;
 [[ECE4580:Modules|ECE4580 Learning Modules]]
 ;#;