(1) Maximum the margin SVM is very easy to understand on the graph,, we just need to find the a separate plane which maximum the margin. see the graph below: (2) How to calculate/find the max Margin Assuming hard-margin issue for the simplicity of math, the separate plane can be expressed as: w*x -b = 0 where • Read More »

The big picture is: a quadratic programming problem can be reduced to be a linear programming problem. Here is how: (1) KTT conditions For any non-linear programming: max: f(x), s.t: g(x) <=0 It has been proved that it needs to meet Karush–Kuhn–Tucker (KKT) conditions provided that some regularity conditions are satisfied how it is being proved? it is • Read More »

While searching a command line to write something into a UNIX Domain Socket, I found this great tool called socat which is similar to netcat/samplicator but has more features. The man page said: Socat is a command line based utility that establishes two bidirectional byte streams and transfers data between them. Because the streams can • Read More »

Why study the linear programming (LP) ? LP has a lot of use cases, one of them is the SVM ( support vector machine). The SVM ‘s Lagrangian dual can give the lower bound of SVM, this Lagrangian dual can be solved by quadratic programming. The KKT conditions of this quadratic programming can be solved by • Read More »

GDB cheatsheat There is a good gdb cheatsheet at http://darkdust.net/files/GDB%20Cheat%20Sheet.pdf The output format for print, x is missing there, somehow I think it is useful to add the output formats: x,d,u,o,t,a,f,s,z,r please see https://sourceware.org/gdb/onlinedocs/gdb/Output-Formats.html How do we usually run gdb commands normally we compile the source code with CPPFLAG -g ( please no -O, otherwise some code are • Read More »

In logistic regression, we just assume the probability of x to be classified as 1 is : P( y = 1 | x ) = 1 / ( 1 + exp ( -w^T x) ) = hw(x) w is the parameter vector that we need to learn and optimize from the training sets. This is • Read More »

Bayes theorem: where A and B are events and P(B) ≠ 0. P(A) and P(B) are the probabilities of observing A and B without regard to each other. P(A | B), a conditional probability, is the probability of observing event A given that B is true. P(B | A) is the probability of observing event B given that A • Read More »

Decision tree works just like computer language if. In AI/ML world, the problem is usually like this: Given training set with features [( f1,f2 ….), ….] and known category/label [c1, ….], how can we learn from this training set/data and design a decision tree , so that for any new data, we can predict which • Read More »

The std::unique_lock class is a lot more flexible when dealing with mutex locks. It has the same interface as std::lock_guard but provides additional methods for explicitly locking and unlocking mutexes and deferring locking on construction As a general rule, std::lock_guard should be preferably used when the additional features of std::unique_lock are not needed use cases: • Read More »