Inductive learning
From observation to conclusion.
In my opinion, it is similar to data-driven learning. Most of popular ML algorithms are belong to this.
deductive learning
From conclusion to observation.
In my opinion, it is similar to expert systems. We give rules (conclusion) , and then get the answer ( observation)
transductive learning
In my opinion, the difference between transductive learning and inductive learning, is Transductive learning include all labeled data (x,y) and unlabeled data (x') to predict y' without build the map function f (http://www.kyb.mpg.de/fileadmin/user_upload/files/publications/pdfs/pdf2527.pdf).
Follows are from Quora (https://www.quora.com/What-is-the-difference-between-inductive-and-transductive-learning):
Inductive learning is traditional supervised learning. We learn a model from labeled examples, and we try to predict the labels of examples we have not seen or know about.
Transductive learning is less ambitious. We learn on lots of examples, but we only try to predict on a known (test) set of unlabeled examples.
Transductive learning assumes that even though we don't have the labels of the test set, we have some other information we can take advantage of information.
Transductive learning (also known as semi-supervised learning, in my opinion, Transductive learning is very similar to semi-supervised learning, but it need not build function f) says "Wait: maybe the other 98,000 images don't have labels, but they tell me something about the problem space. Maybe I can still use them to help improve my accuracy. "
Example:
In particular, in an Transductive SVM, this extra information is knowing how many test samples that are positively labeled, and how many are negatively labeled--even though we don't which are which.
active learning
In my opinion, active learning is that our algorithm can timely ask human help what the label of the unlabeled data is? Of course, we human experts tell all of the label, it make nonsense for the algorithm to learn, i.e., algorithm just ask a small part of the unlabeled data, Under this timely small help, active learning algorithm will get a big boost.
One Example to expain above concepts:
Imagine you have a training data, but only a subset of it has labels.
For example, say you are trying to classify whether an image has a flower in it or not. You have 100,000 images, but you only have 1,000 images that you know definitively contain a flower; and another 1,000 that you know don't contain a flower. The other 98,000 you have no idea about -- maybe they have flowers, maybe they don't.
Inductive learning works by looking at the 2,000 labeled examples and building a classifier on this. Transductive learning (also known as semi-supervised learning) says "Wait: maybe the other 98,000 images don't have labels, but they tell me something about the problem space. Maybe I can still use them to help improve my accuracy. "
There's one more really interesting type of learning, which is active learning. That is when you look through the 98,000 examples and can select a subset and request labels from an oracle. So the algorithm might say "OK, of those 98,000, can you label this set of 50 images for me? That will help me build a better classifier."
