The aim of the current paper was to analyze the process of knowledge growth and scientific discovery by historically reviewing the philosophy of science. After re-assessing inductivism, logical positivism, falsificationism, Kuhn's contribution, Feyerabend's position, Toulmin's model, and Shapere's theory, the following conclusions were derived. First, the so-called "inductive reasoning" is essentially a "hypothesis-testing" process. Second, it should not be expected that important scientific discovery could be achieved simply by mechanistic, mysterious, or accidental process. Third, scientific discovery presupposes scientific creativity. Forth, although there is no logic of scientific discovery, there usually are good reasons for it. In other words, the process of scientific discovery is rational. Fifth, paradigms can facilitate as well as constrain creativity. In order to create new paradigms, learners should be able to understand paradigms, find out counter examples, detect errors, tolerate crisis, and challenge existing paradigms. Sixth, the fundamental process of knowledge growth is basically a dynamic, iterating, bootstrapping, and self-organizing process of creative and critical thinking, i.e., of variation and selection. Some implications of the above conclusions for learning, instruction, and research were also discussed.