Wittgenstein is an odd case. I've always had the impression that his early work, the Tractatus Logico-Philosophicus, is genius—and not only because I find it very difficult to understand. Its form seems like that of an artistic masterpiece, and its profundity, frankly, is probably too deep for me to fully grasp. Not that I think its positions are right. They can't be, inasmuch as they ultimately seem to contradict themselves, as he indeed recognizes in proposition 6.54.
On the other hand, much of Wittgenstein's later work, in the Philosophical Investigations, strikes me as utterly wrongheaded and silly. For one thing, its tendency to behaviorism is wrong. Empiricism, of which behaviorism is an extreme form, is a false philosophy, contradicted, e.g., by cognitive science. Also, I just don't understand why the famous, and bizarre, and difficult-to-parse, arguments on rule-following and the supposed impossibility of a private language are as profound as Wittgenstein devotees always insist. I've briefly discussed the private-language argument elsewhere and go into a little more critical detail below. In fact, as emerges in the following discussion, I suspect Wittgenstein's puzzlement over rules and so forth grew out of his mistaken empiricism; for I don't see how any puzzles arise when we consider that we're following genetically innate rules of language and other cognitive systems when we talk and think. Since empiricism rejects theories of "innateness," Wittgenstein naturally had trouble understanding what it means to follow linguistic and other rules. But surely you have only to adopt a Chomskyan, rationalist, cognitive-science point of view to escape all the confusions and doubts.
(Needless to say, linguistics and cognitive science haven't fully explicated the nature of language or anything else. But I don't see why they can't in principle.)
[From 2009.] Reading volume 2 of Scott Soames’s Philosophical Analysis in the Twentieth Century, a detailed discussion of analytic philosophy from the late Wittgenstein to Kripke.
“One of the themes of the Philosophical Investigations is that terms like ‘think’ and ‘understand,’ which appear to refer to private mental events or processes, should really be understood as standing for complex behavioral and social dispositions, standardly including dispositions to use language.” That’s wrong. Behaviorism is wrong, as Chomsky and others have shown. Besides, it isn’t really either/or, either mental events or behavior. It’s both (though primarily mental events). And that’s the paradox. Understanding and thinking consist mostly in a private mental state but also partly in behavior. My understanding of a math problem consists both in my conscious state—the ‘click’ of understanding—and in my successful completion of the problem. (Chomsky might disagree with that—or he’d add, rightly, that ultimately understanding in this case has to do with the correct application of unconscious computational rules. But does this element apply to all cases of understanding? I haven’t read enough of cognitive science to know much about “computational-representationalism.”)
A few years ago I criticized some of Wittgenstein’s conclusions—or those of his commentators—with respect to rule-following. I was reminded of what I wrote when reading Soames’s summary a moment ago: “In [sections 82 to 85 of the Investigations], Wittgenstein is telling us that our use of words and other symbols is not always guided by rules; sometimes it is simply automatic.” But this automatic quality (I wrote) is precisely how we know we’re following a rule! We may not be able to say what the rule is—it may be something like a series of unconscious computations or whatever, à la Chomsky—but skepticism about rule-following certainly isn’t justified by our so-called “blindness” when following the rules of language. And Soames is wrong to say that “sometimes” our use of words and symbols is automatic; rather, it is always, or at least almost always, more or less automatic. And that’s because we’re being guided by our unconscious cognitive structures and internalizations. We have no choice in the matter. Soames continues: “Moreover, even when we do have a rule in mind that we are attempting to follow, the rule itself is something that, in principle, might be subject to different interpretations. As a matter of simple empirical fact, our interpretations are often instinctive and unthinking, in the way that our interpretation of a directional sign with a finger pointing in one direction is. But even if, in some cases, our interpretation of one rule requires another rule to guide us, at some point the appeal to rules to interpret symbols has to give out, and we will find ourselves using words without thinking. These are cases in which we use words without following rules at all.” No! We don’t have to be explicitly conscious of the exact rule in question in order to be following one. In fact, it’s precisely in those moments when we’re not conscious of what we’re doing that our rule-following is most perfect and complete. As Nietzsche said, animals operate the more perfectly, the less conscious they are.
Wittgenstein is arguing that the referentialist theory of meaning is wrong: a word’s meaning cannot simply be some object to which it refers. (But he also opposes Bertrand Russell’s descriptivism.) The private language argument is a special case of his broader argument: the meaning of an expression cannot consist simply in the (kind of) private sensation it stands for. In order for the word to have meaning, there has to be a public or independent criterion for determining whether it is used correctly. But if its use is entirely private, then whatever seems right to the user is right, which means that there is no real criterion for its correct use. –This was all anticipated by Hegel. A language of “sensuous certainty” is not a genuine language, because in order for there to be communication there has to be conceptual mediation. An unmediated language would consist in brute sounds “referring” to the immediately present: even the expressions ‘here,’ ‘now,’ ‘this’ are not “brute” enough. So, strictly speaking, the meaning of any word in a natural language, even a name, cannot be exhausted in an object to which it refers, because then it is not implicitly being related to other concepts. Saussure said it too: linguistic meaning consists in differentiation between concepts. Such differentiation(s)—i.e., concepts themselves—are precisely the “rules” one is following when using language. Kant said it: a concept is essentially a rule. The rule of its use, and thus the criterion for its correct use, is implicit in the concept itself, which means that any speaker who fully understands the concept understands how to use it. It’s superfluous to seek the criterion for correct use in the linguistic community’s agreement with the speaker’s use (and this is what Wittgenstein does; he insists that others be able to verify that the speaker has used the word correctly), because each language-user is his own community. He is his own other; he has internalized other people’s reactions to him, and this is how he is able to use language. G. H. Mead said it. People take the role of the other when they speak, implicitly anticipating the other’s reactions to their words; hence their ability to judge at every single moment whether they are using words correctly. Language presupposes both conceptual and interpersonal mediation (differentiation) in one consciousness.
Speaking loosely, we might say that the meaning of the name ‘Saul Kripke’ is its referent, the man, or that the meaning of ‘pain’ is a private sensation, but, strictly speaking, the intentional act of positing a reference-relation when using the word isn’t that which solely gives it its meaning. It derives its meaningfulness from its place in the conceptual and interpersonal universe that exists in one’s head and in one’s society. (Kripke’s “rigid designator” theory meets Frege’s descriptivist theory, his sense–reference distinction.)
So, on one weak interpretation at least, Wittgenstein’s private language argument is right, as are many of his ideas about linguistic meaning. To say that a word’s or a sentence’s meaning is just its reference is misleading if you don’t also say its meaningfulness comes from its relations to other expressions, from rules that govern its use, and so on.
—All this is probably meaningless to you if you haven’t read Soames’s book.
I’ve always had a positive view of H. P. Grice and the field of linguistics he founded called pragmatics. A systematic, logically rigorous attempt to develop a sophisticated understanding of how we use language. Conventional and conversational implicature: very important notions. (And conversational impliciture too.) Soames fills out the history a little. (I studied at an analytic department, so history was ignored.)
I have to confess to a certain fascination with Quine’s meaning-eliminativism etc. and Wittgenstein’s thoughts on rule-following etc. I can’t help thinking that certain truths have been hit upon. I’ve discussed Wittgenstein plenty over the years, but I haven’t said much about Quine. He is, if anything, more radical than Wittgenstein, since he denies meaning and reference altogether. In fact, “the same considerations that lead to eliminativism about meaning and reference lead to eliminativism about propositional attitudes such as believing that P, asserting that P, and wondering whether P… Just as there is, in reality, no such thing as ordinary meaning and reference, so there is no such thing as ordinary belief and assertion. Hence, there is no place for these notions in a truly objective and scientific description of the world.” Whatever Quine’s flawed reasons were for adopting these paradoxical positions, I can’t help thinking he was on to something. He merely took it too far. In my paper on him and Kant I tried to sort of articulate what I thought was defensible in his position, but of course more can be said. —By the way, Chomsky would disagree with all this, but I haven’t read enough Chomsky to take him into consideration. Right now I’m concerned with the “phenomenological” level, so to speak, the level of consciousness and behavior.— Meaning, reference, and ‘facts’ about belief, assertion, etc., are not facts in the ordinary commonsense, physical or scientific ways. Obama went to Egypt yesterday. I believe that Céline doesn’t appreciate me. These facts are of two very different kinds—and not just in the obvious ways. The second one actually isn’t even a full-fledged fact: do I truly believe it? Maybe not. Maybe I just think I do, or maybe I’ll change my mind in the next instant. Maybe I don’t think I’m so great after all, maybe I’m deluding myself, maybe I have no idea what I believe about Céline’s attitude towards me, maybe I have no belief whatsoever about her attitude. It seems to me I do, but maybe consciousness is just playing with itself or I’m just playing with words and on some deeper level there is no “belief” coming into play at all. What is belief, anyway? What is intention? What is meaning? A sort of ‘directing’ of consciousness towards some given thing, perhaps combined with a corresponding type of behavior. An ‘investing’ of consciousness in something, a momentary mental commitment to something. But it’s all much more elusive and illusive than ordinary facts like the moon’s revolving around the Earth.
It’s because Quine thinks physical facts are the only genuine facts that he rejects the idea that there is a fact of the matter about meaning, belief and so on in any given case. “Brentano’s thesis [he writes] of the irreducibility of intentional idioms [to non-intentional notions] is of a piece with the thesis of indeterminacy of translation [and thus of meaning].” He goes on: “One may accept the Brentano thesis either as showing the indispensability of intentional idioms and the importance of an autonomous science of intention, or as showing the baselessness of intentional idioms and the emptiness of a science of intention. My attitude, unlike Brentano’s, is the second.” Mine is the first. After all, it’s obvious that Quine’s argument is self-undermining, since if belief, assertion, translation and ultimately meaning are indeterminate, then Quine’s position itself is indeterminate, which means it has no content, which means it is meaningless. If it’s right…then it’s meaningless. We can’t do away with meaning and assertion and truth and all that. But we can confront their elusive, semi-behavioral character.
 [See the "Postscript" below.]
 It’s astonishing to me that professional philosophers haven’t understood this fact, or its implications.
 Quine adopts a complicated strategy to get out of this paradox, but it isn’t convincing.
Postscript: notes from 2006.— You can read about Wittgenstein all day and come away with nothing but a question-mark. All this mumbo-jumbo about conventions, games, linguistic “rules”—Why in the hell have philosophers seized on this “rules” idea like it’s too brilliant for words?!—all this shit cannot be as ingenious as everyone says if it can’t even be coherently expressed! Whenever you read about the “private language argument”—of which, by the way, there are as many interpretations as there are interpreters—you’re assured it has some kind of earth-shaking significance even as the interpreter’s account fails to be minimally intelligible or defensible. Any objection you have to it—to anything Wittgenstein says—is answered with “You’re misunderstanding him!” “But—” “No, you’re misunderstanding him!” Chomsky is right that ideas worth expressing can be expressed in fairly clear language.
[...] “The basic point is that I follow rules ‘blindly’, without any justification for the choice I make” (p. 81 of Kripke's Wittgenstein on Rules and Private Language). This is true in a sense, but misleading. In the cases Kripke has been discussing, we have no choice in the matter. We don’t make an arbitrary, unjustified choice; we are forced to act a certain way. Because we can’t even conceive of an alternative, we’re certain we’re right in our rule-following. A rule, as Wittgenstein says, must guide us; so if whatever seems right to us is right—if, that is, the only criterion is our feeling of certainty—it would appear we aren’t following a rule (because it appears that we aren’t being guided). But he overlooks that we are being guided. Not, indeed, in the choice we make; rather, in the range of possible choices that are laid out for us. And there is no range: when we see a plus sign, we have no choice in the matter of how to interpret it (unless, of course, we self-consciously stipulate it will not mean addition—but in this case we know we are flouting the rule). Thus, we are being guided. Which means we are following a rule. This is why we can speak of ‘right’ here, why there is a fact of the matter. Our rule-following is right because it has been imposed on us, outside of our will; it seems right to us precisely because we feel that we lack a choice—i.e., because we know that, in a different sense of ‘seems’, whatever ‘seems’ right to us is not (necessarily) right.
“To think one is obeying a rule is not to obey a rule. Hence it is not possible to obey a rule ‘privately’; otherwise thinking one was obeying a rule would be the same thing as obeying it.” The first sentence is true on one understanding but not on another. If my obeying is not forced on me, then my thinking I am obeying is indeed not obeying. But if it is forced on me—if I am ‘guided’ into obeying it, in an a priori-ish way—then I am necessarily obeying a rule, and so I think I am and cannot be wrong (because I know I am being forced to act in this way).
Hence, it is possible to obey a rule privately (if I understand that word correctly)—namely, if the rule is a priori. And one reason it may be a priori is if it’s innately programmed into the cognitive architecture of the brain.
But since Wittgenstein was skeptical of the a priori (—a lot of his paradoxical discussions are motivated by familiar empiricist qualms about most things a priori), he was blind to the above arguments.