Olfactory hallucinations are not that uncommon. Sacks mentions that they can be caused simply by vivid memory; perhaps relatedly, people with PTSD have them fairly often.

Sacks hears about hallucinations a lot because, I suspect, he specifically asks people about them (which most doctors don't) and presents his questions in a non-pathologizing way, I would guess with a side of "isn't the human brain fascinating?"