The cannabinoid CB1 receptor is a G protein coupled receptor and plays an important role in many biological processes and physiological functions. A variety of CB1 receptor agonists and antagonists, including endocannabinoids, phytocannabinoids, and synthetic cannabinoids, have been discovered or developed over the past 20 years. In 2005, it was discovered that the CB1 receptor contains allosteric site(s) that can be recognized by small molecules or allosteric modulators. A number of CB1 receptor allosteric modulators, both positive and negative, have since been reported and importantly, they display pharmacological characteristics that are distinct from those of orthosteric agonists and antagonists. Given the psychoactive effects commonly associated with CB1 receptor agonists and antagonists/inverse agonists, allosteric modulation may offer an alternate approach to attain potential therapeutic benefits while avoiding inherent side effects of orthosteric ligands. This review details the complex pharmacological profiles of these allosteric modulators, their structure–activity relationships, and efforts in elucidating binding modes and mechanisms of actions of reported CB1 allosteric modulators. The ultimate development of CB1 receptor allosteric ligands could potentially lead to improved therapies for CB1-mediated neurological disorders.