Demonstration of ultra-low energy switching mechanisms is an imperative for continued improvements in computing devices. Ferroelectric (FE) and multiferroic (MF) orders and their manipulation promises an ideal combination of state variables to reach atto-Joule range for logic and memory (i.e., ~ 30X lower switching energy than nanoelectronics). In BiFeO3 the coupling between the antiferromagnetic (AFM) and FE orders is robust at room temperature, scalable in voltage, stabilized by the FE order, and can be integrated into a fabrication process for a beyond-CMOS era. The presence of the AFM order and a canted magnetic moment in this system causes exchange interaction with a ferromagnet such as CoFe or LSMO. While previous work has shown that exchange coupling (uniaxial anisotropy) can be controlled with an electric field, several puzzling issues remain. Perhaps the most intriguing among them is that the BiFeO3-CoFe bilayer did not demonstrate any electrically controlled directional anisotropy, i.e., an exchange bias, which is a potential mechanism for 180o magnetic reversal and is independent of switching kinetics. However, what is needed/preferred for logic and memory is a magneto-electric mechanism that works analogous to an applied field, i.e a uni-directional anisotropy that is voltage modulated. Here, we present the evidence of electrical control of exchange bias of a laterally scaled spin valve that is exchange-coupled to BiFeO3 at room temperature. We show that the exchange bias in this bilayer is thermally robust, electrically controlled and reversible. We anticipate that magneto-electricity at such scaled dimensions provides a powerful pathway for computing beyond the modern nanoelectronics transistors by enabling a new class of non-volatile, ultra-low energy computing elements.