Who can ignore the fact that self-driving cars will be on our roads, whether we like it or not?
It's not just about the autopilot: new discoveries from engineering, psychology, maths and more
Abstract: Publication date: Available online 10 December 2016 Source:Neural Networks Author(s): Jihun Kim, Jonghong Kim, Gil-Jin Jang, Minho Lee Deep learning has received significant attention recently as a promising solution to many problems in the area of artificial intelligence. Among several deep learning architectures, convolutional neural networks (CNNs) demonstrate superior performance when compared to other machine learning methods in the applications of object detection and recognition. We use a CNN for image enhancement and the detection of driving lanes on motorways. In general, the process of lane detection consists of edge extraction and line detection. A CNN can be used to enhance the input images before lane detection by excluding noise and obstacles that are irrelevant to the edge detection result. However, training conventional CNNs requires considerable computation and a big dataset. Therefore, we suggest a new learning algorithm for CNNs using an extreme learning machine (ELM). The ELM is a fast learning method used to calculate network weights between output and hidden layers in a single iteration and thus, can dramatically reduce learning time while producing accurate results with minimal training data. A conventional ELM can be applied to networks with a single hidden layer; as such, we propose a stacked ELM architecture in the CNN framework. Further, we modify the backpropagation algorithm to find the targets of hidden layers and effectively learn network weights while maintaining performance. Experimental results confirm that the proposed method is effective in reducing learning time and improving performance.
Pub.: 17 Dec '16, Pinned: 13 Jan '17
Abstract: The detection of small road hazards, such as lost cargo, is a vital capability for self-driving cars. We tackle this challenging and rarely addressed problem with a vision system that leverages appearance, contextual as well as geometric cues. To utilize the appearance and contextual cues, we propose a new deep learning-based obstacle detection framework. Here a variant of a fully convolutional network is used to predict a pixel-wise semantic labeling of (i) free-space, (ii) on-road unexpected obstacles, and (iii) background. The geometric cues are exploited using a state-of-the-art detection approach that predicts obstacles from stereo input images via model-based statistical hypothesis tests. We present a principled Bayesian framework to fuse the semantic and stereo-based detection results. The mid-level Stixel representation is used to describe obstacles in a flexible, compact and robust manner. We evaluate our new obstacle detection system on the Lost and Found dataset, which includes very challenging scenes with obstacles of only 5 cm height. Overall, we report a major improvement over the state-of-the-art, with relative performance gains of up to 50%. In particular, we achieve a detection rate of over 90% for distances of up to 50 m. Our system operates at 22 Hz on our self-driving platform.
Pub.: 20 Dec '16, Pinned: 11 Jan '17
Abstract: The emergence of in-vehicle entertainment systems and self-driving vehicles, and the latters' need for high-resolution, up-to-date maps, will bring a further increase in the amount of data vehicles consume. Considering how difficult Wi-Fi offloading in vehicular environments is, the bulk of this additional load will be served by cellular networks. Cellular networks, in turn, will resort to caching at the network edge in order to reduce the strain on their core network – an approach also known as mobile edge computing, or “fog computing”.
Pub.: 09 Dec '16, Pinned: 20 Dec '16
Abstract: Authors: Hao Li ; Kun Fu ; Menglong Yan ; Xian Sun ; Hao Sun ; Wenhui Diao Article URL: http://www.tandfonline.com/doi/full/10.1080/2150704X.2016.1258127?ai=z4&mi=3fqos0&af=R Citation: Remote Sensing Letters Publication Date: 2016-12-07T04:04:22Z Journal: Remote Sensing Letters
Pub.: 07 Dec '16, Pinned: 19 Dec '16
Abstract: Named Data Networking (NDN) has been recently proposed as a prominent solution for content delivery in the Internet of Vehicles (IoV), where cars equipped with a variety of wireless communication technologies exchange information aimed to support safety, traffic efficiency, monitoring and infotainment applications. The main NDN tenets, i.e., name-based communication and in-network caching, perfectly fit the demands of time- and spatially-relevant content requested by vehicles regardless of their provenance. However, existing vehicular NDN solutions have not been targeted to wisely ensure prioritized traffic treatment based on the specific needs of heterogeneous IoV content types. In this work, we propose a holistic NDN solution that, according to the demands of data traffic codified in NDN content names, dynamically shapes the NDN forwarding decisions to ensure the appropriate prioritization. Specifically, our proposal first selects the outgoing interface(s) (i.e., 802.11, LTE) for NDN packets and then properly tunes the timing of the actual transmissions. Simulation results show that the proposed enhancements succeed in achieving differentiated traffic treatment, while keeping traffic load under control.
Pub.: 22 Nov '16, Pinned: 19 Dec '16
Abstract: Abstract This paper investigates the problem of fuel-efficient and safe control of autonomous vehicle platoons. We present a two-part hierarchical control method that can guarantee platoon stability with minimal fuel consumption. The first part vehicle controller is derived in the context of receding horizon optimal control by constructing and solving an optimization problem of overall fuel consumption. The Second part platoon controller is a complementation of the first part, which is given on the basis of platoon stability analysis. The effectiveness of the presented platoon control method is demonstrated by both numerical simulations and experiments with laboratory-scale Arduino cars.AbstractThis paper investigates the problem of fuel-efficient and safe control of autonomous vehicle platoons. We present a two-part hierarchical control method that can guarantee platoon stability with minimal fuel consumption. The first part vehicle controller is derived in the context of receding horizon optimal control by constructing and solving an optimization problem of overall fuel consumption. The Second part platoon controller is a complementation of the first part, which is given on the basis of platoon stability analysis. The effectiveness of the presented platoon control method is demonstrated by both numerical simulations and experiments with laboratory-scale Arduino cars.
Pub.: 17 Nov '16, Pinned: 15 Dec '16
Abstract: Publication date: January 2017 Source:Transportation Research Part F: Traffic Psychology and Behaviour, Volume 44 Author(s): Lionel Bringoux, Jocelyn Monnoyer, Patricia Besson, Christophe Bourdin, Sébastien Denjean, Erick Dousset, Cédric Goulon, Richard Kronland-Martinet, Pierre Mallet, Tanguy Marqueste, Cécile Martha, Vincent Roussarie, Jean-François Sciabica, Anca Stratulat Although discrete auditory stimuli have been found useful for emergency braking, the role of continuous speed-related auditory feedback has not been investigated yet. This point may though be of importance in electric vehicles in which acoustic cues are drastically changed. The present study addressed this question through two experiments. In experiment 1, 12 usual drivers were exposed to naturalistic auditory feedback mimicking those issued from electric cars, while facing dynamic visual scenes in a 3D driving simulator. After being passively travelled up to a sustained constant speed, subjects had to stop their car in front of a traffic light that unexpectedly turned to red. Modifications of the speed-related auditory feedback did not impact braking initiation and regulation. In experiment 2, synthesized auditory feedback based on the Shepard-Risset glissando was provided to a new sample of 15 usual drivers in the same task. Pitch variations of this acoustic stimulus, although not scaled to an absolute speed, were manipulated as a function of visual speed changes. Changing the mapping between pitch variations of the synthesized auditory feedback and visual speed changes induced adjustments on braking which depended on acceleration/deceleration feedback. These findings stressed the importance of the acoustic content and its dynamics for car speed control.
Pub.: 20 Nov '16, Pinned: 15 Dec '16
Abstract: This paper presents a Grammar-aware Driver Parsing (GDP) algorithm, with deep features, to provide a novel driver behavior situational awareness system (DB-SAW). A deep model is first trained to extract highly discriminative features of the driver. Then, a grammatical structure on the deep features is defined to be used as prior knowledge for a semi-supervised proposal candidate generation. The Region with Convolutional Neural Networks (R-CNN) method is ultimately utilized to precisely segment parts of the driver. The proposed method not only aims to automatically find parts of the driver in challenging “drivers in the wild” databases, i.e. the standardized Strategic Highway Research Program (SHRP-2) and the challenging Vision for Intelligent Vehicles and Application (VIVA), but is also able to investigate seat belt usage and the position of the driver's hands (on a phone vs on a steering wheel). We conduct experiments on various applications and compare our GDP method against other state-of-the-art detection and segmentation approaches, i.e. SDS , CRF-RNN , DJTL , and R-CNN  on SHRP-2 and VIVA databases.
Pub.: 02 Dec '16, Pinned: 05 Dec '16
Abstract: Abstract The possibility of using data from multiplex networks on vehicles in road tests, in the development of intelligent transportation systems, and in control systems for autonomous (self-driving) vehicles is considered.AbstractThe possibility of using data from multiplex networks on vehicles in road tests, in the development of intelligent transportation systems, and in control systems for autonomous (self-driving) vehicles is considered.
Pub.: 01 Oct '16, Pinned: 05 Dec '16
Abstract: With the development of state-of-art deep reinforcement learning, we can efficiently tackle continuous control problems. But the deep reinforcement learning method for continuous control is based on historical data, which would make unpredicted decisions in unfamiliar scenarios. Combining deep reinforcement learning and safety based control can get good performance for self-driving and collision avoidance. In this passage, we use the Deep Deterministic Policy Gradient algorithm to implement autonomous driving without vehicles around. The vehicle can learn the driving policy in a stable and familiar environment, which is efficient and reliable. Then we use the artificial potential field to design collision avoidance algorithm with vehicles around. The path tracking method is also taken into consideration. The combination of deep reinforcement learning and safety based control performs well in most scenarios.
Pub.: 01 Dec '16, Pinned: 05 Dec '16
Abstract: In this paper we introduce the TorontoCity benchmark, which covers the full greater Toronto area (GTA) with 712.5 $km^2$ of land, 8439 $km$ of road and around 400,000 buildings. Our benchmark provides different perspectives of the world captured from airplanes, drones and cars driving around the city. Manually labeling such a large scale dataset is infeasible. Instead, we propose to utilize different sources of high-precision maps to create our ground truth. Towards this goal, we develop algorithms that allow us to align all data sources with the maps while requiring minimal human supervision. We have designed a wide variety of tasks including building height estimation (reconstruction), road centerline and curb extraction, building instance segmentation, building contour extraction (reorganization), semantic labeling and scene type classification (recognition). Our pilot study shows that most of these tasks are still difficult for modern convolutional neural networks.
Pub.: 01 Dec '16, Pinned: 05 Dec '16
Abstract: The objective of this study was to evaluate the effectiveness of forward collision warning (FCW) alone, a low-speed autonomous emergency braking (AEB) system operational at speeds up to 19mph that does not warn the driver prior to braking, and FCW with AEB that operates at higher speeds in reducing front-to-rear crashes and injuries. Poisson regression was used to compare rates of police-reported crash involvements per insured vehicle year in 22 U.S. states during 2010-2014 between passenger vehicle models with FCW alone or with AEB and the same models where the optional systems were not purchased, controlling for other factors affecting crash risk. Similar analyses compared rates between Volvo 2011-2012 model S60 and 2010-2012 model XC60 vehicles with a standard low-speed AEB system to those of other luxury midsize cars and SUVs, respectively, without the system. FCW alone, low-speed AEB, and FCW with AEB reduced rear-end striking crash involvement rates by 27%, 43%, and 50%, respectively. Rates of rear-end striking crash involvements with injuries were reduced by 20%, 45%, and 56%, respectively, by FCW alone, low-speed AEB, and FCW with AEB, and rates of rear-end striking crash involvements with third-party injuries were reduced by 18%, 44%, and 59%, respectively. Reductions in rear-end striking crashes with third-party injuries were marginally significant for FCW alone, and all other reductions were statistically significant. FCW alone and low-speed AEB reduced rates of being rear struck in rear-end crashes by 13% and 12%, respectively, but FCW with AEB increased rates of rear-end struck crash involvements by 20%. Almost 1 million U.S. police-reported rear-end crashes in 2014 and more than 400,000 injuries in such crashes could have been prevented if all vehicles were equipped with FCW and AEB that perform similarly as systems did for study vehicles.
Pub.: 30 Nov '16, Pinned: 30 Nov '16
Abstract: Of the many technologies being explored to address sustainability and environmental issues, electric cars are considered to be the most promising alternative to vehicles powered by IC engines. This paper studies the instability control of electric vehicles propelled by permanent magnet synchronous motors (PMSMs). The nonlinear characteristics of a surface-mounted PMSM model are studied under three different assumed driving conditions. To mitigate undesirable dynamic instabilities including hyperchaotic responses that are frequented at low and high speeds, so as to extend the operating range of the PMSM system, a novel control scheme that exerts simultaneous control in both the time and frequency domains is developed and subsequently validated. The control approach has its foundation established in discrete wavelet transformation and adaptive control. Its physical implementation consists of an adaptive controller and an adaptive filter both implemented in the wavelet domain. Numerical results demonstrate the effectiveness of the controller design in restoring PMSM instability with low-amplitude limit-cycle in response to a properly specified reference signal. Of the many technologies being explored to address sustainability and environmental issues, electric cars are considered to be the most promising alternative to vehicles powered by IC engines. This paper studies the instability control of electric vehicles propelled by permanent magnet synchronous motors (PMSMs). The nonlinear characteristics of a surface-mounted PMSM model are studied under three different assumed driving conditions. To mitigate undesirable dynamic instabilities including hyperchaotic responses that are frequented at low and high speeds, so as to extend the operating range of the PMSM system, a novel control scheme that exerts simultaneous control in both the time and frequency domains is developed and subsequently validated. The control approach has its foundation established in discrete wavelet transformation and adaptive control. Its physical implementation consists of an adaptive controller and an adaptive filter both implemented in the wavelet domain. Numerical results demonstrate the effectiveness of the controller design in restoring PMSM instability with low-amplitude limit-cycle in response to a properly specified reference signal.
Pub.: 01 Dec '16, Pinned: 28 Nov '16
Abstract: Improper functioning of traffic signals at the intersections result in extreme congestion leading to increase in overall journey time and wastage of precious fuel. Various algorithms have been proposed in literature for alleviating the problem of congestion. Fixed-time, non-preemptive and preemptive approaches work towards reduction of queue length at the intersections to decrease the overall waiting time on roads. High traffic volume on the road results in large queue length which takes huge amount of time to process using a single processor. Hence, there is a need for fast processing which can be obtained by parallelizing the algorithm.
Pub.: 21 Nov '16, Pinned: 25 Nov '16
Abstract: In the last years, and thanks to improvements on computing and communications technologies, wireless networks formed by vehicles (called vehicular networks) have emerged as a key topic of interest. In these networks, the vehicles can exchange data by using short-range radio signals in order to get useful information related to traffic conditions, road safety, and other aspects. The availability of different types of sensors can be exploited by the vehicles to measure many parameters from their surroundings. These data can then be shared with other drivers who, on the other side, could also explicitly submit queries to retrieve information available in the network. This can be a challenging task, since the data is scattered among the vehicles belonging to the network and the communication links among them have usually a short life due to their constant movement.
Pub.: 17 Nov '16, Pinned: 25 Nov '16
Abstract: Vehicular traffic monitoring is a major enabler for a whole range of Intelligent Transportation System services. Real time, high spatial and temporal resolution vehicular traffic monitoring is becoming a reality thanks to the variety of communication platforms that are being deployed. Dedicated Short Range Communications (DSRC) and cellular communications like Long Term Evolution (LTE) are the major technologies. The former is specifically tailored for Vehicular Ad-hoc Network, the second one is pervasive. We propose a fully distributed Floating Car Data (FCD) collection protocol that exploits the heterogeneous network provided by DSRC and LTE. The proposed approach adapts automatically to the penetration degree of DSRC, achieving the maximum possible LTE offloading, given the VANET connectivity achieved via DSRC. Extensive simulations in real urban scenarios are used to evaluate the protocol performance and LTE offloading, as compared to baseline and literature approaches.
Pub.: 17 Nov '16, Pinned: 24 Nov '16
Abstract: Abstract Recently, pedestrian detection systems have become an important technology in the development of the advanced driver assistance system (ADAS) for the autonomous car. The histogram of oriented gradients (HOG) is currently the most basic algorithm for detecting pedestrians, but it treats the entire body of the pedestrian as one single feature. In other words, if the entire body of the pedestrian is not visible, the detection rate under HOG decreases markedly. To solve this problem, we propose a detection system using a deformable part model (DPM) that divides the pedestrian data into two parts using a latent support vector machine (SVM)-based machine-learning technique. Experimental results show that our approach achieves better performance in a detection system than the existing method. In practice, there are many occlusions in the environment in front of the vehicle. For example, the surrounding transport facilities, such as a car or another obstacle, can occlude a pedestrian. These occlusions can increase the false detection rate and cause difficulties during the detection process. Our proposed method uses a different approach and can easily be applied in real-world scenarios, regardless of occlusions.AbstractRecently, pedestrian detection systems have become an important technology in the development of the advanced driver assistance system (ADAS) for the autonomous car. The histogram of oriented gradients (HOG) is currently the most basic algorithm for detecting pedestrians, but it treats the entire body of the pedestrian as one single feature. In other words, if the entire body of the pedestrian is not visible, the detection rate under HOG decreases markedly. To solve this problem, we propose a detection system using a deformable part model (DPM) that divides the pedestrian data into two parts using a latent support vector machine (SVM)-based machine-learning technique. Experimental results show that our approach achieves better performance in a detection system than the existing method. In practice, there are many occlusions in the environment in front of the vehicle. For example, the surrounding transport facilities, such as a car or another obstacle, can occlude a pedestrian. These occlusions can increase the false detection rate and cause difficulties during the detection process. Our proposed method uses a different approach and can easily be applied in real-world scenarios, regardless of occlusions.
Pub.: 25 Oct '16, Pinned: 20 Nov '16
Abstract: Vehicle lightweighting is arguably the most keenly discussed topic in today's automotive industry. The cars that roll off the manufacturer production lines in the future need to become lighter to meet Global Emissions Targets that are fast approaching and focus on strict fuel efficiency and CO2 emissions targets. But, finding the best material solutions to reduce weight – safely and affordably – is a sizeable challenge.
Pub.: 11 Nov '16, Pinned: 20 Nov '16
Abstract: Mean and variation models are of traditional model. It is increasingly used to explain the stochastic phenomenon such as fluctuations of freeway traffic from Monday to Sunday. Different from the existing method, this paper proposes a novel fractional dimension derivation along with the estimation algorithm. It involves the fractional variation definitions with respect to the prediction of the vehicle speed, which are used to warn upcoming collisions between cars or with pedestrians. The proposed real time algorithm provides additional granularity in terms of the complex fractional variation, that eventually leading to the revealing of the driver's behaviour, from both real and imaginary angles. More precisely, the real part reflects the ways of changing speed, and the imaginary part is related to the ways of switching lanes. The calculation results demonstrate that the complex model is able to distinguish the subtle difference between the offensive and the polite drivers. We take the preliminary road measurement as an input to the MATLAB simulation, and show that the new method is able to see the differences of the speeding habit. It has potential for predicting the collisions in real time.
Pub.: 01 Nov '16, Pinned: 20 Nov '16
Abstract: Publication date: November 2016 Source:Transportation Research Part F: Traffic Psychology and Behaviour, Volume 43 Author(s): Zhenji Lu, Riender Happee, Christopher D.D. Cabrall, Miltos Kyriakidis, Joost C.F. de Winter The topic of transitions in automated driving is becoming important now that cars are automated to ever greater extents. This paper proposes a theoretical framework to support and align human factors research on transitions in automated driving. Driving states are defined based on the allocation of primary driving tasks (i.e., lateral control, longitudinal control, and monitoring) between the driver and the automation. A transition in automated driving is defined as the process during which the human-automation system changes from one driving state to another, with transitions of monitoring activity and transitions of control being among the possibilities. Based on ‘Is the transition required?’, ‘Who initiates the transition?’, and ‘Who is in control after the transition?’, we define six types of control transitions between the driver and automation: (1) Optional Driver-Initiated Driver-in-Control, (2) Mandatory Driver-Initiated Driver-in-Control, (3) Optional Driver-Initiated Automation-in-Control, (4) Mandatory Driver-Initiated Automation-in-Control, (5) Automation-Initiated Driver-in-Control, and (6) Automation-Initiated Automation-in-Control. Use cases per transition type are introduced. Finally, we interpret previous experimental studies on transitions using our framework and identify areas for future research. We conclude that our framework of driving states and transitions is an important complement to the levels of automation proposed by transportation agencies, because it describes what the driver and automation are doing, rather than should be doing, at a moment of time.
Pub.: 07 Nov '16, Pinned: 20 Nov '16
Abstract: Publication date: January 2017 Source:Transportation Research Part F: Traffic Psychology and Behaviour, Volume 44 Author(s): Ines Kawgan-Kagan, Stephan Daubitz Understanding of acceptance of electric mobility has been typically discussed by a comparison of vehicles with different types of propulsion engines, battery electric vehicles and vehicles with an internal combustion engine. Nevertheless, electric mobility comprehends a combination of public transport and electric vehicles. The aim of this paper is to understand peoples’ outlook on electric mobility by identifying shared aspects of the assessment of battery electric vehicles and different user perspectives on transportation. A special research design in the form of repertory grids provides an opportunity to study the underlying causes of the cognitive perceptions and emotions relating to electric mobility. Cognitive interviews motivate respondents to reflect beyond the insights provided by standard forms of interview. Especially for the topic of battery electric vehicles, prejudices - for instance, those propagated by the media - are discarded and the actual requirements and patterns of mobility become visible. The special tasks involved in the interviews lead, for example, to deliberation on how to integrate battery charging processes into existing mobility patterns. This special method reveals that individuals take an interest in more characteristics of modes of transport than those that are usually analysed when researching electric mobility. In addition, three anticipation clusters can be identified for individuals with a higher affinity for cars. First, the perception of battery electric vehicles shows high levels of similarity to cars with internal combustion engines and that differentiating between types of engines is meaningless. Second, battery electric vehicles are perceived as a part of urban public transport. Third, battery electric vehicles are viewed as similar to pedelecs and segways, whereas questions of range, innovation and environmental aspects play a greater role in perceptions. These results lead to the conclusion that when studying the acceptance of BEVs, a comparison between cars with internal combustion engines and battery electric vehicles is not sufficient to grasp the complete user perspective. An analysis within the framework of a wider range of modes of transport is required in order to address people’s transportation needs.
Pub.: 13 Nov '16, Pinned: 20 Nov '16
Abstract: This paper proposes a new method, that we call VisualBackProp, for visualizing which sets of pixels of the input image contribute most to the predictions made by the convolutional neural network (CNN). The method heavily hinges on exploring the intuition that the feature maps contain less and less irrelevant information to the prediction decision when moving deeper into the network. The technique we propose was developed as a debugging tool for CNN-based systems for steering self-driving cars and is therefore required to run in real-time, i.e. it was designed to require less computation than a forward propagation. This makes the presented visualization method a valuable debugging tool which can be easily used during both training and inference. We furthermore justify our approach with theoretical arguments and theoretically confirm that the proposed method identifies sets of input pixels, rather than individual pixels, that collaboratively contribute to the prediction. Our theoretical findings stand in agreement with experimental results. The empirical evaluation shows the plausibility of the proposed approach on road data.
Pub.: 16 Nov '16, Pinned: 20 Nov '16
Abstract: Publication date: December 2016 Source:Transportation Research Part C: Emerging Technologies, Volume 73 Author(s): Gaetano Fusco, Chiara Colombaroni, Natalia Isaenko Big data from floating cars supply a frequent, ubiquitous sampling of traffic conditions on the road network and provide great opportunities for enhanced short-term traffic predictions based on real-time information on the whole network. Two network-based machine learning models, a Bayesian network and a neural network, are formulated with a double star framework that reflects time and space correlation among traffic variables and because of its modular structure is suitable for an automatic implementation on large road networks. Among different mono-dimensional time-series models, a seasonal autoregressive moving average model (SARMA) is selected for comparison. The time-series model is also used in a hybrid modeling framework to provide the Bayesian network with an a priori estimation of the predicted speed, which is then corrected exploiting the information collected on other links. A large floating car data set on a sub-area of the road network of Rome is used for validation. To account for the variable accuracy of the speed estimated from floating car data, a new error indicator is introduced that relates accuracy of prediction to accuracy of measure. Validation results highlighted that the spatial architecture of the Bayesian network is advantageous in standard conditions, where a priori knowledge is more significant, while mono-dimensional time series revealed to be more valuable in the few cases of non-recurrent congestion conditions observed in the data set. The results obtained suggested introducing a supervisor framework that selects the most suitable prediction depending on the detected traffic regimes.
Pub.: 11 Nov '16, Pinned: 15 Nov '16
Abstract: Autonomous driving is a multi-agent setting where the host vehicle must apply sophisticated negotiation skills with other road users when overtaking, giving way, merging, taking left and right turns and while pushing ahead in unstructured urban roadways. Since there are many possible scenarios, manually tackling all possible cases will likely yield a too simplistic policy. Moreover, one must balance between unexpected behavior of other drivers/pedestrians and at the same time not to be too defensive so that normal traffic flow is maintained. In this paper we apply deep reinforcement learning to the problem of forming long term driving strategies. We note that there are two major challenges that make autonomous driving different from other robotic tasks. First, is the necessity for ensuring functional safety - something that machine learning has difficulty with given that performance is optimized at the level of an expectation over many instances. Second, the Markov Decision Process model often used in robotics is problematic in our case because of unpredictable behavior of other agents in this multi-agent scenario. We make three contributions in our work. First, we show how policy gradient iterations can be used without Markovian assumptions. Second, we decompose the problem into a composition of a Policy for Desires (which is to be learned) and trajectory planning with hard constraints (which is not learned). The goal of Desires is to enable comfort of driving, while hard constraints guarantees the safety of driving. Third, we introduce a hierarchical temporal abstraction we call an "Option Graph" with a gating mechanism that significantly reduces the effective horizon and thereby reducing the variance of the gradient estimation even further.
Pub.: 11 Oct '16, Pinned: 10 Nov '16
Join Sparrho today to stay on top of science
Discover, organise and share research that matters to you