Marco Braun

Search for another author

Author Description

Ask a Question About This Mathematician

All published works (7)

Deep Learning Method for Cell-Wise Object Tracking, Velocity Estimation and Projection of Sensor Data over Time

2022-10-08

Marco Braun , Moritz Luszek , Mirko Meuter +3 more

Current Deep Learning methods for environment segmentation and velocity estimation rely on Convolutional Recurrent Neural Networks to exploit spatio-temporal relationships within obtained sensor data. These approaches derive scene dynamics implicitly … Current Deep Learning methods for environment segmentation and velocity estimation rely on Convolutional Recurrent Neural Networks to exploit spatio-temporal relationships within obtained sensor data. These approaches derive scene dynamics implicitly by correlating novel input and memorized data utilizing ConvNets. We show how ConvNets suffer from architectural restrictions for this task. Based on these findings, we then provide solutions to various issues on exploiting spatio-temporal correlations in a sequence of sensor recordings by presenting a novel Recurrent Neural Network unit utilizing Transformer mechanisms. Within this unit, object encodings are tracked across consecutive frames by correlating key-query pairs derived from sensor inputs and memory states, respectively. We then use resulting tracking patterns to obtain scene dynamics and regress velocities. In a last step, the memory state of the Recurrent Neural Network is projected based on extracted velocity estimates to resolve aforementioned spatio-temporal misalignment.

Chat View PDF

Quantification of Uncertainties in Deep Learning-based Environment Perception

2021-08-23

Marco Braun , Moritz Luszek , Jan Siegemund +2 more

In this work, we introduce a novel Deep Learning-based method to perceive the environment of a vehicle based on radar scans while accounting for uncertainties in its predictions. The environment … In this work, we introduce a novel Deep Learning-based method to perceive the environment of a vehicle based on radar scans while accounting for uncertainties in its predictions. The environment of the host vehicle is segmented into equally sized grid cells which are classified individually. Complementary to the segmentation output, our Deep Learning-based algorithm is capable of differentiating uncertainties in its predictions as being related to an inadequate model (epistemic uncertainty) or noisy data (aleatoric uncertainty). To this end, weights are described as probability distributions accounting for uncertainties in the model parameters. Distributions are learned in a supervised fashion using gradient descent. We prove that uncertainties in the model output correlate with the precision of its predictions. Compared to previous concepts, we show superior performance of our approach to reliably perceive the environment of a vehicle.

Chat View PDF

The effect of price-based demand response on carbon emissions in European electricity markets: The importance of adequate carbon prices

2021-05-05

Markus Fleschutz , Markus Bohlayer , Marco Braun +2 more

Price-based demand response (PBDR) has recently been attributed great economic but also environmental potential. However, the determination of its short-term effects on carbon emissions requires the knowledge of marginal emission … Price-based demand response (PBDR) has recently been attributed great economic but also environmental potential. However, the determination of its short-term effects on carbon emissions requires the knowledge of marginal emission factors (MEFs), which compared to grid mix emission factors (XEFs), are cumbersome to calculate due to the complex characteristics of national electricity markets. This study, therefore, proposes two merit order-based methods to approximate hourly MEFs and applies them to readily available datasets from 20 European countries for the years 2017–2019. Based on the calculated electricity prices, standardized daily load shifts were simulated which indicated that carbon emissions increased for 8 of the 20 countries and by 2.1% on average. Thus, under specific circumstances, PBDR leads to carbon emissions increases, mainly due to the economic advantage fuel sources such as lignite and coal have in the merit order. MEF-based load shifts reduced the mean resulting carbon emissions by 35%, albeit with 56% lower monetary cost savings compared to price-based load shifts. Finally, by repeating the load shift simulations for different carbon price levels, the impact of the carbon price on the resulting carbon emissions was analyzed. The Spearman correlation coefficient between carbon intensity and marginal cost along the German merit order substantially increased with increasing carbon price. The coefficients were -0.13 for the 2019 carbon price of 24.9 €/t, 0 for 42.6 €/t, and 0.4 for 100.0 €/t. Therefore, with adequate carbon prices, PBDR can be an effective tool for both economical and environmental improvement.

Chat View PDF

Jarzynski Equality for Soret Equilibria -- Virtues of Virtual Potentials

2020-01-01

Tobias Thalheim , Marco Braun , Gianmaria Falasco +2 more

The Jarzynski equality relates the free energy difference between two equilibrium states to the fluctuating irreversible work afforded to switch between them. The prescribed fixed temperature for the equilibrium states … The Jarzynski equality relates the free energy difference between two equilibrium states to the fluctuating irreversible work afforded to switch between them. The prescribed fixed temperature for the equilibrium states implicitly constrains the dissipative switching process that can take the system far from equilibrium. Here, we demonstrate theoretically and experimentally that such a relation also holds for the nonisothermal case, where the initial stationary state is not in equilibrium and the switching is effected by dynamically changing temperature gradients instead of a conservative force. Our demonstration employs a single colloidal particle trapped by optically induced thermophoretic drift currents. It relies on identifying suitable equivalents of classical work and heat and our ability to measure their distributions and express them in terms of a virtual potential.

Chat View PDF

A whole-year simulation study on nonlinear mixed-integer model predictive control for a thermal energy supply system with multi-use components

2019-11-20

Adrian Bürger , Markus Bohlayer , Sarah G. Hoffmann +3 more

Chat View PDF

A whole-year simulation study on nonlinear mixed-integer model predictive control for a thermal energy supply system with multi-use components

2019-05-22

Adrian Bürger , Markus Bohlayer , Sarah G. Hoffmann +3 more

This work presents a whole-year simulation study on nonlinear mixed-integer Model Predictive Control (MPC) for a complex thermal energy supply system which consists of a heat pump, stratified water storages, … This work presents a whole-year simulation study on nonlinear mixed-integer Model Predictive Control (MPC) for a complex thermal energy supply system which consists of a heat pump, stratified water storages, free cooling facilities, and a large underground thermal storage. For solution of the arising Mixed-Integer Non-Linear Programs (MINLPs) we apply an existing general and optimal-control-suitable decomposition approach. To compensate deviation of forecast inputs from measured disturbances, we introduce a moving horizon estimation step within the MPC strategy. The MPC performance for this study, which consists of more than 50,000 real time suitable MINLP solutions, is compared to an elaborate conventional control strategy for the system. It is shown that MPC can significantly reduce the yearly energy consumption while providing a similar degree of constraint satisfaction, and autonomously identify previously unknown, beneficial operation modes.

Chat View PDF

Simulation of a Vehicle Refrigeration Cycle with Dymola/Modelica

2005-04-11

Marco Braun , Roland Caesar , Dirk Limperich +2 more

Development times in the automotive industry are becoming increasingly shorter. For this reason, design decisions based on simulation results must be made at an early development stage. The dynamic simulation … Development times in the automotive industry are becoming increasingly shorter. For this reason, design decisions based on simulation results must be made at an early development stage. The dynamic simulation of an automotive refrigeration cycle with Dymola/Modelica as part of the design process will be described in the following paper. The component supplier's expertise as well as the automotive manufacturer's knowledge of vehicle parameters in one simulation platform will also be discussed.

Common Coauthors

Coauthor	Papers Together
Markus Bohlayer	3
Moritz Luszek	2
Angelika Altmann‐Dieses	2
Kevin Kollek	2
Adrian Bürger	2
Moritz Diehl	2
Sarah G. Hoffmann	2
Anton Kummert	2
Gianmaria Falasco	1
Mirko Meuter	1
Roland Caesar	1
Gerhard Schmitz	1
Katrin Prölß	1
Tobias Thalheim	1
Dominic Spata	1
Klaus Kroy	1
Gregor P. Henze	1
Dirk Limperich	1
Frank Cichos	1
Michael D. Murphy	1
Markus Fleschutz	1
Jan Siegemund	1

Commonly Cited References

An integrated model predictive control approach for optimal HVAC and energy storage operation in large-scale buildings

2019-02-15

Gianni Bianchini , Marco Casini , Daniele Pepe +2 more

Chat View PDF

Self Supervised Occupancy Grid Learning from Sparse Radar for Autonomous Driving

2019-03-31

Liat Sless , Guy Cohen , Bat El Shlomo +1 more

Occupancy grid mapping is an important component in road scene understanding for autonomous driving. It encapsulates information of the drivable area, road obstacles and enables safe autonomous driving. Radars are … Occupancy grid mapping is an important component in road scene understanding for autonomous driving. It encapsulates information of the drivable area, road obstacles and enables safe autonomous driving. Radars are an emerging sensor in autonomous vehicle vision, becoming more widely used due to their long range sensing, low cost, and robustness to severe weather conditions. Despite recent advances in deep learning technology, occupancy grid mapping from radar data is still mostly done using classical filtering this http URL this work, we propose learning the inverse sensor model used for occupancy grid mapping from clustered radar data. This is done in a data driven approach that leverages computer vision techniques. This task is very challenging due to data sparsity and noise characteristics of the radar sensor. The problem is formulated as a semantic segmentation task and we show how it can be learned using lidar data for generating ground truth. We show both qualitatively and quantitatively that our learned occupancy net outperforms classic methods by a large margin using the recently released NuScenes real-world driving data.

Chat View PDF

Optimal control via collocation and non-linear programming

1975-05-01

T. H. Tsang , D. M. Himmelblau , Thomas F. Edgar

Abstract The collocation method meshed with non-linear programming techniques provides an efficient strategy for the numerical solution of optimal control problems. Good accuracy can be obtained for the state and … Abstract The collocation method meshed with non-linear programming techniques provides an efficient strategy for the numerical solution of optimal control problems. Good accuracy can be obtained for the state and the control trajectories as well as for the value of the objective function. In addition, the control strategy can be quite flexible in form. However, it is necessary to select the appropriate number of collocation points and number of parameters in the approximating functions with care.

On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming

2005-04-28

Andreas Wächter , Lorenz T. Biegler

A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems *

1984-07-01

H.G. Bock , K.J. Plitt

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

2017-01-01

Alex Kendall , Yarin Gal

There are two major types of uncertainty one can model. Aleatoric uncertainty captures noise inherent in the observations. On the other hand, epistemic uncertainty accounts for uncertainty in the model … There are two major types of uncertainty one can model. Aleatoric uncertainty captures noise inherent in the observations. On the other hand, epistemic uncertainty accounts for uncertainty in the model -- uncertainty which can be explained away given enough data. Traditionally it has been difficult to model epistemic uncertainty in computer vision, but with new Bayesian deep learning tools this is now possible. We study the benefits of modeling epistemic vs. aleatoric uncertainty in Bayesian deep learning models for vision tasks. For this we present a Bayesian deep learning framework combining input-dependent aleatoric uncertainty together with epistemic uncertainty. We study models under the framework with per-pixel semantic segmentation and depth regression tasks. Further, our explicit uncertainty formulation leads to new loss functions for these tasks, which can be interpreted as learned attenuation. This makes the loss more robust to noisy data, also giving new state-of-the-art results on segmentation and depth regression benchmarks.

Chat View PDF

Combinatorial integral approximation

2011-04-15

Sebastian Säger , Michael N. Jung , Christian Kirches

Chat View PDF

CasADi: a software framework for nonlinear optimization and optimal control

2018-07-11

Joel A.E. Andersson , Joris Gillis , Greg Horn +2 more

Chat View PDF

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

2017-04-27

Liang-Chieh Chen , George Papandreou , Iasonas Kokkinos +2 more

In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we … In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we highlight convolution with upsampled filters, or 'atrous convolution', as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second, we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third, we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed "DeepLab" system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7 percent mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.

Chat View PDF

Deep Bayesian Active Learning with Image Data

2017-01-01

Yarin Gal , Riashat Islam , Zoubin Ghahramani

Even though active learning forms an important pillar of machine learning, deep learning tools are not prevalent within it. Deep learning poses several difficulties when used in an active learning … Even though active learning forms an important pillar of machine learning, deep learning tools are not prevalent within it. Deep learning poses several difficulties when used in an active learning setting. First, active learning (AL) methods generally rely on being able to learn and update models from small amounts of data. Recent advances in deep learning, on the other hand, are notorious for their dependence on large amounts of data. Second, many AL acquisition functions rely on model uncertainty, yet deep learning methods rarely represent such model uncertainty. In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way. We develop an active learning framework for high dimensional data, a task which has been extremely challenging so far, with very sparse existing literature. Taking advantage of specialised models such as Bayesian convolutional neural networks, we demonstrate our active learning techniques with image data, obtaining a significant improvement on existing active learning approaches. We demonstrate this on both the MNIST dataset, as well as for skin cancer diagnosis from lesion images (ISIC2016 task).

Chat View PDF

Spatially supervised recurrent convolutional neural networks for visual object tracking

2017-05-01

Guanghan Ning , Zhi Zhang , Chen Huang +4 more

In this paper, we develop a new approach of spatially supervised recurrent convolutional neural networks for visual object tracking. Our recurrent convolutional network exploits the history of locations as well … In this paper, we develop a new approach of spatially supervised recurrent convolutional neural networks for visual object tracking. Our recurrent convolutional network exploits the history of locations as well as the distinctive visual features learned by the deep neural networks. Inspired by recent bounding box regression methods for object detection, we study the regression capability of Long Short-Term Memory (LSTM) in the temporal domain, and propose to concatenate high-level visual features produced by convolutional networks with region information. In contrast to existing deep learning based trackers that use binary classification for region candidates, we use regression for direct prediction of the tracking locations both at the convolutional layer and at the recurrent unit. Our experimental results on challenging benchmark video tracking datasets show that our tracker is competitive with state-of-the-art approaches while maintaining low computational cost.

Chat View PDF

You Only Look Once: Unified, Real-Time Object Detection

2016-06-01

Joseph Redmon , Santosh Divvala , Ross Girshick +1 more

We present YOLO, a new approach to object detection. Prior work on object detection repurposes classifiers to perform detection. Instead, we frame object detection as a regression problem to spatially … We present YOLO, a new approach to object detection. Prior work on object detection repurposes classifiers to perform detection. Instead, we frame object detection as a regression problem to spatially separated bounding boxes and associated class probabilities. A single neural network predicts bounding boxes and class probabilities directly from full images in one evaluation. Since the whole detection pipeline is a single network, it can be optimized end-to-end directly on detection performance. Our unified architecture is extremely fast. Our base YOLO model processes images in real-time at 45 frames per second. A smaller version of the network, Fast YOLO, processes an astounding 155 frames per second while still achieving double the mAP of other real-time detectors. Compared to state-of-the-art detection systems, YOLO makes more localization errors but is less likely to predict false positives on background. Finally, YOLO learns very general representations of objects. It outperforms other detection methods, including DPM and R-CNN, when generalizing from natural images to other domains like artwork.

Chat View PDF

Online Multi-Target Tracking Using Recurrent Neural Networks

2017-02-12

Anton Milan , S. Hamid Rezatofighi , Anthony Dick +2 more

We present a novel approach to online multi-target tracking based on recurrent neural networks (RNNs). Tracking multiple objects in real-world scenes involves many challenges, including a) an a-priori unknown and … We present a novel approach to online multi-target tracking based on recurrent neural networks (RNNs). Tracking multiple objects in real-world scenes involves many challenges, including a) an a-priori unknown and time-varying number of targets, b) a continuous state estimation of all present targets, and c) a discrete combinatorial problem of data association. Most previous methods involve complex models that require tedious tuning of parameters. Here, we propose for the first time, an end-to-end learning approach for online multi-target tracking. Existing deep learning methods are not designed for the above challenges and cannot be trivially applied to the task. Our solution addresses all of the above points in a principled way. Experiments on both synthetic and real data show promising results obtained at ~300 Hz on a standard CPU, and pave the way towards future research in this direction.

Chat View PDF

Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling

2018-05-01

Stefan Hoermann , Martin Bach , Klaus Dietmayer

Long-term situation prediction plays a crucial role for intelligent vehicles. A major challenge still to overcome is the prediction of complex downtown scenarios with multiple road users, e.g., pedestrians, bikes, … Long-term situation prediction plays a crucial role for intelligent vehicles. A major challenge still to overcome is the prediction of complex downtown scenarios with multiple road users, e.g., pedestrians, bikes, and motor vehicles, interacting with each other. This contribution tackles this challenge by combining a Bayesian filtering technique for environment representation, and machine learning as long-term predictor. More specifically, a dynamic occupancy grid map is utilized as input to a deep convolutional neural network. This yields the advantage of using spatially distributed velocity estimates from a single time step for prediction, rather than a raw data sequence, alleviating common problems dealing with input time series of multiple sensors. Furthermore, convolutional neural networks have the inherent characteristic of using context information, enabling the implicit modeling of road user interaction. Pixel-wise balancing is applied in the loss function counteracting the extreme imbalance between static and dynamic cells. One of the major advantages is the unsupervised learning character due to fully automatic label generation. The presented algorithm is trained and evaluated on multiple hours of recorded sensor data and compared to Monte-Carlo simulation. Experiments show the ability to model complex interactions.

Chat View PDF

Densely Connected Convolutional Networks

2017-07-01

Gao Huang , Zhuang Liu , Laurens van der Maaten +1 more

Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those … Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper, we embrace this observation and introduce the Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion. Whereas traditional convolutional networks with L layers have L connections-one between each layer and its subsequent layer-our network has L(L+1)/2 direct connections. For each layer, the feature-maps of all preceding layers are used as inputs, and its own feature-maps are used as inputs into all subsequent layers. DenseNets have several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters. We evaluate our proposed architecture on four highly competitive object recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet). DenseNets obtain significant improvements over the state-of-the-art on most of them, whilst requiring less memory and computation to achieve high performance. Code and pre-trained models are available at https://github.com/liuzhuang13/DenseNet.

Chat View PDF

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

2017-01-02

Vijay Badrinarayanan , A. C. Kendall , Roberto Cipolla

We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding … We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [1] . The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the widely adopted FCN [2] and also with the well known DeepLab-LargeFOV [3] , DeconvNet [4] architectures. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. SegNet was primarily motivated by scene understanding applications. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures and can be trained end-to-end using stochastic gradient descent. We also performed a controlled benchmark of SegNet and other architectures on both road scenes and SUN RGB-D indoor scene segmentation tasks. These quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures. We also provide a Caffe implementation of SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet/.

Chat View PDF

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

2015-01-01

Yarin Gal , Zoubin Ghahramani

Deep learning tools have gained tremendous attention in applied machine learning. However such tools for regression and classification do not capture model uncertainty. In comparison, Bayesian models offer a mathematically … Deep learning tools have gained tremendous attention in applied machine learning. However such tools for regression and classification do not capture model uncertainty. In comparison, Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with a prohibitive computational cost. In this paper we develop a new theoretical framework casting dropout training in deep neural networks (NNs) as approximate Bayesian inference in deep Gaussian processes. A direct result of this theory gives us tools to model uncertainty with dropout NNs -- extracting information from existing models that has been thrown away so far. This mitigates the problem of representing uncertainty in deep learning without sacrificing either computational complexity or test accuracy. We perform an extensive study of the properties of dropout's uncertainty. Various network architectures and non-linearities are assessed on tasks of regression and classification, using MNIST as an example. We show a considerable improvement in predictive log-likelihood and RMSE compared to existing state-of-the-art methods, and finish by using dropout's uncertainty in deep reinforcement learning.

Chat View PDF

Tracking by Prediction: A Deep Generative Model for Mutli-person Localisation and Tracking

2018-03-01

Tharindu Fernando , Simon Denman , Sridha Sridharan +1 more

Current multi-person localisation and tracking systems have an over reliance on the use of appearance models for target re-identification and almost no approaches employ a complete deep learning solution for … Current multi-person localisation and tracking systems have an over reliance on the use of appearance models for target re-identification and almost no approaches employ a complete deep learning solution for both objectives. We present a novel, complete deep learning framework for multi-person localisation and tracking. In this context we first introduce a light weight sequential Generative Adversarial Network architecture for person localisation, which overcomes issues related to occlusions and noisy detections, typically found in a multi person environment. In the proposed tracking framework we build upon recent advances in pedestrian trajectory prediction approaches and propose a novel data association scheme based on predicted trajectories. This removes the need for computationally expensive person re-identification systems based on appearance features and generates human like trajectories with minimal fragmentation. The proposed method is evaluated on multiple public benchmarks including both static and dynamic cameras and is capable of generating outstanding performance, especially among other recently proposed deep neural network based approaches.

Chat View PDF

Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

2017-01-01

Alex Kendall , Vijay Badrinarayanan , Roberto Cipolla

© 2017. The copyright of this document resides with its authors. We present a deep learning framework for probabilistic pixel-wise semantic segmentation, which we term Bayesian SegNet. Semantic segmentation is … © 2017. The copyright of this document resides with its authors. We present a deep learning framework for probabilistic pixel-wise semantic segmentation, which we term Bayesian SegNet. Semantic segmentation is an important tool for visual scene understanding and a meaningful measure of uncertainty is essential for decision making. Our contribution is a practical system which is able to predict pixel-wise class labels with a measure of model uncertainty using Bayesian deep learning. We achieve this by Monte Carlo sampling with dropout at test time to generate a posterior distribution of pixel class labels. In addition, we show that modelling uncertainty improves segmentation performance by 2-3% across a number of datasets and architectures such as SegNet, FCN, Dilation Network and DenseNet.

Chat View PDF

Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks

2019-05-01

Marcel Schreiber , Stefan Hoermann , Klaus Dietmayer

We tackle the long-term prediction of scene evolution in a complex downtown scenario for automated driving based on Lidar grid fusion and recurrent neural networks (RNNs). A bird's eye view … We tackle the long-term prediction of scene evolution in a complex downtown scenario for automated driving based on Lidar grid fusion and recurrent neural networks (RNNs). A bird's eye view of the scene, including occupancy and velocity, is fed as a sequence to a RNN which is trained to predict future occupancy. The nature of prediction allows generation of multiple hours of training data without the need of manual labeling. Thus, the training strategy and loss function are designed for long sequences of real-world data (unbalanced, continuously changing situations, false labels, etc.). The deep CNN architecture comprises convolutional long short-term memories (ConvLSTMs) to separate static from dynamic regions and to predict dynamic objects in future frames. Novel recurrent skip connections show the ability to predict small occluded objects, i.e. pedestrians, and occluded static regions. Spatio-temporal correlations between grid cells are exploited to predict multimodal future paths and interactions between objects. Experiments also quantity improvements to our previous network, a Monte Carlo approach, and literature.

Chat View PDF

Probably Unknown: Deep Inverse Sensor Modelling Radar

2019-05-01

Rob Weston , Sarah H. Cen , Paul Newman +1 more

Radar presents a promising alternative to lidar and vision in autonomous vehicle applications, able to detect objects at long range under a variety of weather conditions. However, distinguishing between occupied … Radar presents a promising alternative to lidar and vision in autonomous vehicle applications, able to detect objects at long range under a variety of weather conditions. However, distinguishing between occupied and free space from raw radar power returns is challenging due to complex interactions between sensor noise and occlusion. To counter this we propose to learn an Inverse Sensor Model (ISM) converting a raw radar scan to a grid map of occupancy probabilities using a deep neural network. Our network is selfsupervised using partial occupancy labels generated by lidar, allowing a robot to learn about world occupancy from past experience without human supervision. We evaluate our approach on five hours of data recorded in a dynamic urban environment. By accounting for the scene context of each grid cell our model is able to successfully segment the world into occupied and free space, outperforming standard CFAR filtering approaches. Additionally by incorporating heteroscedastic uncertainty into our model formulation, we are able to quantify the variance in the uncertainty throughout the sensor observation. Through this mechanism we are able to successfully identify regions of space that are likely to be occluded.

Chat View PDF

Development and experimental evaluation of grey-box models of a microscale polygeneration system for application in optimal control

2019-12-27

Parantapa Sawant , Adrian Bürger , Minh Dang Doan +2 more

Chat View PDF

Control of Heat Pumps with CO2 Emission Intensity Forecasts

2020-06-03

Kenneth Leerbeck , Peder Bacher , Rune Grønborg Junker +4 more

An optimized heat pump control for building heating was developed for minimizing CO 2 emissions from related electrical power generation. The control is using weather and CO 2 emission forecasts … An optimized heat pump control for building heating was developed for minimizing CO 2 emissions from related electrical power generation. The control is using weather and CO 2 emission forecasts as inputs to a Model Predictive Control (MPC)—a multivariate control algorithm using a dynamic process model, constraints and a cost function to be minimized. In a simulation study, the control was applied using weather and power grid conditions during a full-year period in 2017–2018 for the power bidding zone DK2 (East, Denmark). Two scenarios were studied; one with a family house and one with an office building. The buildings were dimensioned based on standards and building codes/regulations. The main results are measured as the CO 2 emission savings relative to a classical thermostatic control. Note that this only measures the gain achieved using the MPC control, that is, the energy flexibility, not the absolute savings. The results show that around 16% of savings could have been achieved during the period in well-insulated new buildings with floor heating. Further, a sensitivity analysis was carried out to evaluate the effect of various building properties, for example, level of insulation and thermal capacity. Danish building codes from 1977 and forward were used as benchmarks for insulation levels. It was shown that both insulation and thermal mass influence the achievable flexibility savings, especially for floor heating. Buildings that comply with building codes later than 1979 could provide flexibility emission savings of around 10%, while buildings that comply with earlier codes provided savings in the range of 0–5% depending on the heating system and thermal mass.

Chat View PDF

MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird’s Eye View Maps

2020-06-01

Pengxiang Wu , Siheng Chen , Dimitris Metaxas

The ability to reliably perceive the environmental states, particularly the existence of objects and their motion behavior, is crucial for autonomous driving. In this work, we propose an efficient deep … The ability to reliably perceive the environmental states, particularly the existence of objects and their motion behavior, is crucial for autonomous driving. In this work, we propose an efficient deep model, called MotionNet, to jointly perform perception and motion prediction from 3D point clouds. MotionNet takes a sequence of LiDAR sweeps as input and outputs a bird's eye view (BEV) map, which encodes the object category and motion information in each grid cell. The backbone of MotionNet is a novel spatio-temporal pyramid network, which extracts deep spatial and temporal features in a hierarchical fashion. To enforce the smoothness of predictions over both space and time, the training of MotionNet is further regularized with novel spatial and temporal consistency losses. Extensive experiments show that the proposed method overall outperforms the state-of-the-arts, including the latest scene-flow- and 3D-object-detection-based methods. This indicates the potential value of the proposed method serving as a backup to the bounding-box-based system, and providing complementary information to the motion planner in autonomous driving. Code is available at https://www.merl.com/research/license#MotionNet.

Chat View PDF

Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision

2020-06-01

Fredrik Gustafsson , Martin Danelljan , Thomas B. Schön

While deep neural networks have become the go-to approach in computer vision, the vast majority of these models fail to properly capture the uncertainty inherent in their predictions. Estimating this … While deep neural networks have become the go-to approach in computer vision, the vast majority of these models fail to properly capture the uncertainty inherent in their predictions. Estimating this predictive uncertainty can be crucial, for example in automotive applications. In Bayesian deep learning, predictive uncertainty is commonly decomposed into the distinct types of aleatoric and epistemic uncertainty. The former can be estimated by letting a neural network output the parameters of a certain probability distribution. Epistemic uncertainty estimation is a more challenging problem, and while different scalable methods recently have emerged, no extensive comparison has been performed in a real-world setting. We therefore accept this task and propose a comprehensive evaluation framework for scalable epistemic uncertainty estimation methods in deep learning. Our proposed framework is specifically designed to test the robustness required in real-world computer vision applications. We also apply this framework to provide the first properly extensive and conclusive comparison of the two current state-of-the- art scalable methods: ensembling and MC-dropout. Our comparison demonstrates that ensembling consistently provides more reliable and practically useful uncertainty estimates. Code is available at https://github.com/fregu856/evaluating_bdl.

Chat View PDF

Motion Estimation in Occupancy Grid Maps in Stationary Settings Using Recurrent Neural Networks

2020-05-01

Marcel Schreiber , Vasileios Belagiannis , Claudius Gläser +1 more

In this work, we tackle the problem of modeling the vehicle environment as dynamic occupancy grid map in complex urban scenarios using recurrent neural networks. Dynamic occupancy grid maps represent … In this work, we tackle the problem of modeling the vehicle environment as dynamic occupancy grid map in complex urban scenarios using recurrent neural networks. Dynamic occupancy grid maps represent the scene in a bird's eye view, where each grid cell contains the occupancy prob-ability and the two dimensional velocity. As input data, our approach relies on measurement grid maps, which contain occupancy probabilities, generated with lidar measurements. Given this configuration, we propose a recurrent neural net-work architecture to predict a dynamic occupancy grid map, i.e. filtered occupancy and velocity of each cell, by using a sequence of measurement grid maps. Our network architecture contains convolutional long-short term memories in order to sequentially process the input, makes use of spatial context, and captures motion. In the evaluation, we quantify improvements in estimating the velocity of braking and turning vehicles compared to the state-of-the-art. Additionally, we demonstrate that our approach provides more consistent velocity estimates for dynamic objects, as well as, less erroneous velocity estimates in static area.

Chat View PDF

Exploiting Multi-Layer Grid Maps for Surround-View Semantic Segmentation of Sparse LiDAR Data

2020-10-19

Frank Bieder , Sascha Wirges , Johannes Janosovits +3 more

In this paper, we consider the transformation of laser range measurements into a top-view grid map representation to approach the task of LiDAR-only semantic segmentation. Since the recent publication of … In this paper, we consider the transformation of laser range measurements into a top-view grid map representation to approach the task of LiDAR-only semantic segmentation. Since the recent publication of the SemanticKITTI data set, researchers are now able to study semantic segmentation of urban LiDAR sequences based on a reasonable amount of data. While other approaches propose to directly learn on the 3D point clouds, we are exploiting a grid map framework to extract relevant information and represent them by using multi-layer grid maps. This representation allows us to use well-studied deep learning architectures from the image domain to predict a dense semantic grid map using only the sparse input data of a single LiDAR scan. We compare single-layer and multi-layer approaches and demonstrate the benefit of a multi-layer grid map input. Since the grid map representation allows us to predict a dense, 360° semantic environment representation, we further develop a method to combine the semantic information from multiple scans and create dense ground truth grids. This method allows us to evaluate and compare the performance of our models not only based on grid cells with a detection, but on the full visible measurement range.

Chat View PDF

Quantification of Uncertainties in Deep Learning-based Environment Perception

2021-08-23

Marco Braun , Moritz Luszek , Jan Siegemund +2 more

Chat View PDF

Nonlinear Programming

2010-01-01

Osman Güler

Fully Convolutional Networks for Semantic Segmentation

2014-01-01

J. D. Long , Evan Shelhamer , Trevor Darrell

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic segmentation. Our key insight … Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic segmentation. Our key insight is to build "fully convolutional" networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet, the VGG net, and GoogLeNet) into fully convolutional networks and transfer their learned representations by fine-tuning to the segmentation task. We then define a novel architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional network achieves state-of-the-art segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012), NYUDv2, and SIFT Flow, while inference takes one third of a second for a typical image.

Chat View PDF

Attention Is All You Need

2017-01-01

Ashish Vaswani , Noam Shazeer , Niki Parmar +5 more

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an … The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

Chat View PDF

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

2015-06-06

Yarin Gal , Zoubin Ghahramani

Chat View PDF

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

2015-01-01

Xingjian Shi , Zhourong Chen , Hao Wang +3 more

The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time. Very few previous studies have examined this … The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time. Very few previous studies have examined this crucial and challenging weather forecasting problem from the machine learning perspective. In this paper, we formulate precipitation nowcasting as a spatiotemporal sequence forecasting problem in which both the input and the prediction target are spatiotemporal sequences. By extending the fully connected LSTM (FC-LSTM) to have convolutional structures in both the input-to-state and state-to-state transitions, we propose the convolutional LSTM (ConvLSTM) and use it to build an end-to-end trainable model for the precipitation nowcasting problem. Experiments show that our ConvLSTM network captures spatiotemporal correlations better and consistently outperforms FC-LSTM and the state-of-the-art operational ROVER algorithm for precipitation nowcasting.

Chat View PDF

FlowNet: Learning Optical Flow with Convolutional Networks

2015-12-01

Alexey Dosovitskiy , Philipp Fischer , Eddy Ilg +6 more

Convolutional neural networks (CNNs) have recently been very successful in a variety of computer vision tasks, especially on those linked to recognition. Optical flow estimation has not been among the … Convolutional neural networks (CNNs) have recently been very successful in a variety of computer vision tasks, especially on those linked to recognition. Optical flow estimation has not been among the tasks CNNs succeeded at. In this paper we construct CNNs which are capable of solving the optical flow estimation problem as a supervised learning task. We propose and compare two architectures: a generic architecture and another one including a layer that correlates feature vectors at different image locations. Since existing ground truth data sets are not sufficiently large to train a CNN, we generate a large synthetic Flying Chairs dataset. We show that networks trained on this unrealistic data still generalize very well to existing datasets such as Sintel and KITTI, achieving competitive accuracy at frame rates of 5 to 10 fps.

Chat View PDF

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

2014-01-01

Kyunghyun Cho , Bart van Merriënboer , Çağlar Gülçehre +4 more

Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014. Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014.

Chat View PDF

Fully Convolutional Networks for Semantic Segmentation

2016-05-24

Evan Shelhamer , Jonathan Long , Trevor Darrell

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation. … Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation. Our key insight is to build "fully convolutional" networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet, the VGG net, and GoogLeNet) into fully convolutional networks and transfer their learned representations by fine-tuning to the segmentation task. We then define a skip architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional networks achieve improved segmentation of PASCAL VOC (30% relative improvement to 67.2% mean IU on 2012), NYUDv2, SIFT Flow, and PASCAL-Context, while inference takes one tenth of a second for a typical image.

Chat View PDF

Joint Optical Flow and Temporally Consistent Semantic Segmentation

2016-01-01

Junhwa Hur , Stefan Roth

Chat View PDF

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

2017-07-01

Eddy Ilg , N. Michael Mayer , Tonmoy Saikia +3 more

The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still … The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a subnetwork specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50%. It performs on par with state-of-the-art methods, while running at interactive frame rates. Moreover, we present faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet.

Chat View PDF

Semantic Video Segmentation by Gated Recurrent Flow Propagation

2018-06-01

David Nilsson , Cristian Sminchisescu

Semantic video segmentation is challenging due to the sheer amount of data that needs to be processed and labeled in order to construct accurate models. In this paper we present … Semantic video segmentation is challenging due to the sheer amount of data that needs to be processed and labeled in order to construct accurate models. In this paper we present a deep, end-to-end trainable methodology for video segmentation that is capable of leveraging the information present in unlabeled data, besides sparsely labeled frames, in order to improve semantic estimates. Our model combines a convolutional architecture and a spatiotemporal transformer recurrent layer that is able to temporally propagate labeling information by means of optical flow, adaptively gated based on its locally estimated uncertainty. The flow, the recognition and the gated temporal propagation modules can be trained jointly, end-to-end. The temporal, gated recurrent flow propagation component of our model can be plugged into any static semantic segmentation architecture and turn it into a weakly supervised video processing one. Our experiments in the challenging CityScapes and Camvid datasets, and for multiple deep architectures, indicate that the resulting model can leverage unlabeled temporal frames, next to a labeled one, in order to improve both the video segmentation accuracy and the consistency of its temporal labeling, at no additional annotation cost and with little extra computation.

Chat View PDF

Uncertainty Decomposition in Bayesian Neural Networks with Latent Variables

2017-01-01

Stefan Depeweg , José Miguel Hernández-Lobato , Finale Doshi‐Velez +1 more

Bayesian neural networks (BNNs) with latent variables are probabilistic models which can automatically identify complex stochastic patterns in the data. We describe and study in these models a decomposition of … Bayesian neural networks (BNNs) with latent variables are probabilistic models which can automatically identify complex stochastic patterns in the data. We describe and study in these models a decomposition of predictive uncertainty into its epistemic and aleatoric components. First, we show how such a decomposition arises naturally in a Bayesian active learning scenario by following an information theoretic approach. Second, we use a similar decomposition to develop a novel risk sensitive objective for safe reinforcement learning (RL). This objective minimizes the effect of model bias in environments whose stochastic dynamics are described by BNNs with latent variables. Our experiments illustrate the usefulness of the resulting decomposition in active learning and safe RL settings.

Chat View PDF

Real-time carbon accounting method for the European electricity markets

2019-06-05

Bo Tranberg , Olivier Corradi , Bruno Lajoie +3 more

Electricity accounts for 25% of global greenhouse gas emissions. Reducing emissions related to electricity consumption requires accurate measurements readily available to consumers, regulators and investors. In this case study, we … Electricity accounts for 25% of global greenhouse gas emissions. Reducing emissions related to electricity consumption requires accurate measurements readily available to consumers, regulators and investors. In this case study, we propose a new real-time consumption-based accounting approach based on flow tracing. This method traces power flows from producer to consumer thereby representing the underlying physics of the electricity system, in contrast to the traditional input-output models of carbon accounting. With this method we explore the hourly structure of electricity trade across Europe in 2017, and find substantial differences between production and consumption intensities. This emphasizes the importance of considering cross-border flows for increased transparency regarding carbon emission accounting of electricity.

Chat View PDF

Weight Uncertainty in Neural Networks

2015-01-01

Charles Blundell , Julien Cornebise , Koray Kavukcuoglu +1 more

We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by … We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the marginal likelihood. We show that this principled kind of regularisation yields comparable performance to dropout on MNIST classification. We then demonstrate how the learnt uncertainty in the weights can be used to improve generalisation in non-linear regression problems, and how this weight uncertainty can be used to drive the exploration-exploitation trade-off in reinforcement learning.

Chat View PDF