HoD-Net: High-Order Differentiable Deep Neural Networks and Applications
HoD-Net: High-Order Differentiable Deep Neural Networks and Applications
We introduce a deep architecture named HoD-Net to enable high-order differentiability for deep learning. HoD-Net is based on and generalizes the complex-step finite difference (CSFD) method. While similar to classic finite difference, CSFD approaches the derivative of a function from a higher-dimension complex domain, leading to highly accurate and robust …