Rough Transformers: Lightweight Continuous-Time Sequence Modelling with Path Signatures
Rough Transformers: Lightweight Continuous-Time Sequence Modelling with Path Signatures
Time-series data in real-world settings typically exhibit long-range dependencies and are observed at non-uniform intervals. In these settings, traditional sequencebased recurrent models struggle. To overcome this, researchers often replace recurrent architectures with Neural ODE-based models to account for irregularly sampled data and use Transformer-based architectures to account for long-range dependencies. …