Ruyu Liu

Search for another author

Author Description

Ask a Question About This Mathematician

Chaos detection of Duffing system with fractional-order derivative by Melnikov method

2019-12-01

Jiangchuan Niu , Ruyu Liu , Yongjun Shen +1 more

The chaos detection of the Duffing system with the fractional-order derivative subjected to external harmonic excitation is investigated by the Melnikov method. In order to apply the Melnikov method to … The chaos detection of the Duffing system with the fractional-order derivative subjected to external harmonic excitation is investigated by the Melnikov method. In order to apply the Melnikov method to detect the chaos of the Duffing system with the fractional-order derivative, it is transformed into the first-order approximate equivalent integer-order system via the harmonic balance method, which has the same steady-state amplitude-frequency response equation with the original system. Also, the amplitude-frequency response of the Duffing system with the fractional-order derivative and its first-order approximate equivalent integer-order system are compared by the numerical solutions, and they are in good agreement. Then, the analytical chaos criterion of the Duffing system with the fractional-order derivative is obtained by the Melnikov function. The bifurcation and chaos of the Duffing system with the fractional-order derivative and an integer-order derivative are analyzed in detail, and the chaos criterion obtained by the Melnikov function is verified by using bifurcation analysis and phase portraits. The analysis results show that the Melnikov method is effective to detect the chaos in the Duffing system with the fractional-order derivative by transforming it into an equivalent integer-order system.

Stability and bifurcation analysis of single-degree-of-freedom linear vibro-impact system with fractional-order derivative

2019-04-02

Jiangchuan Niu , Ruyu Liu , Yongjun Shen +1 more

Collaborative Visual Inertial SLAM for Multiple Smart Phones

2021-05-30

Jialing Liu , Ruyu Liu , Kaiqi Chen +2 more

The efficiency and accuracy of mapping are crucial in a large scene and long-term AR applications. Multi-agent cooperative SLAM is the precondition of multi-user AR interaction. The cooperation of multiple … The efficiency and accuracy of mapping are crucial in a large scene and long-term AR applications. Multi-agent cooperative SLAM is the precondition of multi-user AR interaction. The cooperation of multiple smart phones has the potential to improve efficiency and robustness of task completion and can complete tasks that a single agent cannot do. However, it depends on robust communication, efficient location detection, robust mapping, and efficient information sharing among agents. We propose a multi-intelligence collaborative monocular visual-inertial SLAM deployed on multiple ios mobile devices with a centralized architecture. Each agent can independently explore the environment, run a visual-inertial odometry module online, and then send all the measurement information to a central server with higher computing resources. The server manages all the information received, detects overlapping areas, merges and optimizes the map, and shares information with the agents when needed. We have verified the performance of the system in public datasets and real environments. The accuracy of mapping and fusion of the proposed system is comparable to VINS-Mono which requires higher computing resources.

View PDF Chat

Semantic Visual Simultaneous Localization and Mapping: A Survey

2022-01-01

Kaiqi Chen , Jianhua Zhang , Jialing Liu +3 more

Visual Simultaneous Localization and Mapping (vSLAM) has achieved great progress in the computer vision and robotics communities, and has been successfully used in many fields such as autonomous robot navigation … Visual Simultaneous Localization and Mapping (vSLAM) has achieved great progress in the computer vision and robotics communities, and has been successfully used in many fields such as autonomous robot navigation and AR/VR. However, vSLAM cannot achieve good localization in dynamic and complex environments. Numerous publications have reported that, by combining with the semantic information with vSLAM, the semantic vSLAM systems have the capability of solving the above problems in recent years. Nevertheless, there is no comprehensive survey about semantic vSLAM. To fill the gap, this paper first reviews the development of semantic vSLAM, explicitly focusing on its strengths and differences. Secondly, we explore three main issues of semantic vSLAM: the extraction and association of semantic information, the application of semantic information, and the advantages of semantic vSLAM. Then, we collect and analyze the current state-of-the-art SLAM datasets which have been widely used in semantic vSLAM systems. Finally, we discuss future directions that will provide a blueprint for the future development of semantic vSLAM.

View PDF Chat

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

2024-02-20

Ruyu Liu , Shaohua Pan , Yuqia Wu +1 more

Abstract This paper focuses on the minimization of a sum of a twice continuously differentiable function f and a nonsmooth convex function. An inexact regularized proximal Newton method is proposed … Abstract This paper focuses on the minimization of a sum of a twice continuously differentiable function f and a nonsmooth convex function. An inexact regularized proximal Newton method is proposed by an approximation to the Hessian of f involving the $$\varrho $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ϱ</mml:mi> </mml:math> th power of the KKT residual. For $$\varrho =0$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>ϱ</mml:mi> <mml:mo>=</mml:mo> <mml:mn>0</mml:mn> </mml:mrow> </mml:math> , we justify the global convergence of the iterate sequence for the KL objective function and its R-linear convergence rate for the KL objective function of exponent 1/2. For $$\varrho \in (0,1)$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>ϱ</mml:mi> <mml:mo>∈</mml:mo> <mml:mo>(</mml:mo> <mml:mn>0</mml:mn> <mml:mo>,</mml:mo> <mml:mn>1</mml:mn> <mml:mo>)</mml:mo> </mml:mrow> </mml:math> , by assuming that cluster points satisfy a locally Hölderian error bound of order q on a second-order stationary point set and a local error bound of order $$q>1\!+\!\varrho $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>q</mml:mi> <mml:mo>></mml:mo> <mml:mn>1</mml:mn> <mml:mspace/> <mml:mo>+</mml:mo> <mml:mspace/> <mml:mi>ϱ</mml:mi> </mml:mrow> </mml:math> on the common stationary point set, respectively, we establish the global convergence of the iterate sequence and its superlinear convergence rate with order depending on q and $$\varrho $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ϱ</mml:mi> </mml:math> . A dual semismooth Newton augmented Lagrangian method is also developed for seeking an inexact minimizer of subproblems. Numerical comparisons with two state-of-the-art methods on $$\ell _1$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>ℓ</mml:mi> <mml:mn>1</mml:mn> </mml:msub> </mml:math> -regularized Student’s t -regressions, group penalized Student’s t -regressions, and nonconvex image restoration confirm the efficiency of the proposed method.

View PDF Chat

Collaborative Visual Inertial SLAM for Multiple Smart Phones

2021-06-23

Jialing Liu , Ruyu Liu , Kaiqi Chen +2 more

View PDF Chat

360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network

2024-05-08

Yichen Chen , Yuqi Pan , Ruyu Liu +4 more

View PDF Chat

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

2022-01-01

Ruyu Liu , Shaohua Pan , Yuqia Wu +1 more

This paper focuses on the minimization of a sum of a twice continuously differentiable function $f$ and a nonsmooth convex function. An inexact regularized proximal Newton method is proposed by … This paper focuses on the minimization of a sum of a twice continuously differentiable function $f$ and a nonsmooth convex function. An inexact regularized proximal Newton method is proposed by an approximation to the Hessian of $f$ involving the $\varrho$th power of the KKT residual. For $\varrho=0$, we justify the global convergence of the iterate sequence for the KL objective function and its R-linear convergence rate for the KL objective function of exponent $1/2$. For $\varrho\in(0,1)$, by assuming that cluster points satisfy a locally H\"{o}lderian error bound of order $q$ on a second-order stationary point set and a local error bound of order $q>1\!+\!\varrho$ on the common stationary point set, respectively, we establish the global convergence of the iterate sequence and its superlinear convergence rate with order depending on $q$ and $\varrho$. A dual semismooth Newton augmented Lagrangian method is also developed for seeking an inexact minimizer of subproblems. Numerical comparisons with two state-of-the-art methods on $\ell_1$-regularized Student's $t$-regressions, group penalized Student's $t$-regressions, and nonconvex image restoration confirm the efficiency of the proposed method.

Weighted Estimate with Multi-Weight for Multilinear Calderón-Zygmund Operators

2010-01-01

Ruyu Liu

In this paper,a weighted weak type endpoint estimate with general weights for multilinear Calderon-Zygmund operators is established.As an application,by dual argument and employing the maximal operator with Orlicz function,a weighted … In this paper,a weighted weak type endpoint estimate with general weights for multilinear Calderon-Zygmund operators is established.As an application,by dual argument and employing the maximal operator with Orlicz function,a weighted estimate with multi-weight is given for the multilinear Calderon-Zygmund operators.

Collaborative Visual Inertial SLAM for Multiple Smart Phones

2021-01-01

Jialing Liu , Ruyu Liu , Kaiqi Chen +2 more

View PDF Chat

An inexact LPA for DC composite optimization and application to matrix completions with outliers

2023-01-01

Ting Tao , Ruyu Liu , Lianghai Xiao +1 more

This paper concerns a class of DC composite optimization problems which, as an extension of convex composite optimization problems and DC programs with nonsmooth components, often arises in robust factorization … This paper concerns a class of DC composite optimization problems which, as an extension of convex composite optimization problems and DC programs with nonsmooth components, often arises in robust factorization models of low-rank matrix recovery. For this class of nonconvex and nonsmooth problems, we propose an inexact linearized proximal algorithm (iLPA) by computing in each step an inexact minimizer of a strongly convex majorization constructed with a partial linearization of their objective functions at the current iterate, and establish the convergence of the generated iterate sequence under the Kurdyka-\L\"ojasiewicz (KL) property of a potential function. In particular, by leveraging the composite structure, we provide a verifiable condition for the potential function to have the KL property of exponent $1/2$ at the limit point, so for the iterate sequence to have a local R-linear convergence rate. Finally, we apply the proposed iLPA to a robust factorization model for matrix completions with outliers and non-uniform sampling, and numerical comparison with the Polyak subgradient method confirms its superiority in terms of computing time and quality of solutions.

View PDF Chat

A VMiPG method for composite optimization with nonsmooth term having no closed-form proximal mapping

2023-01-01

T. Zhang , Shaohua Pan , Ruyu Liu

This paper concerns the minimization of the sum of a twice continuously differentiable function $f$ and a nonsmooth convex function $g$ without closed-form proximal mapping. For this class of nonconvex … This paper concerns the minimization of the sum of a twice continuously differentiable function $f$ and a nonsmooth convex function $g$ without closed-form proximal mapping. For this class of nonconvex and nonsmooth problems, we propose a line-search based variable metric inexact proximal gradient (VMiPG) method with uniformly bounded positive definite variable metric linear operators. This method computes in each step an inexact minimizer of a strongly convex model such that the difference between its objective value and the optimal value is controlled by its squared distance from the current iterate, and then seeks an appropriate step-size along the obtained direction with an armijo line-search criterion. We prove that the iterate sequence converges to a stationary point when $f$ and $g$ are definable in the same o-minimal structure over the real field $(\mathbb{R},+,\cdot)$, and if addition the objective function $f+g$ is a KL function of exponent $1/2$, the convergence has a local R-linear rate. The proposed VMiPG method with the variable metric linear operator constructed by the Hessian of the function $f$ is applied to the scenario that $f$ and $g$ have common composite structure, and numerical comparison with a state-of-art variable metric line-search algorithm indicates that the Hessian-based VMiPG method has a remarkable advantage in terms of the quality of objective values and the running time for those difficult problems such as high-dimensional fused weighted-lasso regressions.

View PDF Chat

An inexact $q$-order regularized proximal Newton method for nonconvex composite optimization

2023-01-01

Ruyu Liu , Shaohua Pan , Yitian Qian

This paper concerns the composite problem of minimizing the sum of a twice continuously differentiable function $f$ and a nonsmooth convex function. For this class of nonconvex and nonsmooth problems, … This paper concerns the composite problem of minimizing the sum of a twice continuously differentiable function $f$ and a nonsmooth convex function. For this class of nonconvex and nonsmooth problems, by leveraging a practical inexactness criterion and a novel selection strategy for iterates, we propose an inexact $q\in[2,3]$-order regularized proximal Newton method, which becomes an inexact cubic regularization (CR) method for $q=3$. We justify that its iterate sequence converges to a stationary point for the KL objective function, and if the objective function has the KL property of exponent $\theta\in(0,\frac{q-1}{q})$, the convergence has a local $Q$-superlinear rate of order $\frac{q-1}{\theta q}$. In particular, under a locally H\"{o}lderian error bound of order $\gamma\in(\frac{1}{q-1},1]$ on a second-order stationary point set, the iterate sequence converges to a second-order stationary point with a local $Q$-superlinear rate of order $\gamma(q\!-\!1)$, which is specified as $Q$-quadratic rate for $q=3$ and $\gamma=1$. This is the first practical inexact CR method with $Q$-quadratic convergence rate for nonconvex composite optimization. We validate the efficiency of the proposed method with ZeroFPR as the solver of subproblems by applying it to convex and nonconvex composite problems with a highly nonlinear $f$.

360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network

2024-01-01

Yichen Chen , Yiqi Pan , Ruyu Liu +4 more

To enhance the performance and effect of AR/VR applications and visual assistance and inspection systems, visual simultaneous localization and mapping (vSLAM) is a fundamental task in computer vision and robotics. … To enhance the performance and effect of AR/VR applications and visual assistance and inspection systems, visual simultaneous localization and mapping (vSLAM) is a fundamental task in computer vision and robotics. However, traditional vSLAM systems are limited by the camera's narrow field-of-view, resulting in challenges such as sparse feature distribution and lack of dense depth information. To overcome these limitations, this paper proposes a 360ORB-SLAM system for panoramic images that combines with a depth completion network. The system extracts feature points from the panoramic image, utilizes a panoramic triangulation module to generate sparse depth information, and employs a depth completion network to obtain a dense panoramic depth map. Experimental results on our novel panoramic dataset constructed based on Carla demonstrate that the proposed method achieves superior scale accuracy compared to existing monocular SLAM methods and effectively addresses the challenges of feature association and scale ambiguity. The integration of the depth completion network enhances system stability and mitigates the impact of dynamic elements on SLAM performance.

View PDF Chat

A VMiPG Method for Composite Optimization with Nonsmooth Term Having No Closed-form Proximal Mapping

2024-11-04

T. Zhang , Shaohua Pan , Ruyu Liu

Can Students Beyond The Teacher? Distilling Knowledge from Teacher's Bias

2024-12-13

Jianhua Zhang , Yi Qin Gao , Ruyu Liu +3 more

Knowledge distillation (KD) is a model compression technique that transfers knowledge from a large teacher model to a smaller student model to enhance its performance. Existing methods often assume that … Knowledge distillation (KD) is a model compression technique that transfers knowledge from a large teacher model to a smaller student model to enhance its performance. Existing methods often assume that the student model is inherently inferior to the teacher model. However, we identify that the fundamental issue affecting student performance is the bias transferred by the teacher. Current KD frameworks transmit both right and wrong knowledge, introducing bias that misleads the student model. To address this issue, we propose a novel strategy to rectify bias and greatly improve the student model's performance. Our strategy involves three steps: First, we differentiate knowledge and design a bias elimination method to filter out biases, retaining only the right knowledge for the student model to learn. Next, we propose a bias rectification method to rectify the teacher model's wrong predictions, fundamentally addressing bias interference. The student model learns from both the right knowledge and the rectified biases, greatly improving its prediction accuracy. Additionally, we introduce a dynamic learning approach with a loss function that updates weights dynamically, allowing the student model to quickly learn right knowledge-based easy tasks initially and tackle hard tasks corresponding to biases later, greatly enhancing the student model's learning efficiency. To the best of our knowledge, this is the first strategy enabling the student model to surpass the teacher model. Experiments demonstrate that our strategy, as a plug-and-play module, is versatile across various mainstream KD frameworks. We will release our code after the paper is accepted.

View PDF Chat

Can Students Beyond the Teacher? Distilling Knowledge from Teacher’s Bias

2025-04-11

Jianhua Zhang , Yi Qin Gao , Ruyu Liu +3 more

View PDF Chat

An Inexact ${q}$-Order Regularized Proximal Newton Method for Nonconvex Composite Optimization

2025-04-28

Ruyu Liu , PAN SHAO-HUA , Yitian Qian

SVD-KD: SVD-based hidden layer feature extraction for Knowledge distillation

2025-05-01

Jianhua Zhang , Yi Qin Gao , Mian Zhou +4 more

PULLM: A Multimodal Framework for Enhanced 3D Point Cloud Upsampling Using Large Language Models

2025-03-31

Zhi-yong Zhang , Ruyu Liu , Xiufeng Liu +4 more

Semantic Visual Simultaneous Localization and Mapping: A Survey

2025-01-01

Kaiqi Chen , Junhao Xiao , Jialing Liu +6 more

View PDF Chat

SVD-KD: SVD-based hidden layer feature extraction for Knowledge distillation

2025-05-01

Jianhua Zhang , Yi Qin Gao , Mian Zhou +4 more

An Inexact ${q}$-Order Regularized Proximal Newton Method for Nonconvex Composite Optimization

2025-04-28

Ruyu Liu , PAN SHAO-HUA , Yitian Qian

Can Students Beyond the Teacher? Distilling Knowledge from Teacher’s Bias

2025-04-11

Jianhua Zhang , Yi Qin Gao , Ruyu Liu +3 more

View PDF Chat

PULLM: A Multimodal Framework for Enhanced 3D Point Cloud Upsampling Using Large Language Models

2025-03-31

Zhi-yong Zhang , Ruyu Liu , Xiufeng Liu +4 more

Semantic Visual Simultaneous Localization and Mapping: A Survey

2025-01-01

Kaiqi Chen , Junhao Xiao , Jialing Liu +6 more

View PDF Chat

Can Students Beyond The Teacher? Distilling Knowledge from Teacher's Bias

2024-12-13

Jianhua Zhang , Yi Qin Gao , Ruyu Liu +3 more

View PDF Chat

A VMiPG Method for Composite Optimization with Nonsmooth Term Having No Closed-form Proximal Mapping

2024-11-04

T. Zhang , Shaohua Pan , Ruyu Liu

360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network

2024-05-08

Yichen Chen , Yuqi Pan , Ruyu Liu +4 more

View PDF Chat

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

2024-02-20

Ruyu Liu , Shaohua Pan , Yuqia Wu +1 more

View PDF Chat

360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network

2024-01-01

Yichen Chen , Yiqi Pan , Ruyu Liu +4 more

View PDF Chat

An inexact LPA for DC composite optimization and application to matrix completions with outliers

2023-01-01

Ting Tao , Ruyu Liu , Lianghai Xiao +1 more

View PDF Chat

A VMiPG method for composite optimization with nonsmooth term having no closed-form proximal mapping

2023-01-01

T. Zhang , Shaohua Pan , Ruyu Liu

View PDF Chat

An inexact $q$-order regularized proximal Newton method for nonconvex composite optimization

2023-01-01

Ruyu Liu , Shaohua Pan , Yitian Qian

Semantic Visual Simultaneous Localization and Mapping: A Survey

2022-01-01

Kaiqi Chen , Jianhua Zhang , Jialing Liu +3 more

View PDF Chat

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

2022-01-01

Ruyu Liu , Shaohua Pan , Yuqia Wu +1 more

Collaborative Visual Inertial SLAM for Multiple Smart Phones

2021-06-23

Jialing Liu , Ruyu Liu , Kaiqi Chen +2 more

View PDF Chat

Collaborative Visual Inertial SLAM for Multiple Smart Phones

2021-05-30

Jialing Liu , Ruyu Liu , Kaiqi Chen +2 more

View PDF Chat

Collaborative Visual Inertial SLAM for Multiple Smart Phones

2021-01-01

Jialing Liu , Ruyu Liu , Kaiqi Chen +2 more

View PDF Chat

Chaos detection of Duffing system with fractional-order derivative by Melnikov method

2019-12-01

Jiangchuan Niu , Ruyu Liu , Yongjun Shen +1 more

Stability and bifurcation analysis of single-degree-of-freedom linear vibro-impact system with fractional-order derivative

2019-04-02

Jiangchuan Niu , Ruyu Liu , Yongjun Shen +1 more

Weighted Estimate with Multi-Weight for Multilinear Calderón-Zygmund Operators

2010-01-01

Ruyu Liu

Coauthor	Papers Together
Jianhua Zhang	7
Shaohua Pan	6
Shengyong Chen	5
Kaiqi Chen	5
Jialing Liu	4
Yi Qin Gao	3
Yitian Qian	2
Shaopu Yang	2
Xiaoqi Yang	2
Cheng Xu	2
Jiangchuan Niu	2
Houxiang Zhang	2
T. Zhang	2
Dongyan Guo	2
Guodao Zhang	2
Qiyi Tong	2
Jianhua Zhang	2
Yichen Chen	2
Jianhua Zhang	2
Yuqia Wu	2
Yongjun Shen	2
Yiqi Pan	1
Yuqi Pan	1
Yunrui Zhu	1
Lianghai Xiao	1
Jialing Liu	1
Ting Tao	1
Chaochao Wang	1
Mian Zhou	1
Arash Ajoudani	1
Yanyan Yang	1
Haoyu Zhang	1
Haoyu Zhang	1
Junhao Xiao	1
Xiufeng Liu	1
Bo Sun	1
Saša V. Nikolić	1
Dongyan Guo	1
Heng Zhang	1
PAN SHAO-HUA	1
Zhi-yong Zhang	1
Xu Cheng	1
Bo Sun	1

Robust inversion, dimensionality reduction, and randomized sampling

2012-06-28

Aleksandr Y. Aravkin , Michael P. Friedlander , Felix J. Herrmann +1 more

View PDF Chat

On the convergence of a linesearch based proximal-gradient method for nonconvex optimization

2017-01-31

Silvia Bonettini , Ignace Loris , Federica Porta +2 more

We consider a variable metric linesearch based proximal gradient method for the minimization of the sum of a smooth, possibly nonconvex function plus a convex, possibly nonsmooth term. We prove … We consider a variable metric linesearch based proximal gradient method for the minimization of the sum of a smooth, possibly nonconvex function plus a convex, possibly nonsmooth term. We prove convergence of this iterative algorithm to a critical point if the objective function satisfies the Kurdyka-Lojasiewicz property at each point of its domain, under the assumption that a limit point exists. The proposed method is applied to a wide collection of image processing problems and our numerical tests show that our algorithm results to be flexible, robust and competitive when compared to recently proposed approaches able to address the optimization problems arising in the considered applications.

View PDF Chat

Proximal alternating linearized minimization for nonconvex and nonsmooth problems

2013-07-25

Jérôme Bolte , Shoham Sabach , Marc Teboulle

Globalized inexact proximal Newton-type methods for nonconvex composite functions

2020-11-16

Christian Kanzow , Theresa Lechner

Abstract Optimization problems with composite functions consist of an objective function which is the sum of a smooth and a (convex) nonsmooth term. This particular structure is exploited by the … Abstract Optimization problems with composite functions consist of an objective function which is the sum of a smooth and a (convex) nonsmooth term. This particular structure is exploited by the class of proximal gradient methods and some of their generalizations like proximal Newton and quasi-Newton methods. The current literature on these classes of methods almost exclusively considers the case where also the smooth term is convex. Here we present a globalized proximal Newton-type method which allows the smooth term to be nonconvex. The method is shown to have nice global and local convergence properties, and some numerical results indicate that this method is very promising also from a practical point of view.

View PDF Chat

A family of inexact SQA methods for non-smooth convex minimization with provable convergence guarantees based on the Luo–Tseng error bound property

2018-04-30

Man–Chung Yue , Zirui Zhou , Anthony Man–Cho So

Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods

2011-08-19

Hédy Attouch , Jérôme Bolte , B. F. Svaiter

View PDF Chat

Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality

2010-05-01

Hédy Attouch , Jérôme Bolte , Patrick Redont +1 more

We study the convergence properties of an alternating proximal minimization algorithm for nonconvex structured functions of the type: L(x,y)=f(x)+Q(x,y)+g(y), where f and g are proper lower semicontinuous functions, defined on … We study the convergence properties of an alternating proximal minimization algorithm for nonconvex structured functions of the type: L(x,y)=f(x)+Q(x,y)+g(y), where f and g are proper lower semicontinuous functions, defined on Euclidean spaces, and Q is a smooth function that couples the variables x and y. The algorithm can be viewed as a proximal regularization of the usual Gauss-Seidel method to minimize L. We work in a nonconvex setting, just assuming that the function L satisfies the Kurdyka-Łojasiewicz inequality. An entire section illustrates the relevancy of such an assumption by giving examples ranging from semialgebraic geometry to “metrically regular” problems. Our main result can be stated as follows: If L has the Kurdyka-Łojasiewicz property, then each bounded sequence generated by the algorithm converges to a critical point of L. This result is completed by the study of the convergence rate of the algorithm, which depends on the geometrical properties of the function L around its critical points. When specialized to [Formula: see text] and to f, g indicator functions, the algorithm is an alternating projection mehod (a variant of von Neumann's) that converges for a wide class of sets including semialgebraic and tame sets, transverse smooth manifolds or sets with “regular” intersection. To illustrate our results with concrete problems, we provide a convergent proximal reweighted ℓ 1 algorithm for compressive sensing and an application to rank reduction problems.

View PDF Chat

A globally convergent proximal Newton-type method in nonsmooth convex optimization

2022-03-22

Boris S. Mordukhovich , Xiaoming Yuan , Shangzhi Zeng +1 more

View PDF Chat

Variational Analysis

1998-01-01

R. T. Rockafellar , Roger J.‐B. Wets

Calculus of the Exponent of Kurdyka–Łojasiewicz Inequality and Its Applications to Linear Convergence of First-Order Methods

2017-08-10

Guoyin Li , Ting Kei Pong

View PDF Chat

A new collection of real world applications of fractional calculus in science and engineering

2018-04-24

HongGuang Sun , Yong Zhang , Dumitru Băleanu +2 more

Solving High-Order Portfolios via Successive Convex Approximation Algorithms

2021-01-01

Rui Zhou , Daniel P. Palomar

The first moment and second central moments of the portfolio return, a.k.a. mean and variance, have been widely employed to assess the expected profit and risk of the portfolio. Investors … The first moment and second central moments of the portfolio return, a.k.a. mean and variance, have been widely employed to assess the expected profit and risk of the portfolio. Investors pursue higher mean and lower variance when designing the portfolios. The two moments can well describe the distribution of the portfolio return when it follows the Gaussian distribution. However, the real world distribution of assets return is usually asymmetric and heavy-tailed, which is far from being a Gaussian distribution. The asymmetry and the heavy-tailedness are characterized by the third and fourth central moments, i.e., skewness and kurtosis, respectively. Higher skewness and lower kurtosis are preferred to reduce the probability of extreme losses. However, incorporating high-order moments in the portfolio design is very difficult due to their non-convexity and rapidly increasing computational cost with the dimension. In this paper, we propose a very efficient and convergence-provable algorithm framework based on the successive convex approximation (SCA) algorithm to solve high-order portfolios. The efficiency of the proposed algorithm framework is demonstrated by the numerical experiments.

View PDF Chat

Kurdyka–Łojasiewicz Exponent via Inf-projection

2021-07-09

Peiran Yu , Guoyin Li , Ting Kei Pong

View PDF Chat

An efficient DC programming approach for portfolio decision with higher moments

2010-12-22

Tao Pham Dinh , Yi-Shuai Niu

ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM

2021-05-25

Carlos Campos , Richard Elvira , Juan J. Gómez Rodríguez +2 more

This paper presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. The first main … This paper presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. The first main novelty is a feature-based tightly-integrated visual-inertial SLAM system that fully relies on Maximum-a-Posteriori (MAP) estimation, even during the IMU initialization phase. The result is a system that operates robustly in real-time, in small and large, indoor and outdoor environments, and is 2 to 5 times more accurate than previous approaches. The second main novelty is a multiple map system that relies on a new place recognition method with improved recall. Thanks to it, ORB-SLAM3 is able to survive to long periods of poor visual information: when it gets lost, it starts a new map that will be seamlessly merged with previous maps when revisiting mapped areas. Compared with visual odometry systems that only use information from the last few seconds, ORB-SLAM3 is the first system able to reuse in all the algorithm stages all previous information. This allows to include in bundle adjustment co-visible keyframes, that provide high parallax observations boosting accuracy, even if they are widely separated in time or if they come from a previous mapping session. Our experiments show that, in all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature, and significantly more accurate. Notably, our stereo-inertial SLAM achieves an average accuracy of 3.6 cm on the EuRoC drone and 9 mm under quick hand-held motions in the room of TUM-VI dataset, a setting representative of AR/VR scenarios. For the benefit of the community we make public the source code.

View PDF Chat

On the Fractional Order Model of Viscoelasticity

2005-01-01

Klas Adolfsson , Mikael Enelund , Peter Olsson

An Invitation to Tame Optimization

2009-01-01

A. D. Ioffe

The word “tame” is used in the title in the same context as in expressions like “convex optimization,” “nonsmooth optimization,” etc.—as a reference to the class of objects involved in … The word “tame” is used in the title in the same context as in expressions like “convex optimization,” “nonsmooth optimization,” etc.—as a reference to the class of objects involved in the formulation of optimization problems. Definable and tame functions and mappings associated with various o-minimal structures (e.g. semilinear, semialgebraic, globally subanalytic, and others) have a number of remarkable properties which make them an attractive domain for various applications. This relates both to the power of results that can be obtained and the power of available analytic techniques. The paper surveys certain ideas and recent results, some new, which have been or (hopefully) can be productively used in studies relating to variational analysis and nonsmooth optimization.

A Semismooth Newton Method with Multidimensional Filter Globalization for $l_1$-Optimization

2014-01-01

Andre Milzarek , Michael Ulbrich

Due to their property of enhancing the sparsity of solutions, $l_1$-regularized optimization problems have developed into a highly dynamic research area with a wide range of applications. We present a … Due to their property of enhancing the sparsity of solutions, $l_1$-regularized optimization problems have developed into a highly dynamic research area with a wide range of applications. We present a class of methods for $l_1$-regularized optimization problems that are based on a combination of semismooth Newton steps, a filter globalization, and shrinkage/thresholding steps. A multidimensional filter framework is used to control the acceptance and to evaluate the quality of the semismooth Newton steps. If the current Newton iterate is rejected a shrinkage/thresholding-based step with quasi-Armijo stepsize rule is used instead. Global convergence and transition to local q-superlinear convergence for both convex and nonconvex objective functions are established. We present numerical results and comparisons with several state-of-the-art methods that show the efficiency and competitiveness of the proposed method.

Data-Efficient Decentralized Visual SLAM

2018-05-01

Titus Cieslewski , Siddharth Choudhary , Davide Scaramuzza

Decentralized visual simultaneous localization and mapping (SLAM) is a powerful tool for multi-robot applications in environments where absolute positioning is not available. Being visual, it relies on cheap, lightweight and … Decentralized visual simultaneous localization and mapping (SLAM) is a powerful tool for multi-robot applications in environments where absolute positioning is not available. Being visual, it relies on cheap, lightweight and versatile cameras, and, being decentralized, it does not rely on communication to a central entity. In this work, we integrate state-of-the-art decentralized SLAM components into a new, complete decentralized visual SLAM system. To allow for data association and optimization, existing decentralized visual SLAM systems exchange the full map data among all robots, incurring large data transfers at a complexity that scales quadratically with the robot count. In contrast, our method performs efficient data association in two stages: first, a compact full-image descriptor is deterministically sent to only one robot. Then, only if the first stage succeeded, the data required for relative pose estimation is sent, again to only one robot. Thus, data association scales linearly with the robot count and uses highly compact place representations. For optimization, a state-of-the-art decentralized pose-graph optimization method is used. It exchanges a minimum amount of data which is linear with trajectory overlap. We characterize the resulting system and identify bottlenecks in its components. The system is evaluated on publicly available datasets and we provide open access to the code. Supplementary Material Data and code are at: https://github.com/uzh-rpg/dslam_open.

View PDF Chat

On the convergence of the proximal algorithm for nonsmooth functions involving analytic features

2007-05-03

Hédy Attouch , Jérôme Bolte

View PDF Chat

Efficient decentralized visual place recognition from full-image descriptors

2017-12-01

Titus Cieslewski , Davide Scaramuzza

In this paper, we discuss the adaptation of our decentralized place recognition method described in [1] to full image descriptors. As we had shown, the key to making a scalable … In this paper, we discuss the adaptation of our decentralized place recognition method described in [1] to full image descriptors. As we had shown, the key to making a scalable decentralized visual place recognition lies in exploting deterministic key assignment in a distributed key-value map. Through this, it is possible to reduce bandwidth by up to a factor of n, the robot count, by casting visual place recognition to a key-value lookup problem. In [1], we exploited this for the bag-of-words method [3], [4]. Our method of casting bag-of-words, however, results in a complex decentralized system, which has inherently worse recall than its centralized counterpart. In this paper, we instead start from the recent full-image description method NetVLAD [5]. As we show, casting this to a key-value lookup problem can be achieved with k-means clustering, and results in a much simpler system than [1]. The resulting system still has some flaws, albeit of a completely different nature: it suffers when the environment seen during deployment lies in a different distribution in feature space than the environment seen during training.

View PDF Chat

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

2024-02-20

Ruyu Liu , Shaohua Pan , Yuqia Wu +1 more

View PDF Chat

A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

2009-01-01

Amir Beck , Marc Teboulle

We consider the class of iterative shrinkage-thresholding algorithms (ISTA) for solving linear inverse problems arising in signal/image processing. This class of methods, which can be viewed as an extension of … We consider the class of iterative shrinkage-thresholding algorithms (ISTA) for solving linear inverse problems arising in signal/image processing. This class of methods, which can be viewed as an extension of the classical gradient algorithm, is attractive due to its simplicity and thus is adequate for solving large-scale problems even with dense matrix data. However, such methods are also known to converge quite slowly. In this paper we present a new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically. Initial promising numerical results for wavelet-based image deblurring demonstrate the capabilities of FISTA which is shown to be faster than ISTA by several orders of magnitude.

Active Rendezvous for Multi-robot Pose Graph Optimization Using Sensing over Wi-Fi

2022-01-01

Weiying Wang , Ninad Jadhav , Paul Vohs +3 more

View PDF Chat

VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator

2018-07-27

Tong Qin , Peiliang Li , Shaojie Shen

A monocular visual-inertial system (VINS), consisting of a camera and a low-cost inertial measurement unit (IMU), forms the minimum sensor suite for metric six degrees-of-freedom (DOF) state estimation. However, the … A monocular visual-inertial system (VINS), consisting of a camera and a low-cost inertial measurement unit (IMU), forms the minimum sensor suite for metric six degrees-of-freedom (DOF) state estimation. However, the lack of direct distance measurement poses significant challenges in terms of IMU processing, estimator initialization, extrinsic calibration, and nonlinear optimization. In this work, we present VINS-Mono: a robust and versatile monocular visual-inertial state estimator.Our approach starts with a robust procedure for estimator initialization and failure recovery. A tightly-coupled, nonlinear optimization-based method is used to obtain high accuracy visual-inertial odometry by fusing pre-integrated IMU measurements and feature observations. A loop detection module, in combination with our tightly-coupled formulation, enables relocalization with minimum computation overhead.We additionally perform four degrees-of-freedom pose graph optimization to enforce global consistency. We validate the performance of our system on public datasets and real-world experiments and compare against other state-of-the-art algorithms. We also perform onboard closed-loop autonomous flight on the MAV platform and port the algorithm to an iOS-based demonstration. We highlight that the proposed work is a reliable, complete, and versatile system that is applicable for different applications that require high accuracy localization. We open source our implementations for both PCs and iOS mobile devices.

View PDF Chat

A nonsmooth version of Newton's method

1993-01-01

Liqun Qi , Jie Sun

Inexact Successive quadratic approximation for regularized optimization

2019-01-25

Ching-pei Lee , Stephen J. Wright

View PDF Chat

A method to stochastic dynamical systems with strong nonlinearity and fractional damping

2015-11-12

Yong Xu , Yongge Li , Di Liu

Hölder Metric Subregularity with Applications to Proximal Point Method

2012-01-01

Guoyin Li , Boris S. Mordukhovich

This paper is mainly devoted to the study and applications of Hölder metric subregularity (or metric $q$-subregularity of order $q\in(0,1]$) for general set-valued mappings between infinite-dimensional spaces. Employing advanced techniques … This paper is mainly devoted to the study and applications of Hölder metric subregularity (or metric $q$-subregularity of order $q\in(0,1]$) for general set-valued mappings between infinite-dimensional spaces. Employing advanced techniques of variational analysis and generalized differentiation, we derive neighborhood and point-based sufficient conditions as well as necessary conditions for $q$-metric subregularity with evaluating the exact subregularity bound, which are new even for the conventional (first-order) metric subregularity in both finite and infinite dimensions. In this way we also obtain new fractional error bound results for composite polynomial systems with explicit calculating fractional exponents. Finally, metric $q$-subregularity is applied to conduct a quantitative convergence analysis of the classical proximal point method (PPM) for finding zeros of maximal monotone operators on Hilbert spaces.

View PDF Chat

A coordinate gradient descent method for nonsmooth separable minimization

2007-07-31

Paul Tseng , Sangwoon Yun

Tame functions are semismooth

2007-07-20

Jérôme Bolte , Aris Daniilidis , Adrian S. Lewis

Optimization Methods for Large-Scale Machine Learning

2018-01-01

Léon Bottou , Frank E. Curtis , Jorge Nocedal

This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification … This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. Based on this viewpoint, we present a comprehensive theory of a straightforward, yet versatile SG algorithm, discuss its practical behavior, and highlight opportunities for designing algorithms with improved performance. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques that diminish noise in the stochastic directions and methods that make use of second-order derivative approximations.

View PDF Chat

An inexact successive quadratic approximation method for L-1 regularized optimization

2015-08-25

Richard H. Byrd , Jorge Nocedal , Figen Öztoprak

Variable Metric Inexact Line-Search-Based Methods for Nonsmooth Optimization

2016-01-01

Silvia Bonettini , Ignace Loris , Federica Porta +1 more

We develop a new proximal-gradient method for minimizing the sum of a differentiable, possibly nonconvex, function plus a convex, possibly nondifferentiable, function. The key features of the proposed method are … We develop a new proximal-gradient method for minimizing the sum of a differentiable, possibly nonconvex, function plus a convex, possibly nondifferentiable, function. The key features of the proposed method are the definition of a suitable descent direction, based on the proximal operator associated to the convex part of the objective function, and an Armijo-like rule to determine the stepsize along this direction ensuring the sufficient decrease of the objective function. In this frame, we especially address the possibility of adopting a metric which may change at each iteration and an inexact computation of the proximal point defining the descent direction. For the more general nonconvex case, we prove that all limit points of the iterates sequence are stationary, while for convex objective functions we prove the convergence of the whole sequence to a minimizer, under the assumption that a minimizer exists. In the latter case, assuming also that the gradient of the smooth part of the objective function is Lipschitz, we also give a convergence rate estimate, showing the ${\mathcal O}(\frac 1 k)$ complexity with respect to the function values. We also discuss verifiable sufficient conditions for the inexact proximal point and present the results of two numerical tests on total-variation-based image restoration problems, showing that the proposed approach is competitive with other state-of-the-art methods.

View PDF Chat

A Highly Efficient Semismooth Newton Augmented Lagrangian Method for Solving Lasso Problems

2018-01-01

Xudong Li , Defeng Sun , Kim-Chuan Toh

We develop a fast and robust algorithm for solving large-scale convex composite optimization models with an emphasis on the $\ell_1$-regularized least squares regression (lasso) problems. Despite the fact that there … We develop a fast and robust algorithm for solving large-scale convex composite optimization models with an emphasis on the $\ell_1$-regularized least squares regression (lasso) problems. Despite the fact that there exist a large number of solvers in the literature for the lasso problems, we found that no solver can efficiently handle difficult large-scale regression problems with real data. By leveraging on available error bound results to realize the asymptotic superlinear convergence property of the augmented Lagrangian algorithm, and by exploiting the second order sparsity of the problem through the semismooth Newton method, we are able to propose an algorithm, called Ssnal, to efficiently solve the aforementioned difficult problems. Under very mild conditions, which hold automatically for lasso problems, both the primal and the dual iteration sequences generated by Ssnal possess a fast linear convergence rate, which can even be superlinear asymptotically. Numerical comparisons between our approach and a number of state-of-the-art solvers, on real data sets, are presented to demonstrate the high efficiency and robustness of our proposed algorithm in solving difficult large-scale lasso problems.

View PDF Chat

Stationary response of nonlinear system with Caputo-type fractional derivative damping under Gaussian white noise excitation

2014-09-01

Yong-Ge Yang , Wei Xu , Wantao Jia +1 more

On fallacies in the decision between the Caputo and Riemann–Liouville fractional derivatives for the analysis of the dynamic response of a nonlinear viscoelastic oscillator

2012-07-07

Yury A. Rossikhin , Marina V. Shitikova

Chaos in Chen's system with a fractional order

2004-03-25

Changpin Li , Guojun Peng

Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity

2010-01-09

Coralia Cartis , Nicholas I. M. Gould , Philippe L. Toint