Function minimization by conjugate gradients

The modification conjugate coefficient for conjugate gradient method to solving unconstrained optimization problems

2024-01-01

In this manuscript, a novel conjugate coefficient is derived through the utilization of equality conjugate gradient direction in conjunction with a modified quasi-Newton (QN) direction. The demonstrated results establish that … In this manuscript, a novel conjugate coefficient is derived through the utilization of equality conjugate gradient direction in conjunction with a modified quasi-Newton (QN) direction. The demonstrated results establish that this innovative approach ensures both sufficient descent and global convergence, meeting the criteria of the Wolf line search condition and inexact line search. Through numerical experiments, we employ the proposed method on a diverse set of established test functions, clearly demonstrating its enhanced efficiency. A comparative analysis is conducted against the FR CG method using the Dolan-More performance profile based on the number of function evaluations (NOF), number of iterations (NOI), and CPU time.

View PDF Chat

Two hybrid conjugate gradient based algorithms on Riemannian manifolds with adaptive restart strategy for nonconvex optimization problems

2024-12-01

Minghu Jiang , Yun Wang , Hu Shao +2 more

A Perry-type derivative-free algorithm for solving nonlinear system of equations and minimizing ℓ<sub>1</sub>regularized problem

2020-08-24

Aliyu Muhammed Awwal , Poom Kumam , Hassan Mohammad +2 more

In this paper, we propose a Perry-type derivative-free algorithm for solving systems of nonlinear equations. The algorithm is based on the well-known BFGS quasi-Newton method with a modified Perry's parameter. … In this paper, we propose a Perry-type derivative-free algorithm for solving systems of nonlinear equations. The algorithm is based on the well-known BFGS quasi-Newton method with a modified Perry's parameter. The global convergence of the algorithm is established without assumption on the regularity or boundedness of the solution set. Meanwhile, the sequence of iterates generated by the algorithm converges globally to the solution of the problem provided that the function is Lipschitz continuous and monotone. Preliminary numerical experiments on some collection of general nonlinear equations and convex constrained nonlinear monotone equations demonstrate the efficiency of the algorithm. Moreover, we successfully apply the proposed algorithm to solve signal recovery problem.

The Global Convergence of a New Mixed Conjugate Gradient Method for Unconstrained Optimization

2012-01-01

Yueting Yang , Mingyuan Cao

We propose and generalize a new nonlinear conjugate gradient method for unconstrained optimization. The global convergence is proved with the Wolfe line search. Numerical experiments are reported which support the … We propose and generalize a new nonlinear conjugate gradient method for unconstrained optimization. The global convergence is proved with the Wolfe line search. Numerical experiments are reported which support the theoretical analyses and show the presented methods outperforming CGDESCENT method.

View PDF Chat

A Modified Nonlinear Conjugate Gradient Method for Engineering Computation

2017-01-01

Tiefeng Zhu , Zaizai Yan , Xiuyun Peng

A general criterion for the global convergence of the nonlinear conjugate gradient method is established, based on which the global convergence of a new modified three‐parameter nonlinear conjugate gradient method … A general criterion for the global convergence of the nonlinear conjugate gradient method is established, based on which the global convergence of a new modified three‐parameter nonlinear conjugate gradient method is proved under some mild conditions. A large amount of numerical experiments is executed and reported, which show that the proposed method is competitive and alternative. Finally, one engineering example has been analyzed for illustrative purposes.

View PDF Chat

A Low-Rank Tensor Factorization Using Implicit Similarity in Trust Relationships

2020-03-09

Pei Ma , Liejun Wang , Jiwei Qin

Low-rank tensor factorization can not only mine the implicit relationships between data but also fill in the missing data when working with complex data. Compared with the traditional collaborative filtering … Low-rank tensor factorization can not only mine the implicit relationships between data but also fill in the missing data when working with complex data. Compared with the traditional collaborative filtering (CF) algorithm, the changes are essentially proposed, from traditional matrix analysis to three-dimensional spatial analysis. Based on low-rank tensor factorization, this paper proposes a recommendation model that comprehensively considers local information and global information, in other words, combining the similarity between trust users and low-rank tensor factorization. First, the similarity between trusted users is measured to capture local information between users by trusting similar preferences of users when selecting items. Then, the users’ similarity is integrated into the tensor, and the low-rank tensor factorization is used to better maintain and describe the internal structure of the data to obtain global information. Furthermore, based on the idea of the alternating least squares method, the conjugate gradient (CG) optimization algorithm for the model of this paper is designed. The local and global information is used to generate the optimal expected result in an iterative process. Finally, we conducted a large number of comparative experiments on the Ciao dataset and the FilmTrust dataset. Experimental results show that the algorithm has less precision loss under the data set with lower density. Thus, not only can a perfect compromise between accuracy and coverage be achieved, but also the computational complexity can be reduced to meet the need for real-time results.

View PDF Chat

Analysis and Prediction of COVID-19 using SIR, SEIR, and Machine Learning Models: Australia, Italy, and UK Cases

2020-10-13

Iman Rahimi , Amir H. Gandomi , Fang Chen

Abstract The novel Coronavirus disease, known as COVID-19, is an outbreak that started in Wuhan, one of the Central Chinese cities. In this report, a short analysis focusing on Australia, … Abstract The novel Coronavirus disease, known as COVID-19, is an outbreak that started in Wuhan, one of the Central Chinese cities. In this report, a short analysis focusing on Australia, Italy, and the United Kingdom has been conducted. The analysis includes confirmed and recovered cases and deaths, the growth rate in Australia as compared with Italy and the United Kingdom, and the outbreak in different Australian cities. Mathematical approaches based on the susceptible, infected, and recovered case (SIR) and susceptible, exposed, infected, and recovered (SEIR) models were proposed to predict the epidemiology in the countries. Since the performance of the classic form of SIR and SEIR depends on parameter settings, some optimization algorithms, namely, the Broyden–Fletcher–Goldfarb–Shanno (BFGS), conjugate gradients (CG), L-BFGS-B, and Nelder-Mead are proposed to optimize the parameters of SIR and SEIR models and improve its predictive capabilities. The results of optimized SIR and SEIR models are compared with the Prophet algorithm and logistic function as two known ML algorithms. The results show that different algorithms display different behaviours in different countries. However, the improved version of the SIR and SEIR models have a better performance compared with other mentioned algorithms described in this study. Moreover, the Prophet algorithm works better for Italy and the United Kingdom cases than for Australian cases and Logistic function compared with Prophet algorithm has a better performance in these cases. It seems that Prophet algorithm is suitable for data with increasing trend in pandemic situations. Optimization of the SIR and SEIR models parameters has yielded a significant improvement in the prediction accuracy of the models. Although there are several algorithms for prediction of this Pandemic, there is no certain algorithm that would be the best one for all cases.

A new hybrid conjugate gradient algorithm based on the Newton direction to solve unconstrained optimization problems

2023-02-27

Naima Hamel , Noureddine Benrabia , Mourad Ghiat +1 more

A Global Convergence of LS-CD Hybrid Conjugate Gradient Method

2013-10-22

Xiangfei Yang , Zhijun Luo , Xiaoyu Dai

Conjugate gradient method is one of the most effective algorithms for solving unconstrained optimization problem. In this paper, a modified conjugate gradient method is presented and analyzed which is a … Conjugate gradient method is one of the most effective algorithms for solving unconstrained optimization problem. In this paper, a modified conjugate gradient method is presented and analyzed which is a hybridization of known LS and CD conjugate gradient algorithms. Under some mild conditions, the Wolfe-type line search can guarantee the global convergence of the LS-CD method. The numerical results show that the algorithm is efficient.

View PDF Chat

A new algorithm of nonlinear conjugate gradient method with strong convergence

2008-01-01

Zhen-Jun Shi , Jinhua Guo

View PDF Chat

A Conjugate Gradient Algorithm under Yuan-Wei-Lu Line Search Technique for Large-Scale Minimization Optimization Models

2018-01-01

Xiangrong Li , Songhua Wang , Zhongzhou Jin +1 more

This paper gives a modified Hestenes and Stiefel (HS) conjugate gradient algorithm under the Yuan-Wei-Lu inexact line search technique for large-scale unconstrained optimization problems, where the proposed algorithm has the … This paper gives a modified Hestenes and Stiefel (HS) conjugate gradient algorithm under the Yuan-Wei-Lu inexact line search technique for large-scale unconstrained optimization problems, where the proposed algorithm has the following properties: <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M1"><mml:mo stretchy="false">(</mml:mo><mml:mn fontstyle="italic">1</mml:mn><mml:mo stretchy="false">)</mml:mo></mml:math> the new search direction possesses not only a sufficient descent property but also a trust region feature; <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M2"><mml:mo stretchy="false">(</mml:mo><mml:mn fontstyle="italic">2</mml:mn><mml:mo stretchy="false">)</mml:mo></mml:math> the presented algorithm has global convergence for nonconvex functions; <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M3"><mml:mo stretchy="false">(</mml:mo><mml:mn fontstyle="italic">3</mml:mn><mml:mo stretchy="false">)</mml:mo></mml:math> the numerical experiment showed that the new algorithm is more effective than similar algorithms.

View PDF Chat

Approximation methods with inertial term for large-scale nonlinear monotone equations

2022-07-03

Abdulkarim Hassan Ibrahim , Poom Kumam , Sanja Rapajić +2 more

Two modified DY conjugate gradient methods for unconstrained optimization problems

2020-01-13

Zhibin Zhu , Dongdong Zhang , Shuo Wang

Aggregation on ordinal scales with the Sugeno integral for biomedical applications

2019-06-11

Gleb Beliakov , Marek Gągolewski , James N. K. Liu

A fast inertial self-adaptive projection based algorithm for solving large-scale nonlinear monotone equations

2023-01-24

N. Zhang , J.K. Liu , Liyuan Zhang +1 more

COMPLEXITY OF FIRST-ORDER METHODS FOR DIFFERENTIABLE CONVEX OPTIMIZATION

2014-11-15

Clóvis C. Gonzaga , Elizabeth W. Karas

This is a short tutorial on complexity studies for differentiable convex optimization. A complexity study is made for a class of problems, an "oracle" that obtains information about the problem … This is a short tutorial on complexity studies for differentiable convex optimization. A complexity study is made for a class of problems, an "oracle" that obtains information about the problem at a given point, and a stopping rule for algorithms. These three items compose a scheme, for which we study the performance of algorithms and problem complexity. Our problem classes will be quadratic minimization and convex minimization in ℝn. The oracle will always be first order. We study the performance of steepest descent and Krylov spacemethods for quadratic function minimization and Nesterov’s approach to the minimization of differentiable convex functions.

View PDF Chat

Preconditioned conjugate gradient algorithms for nonconvex problems

2004-01-01

R. Pytlak , Tomasz Tarnawski

The paper describes a new conjugate gradient algorithm for large scale nonconvex problems. In order to speed up the convergence the algorithm employs a scaling matrix which transforms the space … The paper describes a new conjugate gradient algorithm for large scale nonconvex problems. In order to speed up the convergence the algorithm employs a scaling matrix which transforms the space of original variables into the space in which Hessian matrices of functionals describing the problems have more clustered eigenvalues. This is done efficiently by applying limited memory BFGS updating matrices. Once the scaling matrix is calculated, the next few iterations of the conjugate gradient algorithms are performed in the transformed space. We believe that the preconditioned conjugate gradient algorithm gives more flexibility in achieving balance between the computing time and the number of function evaluations in comparison to a limited memory BFGS algorithm. We give some numerical results which support our claim

Two classes of spectral conjugate gradient methods for unconstrained optimizations

2022-02-18

Jinbao Jian , Pengjie Liu , Xianzhen Jiang +1 more

DHO conjugate gradient method for unconstrained optimization

2020-09-01

Dlovan Haji Omar

In this paper, we suggest a new coefficient conjugate gradient method for nonlinear unconstrained optimization by using two parameters one of them is parameter of (FR) and the other one … In this paper, we suggest a new coefficient conjugate gradient method for nonlinear unconstrained optimization by using two parameters one of them is parameter of (FR) and the other one is parameter of (CD), we give a descent condition of the suggested method.

An inverse problem in estimating the time dependent source term and initial temperature simultaneously by the polynomial regression and conjugate gradient method

2020-01-01

Arzu Erdem

From the final and interior temperature measurements identifying the source term with initial temperature simultaneously is an inverse heat conduction problem which is a kind of ill-posed. The optimal control … From the final and interior temperature measurements identifying the source term with initial temperature simultaneously is an inverse heat conduction problem which is a kind of ill-posed. The optimal control framework has been found to be effective in dealing with these problems. However, they require to find the gradient information. This idea has been employed in this research. We derive the gradient of Tikhonov functional and establish the stability of the minimizer from the necessary condition. The stability and effectiveness of evolutionary algorithm are presented for various test examples.

View PDF Chat

New hybrid conjugate gradient method under strong Wolfe line search

2022-01-01

Ain Aqiela Azamuddin , Nurul Aini , Mohd Rivaie +2 more

DEVELOPING A NEW CONJUGATE GRADIENT ALGORITHM WITH THE BENEFIT OF SOME DESIRABLE PROPERTIES OF THE NEWTON ALGORITHM FOR UNCONSTRAINED OPTIMIZATION

2023-12-04

Naima Hamel , Noureddine Benrabia , Mourad Ghiat +1 more

The conjugate gradient method and the Newton method are both numerical optimization techniques. In this paper, we aim to combine some desirable characteristics of these two methods while avoiding their … The conjugate gradient method and the Newton method are both numerical optimization techniques. In this paper, we aim to combine some desirable characteristics of these two methods while avoiding their drawbacks, more specifically, we aim to develop a new optimization algorithm that preserves some essential features of the conjugate gradient algorithm, including the simplicity, the low memory requirements, the ability to solve large scale problems and the convergence to the solution regardless of the starting vector (global convergence). At the same time, this new algorithm approches the quadratic convergence behavior of the Newton method in the numerical sense while avoiding the computational cost of evaluating the Hessian matrix directly and the sensitivity of the selected starting vector. To do this, we propose a new hybrid conjugate gradient method by linking (CD) and (WYL) methods in a convex blend, the hybridization paramater is computed so that the new search direction accords with the Newton direction, but avoids the computational cost of evaluating the Hessian matrix directly by using the secant equation. This makes the proposed algorithm useful for solving large scale optimization problems. The sufficient descent condition is verified, also the global convergence is proved under a strong Wolfe Powel line search. The numerical tests show that, the proposed algorithm provides the quadratic convergence behavior and confirm its efficiency as it outperformed both the (WYL) and (CD) algorithms.

A New Formula for Conjugate Gradient in Unconstrained Optimization

2020-05-19

Hussein Wali , Khalil K. Abbo

The conjugate gradient method is an important part of the methods of optimization that are not constrained by local convergence characteristics. In this research, a new formula for the conjugated … The conjugate gradient method is an important part of the methods of optimization that are not constrained by local convergence characteristics. In this research, a new formula for the conjugated coefficient is derived depending on the linear structure. The new method fulfills the regression requirement. In addition, using the Wolff search line terms, the overall convergence of the new method has been demonstrated. At the end of the research were presented numerical results that show the effectiveness of the proposed method.

View PDF Chat

A Sequential Subspace Quasi-Newton Method for Large-Scale Convex Optimization

2020-07-01

Aleksandr Senov , Oleg Granichin , Olga Granichina

Large-scale optimization plays important role in many control and learning problems. Sequential subspace optimization is a novel approach particularly suitable for large- scale optimization problems. It is based on sequential … Large-scale optimization plays important role in many control and learning problems. Sequential subspace optimization is a novel approach particularly suitable for large- scale optimization problems. It is based on sequential reduction of the initial optimization problem to optimization problems in a low-dimensional space. In this paper we consider a problem of multidimensional convex real-valued function optimization. In a framework of sequential subspace optimization we develop a new method based on a combination of quasi-Newton and conjugate gradient method steps. We provide its formal justification and derive several of its theoretical properties. In particular, for quadratic programming problem we prove linear convergence in a finite number of steps. We demonstrate superiority of the proposed algorithm over common state of the art methods by carrying out comparative analysis on both modelled and real- world optimization problems.

Two spectral conjugate gradient methods for unconstrained optimization problems

2022-04-11

Zhibin Zhu , Ai Long , Tian Wang

An inverse problem in estimating the time dependent source term and initial temperature simultaneously by the polynomial regression and conjugate gradient method

2020-06-10

Arzu Erdem

From the final and interior temperature measurements identifying the source term with initial temperature simultaneously is an inverse heat conduction problem which is a kind of ill-posed. The optimal control … From the final and interior temperature measurements identifying the source term with initial temperature simultaneously is an inverse heat conduction problem which is a kind of ill-posed. The optimal control framework has been found to be effective in dealing with these problems. However, they require to find the gradient information. This idea has been employed in this research. We derive the gradient of Tikhonov functional and establish the stability of the minimizer from the necessary condition. The stability and effectiveness of the evolutionary algorithm are presented for various test examples.

A STEP-LENGTH FORMULA FOR CONJUGATE GRADIENT METHODS

2020-04-30

Adam Ajimoti Ishaq , Tolulope Latunde , Kazeem Babatunde Akande

A new step-length formula is proposed for the conjugate gradient methods. The unified formula for obtaining step-length does not involve any matrix operation. Numerical results obtained are graphically illustrated using … A new step-length formula is proposed for the conjugate gradient methods. The unified formula for obtaining step-length does not involve any matrix operation. Numerical results obtained are graphically illustrated using performance profiling software. This showed that the new formula performs efficiently in terms of computational efforts and execution time compared with some existing formulae for obtaining the step-length without line search procedures.

View PDF Chat

Numerical methods for large-scale nonlinear optimization

2005-04-19

Nicholas I. M. Gould , Dominique Orban , Philippe L. Toint

Recent developments in numerical methods for solving large differentiable nonlinear optimization problems are reviewed. State-of-the-art algorithms for solving unconstrained, bound-constrained, linearly constrained and non-linearly constrained problems are discussed. As well … Recent developments in numerical methods for solving large differentiable nonlinear optimization problems are reviewed. State-of-the-art algorithms for solving unconstrained, bound-constrained, linearly constrained and non-linearly constrained problems are discussed. As well as important conceptual advances and theoretical aspects, emphasis is also placed on more practical issues, such as software availability.

View PDF Chat

A NEW COEFFICIENT OF CONJUGATE GRADIENT METHODS FOR NONLINEAR UNCONSTRAINED OPTIMIZATION

2016-06-12

Nur Syarafina Mohamed , Mustafa Mamat , Fatma Susilawati Mohamad +1 more

Conjugate gradient (CG) methods are widely used in solving nonlinear unconstrained optimization problems such as designs, economics, physics and engineering due to its low computational memory requirement. In this paper, … Conjugate gradient (CG) methods are widely used in solving nonlinear unconstrained optimization problems such as designs, economics, physics and engineering due to its low computational memory requirement. In this paper, a new modifications of CG coefficient ( ) which possessed global convergence properties is proposed by using exact line search. Based on the number of iterations and central processing unit (CPU) time, the numerical results show that the new performs better than some other well known CG methods under some standard test functions.

View PDF Chat

A modification of classical conjugate gradient method using strong Wolfe line search

2016-01-01

Syazni Shoid , Mohd Rivaie , Mustafa Mamat

Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Share Icon Share Twitter Facebook Reddit LinkedIn Tools Icon Tools Reprints and Permissions Cite Icon Cite … Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Share Icon Share Twitter Facebook Reddit LinkedIn Tools Icon Tools Reprints and Permissions Cite Icon Cite Search Site Citation Syazni Shoid, Mohd. Rivaie, Mustafa Mamat; A modification of classical conjugate gradient method using strong Wolfe line search. AIP Conf. Proc. 2 June 2016; 1739 (1): 020071. https://doi.org/10.1063/1.4952551 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAIP Publishing PortfolioAIP Conference Proceedings Search Advanced Search |Citation Search

Newton and conjugate gradient for harmonic maps from the disc into the sphere

2004-01-01

Morgan Pierre

We compute numerically the minimizers of the Dirichlet energy among maps from the unit disc into the unit sphere that satisfy a boundary condition and a degree condition. We use … We compute numerically the minimizers of the Dirichlet energy among maps from the unit disc into the unit sphere that satisfy a boundary condition and a degree condition. We use a Sobolev gradient algorithm for the minimization and we prove that its continuous version preserves the degree. For the discretization of the problem we use continuous P1 finite elements. We propose an original mesh-refining strategy needed to preserve the degree with the discrete version of the algorithm (which is a preconditioned projected gradient). In order to improve the convergence, we generalize to manifolds the classical Newton and conjugate gradient algorithms. We give a proof of the quadratic convergence of the Newton algorithm for manifolds in a general setting.

View PDF Chat

A NEW THREE–TERM CONJUGATE GRADIENT METHOD WITH DESCENT DIRECTION FOR UNCONSTRAINED OPTIMIZATION

2016-05-19

Xiaoliang Dong , Hongwei Liu , Yubo He +2 more

In this paper, we propose a three–term PRP–type conjugate gradient method which always satisfies the sufficient descent condition independently of line searches employed. An important property of our method is … In this paper, we propose a three–term PRP–type conjugate gradient method which always satisfies the sufficient descent condition independently of line searches employed. An important property of our method is that its direction is closest to the direction of the Newton method or satisfies conjugacy condition as the iterations evolve. In addition, under mild condition, we prove global convergence properties of the proposed method. Numerical comparison illustrates that our proposed method is efficient for solving the optimization problems.

View PDF Chat

A Modified Structured Spectral HS Method for Nonlinear Least Squares Problems and Applications in Robot Arm Control

2023-07-21

Rabiu Bashir Yunus , Nooraini Zainuddin , Hanita Daud +3 more

This paper proposes a modification to the Hestenes-Stiefel (HS) method by devising a spectral parameter using a modified secant relation to solve nonlinear least-squares problems. Notably, in the implementation, the … This paper proposes a modification to the Hestenes-Stiefel (HS) method by devising a spectral parameter using a modified secant relation to solve nonlinear least-squares problems. Notably, in the implementation, the proposed method differs from existing approaches, in that it does not require a safeguarding strategy and its Hessian matrix is positive and definite throughout the iteration process. Numerical experiments are conducted on a range of test problems, with 120 instances to demonstrate the efficacy of the proposed algorithm by comparing it with existing techniques in the literature. However, the results obtained validate the effectiveness of the proposed method in terms of the standard metrics of comparison. Additionally, the proposed approach is applied to address motion-control problems in a robotic model, resulting in favorable outcomes in terms of the robot’s motion characteristics.

View PDF Chat

A Global Convergence of Spectral Conjugate Gradient Method for Large Scale Optimization

2018-06-01

Ghada M. Al-Naemi

In this paper, we are concerned with the conjugate gradient method for solving unconstrained optimization problems due to its simplicity and don’t store any matrices. We proposed two spectral modifications … In this paper, we are concerned with the conjugate gradient method for solving unconstrained optimization problems due to its simplicity and don’t store any matrices. We proposed two spectral modifications to the conjugate descent (CD). These two proposed methods produces sufficient descent directions for the objective function at every iteration with strong Wolfe line searches and with inexact line search, and also they are globally convergent for general non-convex functions can be guaranteed. Numerical results show the efficiency of these two proposed methods.

View PDF Chat

New spectral LS conjugate gradient method for nonlinear unconstrained optimization

2023-01-02

Ahmed Anwer Mustafa

In this work, we propose a novel algorithm to perform spectral conjugate gradient descent for an unconstrained, nonlinear optimization problem. First, we theoretically prove that the proposed method satisfies the … In this work, we propose a novel algorithm to perform spectral conjugate gradient descent for an unconstrained, nonlinear optimization problem. First, we theoretically prove that the proposed method satisfies the sufficient descent condition, the conjugacy condition, and the global convergence theorem. The experimental setup uses Powell's conjugacy condition coupled with a cubic polynomial line search using strong Wolfe conditions to ensure quick convergence. The experimental results demonstrate that the proposed method shows superior performance in terms of the number of iterations to convergence and the number of function evaluations when compared to traditional methods such as Liu and Storey (LS) and Conjugate Descent (CD).

A new hybrid conjugate gradient algorithm for unconstrained optimization

2023-06-01

Imane Hafaidia , Hamza Guebbaï , M. Al-Baali +1 more

It is well known that conjugate gradient methods are useful for solving large-scale unconstrained nonlinear optimization problems. In this paper, we consider combining the best features of two conjugate gradient … It is well known that conjugate gradient methods are useful for solving large-scale unconstrained nonlinear optimization problems. In this paper, we consider combining the best features of two conjugate gradient methods. In particular, we give a new conjugate gradient method, based on the hybridization of the useful DY (Dai-Yuan), and HZ (Hager-Zhang) methods. The hybrid parameters are chosen such that the proposed method satisfies the conjugacy and sufficient descent conditions. It is shown that the new method maintains the global convergence property of the above two methods. The numerical results are described for a set of standard test problems. It is shown that the performance of the proposed method is better than that of the DY and HZ methods in most cases.

View PDF Chat

A semismooth conjugate gradients method – theoretical analysis

2024-05-23

Franz Bethke , Andreas Griewank , Andrea Walther

In large-scale applications, deterministic and stochastic variants of Cauchy's steepest descent method are widely used for the minimization of objectives that are only piecewise smooth. In this paper, we analyse … In large-scale applications, deterministic and stochastic variants of Cauchy's steepest descent method are widely used for the minimization of objectives that are only piecewise smooth. In this paper, we analyse a deterministic descent method based on the generalization of rescaled conjugate gradients proposed by Philip Wolfe in 1975 for objectives that are convex. Without this assumption, the new method exploits semismoothness to obtain pairs of directionally active generalized gradients such that it can only converge to Clarke stationary points. Numerical results illustrate the theoretical findings.

A family of limited memory three term conjugate gradient methods

2024-05-23

Min Li , Minru Bai

Based on several modified Hestenes–Stiefel and Polak–Ribière–Polyak nonlinear conjugate gradient methods, a family of three term limited memory CG methods are developed. When the current search direction falls into the … Based on several modified Hestenes–Stiefel and Polak–Ribière–Polyak nonlinear conjugate gradient methods, a family of three term limited memory CG methods are developed. When the current search direction falls into the subspace spanned by the previous m directions, the algorithm branches to optimize the objective function on the subspace by using L-BFGS method. We use this strategy to avoid potential local loops and accelerate the convergence. The proposed methods are sufficient descent. When the steplength is determined by Wolfe or Armijo line search, we establish the global convergence of the methods in a concise way. Numerical results verify the efficiency of the proposed method for the unconstrained optimization problems in the CUTEst library.

An improved descent Perry‐type algorithm for large‐scale unconstrained nonconvex problems and applications to image restoration problems

2024-07-13

Xiaoliang Wang , Jian Lv , Na Xu

Abstract Conjugate gradient methods are much effective for large‐scale unconstrained optimization problems by their simple computations and low memory requirements. The Perry conjugate gradient method has been considered to be … Abstract Conjugate gradient methods are much effective for large‐scale unconstrained optimization problems by their simple computations and low memory requirements. The Perry conjugate gradient method has been considered to be one of the most efficient methods in the context of unconstrained minimization. However, a globally convergent result for general functions has not been established yet. In this paper, an improved three‐term Perry‐type algorithm is proposed which automatically satisfies the sufficient descent property independent of the accuracy of line search strategy. Under the standard Wolfe line search technique and a modified secant condition, the proposed algorithm is globally convergent for general nonlinear functions without convexity assumption. Numerical results compared with the Perry method for stability, two modified Perry‐type conjugate gradient methods and two effective three‐term conjugate gradient methods for large‐scale problems up to 300,000 dimensions indicate that the proposed algorithm is more efficient and reliable than the other methods for the testing problems. Additionally, we also apply it to some image restoration problems.

Nonlinear conjugate gradient methods for unconstrained set optimization problems whose objective functions have finite cardinality

2024-08-14

Krishan Kumar , Debdas Ghosh , Jen‐Chih Yao +1 more

In this paper, we propose nonlinear conjugate gradient methods for unconstrained set optimization problems in which the objective function is given by a finite number of continuously differentiable vector-valued functions. … In this paper, we propose nonlinear conjugate gradient methods for unconstrained set optimization problems in which the objective function is given by a finite number of continuously differentiable vector-valued functions. First, we provide a general algorithm for the conjugate gradient method using Wolfe line search but without imposing an explicit restriction on the conjugate parameter. Later, we study two variants of the algorithm, namely Fletcher-Reeves and conjugate descent, using two different choices of the conjugate parameter but with the same line search rule. In the general algorithm, the direction of movement at each iterate is identified by finding out a descent direction of a vector optimization problem. This vector optimization problem is identified with the help of the concept of partition set at the current iterate. As this vector optimization problem is different at different iterations, the conventional conjugate gradient method for vector optimization cannot be straightly extended to solve the set optimization problem under consideration. The well-definedness of the methods is provided. Further, we prove the Zoutendijk-type condition, which assists in proving the global convergence of the methods even without a regular condition of the stationary points. No convexity assumption is assumed on the objective function to prove the convergence. Lastly, some numerical examples are illustrated to exhibit the performance of the proposed method. We compare the performance of the proposed conjugate gradient methods with the existing steepest descent method. It is found that the proposed method commonly outperforms the existing steepest descent method for set optimization.

Function minimization by conjugate gradients

Authors

Abstract

Topics

Locations

Ask a Question About This Paper

Summary