Can Language Models Replace Programmers? REPOCOD Says 'Not Yet'
Can Language Models Replace Programmers? REPOCOD Says 'Not Yet'
Large language models (LLMs) have achieved high accuracy, i.e., more than 90% pass@1, in solving Python coding problems in HumanEval and MBPP. Thus, a natural question is, whether LLMs achieve comparable code completion performance compared to human developers? Unfortunately, one cannot answer this question using existing manual crafted or simple …