Programming paper: Large-Language Models (LLMs) are changing the way learners acquire knowledge outside the classroom setting. Previous studies have shown that LLMs seem effective in generating to short and simple questions in introductory CS courses using high-resource programming languages such as Java or Python....

Programming Journal's publication is a resource for developers seeking to expand their knowledge and skills in software development. Through articles, tutorials, and coding challenges, Programming Journal offers insights into programming languages, algorithms, and data structures. Readers can learn about best practices in software engineering, problem-solving techniques, and coding interview preparation to excel in their careers as developers.

The Programming Journal

A research study evaluates 9 state-of-the-art LLMs in the context of a functional programming course using OCaml, a low-resource language. Three benchmarks were created: λCodeGen (homework programming problems), λRepair (programs with syntax/type/logical errors from student submissions), and λExplain (theoretical concept questions). Results show top LLMs perform well overall but solve fewer problems in OCaml compared to Python/Java. LLMs excel at fixing syntax and type errors and answering basic conceptual questions. The benchmarks aim to help instructors, students, and PL researchers understand LLM limitations and opportunities for improvement in low-resource language settings.

Evaluating LLMs in the Context of a Functional Programming Course: A Comprehensive Study