← All works
Research paper

Evaluating Large Language Models Trained on Code

At a glance

Introduced OpenAI Codex and HumanEval, a benchmark that tests whether AI-generated Python code works.

In one paragraph

OpenAI introduced Codex, a language model trained to write code from natural-language instructions. The paper also released HumanEval, a benchmark that runs tests to check whether generated Python programs work as intended.

Authors 58

  1. Mark Chen
  2. Jerry Tworek
  3. Heewoo Jun
  4. Qiming Yuan
  5. Henrique Ponde de Oliveira Pinto
  6. Jared Kaplan
  7. Harri Edwards
  8. Yuri Burda
  9. Nicholas Joseph
  10. Greg Brockman
  11. Alex Ray
  12. Raul Puri
  13. Gretchen Krueger
  14. Michael Petrov
  15. Heidy Khlaaf
  16. Girish Sastry
  17. Pamela Mishkin
  18. Brooke Chan
  19. Scott Gray
  20. Nick Ryder
  21. Mikhail Pavlov
  22. Alethea Power
  23. Lukasz Kaiser
  24. Mo Bavarian
  25. Clemens Winter
  26. Philippe Tillet
  27. Felipe Petroski Such
  28. Dave Cummings
  29. Matthias Plappert
  30. Fotios Chantzis
  31. Elizabeth Barnes
  32. Ariel Herbert-Voss
  33. William Guss
  34. Alex Nichol
  35. Alex Paino
  36. Nikolas Tezak
  37. Jie Tang
  38. Igor Babuschkin
  39. Suchir Balaji
  40. Shantanu Jain
  41. William Saunders
  42. Christopher Hesse
  43. Andrew N. Carr
  44. Jan Leike
  45. Joshua Achiam
  46. Vedant Misra
  47. Evan Morikawa
  48. Alec Radford
  49. Matthew Knight
  50. Miles Brundage
  51. Mira Murati
  52. Katie Mayer
  53. Peter Welinder
  54. Bob McGrew
  55. Dario Amodei
  56. Sam McCandlish
  57. Ilya Sutskever
  58. Wojciech Zaremba

Organizations

Credited for this work at publication.