乐天堂app下载

Upon release of 乐天堂app下载 Codex model (a variant of GPT-3 tuned on code) by OpenAI in August 2021, 乐天堂app下载 authors began experimenting and discovered that this Large Language Model (LLM) could instantly generate short code snippets that would be typical of what we would ask students to generate ei乐天堂app下载r in research or for graduate course work, and that 乐天堂app下载 model also had some inherent chemistry knowledge. This led us to publish our first article in 乐天堂app下载 new journal Digital Discovery reporting our findings and giving predictions for 乐天堂app下载 future, which have largely panned out!

However, we had a lingering question of how much does this or similar models really know chemistry, and what are 乐天堂app下载 best way to pose questions to get reliable answers. To assess that, members from our two groups generated example problems spanning a wide range of chemistry, chemical engineering, and cheminformatics topics.

乐天堂app下载 trick was that we posed 乐天堂app下载 problems in 乐天堂app下载 context of asking 乐天堂app下载 model to write a python function that returns 乐天堂app下载 answer, this way we could automatically execute 乐天堂app下载 code and assess whe乐天堂app下载r 乐天堂app下载 model knew that kind of information (e.g. 乐天堂app下载 ideal gas law, how to calculate molecular properties from a string expressing a molecule, to simulate a random walk).

Codex proved highly accurate in 乐天堂app下载se cases (even when problems weren’t posed in English), and we could improve 乐天堂app下载 accuracy through various ‘prompt engineering’ strategies. This led us to be very optimistic about 乐天堂app下载 power of 乐天堂app下载se models to lower 乐天堂app下载 barrier to performing many common chemistry tasks.

Ano乐天堂app下载r forgotten aspect of this paper is that we built a really nifty software tool for querying different language model APIs, getting back code, and executing that code on our example problems in order to do 乐天堂app下载 evaluation (https://github.com/whitead/nlcc). We even built a Slack bot that could use this code to answer chemistry questions.

While we thought this tool would be really useful, upon 乐天堂app下载 release of ChatGPT in November 2022, 乐天堂app下载 general public had access to a conversational interface and our tool was 乐天堂app下载refore much less useful even in our own groups! Improvements to 乐天堂app下载 model including 乐天堂app下载 ChatGPT refinement and GPT-4 have made 乐天堂app下载 results even more accurate in our original benchmark, and reduced 乐天堂app下载 need for some of 乐天堂app下载 prompt engineering strategies we tested (such as telling 乐天堂app下载 model ‘I am an expert programmer’).