Skip to content

Commit

Permalink
add lisp
Browse files Browse the repository at this point in the history
  • Loading branch information
@TyDunn
TyDunn committed Nov 7, 2023
1 parent 1a411ff commit fc58702
Show file tree
Hide file tree
Showing 2 changed files with 52 additions and 3 deletions.
7 changes: 4 additions & 3 deletions docs/docs/languages/languages.csv
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
language,multiple,babel,mbxp,humaneval_x,so_2023_language_percent,so_2023_language_rank,so_tags,github_prs,github_pushes,github_issues,github_stars,stack_gb,codeparrot_gb,alphacode_gb,codegen_gb,polycoder_gb,subreddit_members,subreddit_url,anecdote_1_content,anecdote_1_author,anecdote_1_url,anecdote_2_content,anecdote_2_author,anecdote_2_url,anecdote_3_content,anecdote_3_author,anecdote_3_url
Erlang,N/A,N/A,N/A,N/A,0.99%,38,"9,621","70,890","249,209","49,786","127,120",0,0,0,0,0,9.5k,https://www.reddit.com/r/erlang,It seems like ChatGPT doesn't know that much Erlang.,u/Ranugad,https://www.reddit.com/r/erlang/comments/11kl57z/comment/jbbw94t,"I recently asked ChatGPT to translate some Erlang code into Elixir. Here’s an edited transcript, for your amusement and edification…",Rich_Morin,https://elixirforum.com/t/asking-chatgpt-to-translate-erlang-to-elixir/53548,I don’t think anything automated is going to work well. ChatGPT might be interesting but you’ll almost certainly have to fix it up quite a bit. https://learnxinyminutes.com/docs/erlang/ gives a quick rundown on erlang syntax/semantics and https://learnyousomeerlang.com/ is a good book on it,u/boy-griv,https://www.reddit.com/r/AskProgramming/comments/10tave8/comment/j78bvj5
Erlang,N/A,N/A,N/A,N/A,0.99%,38,"9,621","70,890","249,209","49,786","127,120","Unspecified",0,0,0,0,9.5k,https://www.reddit.com/r/erlang,It seems like ChatGPT doesn't know that much Erlang.,u/Ranugad,https://www.reddit.com/r/erlang/comments/11kl57z/comment/jbbw94t,"I recently asked ChatGPT to translate some Erlang code into Elixir. Here’s an edited transcript, for your amusement and edification…",Rich_Morin,https://elixirforum.com/t/asking-chatgpt-to-translate-erlang-to-elixir/53548,I don’t think anything automated is going to work well. ChatGPT might be interesting but you’ll almost certainly have to fix it up quite a bit. https://learnxinyminutes.com/docs/erlang/ gives a quick rundown on erlang syntax/semantics and https://learnyousomeerlang.com/ is a good book on it,u/boy-griv,https://www.reddit.com/r/AskProgramming/comments/10tave8/comment/j78bvj5
Julia,11,13,N/A,N/A,1.15%,37,"12,402","39,305","166,898","51,276","52,326",3.09,0.29,0,0,0,23.9k,https://reddit.com/r/julia,"I usually start my own articles with ChatGPT but the truth is that right now, if you want to say something interesting in the Julia space, you mostly need to write it yourself since the volume of content about Julia out there isn’t enough for the outputs of ChatGPT to be very useful since our ecosystem is so small.",u/LoganKilpatrick1,https://www.reddit.com/r/Julia/comments/zzvkso/comment/j2i6knx/,"It wasn't trained on sufficient Julia code. As with any machine learning model, ChatGPT is only able to regurgitate what's been fed into it. Also, this behaviour happens with basically every other topic, too. LLMs work by trying to predict what the next word in a sentence would be based on the previous string of words. If a sentence is incomplete, it's going to add a next word. That word is going to be whichever has the highest confidence score, regardless of low that score may actually be. This results in it just making shit up, but often shit that sounds plausible. We've seen CGPT invent academic articles, books, and even entire people because it makes sense to in the sentence it's generating.`",u/Kichae,https://www.reddit.com/r/Julia/comments/112wlle/comment/j8mpgx5/,"I suspect the current language model behind ChatGPT was fed with a lot of code examples from Stack Exchange, but the Julia community mainly uses Discourse instead, which probably wasn't in the training set: https://discourse.julialang.org/",u/Paravalis,https://www.reddit.com/r/Julia/comments/112wlle/comment/j8qzc0j/
Clojure,N/A,N/A,N/A,N/A,1.26%,36,"17,630","112,757","518,359","84,128","272,970",0,0,0,0,0,31.5k,https://www.reddit.com/r/Clojure,"I've been using Copilot since December 2022. It sucks for Clojure but can be great for other languages like Python, JavaScript, SQL, etc. if you know how to prompt it. As other have mentioned, Copilot excels at reducing boilerplate and picking up on patterns. For example, lets say there is a table of data in a markdown document and you want to convert it to a vector of maps. You can copy/paste the markdown table into your buffer as a comment and just start writing the data structure you want it to be, Copilot will figure it out and complete it. Its also useful for generating random utility functions. Recently in JavaScript, I typed `function lerp` (linear interpolation) and it pretty quickly filled it in. I had an array of hex color values that I wanted to be RGB and I wanted to double the number of values by interpolating between them. All I had to do was type that in a comment and wait a second before it gave me a working rough draft of the function. Copilot can actually do a lot of these things for Clojure but when I was trying to use it I found myself consistently having to fix issues with delimiters, typically round braces. Eventually, I just gave up on it. Maybe I'll give it another shot when Copilot-X releases. ChatGPT is much more useful for Clojure than Copilot. It does hallucinate and get some things wrong but overall its awesome for generating documentation, explaining code, translating diffs into PR notes, and exploring ideas. I've found it very useful for random Java questions and then translating the answers into mostly working Clojure code. These things are handy tools and have quirks but they're going to get better. It's a great time to be a cosmopolitan (polyglot) programmer.",u/noprompt,https://www.reddit.com/r/Clojure/comments/148nhuj/comment/jo2z2n8,"No Clojure. No Julia. No Haskell. No Racket. No Scheme. No Common Lisp. No OCaml. And, as much as I despise Microsoft, No C#. No F#. No Swift. No Objective-C. No Perl. No Datalog. A glaringly lacking choice of languages.",waffletower,https://news.ycombinator.com/item?id=35803856,"FizzBuzz was once a common programming exercise used for screening software developers (maybe it still is?) I told chatGPT to ""Write an efficient fizz buzz function in Clojure"".",@EricTheTurner,https://x.com/EricTheTurner/status/1600344406166380544?s=20
Solidity,N/A,N/A,N/A,N/A,1.33%,35,"6,669",0,0,0,350,0,0,0,0,0,17.0k,https://reddit.com/r/solidity,"ChatGPT is awful at smart contract, the data is years out of date, and it tend to override and make functions that are unnecessary. Even worse it overrides safe good functions for unsafe inefficient functions. Speaking of inefficiency it will seriously de-optimize optimized code, even when asked to gas optimize it.",u/Adrewmc,https://www.reddit.com/r/solidity/comments/142amjb/comment/jn48x8v/,"Despite the mixed results, ChatGPT, aka GPT-3.5, is a step forward in the direction of writing code with an AI assistant. I actually enjoyed doing these little experiments. However, compared to other experiments I did with JavaScript and other languages, a clear takeaway from my efforts is that when it comes to the Web3 space, GPT doesn’t yet have enough accuracy. In fairness, there is far less available Solidity and Web3-related JavaScript code in the wild than there is general-purpose JavaScript code. Plus, the web3 industry is constantly changing, which makes the problems of ChatGPT relying on an old dataset much worse. . On the positive side, generating an ABI from Solidity is something it did well, which shows it can learn from the available snippets the general rules to create something new.",Lorenzo Sicilia,https://outlierventures.io/article/can-chatgpt-really-be-trusted-to-write-a-smart-contract-or-to-refactor-your-existing-solidity-code/,Can someone please make an open coder model trained on Solidity,u/thatdudeiknew,https://www.reddit.com/r/LocalLLaMA/comments/14qednx/comment/jqmq2t5/?utm_source=share&utm_medium=web2x&context=3
Clojure,N/A,N/A,N/A,N/A,1.26%,36,"17,630","112,757","518,359","84,128","272,970","Unspecified",0,0,0,0,31.5k,https://www.reddit.com/r/Clojure,"I've been using Copilot since December 2022. It sucks for Clojure but can be great for other languages like Python, JavaScript, SQL, etc. if you know how to prompt it. As other have mentioned, Copilot excels at reducing boilerplate and picking up on patterns. For example, lets say there is a table of data in a markdown document and you want to convert it to a vector of maps. You can copy/paste the markdown table into your buffer as a comment and just start writing the data structure you want it to be, Copilot will figure it out and complete it. Its also useful for generating random utility functions. Recently in JavaScript, I typed `function lerp` (linear interpolation) and it pretty quickly filled it in. I had an array of hex color values that I wanted to be RGB and I wanted to double the number of values by interpolating between them. All I had to do was type that in a comment and wait a second before it gave me a working rough draft of the function. Copilot can actually do a lot of these things for Clojure but when I was trying to use it I found myself consistently having to fix issues with delimiters, typically round braces. Eventually, I just gave up on it. Maybe I'll give it another shot when Copilot-X releases. ChatGPT is much more useful for Clojure than Copilot. It does hallucinate and get some things wrong but overall its awesome for generating documentation, explaining code, translating diffs into PR notes, and exploring ideas. I've found it very useful for random Java questions and then translating the answers into mostly working Clojure code. These things are handy tools and have quirks but they're going to get better. It's a great time to be a cosmopolitan (polyglot) programmer.",u/noprompt,https://www.reddit.com/r/Clojure/comments/148nhuj/comment/jo2z2n8,"No Clojure. No Julia. No Haskell. No Racket. No Scheme. No Common Lisp. No OCaml. And, as much as I despise Microsoft, No C#. No F#. No Swift. No Objective-C. No Perl. No Datalog. A glaringly lacking choice of languages.",waffletower,https://news.ycombinator.com/item?id=35803856,"FizzBuzz was once a common programming exercise used for screening software developers (maybe it still is?) I told chatGPT to ""Write an efficient fizz buzz function in Clojure"".",@EricTheTurner,https://x.com/EricTheTurner/status/1600344406166380544?s=20
Solidity,N/A,N/A,N/A,N/A,1.33%,35,"6,669",0,0,0,350,0,0,0,0,0,17.0k,https://reddit.com/r/solidity,"ChatGPT is awful at smart contract, the data is years out of date, and it tend to override and make functions that are unnecessary. Even worse it overrides safe good functions for unsafe inefficient functions. Speaking of inefficiency it will seriously de-optimize optimized code, even when asked to gas optimize it.",u/Adrewmc,https://www.reddit.com/r/solidity/comments/142amjb/comment/jn48x8v/,"Despite the mixed results, ChatGPT, aka GPT-3.5, is a step forward in the direction of writing code with an AI assistant. I actually enjoyed doing these little experiments. However, compared to other experiments I did with JavaScript and other languages, a clear takeaway from my efforts is that when it comes to the Web3 space, GPT doesn’t yet have enough accuracy. In fairness, there is far less available Solidity and Web3-related JavaScript code in the wild than there is general-purpose JavaScript code. Plus, the web3 industry is constantly changing, which makes the problems of ChatGPT relying on an old dataset much worse. . On the positive side, generating an ABI from Solidity is something it did well, which shows it can learn from the available snippets the general rules to create something new.",Lorenzo Sicilia,https://outlierventures.io/article/can-chatgpt-really-be-trusted-to-write-a-smart-contract-or-to-refactor-your-existing-solidity-code/,Can someone please make an open coder model trained on Solidity,u/thatdudeiknew,https://www.reddit.com/r/LocalLLaMA/comments/14qednx/comment/jqmq2t5/?utm_source=share&utm_medium=web2x&context=3
Lisp,N/A,N/A,N/A,N/A,1.53%,34,"6,945","8,431","73,903","12,870","47,157","Unspecified",0,0,0,0,37.7k,https://www.reddit.com/r/lisp,"Chat gpt is known to lie and be confident in its incorrectness. Also, try telling it to convert a program from lisp to python that uses advanced features like the condition system.",u/KaranasToll,https://www.reddit.com/r/lisp/comments/138aovs/comment/jixfrkr/,"How do you think the advent of ChatGPT and Copilot would affect the adoption and popularity of Common Lisp, Clojure and Schemes? On one hand, Large Language Models did not have access to these ""niche"" languages for training as much as the more popular alternatives like Python and Typescript so the quality of their output would be worse in comparison. On the other hand, the ""interactive"" aspect of LISP in that you code stuff, test in REPL and code again would not be so unique since the developer can just use the chat system to refine his solution. The other upside that LISPs had over the likes of Rust and C++ is the lack of syntax clutter and cleanness of s-expressions. In this front too, they would hurt from the likes of ChatGPT since the syntactic complexity is handled by the LLM not the developer.",u/friedrichRiemann,https://www.reddit.com/r/lisp/comments/11lwwv1/possible_effects_of_aiassisted_tools_on_lisps/?utm_source=share&utm_medium=web2x&context=3,"I'm an engineer working in the construction field, and I'm currently trying to create a Lisp routine for a project I'm working on. I've been trying to use GPT to generate the code, but I'm having some trouble getting it to work properly. I was wondering if anyone knows of a pre-trained GPT that has been specifically trained on Lisp code. I've been searching online, but I haven't had any luck so far. If anyone knows of a pre-trained GPT with Lisp, or has any tips for training my own GPT on Lisp code, I would really appreciate the help.",/u/Fine_Impression_3171,https://www.reddit.com/r/ChatGPT/comments/12o4k1n/looking_for_pretrained_gpt_with_lisp_autocad/
48 changes: 48 additions & 0 deletions docs/docs/languages/lisp.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Lisp

Lisp is the #34 most popular language according to the [2023 Stack Overflow Developer Survey](https://survey.stackoverflow.co/2023/#section-most-popular-technologies-programming-scripting-and-markup-languages).

## Benchmarks

❌ Lisp is not one of the 19 languages in the [MultiPL-E benchmark](https://blog.continue.dev/an-introduction-to-code-llm-benchmarks-for-software-engineers/#:~:text=couple%20notable%20mentions-,4.%20MultiPL%2DE,-Creator%3A%20Northeastern)

❌ Lisp is not one of the 16 languages in the [BabelCode / TP3 benchmark](https://blog.continue.dev/an-introduction-to-code-llm-benchmarks-for-software-engineers/#:~:text=amazon%2Dscience/mxeval-,12.%20BabelCode%20/%20TP3,-Creator%3A%20Google)

❌ Lisp is not one of the 13 languages in the [MBXP / Multilingual HumanEval benchmark](https://blog.continue.dev/an-introduction-to-code-llm-benchmarks-for-software-engineers/#:~:text=11.%20MBXP%20/%20Multilingual%20HumanEval)

❌ Lisp is not one of the 5 languages in the [HumanEval-X benchmark](https://blog.continue.dev/an-introduction-to-code-llm-benchmarks-for-software-engineers/#:~:text=Some%20multilingual%C2%A0benchmarks-,10.%20HumanEval%2DX,-Creator%3A%20Tsinghua)

## Datasets

✅ Lisp is included in [The Stack dataset](https://arxiv.org/abs/2211.15533)

❌ Lisp is not included in the [CodeParrot dataset](https://huggingface.co/datasets/codeparrot/github-code)

❌ Lisp is not included in the [AlphaCode dataset](https://arxiv.org/abs/2203.07814)

❌ Lisp is not included in the [CodeGen dataset](https://arxiv.org/abs/2203.13474)

❌ Lisp is not included in the [PolyCoder dataset](https://arxiv.org/abs/2202.13169)

## Stack Overflow & GitHub presence

Lisp has 6,945 [tagged questions on Stack Overflow](https://stackoverflow.com/tags)

Lisp projects have had 8,431 [PRs on GitHub since 2014](https://madnight.github.io/githut/#/pull_requests/2023/3)

Lisp projects have had 12,870 [issues on GitHub since 2014](https://madnight.github.io/githut/#/issues/2023/3)

Lisp projects have had 73,903 [pushes on GitHub since 2014](https://madnight.github.io/githut/#/pushes/2023/3)

Lisp projects have had 47,157 [stars on GitHub since 2014](https://madnight.github.io/githut/#/stars/2023/3)

## Anecdotes from developers

[u/KaranasToll](https://www.reddit.com/r/lisp/comments/138aovs/comment/jixfrkr/)
> Chat gpt is known to lie and be confident in its incorrectness. Also, try telling it to convert a program from lisp to python that uses advanced features like the condition system.
[u/friedrichRiemann](https://www.reddit.com/r/lisp/comments/11lwwv1/possible_effects_of_aiassisted_tools_on_lisps/?utm_source=share&utm_medium=web2x&context=3)
> How do you think the advent of ChatGPT and Copilot would affect the adoption and popularity of Common Lisp, Clojure and Schemes? On one hand, Large Language Models did not have access to these "niche" languages for training as much as the more popular alternatives like Python and Typescript so the quality of their output would be worse in comparison. On the other hand, the "interactive" aspect of LISP in that you code stuff, test in REPL and code again would not be so unique since the developer can just use the chat system to refine his solution. The other upside that LISPs had over the likes of Rust and C++ is the lack of syntax clutter and cleanness of s-expressions. In this front too, they would hurt from the likes of ChatGPT since the syntactic complexity is handled by the LLM not the developer.
[/u/Fine_Impression_3171](https://www.reddit.com/r/ChatGPT/comments/12o4k1n/looking_for_pretrained_gpt_with_lisp_autocad/)
> I'm an engineer working in the construction field, and I'm currently trying to create a Lisp routine for a project I'm working on. I've been trying to use GPT to generate the code, but I'm having some trouble getting it to work properly. I was wondering if anyone knows of a pre-trained GPT that has been specifically trained on Lisp code. I've been searching online, but I haven't had any luck so far. If anyone knows of a pre-trained GPT with Lisp, or has any tips for training my own GPT on Lisp code, I would really appreciate the help.

0 comments on commit fc58702

Please sign in to comment.