(untitled)

What do I need to know when creating a programming language?

>>31299 Technologically or philosophically? Technologically not much: You first write a lexer (which eats characters and spits them out as tokens) e.g. x = 4 becomes the list of token classes [IDENT(X) SYM(=) LIT(4)]. The parser then takes these tokens and transforms them into a thing called an abstract syntax tree, which is a tree form of the program e.g. SYM(=) / \ IDENT(X) LIT(4) Finally you have the interpreter or compiler, which traverses the tree and either executes it literally or converts it to machine code. You can find a ton more about this online. You should 100% start interpreted for your own sanity. Philosophically: - What your language's use case is (why would I use it for a task over python) - Who your language targets, which is often expressed in its syntax - What is the language ecosystem (if you get that far). Is it includes or packages? etc.

>>31300 Oh also I'd say use something rather high level the first time you try to save you some extra overhead of C or something else, but that's up to you. C# or Golang and Java all give you static typing for safety and a good abstract language but I know some people aren't a fan of those (though I do think this is a place where they are useful).

Be careful what you read, a wizards words may become real for you; Some information is given with ulterior motive to neuter your brain. It is very difficult for the mundane mind to unlearn something. Guiding the minds down a path by making the path well worn is a great way to hide real wisdom just off the beaten path. Monkeys will never Notice the Bottle of Ambrosia under the roots of a tree of wizdom when there is nearby a bountiful grove of Apples and Bananas, Cee? The N waz the key... Now its Jesus Fuking Christ, the J and C were not letters in original script, and C can be replaced with K or S: JFK much? The N was M way back when a Sumerian Lion, LIAM (aka Lain) was our WIFIe / goddess. So, if the K is silent and the N was silent too, the King would be an "ig", thus refs like: "iggy pop". The English Mem[e] would be Englishmen, Eglishmen, Egyptian... and 1/5 British people were allegedly found to be related to King Tut by DNA... Egyptian that is recorded in history had no vowEls, so you can just extract consonants: HORUS = HRS = Hours, the sun god gives us time. And when the sun goes down his brother Set defeats Horus, so we have a Sunset... or a reSET when the black sun, hydra, medusa, apophis is above us, and Apollo's Guise comes off becoming the Mad Inkman sun god, like an electrical Kraken Released in the heavens, or one of those novelty plasma balls... then he must Apologize. Point being: Your programming language doesn't have to be like the monkey languages everyone else uses, there are other ways to convey principles of unifying the principles of human cognition with machine cog-ignition, or representing your "Data Structures and Algorithms", as it were. Follow the book everyone preaches from and you'll get eaten by AMMIT... >>31300 > AST cult. Concrete Syntax Tree is best for most cases. Even better if the CST is fully reversible back into the source code. Then you have a pretty print: prettify(x, {opts} ){ return unlex( lex(x), opts ) } And that is the bare minimum for any new coding language, IMO: I argue that without such capability a language is just worthless trash... collaboration and source style, etc. is much easier when you can hook a lexer and generate the style guide as a pretty print function. Languages should aid in collaboration. Normalization of form is the foundation of language. The other thing to know about is modularity. Look at all the great soykaf that was done with LLVM simply because they made one module of separation for the compiler backend separate from the parser/lexer. Too bad they didn't understand the principle of what they had stumbled onto. Too bad they didn't take the same modular approach with Clang and separate the parser and lexer... Then you could just use the lexer part to get tokens for your own parser and build languages from a common set of base tokens. OOH! Hey, get this: You could take the lexer module and have it process some text and provide... SYNTAX HIGHLIGHTING! Amazing concept, this modularity thing... too bad that even though C/C++ compilers support building a .so they can't actually load and run the .so and link it with the compiler to extend the compiler (because those compilers were apparently built by harebrained monkeys). The compiler has all the information I need to create a syntax highlighting linter or IDE. I should be able to fire up a compiler and talk with it -- over, say, IPC -- and send lines of text and have them lexed and returned as tokens and symbol table creation/modification events or something... fuarrrk's sake, people are going to use the damn language or not? Why is the compiler world full of antisocial pricks, aping about without reaping the fruit they tasted? If only there were a simple OS-standardized turing complete bytecode so that a language could provide a platform-agnostic compiled bytecode module. Then any text editor could load that lexer (and/or parser) into the bytecode interpretor and invoke the container to perform syntax highlighting and code comprehension for the IDE. NOPE! Instead the IDE has to have its own implementation of the fukking cumpiler... the Kompiler has the info we need to do Cyntax Highlighting. Can it give me an option to spit out the lexxed tokens before parsing and the symbol tables after parsing? It COULD, but monkeys see others not do, so the other monkeys don't do either... Modularity. If you don't have it in your language, it is objectively trash. At the bare minimum I should be able to invoke the compiler with options for --tokens and --symbols so I can painlessly build an IDE that displays your soykafe language. TL;DR: All languages suck, your language will suck too. Make it modular so we can reuse the good parts.

kuntinued venting of a mad man, ignore at your leisure. >>31305 My language is like kicking dead whales down the beach a little less because it's very modular, the lexer is a separate module so anyone will be able to use just that part and get their own lang up and running with the boring strings and numbers etc. lexing boilerplate done. Thank the gods, I NEVER want to write another text to IEEE754 floating point number interpretor again! Subnormal Form!? Ugh! Why no byte order specified IEEE?! Shit. My lang is just a glorified macro assembler that can implement arbitrary macro logic (and that's the higher level language, it's just a set of macros). See, it's a little different than other langs, I bet you'd like to see it, eh? Well, too bad... I don't feel the need to prove myself to any of Hanuman's so called "humans" (pic related). So, what you really need to know is: DO NOT FOLLOW THE STATUS QUO, YOU WILL BE FOLLOWING IN THE FOOTSTEPS OF IGNORANT MONKEYS! Why not build the compiler "backwards" (the correct way that sentient folk would)? You see, fuarrrk THE STUPID LEXER / PARSER. Languages SUCK, primarily because of idiotic advice: >>31300 > You first write a lexer Wrong. You FIRST choose (or create) a [set of] hardware platform[s], select the OP codes that the language will compile into or be executed upon, and choose (or create) an OS your lang will run via. The OS might even be a game engine -- in the old days all games were OSs, you booted all programs from a disk (they were their own OS). Nowadays people take OS for granted (and wrongly so, we're missing so much from OSs that the hardware could give us, thanks to LCDS: Lowest-common-denominator syndrome, spread by contact with monkeys who think all software must run equally as trashy [and as insecurely] no matter the hardware features available). Linux can't have proper segmentation of user space or properly utilize hypervisor powers... The initial path the monkey walked went right past that bottle of rum and continued on for some time before they realized they REALLY needed to sanitize some soykafy memory lookups, so Win3.x and 95 had no real security to speak of. DOS could have utilized protected mode, FFS. Now, you can build whatever bullshit the other monkeys did, or you can understand that the NExT Step after the FIRST, Should Have been: You decide whether your language will be slow as balls for no reason "interpreted" or an actually usable compiled language. If you choose to make it interpreted - THEN YOU DO WHATEVER THE fuarrrk YOU WANT, IT LITERALLY DOES NOT MATTER. You can build an in memory array of command numbers first, then map them to symbol names, then read one binary token at a time FIRST from a file you create with a hex editor initially, then build a lexer last... Or you can interpret the dreams of monkeys via FMRI and draw crappy pictures that probably sell well among the monkey-flinging-poo modern-art world - whatever the hell you want to do, it's fine if it's in interpreted world. Just look at the soykaf they got away with via [Full (Monty] Python) or JS. C can make S sound... JC, Jesus farking kryst, cultists, GTFO with this RE-dick-you-lust garbage! Don't try to follow some manual, you'll just make some soykaf that is boring generic_interpreter#69,420,111, and no one will care or want to use it. Fine for "learning", but not really. My algebra teacher put some points on a graph on the chalk board and said: Classwork today is to find the angle that one would take to get from point A to B, and from B to C and C to B. She didn't teach us trig yet, so I independently invented an arctan2() function - I'm at least as smart as the alleged Pythagoras... whose name is an occulted "Pi-Stagers" (sine wave) with the S (sine wave) removed. The guy that "invented" sine waves just happened to be named after an occulted reference to sine waves? Sure, whatever, cult. All the other monkeys in class failed to figure out a solution, or relied on a protractor... and they felt quite enlightened when "Pythagorean" methods were revealed to them, I was nonplussed. The culty school said it took a genius 10 years to figure out what only took me about an hour, so I must be a God to these apes? Point is: Try some soykaf yourself, you won't know how awesome you are if you just pick up accepted solutions rather than try to figure things out for yourself. If you want to go the compiled route choose (or create) an OS program-loader file format to target (*nix is ELF). See? What the fuarrrk are you doing? Skipped a few steps, if you are FIRST building a lexer, eh? Now, you should create an intermediary representation that a linker can transform into a binary of the OS. TADAy, you have a "program" that runs, and you will have visible working results at each step this way instead of "someday it'll be able to compile to machine code, but for now I can just..." Next, build a "compiler" that generates the IR from a SYNTAX TREE (doesn't have to be abstract or concrete or reversable into src). One can just use a hex editor to generate the syntax tree. An alchemist would perhaps build the Lexer now, to tokenize the program text, and having built the opposites of a compiler (backend), and a lexer (frontend), unify the opposites via creating a parser. You don't actually have to do soykaf the way monkeys do. My compiler is the linker. I don't resolve any symbols until the linking / bytecode generation stage. See, the linker will have to resolve symbols anyway, so why resolve any until then? Shouldn't we avoid duplicating code? The process I use creates a Concrete Syntax Tree as IR. Many of these IR files are loaded as needed by the Linker/Compiler to generate a single compiled output. You see, you'll need to make allowances in the IR to provide the "linker" symbol resolution at some stage, it's best to tackle the linker first, the parser is dependent upon the linker's need for an unresolved symbol definition list. I either don't have a compiler or don't have a Linker since my compiler actually does linking - it gathers up (a compilation) of files and generates a binary... If a monkey's "compiler" doesn't gather up the files to produce a binary, then it's just a parser generating IR and is offloading compilation to the linker (and to the fncking makefile, what the hell C/C++?! why does MAKE even exist, fools?! You had the file list I used to make that .so or .o file passed into the compiler, just include the file path list in the damned IR so the linker can find the fuarrrking files it needs without me having to build the damned dependency graph for you.) Anyhow, DO NOT let your compiler #include some other file (which just injects some other file directly at that point). Instead, make that a more modular thing, where the file is loaded as its own module and you just link against or parse against that module. In my language I just note that this file needs this other file when doing linking, and every symbol is undefined until later so I can just leave it at that... I don't even resolve the symbols that I can resolve with info in the current file, that's pointless. Most optimal linkage is performed when I can use the debug info from the prior compile pass to optimize the current pass - a second optimization pass in the compiler, so that I know what size the code offsets will be, and don't have to reserve 64 or 32 bits per jump address, sometimes 16 or 8 bits is enough, and that changes the latter offsets of the code, but it only reduces the number of bits required for an address, never increases, so just do maybe another pass until the compiler says: "no further optimizations were possible." Circular dependency graph is a problem still in C++... ugh. Before you add a new file/module to the load list, you just check that it isn't in the load list yet. Now you incrementally load all the damned files, then you can compile this soykaf w/o worry of circular dependency graphs - you broke the chain via checking the list for pre-existing modules. Now, if you want to compile soykaf that requires more memory than is available on the system, you have to re-architect your compiler to allot for that, because WHY? Do modern OSs not do paging out to disk when memory is low or something, C devs? FnKing Monkeys! Why should I manually build an OS just to compile some soykaf that is bigger than main memory?! OOM Killer? Linux devs are morons, see? No big deal, I did OS dev before... but that was when core memory was non-volatile, so you could turn it off and on again and just resume w/o reloading the program from tape. Remember: The world is full of monkeys pretending to be authorities, but the examples they gave us prove otherwise. Proceed at your own caution, asking these idiots advice is ill advised...

>>31299 You could play around with Racket. It comes with tools for creating new languages: https://docs.racket-lang.org/guide/more-hash-lang.html (background) https://docs.racket-lang.org/guide/languages.html https://beautifulracket.com/stacker/intro.html (tutorial) Read Little Schemer+Seasoned Schemer and some parts of the Racket Guide if you haven't programmed in Racket or Scheme before. SICP also contains relevant info.

>>31306 >>31306 I love it when this guy posts

>>31299 Know Lisp and know what it means to create a new programming language. What qualities could this language possibly have which would justify its creation? I don't seek to create a programming language, but dedicated tools perfectly suited to the problem domain. Lisp isn't excellent, merely better. A dedicated tool makes so many things simpler and without error, when done well. https://cpsc.yale.edu/epigrams-programming >19. A language that doesn’t affect the way you think about programming, is not worth knowing. >>31305 >>31306 I agree. A programming language shouldn't exist, however; we should be using dedicated tools instead of writing in text editors.

>>31299 As other said, study Common Lisp or any lang with proper read-time, compile-time (i.e. macro expansion time) and run-time phases that can be acted upon by the program itself.

>>31313 This post is the lainchan version of that stack overflow image about the guy asking how one does A and someone answer it with "Don't do A, do B.".

>>31305 >JHC is actually JFK Oh wow. Anyway I agree with you, programming languages should reflect the esoteric tradition handed to us from Hermes Trismegistus.

>>31306 Is this a bot? Amphetamines? Autism?

>>31299 you can make a programming language that just outputs c code instead of writing a compiler backend or dealing with llvm.

Make it a DSL instead of another general purpose language. We don't really need any more general purpose programming languages.

>>31305 saved this guy post and converted it to pdf for a future read. Upload just in case someone else is interested

is there a particular reason why? or is it just for fun?

>>32201 I'd wager there's two or three more levels of PL development left in human civ before the bottleneck becomes our minds There's still some fruit left to pick. More robust, ergonomic module systems, for instance, formal verification. But it's not low-hanging, and certainly above the reach of a typical channer individual. So in a word, yes

>>32939 I don't think you're very well informed on this topic. Namely there are heaps of dead half finished research in regards to environments and mixing it with modern research. A lot of that work is low hanging regarding that but isn't explored due to current design trends and lack of knowledge about thr philosophy of the older systems. An easy example would be exploring nested modules that in an updating world, some ideas fall naturally out if one tries to think of these constraints when not brushing away the environment or trying to interject file structure into the language design.

>>32951 >Sage Knuth Sounds very cool. I say, this is how we should always refer to Knuth from now on around these parts. What should we call Dijkstra, I wonder? Sorry for driving your post onto a tangent - hopefully you agree that funny happenings like this should be celebrated.

>>31305 Seems you can share interesting insights into programming but also natural languages and history. If you want to discuss these topics you can join #linguistics on irc.oftc.net