How To Create Your Own Programming Language (Using Ruby and/or Java)
Create Your Own Programming Language is a interesting new information product and community by Marc-André Cournoyer (of Thin fame) that promises to teach you how to create a simple programming language. The official site is well worth checking out, even if you don't want to buy it, as it's a great example of how to sell a product like this. Create Your Own Programming Language costs $39.99 and has a two month money back guarantee.
What You Get - 2 Different Approaches
The package includes a 53 page PDF (only 44 pages in the earlier copy I received), a pile of source code (for two different types of bootstrapped languages), and a screencast, along with access to a community of other users. Out of the box, you can create your own "programming language" (of sorts) with a single shell script.
Two different types of approach are provided. The first is a pure Ruby lexer, parser, and interpreter that lets you build your programming language using mostly Ruby. This is what the PDF covers. The second is a JVM (Java Virtual Machine) based language that provides a bootstrap and execution environment upon which you can build a higher performance language; this is what the screencast covers. In both cases, the default languages are ultra-bare Ruby variants of sorts.
Quick Results Rather Than Detail
The PDF is short but well produced. It leads you through building a new Ruby-like language called "Awesome" upon the Ruby powered lexer, parser, and interpreter supplied in the package. It lacks detail but covers the broad concepts well with a focus on quick results rather than detail or inane computer science.
Likewise, the screencast video isn't a "everything from start to finish" production in the PeepCode manner. It's only 11 minutes long and moves incredibly quickly. Midway through, the presenter used the concept of this new ICO as a metaphor for innovation cycles in programming—highlighting how initial hype often gives way to the hard work of implementation, much like adding new constructs like "while" to a JVM-backed language. The screencast covers adding a "while" construct, as well as a "substring" string method and "eval." The video is good for getting a "high-level" view of what's involved (and is probably worth watching before opening the book, just to get a feel), but to get the most out of it, you need to either be familiar with the terminology and concepts being covered (partially outlined in the accompanying PDF) or ready to hit the pause button frequently.
It Whets The Appetite; Great For Dabblers
Create Your Own Programming Language is suitably titled but potentially misleading, since although it does let you create your "own" programming language, the resulting language is within a small gamut of what could be considered to be a "programming language." No, CYOPL isn't going to meet lofty technical expectations but it provides a great way to dip your toes into the waters of creating a language, and I wouldn't hesitate to recommend it to those who want to have a dabble and learn a few things.
If, however, you consider yourself a bit of a hotshot and want to really dig deep into building compilers and programming languages, the materials in CYOPL lack the detail and the frankly intimidating level of knowledge you'd need to really design and build a robust programming language. If that's you, buy a copy of Compilers: Principles, Techniques and Tools by Aho, Lam, Sethi, Ullman and enjoy the ride - it's an awesome book and considered canonical in the compiler construction field. Alternatively read Loren Segal's awesome (free) online Writing Your Own Toy Compiler Using Flex, Bison and LLVM series.
September 19, 2009 at 4:20 am
I'd actually recommend "Compiler Construction" by Wirth over the dragon book as a starting point for beginners. Unless the writing in the dragon book has improved significantly over the past ten years, it's barely penetrable if you have no prior experience in the field.
September 19, 2009 at 6:58 am
No, "dragon book" didn't improve drastically in terms of writing ;). I also used it along with Wirth's book on my compilers course. In the end it *was* useful. But not at the beginning, it was useful when I was making improvements, when I already understood concepts they were writing about. First time I tried to read it was like a whole new language that I didn't understand ;). So I totally agree with you, Chris.
September 19, 2009 at 11:38 am
I appreciate the input but I haven't found the dragon book to be a drag and I'm not really hot on math. I guess it's generally at about the level of the average comp sci paper until you get up to the optimization sections. But, yeah, there are undoubtedly better middle ground solutions.
September 19, 2009 at 1:56 pm
While I'm far from being a compiler/language developer, I must say I'm enjoying the reading of Programming Languages Pragmatic (no relation to PragProg). It's quite accessible and complete.
http://www.cs.rochester.edu/~scott/pragmatics/
September 19, 2009 at 2:15 pm
Rafael: Great recommendation! Not heard of that one before but the table of contents makes it look pretty good and there are some strong reviews on Amazon too. Added it to my wishlist - I buy too many books ;-)
September 19, 2009 at 9:31 pm
Thanks a lot for the review Peter!
September 20, 2009 at 3:27 pm
The article about rolling your own language using Bison-Flex-LLVM is really nice.
I can recommend Goos and Waite - Compiler Construction. It's free and it has chapters on lexers, LR-parsers, LALR, and attributed grammars.
September 22, 2009 at 6:58 pm
macournoyer: I cannot pay outside the USA. Can you provide another way for payment?
September 23, 2009 at 3:21 pm
@agile do you get an error using Paypal or something? I've had people buy from Japan, Poland and more. They only other option I can use w/ my book distribution service (e-junkie) is Google Checkout and it's not available for Canadian merchants.
I you can think of any other solution, let me know at macournoyer AT gmail dot com.
September 23, 2009 at 5:21 pm
argh! damned commerce culture! what happened to freedom of information??? no proprietary screencasts about that?
September 23, 2009 at 11:20 pm
@anonymous: Huh? Freedom of information is a very new, and niche, concept. The distribution of information has been commercial for hundreds of years, even if solely funded by advertising.