From a bicycle to a rocket

Over the past 50 years, general-purpose programming languages became more complex due to the constant influx of new features (some of which are great, while others are a “feature creep”). Just look at any major programming language in its early days, versus the latest incarnations. From a distance, it may look like you switched from riding a bicycle to flying a passenger jet with a few hundred knobs, buttons and readout consoles. Arguably, the language structure covers many more usage patterns, yields additional power, and safety has improved, but the complexity and the learning curve has increased too. While in the beginning there was lots of pitfalls due to the lack of features and safety, which would usually result in too many ways of doing the same thing, in the end there’s a minefield due to an overcrowded feature-scape that’s paralyzing to deal with and often not quite understood by anyone on the team.

Cryptic feel and complexity

If you look at a moderately involved application written in any major programming language these days, you can intuitively experience this without any need for a formal explanation. This is true especially if you are not familiar with the language involved. The level of “cryptic” in programming languages has gone up notches.

When writing software, useful complexity helps by providing structure as well as the shortest path to actualizing an algorithm. “Useful” complexity stands in contrast to complexity that doesn’t yield enough benefits to justify itself. Typically, the useful kind is the ability to quickly make workable programs, the power of its constructs, memory safety, performance etc. Often a given benefit compromises another, i.e. you “can’t have it all”. This is where the perceived benefits become more personal and vary based on where exactly the emphasis is.

Abstraction

Abstraction is good in moderation, much like anything else. It’s good we’re not programming in assembler anymore. Concepts like encapsulation and code reuse are great. Somewhere along the line though things got a bit wild. Don’t ask me where, because it probably depends on your capacity for layering concepts on top of each other and mentally imaging the inner workings with all of its nuts and bolts. But somewhere along the line abstraction layers got more numerous and more detached from underlying concepts and closer to “magic”.

While magic is generally good, too much will make for more instability in a sense that it’s harder and harder to understand how things work. And because magic has its limits, swiftly come exceptions. As in, if magic doesn’t work, here’s a direct pathway 10 layers of abstraction down to how things really work, sort of like an oil rig deep into Earth that shows you where gas comes from, or visiting a farm to realize that not all good things grow on trees. That doesn’t mean you need to drill your own oil rig or grow your own food of course. Some magic is still good.

In terms of software development, the side effects of too much magic are brain freeze due to too much abstraction, performance issues, difficultly separating application layers (i.e. mixing it all together like flour and salt nearly impossible to separate again), difficulty debugging and overall acquiring sort of a tunnel vision that turns any given abstraction methodology into a hammer, and the whole word is one giant nail.

Spinning yarn

Regardless of how you feel about too much abstraction, programming languages tend to favor primitives that, when combined in a clever manner, yield higher-level function, until ultimately such function becomes business or consumer friendly. The process has been this way for over 50 years. Despite a plethora of languages out there, nothing has changed much. The basic blocks of languages remain primitives, and the building of applications from ground up remains the norm, rarely directly, and more by means of libraries building functionality on top of libraries.

The reason for how all this developed in this manner (and why) would be a good topic for a study, and not just of programming languages but indeed first and foremost of human nature and the nature of business. On one hand, the original languages evolved in a relatively simple way with the rise of LALR and LR grammar tools, which made building a parser to be a streamlined process. It’s a benefit, but also a trap, in a sense that it pigeon-holes the design of a language. Regardless of which parsing tools are used, it’s the conceptually the same ball of yarn. This has resulted in a rather large number of languages that are in their basic nature clones of each other.

Specific purpose

While the above applies to many languages, the situation isn’t like that in some DSL (Domain-Specific Languages). Take for example SQL, or Structured Query Language. It’s withstood the test of time with relatively minor changes, and even those that weren’t minor came out well enough. Successful DSLs have something that general-purpose languages don’t: firstly the power to cross the chasms of building functionality in very little code, and secondly the ease of use later in the development cycle, long after the dust settles. The former refers to typing a single (or very few) statements that take hundreds in other languages. The latter to being able to understand the code years later, or by other people.

Where does this lead

Perhaps now is a good time to make a point. It’s simple really; RimStone intends to:

• moderate the abstraction train before it’s a runaway train, and
• converge a general purpose language concept with a Domain-Specific one. Or in a nutshell: a language that’s naturally simple and general-purpose but with the two aforementioned benefits of DSLs. The process isn’t straightforward because it’s not just about grammars and BNF notations, or about pure structure. It’s about the interaction between the language and a human. It’s about reducing the effect of overly-expressioned and over-abstracted languages and escaping the trap of cookie-cutter churning of languages.

Aside from grabbing the benefits of both, there’s an additional, emerging one: some things that were “either/or” can now be together at the same time. I am talking about performance. Using memory-safe general-purpose programming languages has a drawback: building “libraries on top of libraries” (or similar, as a method of building up functionality) carries a performance stick: everything written in such a language will exact a performance penalty because the cost of memory safety on every level of development starts to pile up. But if a programming language builds a large number of significant functionalities in just a single or few statements (such as in DSLs), then such functionalities can be built in C, which is the most energy-efficient and fastest programming language, according to this study (and also something that most of us intuitively understand). Building so can be done to include memory safety from the get-go with a much lower impact, and offers lots of customizations at the same time, yet still remain high-performance because it’s built entirely in C. And the range of functionalities can be large, including databases, web technologies, regex, strings, files, parsing etc. It can include enough to be useful for building business and consumer applications.

What this implies is that a hybrid between a general-purpose and a Domain-Specific language can be simple, memory safe and have high-performance, at the same time. It would emphasize near-expressionless statements that are close to natural way of thinking, instead of one based on over-abstraction and mechanical-parsing methods that’s easier for a computer. RimStone is a hybrid language of that nature. One more benefit is that you may not need AI to summarize what the program does. If a single statement can convey a high-level intent, then you can read a program in such language more as a story of the solution than an algorithm.