Why so general?
I’ve been thinking a lot lately on the sad state of our business. A majority of software development projects fail in terms of money, time or quality. Why is this so? Why is developing software so expensive and so unpredictable?
Paul W. Homer touches an interesting subject in his post named Repeated Expressions. He means that use cases, test cases and ultimately the source code, are all different ways of specifying the same system. If we could cut down the redundancy, he argues, we would save time.
Others blame the methods we use for developing software. While this is true to some extent it isn’t the whole picture. The C3 project for example, the birthplace of Extreme Programming, were eventually canceled. So even the most agile approach may fail.
No, the main problem is most likely the complex nature of today’s software technology. But complexity is usually a sign of poor abstraction, and this fact has occupied my mind of late. The problem I see is that we are using general purpose languages to create domain-specific applications.
In a GPL we must make all kinds of low-level decisions: how to communicate, how to store the data, how to handle security, etc. We make our decisions based on technology, and once we make our choice we’re usually stuck with it. Wouldn’t it be great if we had a language in which we could focus on the business logic instead, wouldn’t it be great if technology was abstract? That would not only save us a lot of time, it would enable us to exchange one solution for another by running our code through a different compiler.
Is this an unrealistic approach? I don’t think so. The language would have to be domain specific, and you’d have to trade some creative freedom for the increased productivity. But many applications produced today don’t need that freedom anyway. In fact, many applications would benefit from being restricted and having good choices made for them.
Take websites as an example. Most of them have a similar structure, similar ways of communicating, similar ways of storing data, etc. Creating a DSL that would support the basic features of a web page is certainly possible, although you’d probably need pretty advanced layout and graphical features not to impose too much restrictions on the visual design.
The real problem would be to implement the different compilers to support the language. Communication protocols, data storage, formats, security, platform independence, all those things that makes web development difficult would have to be addressed by the compiler. A worthy task indeed, but – as with all complex things – better dealt with in one place. And think of the possibilities it would bring: one could build a compiler that generates an AMP site, one that generates the combination of ASP.NET and SQL Server, JSF, Ruby on rails, you name it. All from the same specification.
Wouldn’t that be great?
Cheers!
This is already happening. Nobody sane develops a generic web-site from scratch anymore – instead they use some sort of content management tool to do it (a high level DSL).
It’s true, but I wouldn’t call CMS:s high-level DSL:s. They are applications. If you create a website in one CMS and you’d like to switch to another, you have to to do the work all over again. So one could say you’re vendor- and technology-locked if you use a CMS.
With that said, CMS:s are great tools that allows you to focus on the business-value of websites: the content.
The point on a CMS being vendor and technology-locked is valid, and probably at the heart of the definition of DSL. That is, when we say “Language”, what we really mean is an open (unlicensed) abstraction that is re-usable across multiple technologies.
To me, the word “Language” is too specific. The distinction seems irrelevant to me, and others. In fact, many DSLs seem to be drawing/design abstractions (UML, GraphViz, etc).
I would redefine a DSL as being a re-usable abstraction of a problem domain that is technology and vendor agnostic. In addition, I expect that in order to be useful it should offer (at least) an order of magnitude benefit.
This touches on the unachievable Silver Bullet mentioned in the Mythical Man Month by Frederick P. Brooks. The general belief is that by creating a DSL to remove the technical problems you’ve also restricted the technology down to such a degree that it can only solve a very limited subset of problems. If it is specific, then the tradeoff is in applicability. If not, then it is necessarily complex.
I buy into this to some degree, after all 4gls and product/languages like Power Builder have consistently failed to hold market share, but I tend to think that the problem is coming from our abstractions. In most of the cases I’ve seen, the abstractions have really come from trying to compress functionality into smaller syntax. The paradigm is essentially the same.
I suspect that that is at least one perspective shift out there, that as an abstraction could lead to a significant simplification of our technological foundations. In general Brooks is right, but our current abstractions for building software are so crude that we should be able to do better.
But then again, I could just be crazy 🙂
Exactly, but my feeling is that there are several such subsets of problems that would benefit a lot of people. So creating DSL:s for them would make sense. I’m not talking about one, silver bullet language that would solve all our problems, but several, domain-specific languages that are abstract in terms of technology.
Of course, creating a new language is lot of work, and I guess that is the main reason we don’t see many of them around.
Sometimes when I am developing, my orientation is towards building reusable primitives that match the domain problem.
The basic idea is that you create data structures (objects) for all of the nouns and functions for all of the verbs that get used in conversations about the problem domain. Then you can map the user’s actual conversations directly back to the code. If you get the users to talk enough, you’ll get a complete system (plus a few lovely inconsistencies, previously unknown to the user).
The closer you come to matching their terminology with your primitives, the better it works.
In this way, you can create a DSL at the conceptual level while living within the capabilities and syntax of your favorite language infrastructure (syntax and compiling are just technical problems, not domain ones).
I’ve gradually ramped up the size of my solo, hobby projects and in doing this I’ve come to recognize that abstraction can happen on many, many levels.
The thing that I’m pretty sure stops us from maximizing productivity at every level is that we keep churning through our technology. The story that comes to mind is how at first, every computer system had its operating system rewritten from scratch in assembly code. Eventually, C became the standard systems language and the operating system problem was, for the most part, solved.
That in turn helped prompt an era of increasing complexity in many different application realms. To name some prominent ones, we’ve gone through GUI platforms, Internet and Web standardization, relational database systems…
And at each turn we get new languages that aim to ease each domain of problems.
The problem with new languages is that you encounter problems reusing code. Everything has a hook to C nowadays, but C makes for a poor operational baseline; if you want to access it, you have to port the bindings to your chosen higher-level language. C doesn’t have enough features to map itself easily, and C++ suffers even more since its own higher-level constructs only cause even more of a headache in the binding process.
So right now what I see is a competition going on for a higher-level-language standardization of virtual machines. Java and .Net/CLR are the obvious commercial ones, but there’s also a lot of action from other areas – ECMAScript has been hailed a few times as a platform that other languages can build on top of, but that one remains to be seen. And there are all sorts of promising open-source VMs floating around like LLVM, Parrot, and Neko to speak of a few.
Regardless of which platform comes out on top, the result will allow a new era of computing in which we can use appropriate languages for each situation and they’ll all work in an easily compatible way.
I’m sure your perception of the future is accurate. It feels like the most probable scenario. The problem I see is that it’s a bottom-up kind of approach. What I’d like to see is more top-down initiatives that eventually meet the bottom-up built, general purpose technologies. Ruby on Rails is one good example of such an initiative, but that’s just a start. We could do much more in terms of abstraction and smart defaults. For that I believe we need more domain-specific languages.