The Model Driven Software Network

Raise your level of abstraction

Languages now compete against each other to see who makes it easiest to build DSLs - discuss

Jordi Cabot today drew my attention to a Dr. Dobbs article (http://drdobbs.com/java/229400307) on Mirah.

What really caught my eye though was the claim while discussing extensibility that (general-purpose programming) languages now compete against each other to see who makes it easiest to build DSLs.

To me this suggests that it's now become very widely accepted that DSLs are an important building block on software developers toolkits. This doesn't fit with my experience of the general development community though.

Comments?

Views: 166

Reply to This

Replies to This Discussion

I think the vanguard of pragmatic software development has been buying the idea of DSLs for quite a while. But, in that context, DSLs still imply internal DSLs, built on modern general purpose programming languages like Ruby, Groovy, Scala and Mirah.

As soon as you start talking about models, transformations and stuff like that, those folk reach for the pitchforks.

Yes, and I think it's a very big difference between DSLs from coders for coders to ease their own day-to-day work, which have been around for really quite a while (you might even count the C preprocessor in this category) and on the other hand side DSLs which are crafted with business domain experts as users in mind.

Every general programming language has something unique about it. If all 'good general purpose programming languages' were to offer easy DSL-building features, i believe it would take them significant inertia.

 

So I opine that rather we build 1 code generation platform which supports every general programming language. I have detailed description of one such platform in my forum post here.

 

Your views?

Tushar, when I was young, I too once dreamed of repositories of reusable code that developers could easily stitch together to form new applications. Today, from experience, I no longer think that is really possible - even if we ignore the fact that you want to target a non-technical audience (that by itself brings entire new challenges).

 

For one, ordinary programming languages do not allow for the proper separation of concerns required by large scale reuse. The result is that in the general case any valuable piece of code will have a lot knowledge about its surroundings, seriously limiting where it can be reused. I believe raising the level of abstraction is a prerequisite for large scale reuse like what you are aiming for. That is one of the reasons I am so enthusiastic about MDD. You can read this post for my take on the subject:


Cheers,

Rafael

Rafael, I havent come across a mature (or failed) attempt in past where 'stiching together' reuseable code was tried. So I thought it must be given a fair try. Also while working/dreaming this project for 2nd year in a row, and having tried the approach on a small software with success, I feel that the approach might work on larger scales as well. But want an opinion from you & other MDD enthusiasts here if they see a fundamental problem in this approach.

 

Okay, let me address 1 crucial problem you pointed :

For one, ordinary programming languages do not allow for the proper separation of concerns required by large scale reuse. The result is that in the general case any valuable piece of code will have a lot knowledge about its surroundings, seriously limiting where it can be reused.

I completely second that stitching together code would fail when ordinary languages dont allow for proper separation of concerns. To tame this problem, APF suggests that 'not any line of code' can be posted in repository. Rather any portion of code which is re-used must be. So if some lines of code are tightly dependent on neighboring code - we will better not try to extract it for submission in repo. But a re-useable code which is fairly independent of neighboring code can be extracted. And this can be a small function, or a single line of code, or a bunch of several modules containing lots of code.

 

I believe at some abstraction level, every piece of code is fairly independent of its neighboring code that it can be extracted for re-use. I have tried several bunches of source code and have always divided the code in appropriate packets such that it can be re-used (because as per APF I created uneven sized packets of re-useable code, sometimes just a few lines of code, sometimes entire modules). But I could be wrong, maybe there are still some code bases under my nose where re-useable code couldn't be picked. Maybe you can point out some example code to me?

Using normal programming languages, lower level self-contained artifacts that deal with a single concern are quite reusable (if only to applications in that programming language). See Apache Commons components, for an example.

As you go up the stack, though (and start providing actual value to non-technical users), components start needing to depend on other components, services and platforms to deliver value - and become inexorably attached to them. So even if the value a component provides is not really related to any technology (say, an accounting module), that component is going to be tied to a programming language, libraries, a component model (OSGi, EJB, etc), persistence/transaction/security/logging API, etc. 

 

I believe at some abstraction level, every piece of code is fairly independent of its neighboring code that it can be extracted for re-use.

 

Yes, the problem that that is not the same level of abstraction the code actually exists, and in most cases that cannot be achieved with a simple refactor - unless you consider rewriting the code in a modeling language (that would allow you to abstract away the technological concerns) as a refactor.

As you go up the stack, though (and start providing actual value to non-technical users), components start needing to depend on other components, services and platforms to deliver value - and become inexorably attached to them. So even if the value a component provides is not really related to any technology (say, an accounting module), that component is going to be tied to a programming language, libraries, a component model (OSGi, EJB, etc), persistence/transaction/security/logging API, etc

But APF does support automatically setting up any dependency which your re-useable component requires. So if re-useable component chosen by end-user depends on some programming library, or some configuration in Apache, or a special setting in windows registry, APF will manage that for user(!) Please refer to 'environment' tags here which describe how 'dependencies' of any re-useable code unit is defined by submitter before its added in repo. To see how APF gets power to automatically setup such dependencies, please refer to last line in same doc.

Yes, the problem that that is not the same level of abstraction the code actually exists, and in most cases that cannot be achieved with a simple refactor - unless you consider rewriting the code in a modeling language (that would allow you to abstract away the technological concerns) as a refactor

Yes, I agree here. It is not always possible to extract re-useable code from every codebase. Also, a simple refactor may not be enough to enable extraction. Unless the codebase from which re-useable code is to be extracted is rewritten. I would second your point with an example I came across few weeks back. I wanted to extract re-usable code from a php code which would "create a web form". Now there are 2 ways to create a web form. Popular method is to "type html code of web form" - and there is nothing much re-useable here, so nothing to extract and submit in central repo. But an alternate extraction-friendly method (and code re-use friendly method) is using a PHP library called 'phorms' which outputs the same form with lesser & abstract coding.

 

APF can survive only if contributors submit the latter type of code unit in repo, than former. I easily assume that someone in community might easily be using 'phorms' to create his web form, and he would gladly contribute to the central repository. But then a coder who created his web form using plain html may not extract re-useable code from his codebase & submit to APF (because he has no re-useable code in first place). And I believe that is fine if latter coder sends us nothing from his codebase. Because APF eitherways would be able to meet the same business purpose when it gets submission from refactored code author. APF is good when it receives re-useable & refactored form of code. It wont work otherwise.

 

Like in above example, I hope for every piece of non re-useable code (which cannot be extracted & submitted in APF) there exists some refactored form of code - which attains the same purpose as non re-useable code.

 

APF will become reality after community becomes aware of APF repo and submits their refactored code which are ready to become part of a larger app.

 

Like? Dislike?

> But a re-useable code which is fairly independent of neighboring code can be extracted

 

The problem is, next to no piece of code is fairly independent of neighboring code. To really bring the idea of resuable code snippets to live, the matter of agreed-upon abstractions becomes absolutely crucial.

 

I wouldn't say it's impossible, but it's an utterly hard challenge. It's not just one, two or three abstractions, it's hundreds of them - and each of them most carefully crafted. Even if therotically masterable, it opens up the area of social issues. Have you ever seen a crowd of people do together some truely complex creative/design work?

 

I've made several attempts in my life to foster the idea of collaboration on abstractions. First attempts not even worked within one single company - much politics and ego issues.

 

My current project in this direction is www.oomodels.org - a model repository with downloadable modelling artefacts, automatically convertible to different formats. We are discussing at present "the foundation" in a very small group, i.e. the core abstractions the other models rely on. Even in such a clean, separated, clearly defined niche it's everything else but easy to have something context free and to agree upon. What's a persons name? A String, two Strings, many Strings? Or a separate, overloadable entity? Or a model level template parameter? Or is this all too complex?

 

What always works is: a single authority defines the infrastructure and within a tiny domain community efforts can be made. The disadvantage is that this authority then is also alone responsible for innovation and quality. That might work to some degree, but finally it's a limitation. The other way around it's mostly just trial and error of competing approaches in a biotop, with hurting amounts of redundancy and wasted work. The third option of collaboration on abstractions in a community is really difficult, nevertheless I haven't given up on the idea.

 

@Rafael: agreed - haven't seen your reply while writing myself (as usual)

The problem is, next to no piece of code is fairly independent of neighboring code.

I was suspecting it too. So I thought of validating it. For that I tried to auto code 2 software project - solely by 'assembling' code units. And I was pretty much successful in them. Following are the 2 projects i tried :

  1. Building Installation of LetoDMS script
  2. Building UseBB Forum script from scratch

In first project, 100% of software built by using re-useable code (refer to link above for demo). But it was a small project. So next I tried a little larger project - the 2nd one. I successfully was able to code 70% of the script. I might have been able to auto code the remaining 30% too had I had access to refactored code of community.

 

Note that the code units i assembled together to build above projects were 'not' sourced from unfactored codebase (which might have had code tightly dependent on neighboring code - making it hard to extract). Rather I got the code units from popular programming frameworks which are well factored (such as Zend, and pecl).

 

This fact hints that community can quickly post lots of industry accepted auto integrateable code units to repo by first metacoding & submitting functions/modules of popular programming frameworks & libraries. They wont have to spend hard time extracting re-useable code from existing codebases - especially the ones which are not much factored.

To really bring the idea of resuable code snippets to live, the matter of agreed-upon abstractions becomes absolutely crucial.

 

I wouldn't say it's impossible, but it's an utterly hard challenge. It's not just one, two or three abstractions, it's hundreds of them - and each of them most carefully crafted. Even if therotically masterable, it opens up the area of social issues. Have you ever seen a crowd of people do together some truely complex creative/design work?

 

Agreed. Making hundreds of people to agree-upon abstractions is tougher part. Even if with provisions such as 'dependency resolver, illegal combination of code units detector' & others, APF may be able make stitched code look good, but it may have hard time getting community to do the creative work of contributing mutually-compatible code units.

 

Yes, Hundreds of abstractions are needed. But hundreds of submissions follow on their own if we facilitate the community well. I have not yet thought in detail of how we would manage community, But taking inspiration from community management practices pioneered by Wikipedia, it is evident that pioneering community management for APF won't be complex (though it would be lengthy). Community will work constructively & in synergy given the favorable circumstances - like they are in Wikipedia. We can make circumstances favorable by verifying :

  1. How easy we make it for community to learn the system
    (unlike Language Workbenches)
  2. How easy we make it for community to agree on abstraction
    (contrast 'abstraction in plain english' approach of APF with namespace oriented abstraction followed by oomodels.org)
  3. How efficiently  our community management script (like MediaWiki for Wikipedia) helps users rate, lock, update existing submissions

Please suggest if the 5-6 documents uploaded on http://apfwork.org/ do justice to point #1, #2 above. #3 has been discussed too, but is pretty much in infancy.

Tushar,

 let me underline: I like your idea and you seem to have an awareness of the challenge you're up to ("...seeking contribution of Computer Science fanatics..."). Looking at your example, I could start argueing against it (even the simplest string replacement routine can become a hot topic, look at the unfamous history of regexp-syntax-flavours, even after decades the matter is not settled fully), but actually I don't want to take such a boring, reality afflicted position.

My major advice would be: consider and reconsider, over and over, thoroughly, the technical concept (architecture, usability etc.) of your repository as well as the marketing concept (community building). Try to learn from feedback as much as you can, and even more than that. And don't give up early, it is a challenge.

Thanks Andreas,

 

The discussion here in this thread has helped me realize 'community management' as an equally crucial component of the challenge (which otherwise I was leaving for last moment)

 

I am now gearing up for a prototype which facilitates community, and (hopefully) helps people agree on commonalities of repository. I had some ideas few years back of a system which helps people 'plan' & brainstorm things together - without stepping over each other's toes.

 

I am excited to share here the prototype of such solution which hopefully finishes in next few days/weeks.

 

Will ping back soon :)

RSS

Badge

Loading…

© 2019   Created by Mark Dalgarno.   Powered by

Badges  |  Report an Issue  |  Terms of Service