Editor’s Note: Excerpted from their book Domain Specific Modeling: Enabling Full Code Generation , the authors use a fictitious digital watch company to demonstrate the advantages of domain specific modeling and describe how to build a DSM model generator to create digital watch applications. Part 1 focuses on the initial steps in creating an application specific watch modeling language.
In an earlier article , we used a fictitious manufacturer of digital wristwatches called ‘Secio’ to demonstrate the principles of Domain Specific Modeling.
The actual process of creating the modeling language has been made as natural as possible, following the practices that have become established in real projects. The artificial background of the example is perhaps most clearly seen in its limited scale. Conversely, this small size is its strength as a pedagogical tool: small enough to be understood in a relatively short time but large enough to provide realistic insights into every aspect of DSM.
Secio has noticed that producing the software for each watch by hand is becoming a significant bottleneck, as consumers demand functionality beyond simple setting and display of the time. It has also realized that different consumers want different functionality and have different requirements for ease of use versus extensive functionality, physical compactness versus amount of information displayed, and so on.
Therefore, Secio has decided to build its range of watches for next year as a product family. There will be different watches for different consumer types and price points, but the watches will be able to share common parts of the software. Such common parts will include basic framework such as the ability to show numbers on an LCD display, as well as individual applications such as a stopwatch or a countdown timer. The basic framework components will be present in all watches, and either already exist or will be coded by hand. All the individual applications will not be present in all watches, yet it is hoped that later addition of an existing application to an existing watch could be a simple operation.
For a variety of reasons—some technical, some political—Secio has decided to try to create a DSM language for building the watch applications. The main objectives are to reduce the development time and complexity of building watch applications. In particular, the current watch software is one large piece of code with little separation of concerns.
Being able to separate out different parts of the code would improve reuse possibilities, and also allow the developer to concentrate on one area at a time, thereby reducing errors. The separation of behavior into Model (operations on time values), View (display of the time and icons), and Controller (user input on the watch buttons) will form one weapon to divide and conquer the mass of code.
The main weapons though will be the higher level of abstraction and the close fit of the modeling language with the problem domain. Developers will be able to concentrate on the application behavior as perceived by the end user, rather than on the lower-level details of how to implement that behavior.
Behind the stage dressing of Secio’s objectives, the main objective was to provide a fully worked example of DSM. An important component in convincing people that DSM works is making the code generated from example models actually run. This presents several challenges not normally found in a real-world case of DSM. First, the most significant challenge is that the users must have a compiler installed for the language of the generated code.
These days, compilers tend to come as part of a full IDE, requiring a large investment of time, disk space, and possibly money to set up. Second, the user must have a runtime environment compatible with the format produced by the compiler. For both compiler and environment, there may also be various settings such as paths that are specific to each PC, and these settings will normally need to be synchronized with the generated code. Finally, the users will have a variety of different backgrounds, and varying experience with different languages, IDEs, and programming styles.
The need to make an application that would compile and run on many different platforms pointed away from C, the natural language choice for an embedded system. While core watch behavior in C would have been platform independent, the GUI widgets and graphics library in C tend to be platform specific. Also, the majority of commonly used C compilers and IDEs that users were likely to have are commercial products. The installation and use of the freely available compilers are generally sufficiently difficult to deter someone whose only motive would be to see an example application.
While other languages with freely available compilers and runtime environments existed, the mainstream choice—and hence most likely to already exist on users’ computers—was Java. The resulting Java applications would also be small enough to be easily passed to other users, and availability of at least a Java runtime environment on a PC is virtually guaranteed. While Java was not the current language of choice for embedded development, it was originally developed for such devices and would be familiar to a large number of users.
Creating a watch modeling language
The development of the watch modeling language was carried out by Steven Kelly and Risto Pohjonen of MetaCase over a couple of weeks. As this was an example rather than a real-world case, there was no outside customer: as both developers had owned digital watches in the 1980s, they felt qualified to play the role of domain expert.
This distinguishes this from the other example cases, where the authors did not have sufficient domain knowledge at the start of the project to create the language on their own. This case thus brings the authors’ positions closer to those of readers thinking about their own application domains, and gives us a good opportunity to examine the thought processes of a domain expert. As will be seen, creating a DSM language is largely a question of determining what facts need to be recorded about each application in that domain, and where in the modeling language to store these facts.
The total time spent was approximately 10 man-days, including the modeling language, a full set of models, the generator, and the domain framework. Since this was the first Java project for either developer, the time also included learning the language, its libraries, and development environment. Over the subsequent years, a few more days have been spent on upgrading the framework and generator to work on other desktop platforms, cope with later Java versions, produce watches that run on mobile phones, and add support for model-level tracing of a running watch application.
None of these changes has required changing the models, and the result has always been fully running applications, identical in behavior but with environment-related differences in appearance and sometimes in code. Following is the analysis of the domain and the development of the language, in chronological order.
Reusing watch applications
It was soon evident that it would be good to break a watch down into its component applications: time display, a stopwatch, an alarm clock, and so on. Beyond providing a sensible modularization of the whole watch, this would also allow a watch application to be reused in different watches. Since these watches would have different physical designs—displays, buttons, and so on—there was a need for some decoupling of references to these physical elements in the models.
For instance, if a model of a watch application wanted to specify that a certain operation was caused by a certain button, we had to answer questions about whether that button was available in the given physical watch, whether it would be named in the same way or have the same semantics, and what to do if no such button existed.
Thinking about this issue prompted the idea of explicitly modeling a whole group of watches as a family. This would be a separate level of modeling, probably with its own modeling language. Often in DSM such a level exists, but it is not always explicitly modeled: it is enough to simply have several products—watches in this case—each built from its own set of models.
However, making reuse of models explicit generally makes it easier to maintain them and concretely shows the dependencies of the various components. For instance, a change in the Alarm application to require an extra button would affect which physical designs of watch the Alarm could be used in. If Alarm had simply been reused, this effect might not have been obvious. If, however, there were a top-level model showing each member of the watch product family, which physical watch body it used, and which watch applications it contained, the effects of that change would be easier to see.
If there was a need for a mapping between the buttons mentioned in a watch application model and those present in a physical watch, there was also the question of how to model this mapping. Would the mapping be included as part of the top-level family model, or would it be a separate kind of model between the top-level family model and the actual watch applications?
Further, who would be responsible for building these mappings: the watch application designer, the family designer, the designer of a particular watch model, or somebody else? Similarly, would a separate mapping be required for each pair of a watch application model, for example a stopwatch, with a physical watch body, or could one mapping be reused over many watch applications or physical watch bodies?
The question of the mapping was thus difficult in both technical and organizational terms, and also hard to decide at this early stage. While we did not even have modeling languages for watch applications and watch families, it seemed unrealistic to expect to pick a good solution to a problem that would probably only become apparent once several families had been built.
We thus decided to go with the simplest thing that could possibly work: there would be a limited number of named buttons, initially just Up, Down, Mode, and Set. Each physical watch body could contain any combination of these buttons, and similarly each watch application could refer to them directly.
While less flexible than a different mapping for each watch, this had a good chance of working well for both watch modelers and users. Both groups would prefer a consistent semantics for the buttons: if in one watch application Up was used to increase a time value, and in another the same function was achieved using Set, learning to use the watch would be rather difficult!
Now we had a fair idea of the contents and division of labor of the two modeling languages. The family model would contain a number of watches, and each watch would be composed there from a number of watch applications and a physical watch body. A physical watch body would specify a number of buttons. These buttons would also play a key role in the definition of the watch applications: different buttons would cause different actions in different applications.Watch application behavior
While the family model would havebeen easier to work on, we decided to tackle the watch applicationsnext. If DSM were to mean anything, it would have to be able to specifythe differing behaviors of the various applications sufficiently exactlythat we could generate full code directly from the models. One possibletactic at this point would have been to hand code a couple of watchapplications.
This would have given us an idea of what kind ofbehavior they would actually have on a low level, as well as insightsinto what elements of that behavior might repeat over several differentwatch applications. However, as neither developer had programmed in Javaat this point, it was thought that trying to go this way would be a badidea. DSM is generally about abstracting from the best practice, andclearly there was no way our first attempts would fit into thatcategory.
Instead, we decided to concentrate on the actualuser-visible behavior of the watch applications, and trust that ourgeneral development experience and instincts would tell us when we had asufficiently exact specification to enable full code generation.
Thefirst question was thus about the buttons, the only way the end usercan interact with the watch: When a user presses a button in a givenapplication, does that button always have the same effect? In otherwords, could we program a watch as a composition of applications and anapplication as a composition of button behaviors? In some simple cases,this appeared to be true. In more complicated cases, for example settingthe time, it might be true if understood sufficiently broadly.
Forinstance, pressing Set in the Alarm application would start setting thealarm, and pressing Set again would stop setting the alarm. Whilestarting and stopping the set process were two different things, theycould perhaps be understood as a toggling of the set process. Goingfurther, though, within the set process it would be normal to press theMode button to toggle between setting the hours and setting the minutes.However, outside of the set process the Mode button would be expectedto exit the Alarm application and move us to the next application.
Althoughfurther stretching of semantics might have made it possible to modelapplications as simply one behavior per button, that behavior would havebeen more and more conditional on what had occurred before. It thusseemed best to admit that the action taken when a button was presseddepended on what state the application was in. A watch application couldthus be modeled as some kind of state machine. Pressing buttons wouldcause transitions between states, and possibly also other actions.
Atthis point, it would normally have been a good idea to move from modelsthat were either imagined or drawn simply on paper, to actuallybuilding a prototype modeling language. This would generally give abetter basis for further decisions, while also making more concretethose things that seemed already decided—and perhaps revealing problemsin them. Perhaps because both developers had a deep knowledge of DSM,MetaEdit+, and state machines, we actually continued on paper for alittle while longer, looking at the possible kinds of actions.
Theclearest action with a user-visible result was the control of thelittle LCD icons for applications: a stopwatch icon for when thestopwatch was running, an Alarm icon for when the alarm was set to ring,and so on. We decided to make turning the icons on or off an explicitaction, taken when following a transition between two states.
Anotherpossible approach would have been to make each state contain theinformation about the status of each icon, but we decided against thisapproach for two reasons. First, the on/off state of the icons waspersistent when their application was exited and the next applicationstarted, and it was then unclear whether the original application wasstill in some sense live and in a particular state. Second, and moreimportantly, storing the icon status in each state would have meanteither each state having to store the status of each of the icons,leading to maintainability problems if the number of icons increased, oreach application only being able to control one icon, which whileinitially appearing to be true, might not necessarily be so in morecomplicated cases.
It was also thought that the number ofactions needed to make an icon behave as desired would probably be lessthan the total number of states in that application, making modelingeasier if actions were used. We thus decided to represent the icons asobjects in the modeling language, and to allow modelers to change theirstate by drawing a relationship to the icon to turn it on or off. Theresult so far, after 40 minutes of metamodeling, is seen in Figure 1 .
Watchapps and manipulating time. After the work so far, we could model astopwatch that would know which state it was in (say, Stopped orRunning), move between those states when the user pressed a button (say,Up), and that would also turn on or off the stopwatch icon when movingbetween the states. All very well, but it might be useful if thestopwatch actually recorded something about the elapsed time, too!
IfButtons represented a Controller in the MVC triad that Secio envisaged,and Icons represented a View, then being able to record elapsed timewas clearly part of the Model. Storing, recalling, and performing basicarithmetic on time variables were therefore important parts of watchapplication behavior that we needed to represent in some way in themodeling language.
We thought it was fair to assume that anydigital watch company worth its salt would already have existingfunctions to return the current time, along with functions to store andrecall and perform basic arithmetic on time variables. This meant wecould consider the time operations on a high level, rather than theimplementation details of the bits and bytes of data structures to storetime.
The first question to be answered was one about theboundaries of the domain: did we need to handle dates as well as time?Thinking through the various possible watch applications, it seemed thatmost had no connection to dates: an alarm could not be set for a givendate, for instance. In fact, only the basic time display would showdates.
The handling of dates would most likely be identical fromwatch to watch: a source of commonality, not of variability. Inaddition, the behavior of dates is complex, with weekdays, day numbers,months with different numbers of days, leap years, different orderingsof date elements for different countries, and so on. All in all, itlooked like building a modeling language for handling dates would notresult in any productivity improvements for Secio. Rather than introducecomplexity for no gain, we decided to leave dates out at this point andconcentrate on time.
Thinking about the stopwatch application,it seemed there could be two ways to think about time variables and theunderlying platform. In the first way, the platform would just respondwith the current time when asked, and we could store that and then laterask again for the new time and calculate the difference. In the secondway, the platform could offer a service to create a new timer, which itwould then continuously update.
While the second way would makemodeling the stopwatch easier, it seemed unlikely in the resourceconstrained embedded environment of a digital wristwatch: updatingmultiple timers simultaneously would place unnecessary demands onprocessing power and battery life. We would, therefore, need a way ofrepresenting subtraction of time variables, and most likely additiontoo.
Thinking about other watch applications such as the alarmclock, it appeared there might be a need for another time operation.When editing the alarm time, pressing Up would increment (say) theminutes, which could be represented as adding one minute. However, whenthe minutes reached 59, the next press of Up would take them to 00,without incrementing the hours.
In other words, this was not atrue increment of adding one minute, but rather an increment to just oneunit of the whole time, which would roll around at either end of theunit’s range. This could have been represented in the models with acomparison operator and a subtraction, for example: “if minutes equals60 then subtract 60 from minutes.” However, this would be needed forevery edit operation, for every unit of the time, and once for each endof the range. It would also require a modeling concept for comparisons,which we did not seem to need otherwise. We therefore decided to make a“Roll” operation a first-class concept in the modeling language,incrementing or decrementing a particular unit in the whole timevariable.
Now we knew the concepts we wanted for time variablesand operations, how best to represent them in the modeling language?Programmers would be used to a textual representation, and we couldindeed have had a textual DSL providing a natural syntax, restricted tojust these operations. However, because there were so few operationsneeded, and each calculation would need only one or two operations, wedecided to represent time variables as actual objects in the modelinglanguage, and operations as relationships connecting to them. This wouldfit well with the earlier decision to have icons as objects, andactions on them represented as relationships drawn to them from statetransitions. The result so far, after 70 minutes of metamodeling, isseen in Figure 2 .
Atthis point, we had the majority of the concepts of the modelinglanguage and had covered the model and controller parts of the MVC triadfairly thoroughly. We also had addressed some of the View part in theform of the Icons, but the most important view element, the display oftime, was still largely untouched.
The display code of areal-time application is notorious for being difficult to get right. HadSecio been a real organization, its experienced developers would havebeen throwing up their hands at this point and saying “All you’ve doneso far are the easy bits, and we never had problems with those anyway.Getting the display to update smoothly in real time is hard work, andyou’ve not even touched on it. Besides, that’s the area where ourdevelopers make most of their mistakes: they just can’t seem to get thethread synchronization and semaphores right, no matter how many times weexplain it.”
What then can we do for Secio? First, we can take along step away from the implementation details toward the problemdomain and see what actually needs to happen on the watch display fromthe end user’s point of view. Let’s take the most used application,Time. What is it displaying? The current time, of course! Now, the valueof the current time is changing all the time, so are the updates forthat something we need our models to specify?
In a sense, maybenot: what is being displayed, on a high level of abstraction, is simplythe current time. What then about the stopwatch, what is it displaying?That seems to vary a little: initially, it just displays all zeroes;when it is running, it is displaying the elapsed time, which is updatingconstantly but always equal to the current time minus the originalsaved starting time. If we look at World Time, that displays the currentsystem time adjusted by some fixed offset. A countdown timer displaysthe time the alarm will sound minus the current system time.
Thisis interesting: we seem to be able to express quite simply the basicidea of what each application should be displaying. This is actually notall that surprising: if the time value displayed required someimmensely complicated algorithm to calculate, few people would be ableto interpret its values. Of course, actually making the value displayand update smoothly will be tricky—real-time software always is—butperhaps the models themselves can remain quite simple. Could we abstractout the complicated parts of real-time display into the domainframework? This would allow the model to simply specify what to display,while an expert developer’s handwritten code, written once but reusedfor all displays in all watch applications, would handle how to displayit.
In our blessedly deep ignorance of the intricacies (andbugs!) of Java’s thread handling and synchronization, we decided to takethis path. A watch application would specify a calculation to obtainthe time to display, either just one such display function for the wholeapplication (e.g., Time, Alarm) or perhaps different display functionsfor different states (e.g., stopwatch states for being stopped andrunning, or countdown timer states for setting the timer and countingdown).
The modeler’s burden would end there, and not a threadsynchronization or semaphore in sight. The expert developer would write adisplay update function that would run in a separate thread once everyfew milliseconds, ask the application to perform its display functioncalculation, and update the display with the result. Our example modelat this stage, after 105 minutes of metamodeling, is shown in Figure 3.
Odds and ends
Whenbuilding a modeling language, there are always things that get missedon the first pass. Thinking through a concrete example application andits model helps keep things on track, but any given example will rarelycontain an instance of everything needed in the modeling language. Sotoo in this case: stopwatch had been a good example for many areas, butit was missing features that were used in several other watchapplications. We also missed one important part of display functions,which only became apparent in the stopwatch when it was used in severaldifferent watch models.
The most obvious omission from themodeling language was the complete lack of the concept of an Alarm. Ithad been mentioned early on but never focused on for long enough toactually figure out how it should be modeled. Setting the alarm seemedeasy enough: we could edit some time value for it, either the time atwhich the alarm would ring, or the amount of time until it would ring.An alarm ringing would be similar to a button press in that it couldcause a change of state.
More interestingly, this change ofstate could happen at any time, even if the watch was running adifferent application at that point. Clearly, we could not drawtransitions from every state in the entire model in case the alarm rangthere. Instead, we settled on having an alarm symbol, and a singlespecial “alarm” transition from that to a state. Whenever the alarmrang, its application would take control and jump to that state. Onexiting the alarm application, the watch would return to the applicationthat was current when the alarm rang.
In deciding whether to setalarms at a time of day or as an amount of time until the alarm rang,we hit a problem. What would happen if the user changed the watch timeafter setting the alarm? If we stored alarms as a time of day, astandard Alarm Clock application would be fine, but a countdown timerwould suddenly be wrong.
For example, if a countdown timer wasset at 5:00 p.m. for 10 minutes, it would have stored its alarm as 5:10p.m.; if the user moved the clock back an hour at 5:05 p.m. to 4:05 p.m.the countdown alarm would not ring for another 65 minutes. Conversely,if we stored alarms as an amount of time until ringing, the countdowntimer would work fine, but a standard Alarm Clock would ring at thewrong time. We decided to solve this by allowing each Alarm object tospecify whether it responded to changes in local time or not.
Thinkingabout setting alarms also reminded us that we needed some way to make apair of digits on the time display blink, so the watch user would knowwhich time unit he was editing. While we could have avoided adding aconcept by saying that a time unit blinking was just a special case ofan Icon, this would have meant that going from an “edit hours” state toan “edit minutes” state would have had to turn off “blink hours” andturn on “blink minutes.” We would also have to make sure that every pathto “edit hours” would turn on the “blink hours” pseudo-Icon.
Abetter approach seemed to be to recognize from the names of the statesthat blinking of a time unit was actually a feature of the state itself.We thus added a property “blinking” to state, making it a list wherethe user could choose from hours, minutes, or seconds. This also fittedwell with choosing a display function in each state: the actual displaythread would need to know both the function and any blinking time unitto be able to keep the display updated. Having the blinking property inthe display function itself would not have helped, as it would meanhaving to create duplicate display functions differing only in whichtime unit was blinking.
Display functions were also the source ofthe third addition to the modeling language. We realized that themapping of the different time units of a time value to actual digitpairs on the watch display could be different in different physicalwatches, and also different between two watch applications in the samephysical watch. For instance, in a lady’s watch there are often only twodigit pairs.
In the normal time display these are used forhours and minutes, but in a stopwatch and possibly in a countdown timer,they should show minutes and seconds. However, editing a countdowntimer generally only allows choosing hour and minute values, so whileediting those should clearly be shown. Specifying a hard mapping in awatch application from time units to digit pairs would not work though,since it would mean the application would not work so well in adifferent watch model with a different number of digit pairs.
Wetried several different schemes for specifying mappings, includingspecifying which time unit would be shown in the leftmost or rightmostdigit pair. Neither of these gave satisfactory results when we wrote outon paper what would be displayed in various states, applications, andphysical watches.
In the end, we hit on the idea of a displayfunction specifying a “central time unit,” where “central” was a mixtureof “most important” and “should be displayed in the center digit pair.”A little heuristic—for what counts as “center” when there are an evennumber of digit pairs— rounded off the scheme, and we added anappropriate property to display function. Now we had a language thatcould specify all we could think of about watch applications.
Juha-PekkaTolvanen () has been involved in domain-specificlanguages, code generators and related tools since 1991. He works forMetaCase and has acted as a consultant world-wide for modeling languageand code generator development. Juha-Pekka holds a Ph.D. in computerscience from the University of Jyväskylä, Finland.
Steven Kelly is chief technical officer of Metacase and cofounderof the DSM Forum. He is architect and lead developer of MetaEdit+,Metacase’s DSM tool.
Used with permission from Wiley-IEEEComputer Society Press, Copyright 2014, this article was excerpted from Domain-Specific Modeling: Enabling Full Code Generation , by StevenKelly and Juha-Pekka Tolvanen.