The Terms "Model-Based" and "Model-Driven" Considered Harmful

I truly believe that abstraction is at the core of computer science and software engineering. Abstraction is essential to be able to cope with complexity and allows us to build the tremendously complex systems we build today. Yet, it is hard to clearly define what abstraction is.  Florian Deissenboeck and I were attempting this almost 10 years ago in an article that was published in a workshop on the role of abstraction in software engineering. I think we found some interesting ways to think about it, but it was extremely hard to wrap your head around the concept as such.

Similarly, models, which I would define in our context as abstractions of systems that have a specific purpose, are essential for software engineering, maybe for engineering in general. Especially in software engineering, almost everything we deal with is a model. Requirements written down are a model of a part of a system to be. Even the short sentence in a typical user story is a model. The sketch with boxes and arrows on the whiteboard indicating an aspect of a software architecture is a model of the software. The Java source code is a model of the software abstracting e.g. the details of the execution on the machine. I could go on and on. Models are essential for software engineering.

So what does it mean to call something "model-based software engineering" (MBSE) or "model-driven software engineering" (MDSE)? I would argue without models there is no software engineering. Yet, there is a large research community working on MBSE/MDSE. To this day, I have not fully understood what that is, although I have been working on things that were called "model-based" myself.

My first research project in 2002 was a collaboration with BMW in which we tried model-based testing of their MOST network master. We used the approach my valued colleague Alex Pretschner, now professor at TU Munich, built in his PhD project. We invested a lot of time discussing with the BMW engineers to build a detailed model in AutoFOCUS. This model was then quite suitable to generate test cases to be run against the actual network master. Interestingly, we found that many of the defects we found were found during the modelling. My personal observation was that the model became so detailed, it was almost a reimplementation of the network master software. Was there really a conceptual difference between our "model" and the code?

Over the years, I have thought about this a lot. This post is my current status of what I think about  MBSE/MDSE. I do not want to convey that everyone working in that field is stupid and does non-sense research. Quite to the contrary, I do think there is a lot of interesting work going on. My hypothesis, however, is the following:

Using the terms "model-based" or "model-driven" in combination with software engineering or software engineering techniques obscures what these techniques are actually about and are actually capable of. Progress in software engineering is hindered by this division in "model-based" and "code-based".

I sincerely hope that this might start a discussion in our community. To substantiate why this hypothesis may be true, I collected the following five observations:

Nobody knows what MBSE/MDSE really is. There is a great confusion in research and especially practice what this MBSE or MDSE should be. For many, it is working with diagrams instead of textual code. For example, a UML sequence diagram would be a model-based technique but text in a programming language describing the sequence of messages might not. For others, it needs to be mathematically formal. For other still, it is the name for using Simulink to program. A recent study by Andreas Vogelsang et al. showed this perfectly.

Practitioners don't adopt MBSE/MDSE. This point is a bit hard to discuss given the first one. Oftentimes, if practitioners state that they do MBSE/MDSE, they apply a certain tool such as Simulink. Or they have some UML diagrams of the system lying around (usually out of date). In the same study of Vogelsang et al., they investigate drivers and barriers for the adoption of MBSE. They found that "Forces that prevent MBSE adoption mainly relate to immature tooling, uncertainty about the return-on-investment, and fears on migrating existing data and processes. On the other hand, MBSE adoption also has strong drivers and participants have high expectations mainly with respect to managing complexity, adhering to new regulations, and reducing costs." So practitioners think the tools are immature and the whole thing might have a negative ROI. But there's the hope that it might help to manage complexity and reduce costs. This does not seem like a methodology I would like to invest in.

MBSE/MDSE is Formal Methods 2.0. I don't think formalisation and formal analysis is useless. It has its benefit and surely a place in the development of software systems. Yet, it is not the holy grail it was often sold to be. If I play the devil's advocate, it feels like after formal methods failed to be broadly adopted in industry, they are now simply renamed as model-based methods. Yet, putting some nice diagrams on top of a formal analysis most of the times won't make it easier to understand. In contrast, I love the work by people like Daniel Ratiu or Markus Völter on integrating formal verification in DSLs or common programming languages (see e.g. here). Is this "model-based"? Does it really matter whether it is?

Models are positioned as replacing code. I hear that less nowadays, but it is still out there. The story line was that the success of software engineering has always been based on reaching higher levels of abstraction. We went from machine code to assembly to C and Java. Models are supposedly then the next step in which we can abstract away all these technical details and everything will be easier.  I don't believe that at all, unless for very specific instances of "model". Abstraction comes at a cost. What we abstract away can come back and haunt us. For example, although Java hides null pointers from us most of the time, sometimes we suddenly see a null pointer exception pop up. It breaks the abstraction and suddenly makes the underlying details visible. As we are not used to dealing with null pointers in Java, this might be even worse than dealing with them directly in the first place. Furthermore, there are many arguments that can be made that there is a huge tool chain supporting us dealing with source code that is not directly available for various kinds of models. Finally, I fail to see how it is easier to work with graphical diagrams than with plain text.

The research community is split (at least) in two. Again, I will play the devil's advocate and exaggerate to make the point: On the one side, we have the MBSE/MDSE people claiming that models are the future and everybody still working on code just hasn't realised that. On the other side, we have the code people (e.g. in the maintenance community) who only analyse source code and ignore what these MBSE/MDSE people are doing for the most part. In the end, I believe, that leads to duplications of efforts, misunderstandings in reviews of papers and project proposals as well as a very confusing picture for practitioners. I don't think we project a consistent vision of how software engineering should be done in the future.

In summary, I believe that the extensive use of the terms "model-based" and "model-driven" in the software engineering community has done more harm than good. When I read that a method is model-based, I don't know if that means it uses graphical diagrams, it just uses additional artefacts or it is based on formalisations. I would be happy if my little rant here serves as the starting point in our community to bring the two fractions together so that we can work jointly on the next generation of software engineering methods, techniques and tools. 

Is there a future for empirical software engineering? Claes Wohlin's keynote at #ESEIW2016

Claes Wohlin gave a great keynote last week at the International Symposium on Empirical Software Engineering in Ciudad Real, Spain. He emphasised that we need to get better at creating empirical results that are puzzle pieces that we can try to fit together. That doesn't mean that that there be no conflicting results but that those should be easily identifiable. We need to build more theories, add and understand context and document our results in a way that helps in meta-analysis and generalisations.

My sketch notes:

Sketchnotes for talk by @alexbergel on evaluating software visualisations @VIS_VISUS

Alexandre Bergel, professor at the University of Chile, gave a talk yesterday at the Visualisation Institute of the University of Stuttgart. He presented the application of two innovative visualisation techniques to software engineering data. He closed the talk with a demonstration of a visualisation tool they develop.

I believe there is still a lot to explore in how to visualise best the data we have for the tasks to be performed during a software project. Yet, I also think we still don't know enough about the information needs. Maybe Daniel Graziotin will be able to contribute to this (doi: 10.3897/rio.2.e8865).

Here are my sketch notes of the talk:

Agile Hardware – Sketchnotes from Nancy Schooenderwoert's workshop keynote #XP2016

At XP 2016, I also attended the Second International Workshop on Agile Development of Safety-critical Software. Also very interesting workshop with – from my point of view – very timely topics. Very much related to agile and safety is the agile development of hardware which I'm also interested in.

Nancy Schooenderwoert gave an interesting keynote on her experiences. The sketch notes are below:

There must be more than ICSE for academic software engineering research

The International Conference on Software Engineering (ICSE) is the best known conference for academic software engineering research. It gets more and more submissions and the PC chairs are trying different things to deal with it. For this year, Willem Visser and Laurie Williams tried a Program Board approach which worked reasonably well. But it does not fully solve the problem.

Recently, the PC chairs of 2017, Alessandro Orso and Martin Robillard announced the idea to limit the submission to three per author.

Read More

Sketchnotes for "Cloning Considered Harmful" Considered Harmful - A look back

Mike Godfrey gave a talk at SANER 2016 looking back at the 10 year old paper "Cloning Considered Harmful" Considered Harmful which received the Most Influential Paper award. It definitely has influenced the discussion on cloning emphasising that there are valid reasons to clone. The discussion overall is not done, however. We are still working to understand the factors influencing the impact of cloning.

Here are my sketch notes of his talk:

How are functionally similar code clones syntactically different? An empirical study and a benchmark

A blogged previously about this study when we made it available as a pre-print. Now I'm happy to report that it has passed peer reviews and is published in PeerJ Computer Science.

We used solutions to programming problems of Google Code Jam. The solutions to a programming problem have to be functionally similar. Therefore, we can consider them functionally similar clones  (i.e. the code fragments provide similar functionality but are not necessarily syntactically similar). Clones with similar syntax, usually the result of copy & paste, can be found reliably with contemporary clone detection tools. Functionally similar clones, however, are still a large challenge.

We used the clone detection functionality in ConQAT and DECKARD to investigate how much syntactic similarity is there in these functionally similar code clones. They probably have not been created by copy & paste. We found that the syntactic similarity is below 16 %; often it is much smaller. To better understand the remaining differences, we categorised them manually. We found differences in the algorithms, data structures, OO design, I/O and the usage of libraries. While this classification may seem trivial at first sight, it really does capture well what has to be addressed in detecting such clones.

We provide based on this categorisation a freely available benchmark representing these differences along with all the data used in the study. We hope this helps the clone detection research community to move the detection of functionally similar code clones forward.

We plan to use the benchmark ourselves now to work on a new detection approach for functionally similar code clones. If you would like to collaborate, please get in touch!

Also: If you are a practitioner and have seen functionally similar clones in your code, please get in touch as well. We are planning to conduct a study on the relevance of such clones in practice.

We are looking forward to your comments and feedback!

Source: https://peerj.com/articles/cs-49/?td=bl

Supporting requirements updates during software evolution – Talk by Martin Glinz at SE 2016

Martin Glinz from the University of Zürich has given an interesting talk on a paper they presented at the RE conference. If you work in a process in which you update "old" requirements, it is difficult to identify which ones. They tried to identify the affected requirements based on a code change.

I would suspect that this is not really relevant for agile development processes. But in other context, this could really safe time.

ink-image.png
ink-image_1.png


Continuous Software Engineering - Keynote von Willi Hasselbring auf der SE 2016

Die ersten Keynote auf der SE 2016 in Wien wurde von Willi Hasselbring (Universität Kiel) gehalten. Er hat dabei einen Überblick aus seiner Sicht zum Thema Continuous Software Engineering gegeben. Ich fand den Überblick interessant, aber sogar noch etwas zu eng. Agile Software-Entwicklung kam zum Beispiel nur am Rande vor. Unser Continuous Quality Control fand gar keine Erwähnung.

Hier sind meine Sketchnotes:


How are functionally similar code clones different?

A pre-print of a study on the differences between functionally similar code clones just became available at PeerJ. Please have a look. We are looking forward to comments, feedback and questions here or on the pre-print page.

We used solutions to programming problems of Google Code Jam. The solutions to a programming problem have to be functionally similar. Therefore, we can consider them functionally similar clones  (i.e. the code fragments provide similar functionality but are not necessarily syntactically similar). Clones with similar functionality, usually the result of copy & paste, can be found reliably with contemporary clone detection tools. Functionally similar clones, however, are still a large challenge.

We used the clone detection functionality in ConQAT to investigate how much syntactic similarity is there in these functionally similar code clones. They probably have not been created by copy & paste. We found that the syntactic similarity is below 16 %; often it is much smaller. To better understand the remaining differences, we categorised them manually. We found differences in the algorithms, data structures, OO design, I/O and the usage of libraries.

We provide based on this categorisation a freely available benchmark representing these differences along with all the data used in the study. We hope this helps the clone detection research community to move the detection of functionally similar code clones forward.

We are looking forward to your comments and feedback!

Source: https://peerj.com/preprints/1516/?td=bl

Lightning port doesn't keep Lightning cable in place?

Apparently, this is a problem many have already encountered and solved. I have not stumbled across it so far. Here is what my problem was:

At some point in time of using my iPhone 6 (a bit more than half a year old), didn't charge correctly anymore. It happened that in the morning, after supposedly charging it the whole night, the iPhone was almost empty. I then realised that the plug did not go fully in the port. Moreover, it only charged when the plug was slightly skewed in the port.

My initial thought was that this must be a defect with the Lightning port in my iPhone. Therefore, I went to my local GRAVIS store in Stuttgart (which is in walking distance from my home). And the employee there said that he could clean it for 10€. I was a bit surprised that he fully ruled out a port defect. He answered that in 95 % of times, this is from dirt in the port. 

I tried his hint to use a toothpick to clean the port. I found crazy amounts of dust in the port. And now it is fully functional again. Sometimes, problems are much easier to solve than expected.

A quick Web search showed that I was definitely not the first with this problem. For example, Dan Graziano on CNET also describes the toothpick hint:

Lint and other debris from our pockets or purses gets stuck in this port and can build up over time. To be safe, back up your iPhone or iPad to your computer or the cloud. Power it off, and with a normal toothpick, gently remove the lint. You will be amazed how much can get stuck in the port.

As I hadn't seen this solution before, maybe this article is helpful to somebody.