ReqIF’s XML overhead

XML formats can have a tendency to be a little bit verbose (the AUTOSAR exchange format can be an example for this). To get an impression on the overhead of the ReqIF standard, I fed a document from Project Gutenberg to our Requirements Modeling Framework (see Eclipse Proposal). Breaking down “Antarctic Penguins A Study of Their Social Habits” by George Murray Levick into one requirement per paragraph (what text would be better suited for an open source project than a text about penguins), results in a document with 636 requirements.

The original file size is 198936 bytes, the .reqif is 560221 bytes. That is an overhead of (560221-198936)/636 = 568 bytes per requirement.

The paragraph

When seen for the first time, the Adélie penguin gives you the
impression of a very smart little man in an evening dress suit, so
absolutely immaculate is he, with his shimmering white front and black
back and shoulders. He stands about two feet five inches in height,
walking very upright on his little legs.

is in XML:

 

<SPEC-OBJECT IDENTIFIER="_fHa6SLz1EeCXTZW4PC1qig">
          <VALUES>
            <ATTRIBUTE-VALUE-STRING THE-VALUE=" When seen for the first time, the Adélie penguin gives you the impression of a very smart little man in an evening dress suit, so absolutely immaculate is he, with his shimmering white front and black back and shoulders. He stands about two feet five inches in height, walking very upright on his little legs.">
              <DEFINITION>
                <ATTRIBUTE-DEFINITION-STRING-REF>Description</ATTRIBUTE-DEFINITION-STRING-REF>
              </DEFINITION>
            </ATTRIBUTE-VALUE-STRING>
          </VALUES>
        </SPEC-OBJECT>

In addition, to build an hierarchical structure, each requirement/paragaph needs to have a spec hierarchy.

<SPEC-HIERARCHY>
                  <OBJECT>
                    <SPEC-OBJECT-REF>_fHa6SLz1EeCXTZW4PC1qig</SPEC-OBJECT-REF>
                  </OBJECT>
                </SPEC-HIERARCHY>

Bildquellenangabe: tokamuwi  / pixelio.de

Bubble Charts in Excel

Bubble Charts are a nice way to visualize data with three dimensions, however they are not too easy to create in Excel. Assume the following scenario: We want to represent our customers, their distance from our office, the current turnover at the customer and the turnover potential that we see. We want to create a bubble chart like this:

Bubble Chart

Bubble Chart

The X-Axis shows the distance from the customer to the office, the Y-axis the current turnover and the size of the bubbles indicates the potential turnover that we project. You could then decide that the you’d like to focus your marketing activities on customers that have big bubbles on the lower left: They have a big potential for growth and are close to your office. The data set in Excel could look like this:

However, if you try the “insert bubble chart” menu entry of Excel, you will have no way of creating a chart like the above. The trick is to use a VBA script that you find on Tom Hollander’s blog.

This allows you to select the data and then call the macro to create the charts. In my case, I also want to show the names of the customers near the bubbles. This is not in Tom’s macro. To do this, you would have to find the line with “.BubbleSizes” and add to more lines:

          .BubbleSizes = Selection.Cells(r, 4).Address(External:=True)
     .ApplyDataLabels Type:=xlDataLabelsShowLabel
        .DataLabels.ShowSeriesName = True

After that, the bubble chart looks like this:

Bubble Chart with labels

Bubble Chart with labels

You can still tweak it manually or by script.

 

 Header Picture: sabine koriath  / pixelio.de

 

 

ReqIF / RIF and Excel Import and Export

About a week ago we’ve started the Eclipse Project Proposal for a Requirements Modeling Framework. We are having promising results with the data sets that we get from our partners. In addition, we’ve starting implementing some very useful tooling.

As Spreadsheets (i.e. Excel) are very common as a means of data exchange, it would be very useful to have an Excel to ReqIF conversion utility. These would obviously reduce the need for the use of application specific conversion tools, since the data could then be transferred to any tool that supports ReqIF.

In the current import, each line of the Excel table is considered to be a requirement, and the requirement definition for ReqIF is taken from the first line. A small table like

will result in the following parts of the ReqIF-exchange file:

<?xml version="1.0" encoding="UTF-8"?>
<REQ-IF xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.omg.org/spec/ReqIF/20101201/reqif.xsd" xsi:schemaLocation="http://www.omg.org/spec/ReqIF/20101201/reqif.xsd reqif.xsd">
 <CORE-CONTENT>
 <REQ-IF-CONTENT>
 <DATATYPES>
 <DATATYPE-DEFINITION-STRING IDENTIFIER="NormalString"/>
 </DATATYPES>
 <SPEC-TYPES>
 <SPECIFICATION-TYPE IDENTIFIER="StructureType"/>
 <SPEC-OBJECT-TYPE IDENTIFIER="Requirement">
 <SPEC-ATTRIBUTES>
 <ATTRIBUTE-DEFINITION-STRING IDENTIFIER="ID">
 <TYPE>
 <DATATYPE-DEFINITION-STRING-REF>NormalString</DATATYPE-DEFINITION-STRING-REF>
 </TYPE>
 </ATTRIBUTE-DEFINITION-STRING>
 <ATTRIBUTE-DEFINITION-STRING IDENTIFIER="Short">
 <TYPE>
 <DATATYPE-DEFINITION-STRING-REF>NormalString</DATATYPE-DEFINITION-STRING-REF>
 </TYPE>
 </ATTRIBUTE-DEFINITION-STRING>
 <ATTRIBUTE-DEFINITION-STRING IDENTIFIER="Description">
 <TYPE>
 <DATATYPE-DEFINITION-STRING-REF>NormalString</DATATYPE-DEFINITION-STRING-REF>
 </TYPE>
 </ATTRIBUTE-DEFINITION-STRING>
 <ATTRIBUTE-DEFINITION-STRING IDENTIFIER="Author">
 <TYPE>
 <DATATYPE-DEFINITION-STRING-REF>NormalString</DATATYPE-DEFINITION-STRING-REF>
 </TYPE>
 </ATTRIBUTE-DEFINITION-STRING>
 </SPEC-ATTRIBUTES>
 </SPEC-OBJECT-TYPE>
 </SPEC-TYPES>

 

 

The fragment above is the type definition section of ReqIF. After that, every line of the spread sheet is considered to be a requirement (SpecObject in ReqIF-lingo):

<SPEC-OBJECTS>
 <SPEC-OBJECT IDENTIFIER="_anz2kbONEeCkMvpBkQxRFw">
 <VALUES>
 <ATTRIBUTE-VALUE-STRING THE-VALUE="1">
 <DEFINITION>
 <ATTRIBUTE-DEFINITION-STRING-REF>ID</ATTRIBUTE-DEFINITION-STRING-REF>
 </DEFINITION>
 </ATTRIBUTE-VALUE-STRING>
 <ATTRIBUTE-VALUE-STRING THE-VALUE="X">
 <DEFINITION>
 <ATTRIBUTE-DEFINITION-STRING-REF>Short</ATTRIBUTE-DEFINITION-STRING-REF>
 </DEFINITION>
 </ATTRIBUTE-VALUE-STRING>
 <ATTRIBUTE-VALUE-STRING THE-VALUE="Descr">
 <DEFINITION>
 <ATTRIBUTE-DEFINITION-STRING-REF>Description</ATTRIBUTE-DEFINITION-STRING-REF>
 </DEFINITION>
 </ATTRIBUTE-VALUE-STRING>
 <ATTRIBUTE-VALUE-STRING THE-VALUE="graf">
 <DEFINITION>
 <ATTRIBUTE-DEFINITION-STRING-REF>Author</ATTRIBUTE-DEFINITION-STRING-REF>
 </DEFINITION>
 </ATTRIBUTE-VALUE-STRING>
 </VALUES>
 </SPEC-OBJECT>
 <SPEC-OBJECT IDENTIFIER="_an1rwLONEeCkMvpBkQxRFw">
 <VALUES>
 <ATTRIBUTE-VALUE-STRING THE-VALUE="1.1">
 <DEFINITION>
 <ATTRIBUTE-DEFINITION-STRING-REF>ID</ATTRIBUTE-DEFINITION-STRING-REF>
 </DEFINITION>
 </ATTRIBUTE-VALUE-STRING>
 <ATTRIBUTE-VALUE-STRING THE-VALUE="Y">
 <DEFINITION>
 <ATTRIBUTE-DEFINITION-STRING-REF>Short</ATTRIBUTE-DEFINITION-STRING-REF>
 </DEFINITION>
 </ATTRIBUTE-VALUE-STRING>
 <ATTRIBUTE-VALUE-STRING THE-VALUE="C">
 <DEFINITION>
 <ATTRIBUTE-DEFINITION-STRING-REF>Description</ATTRIBUTE-DEFINITION-STRING-REF>
 </DEFINITION>
 </ATTRIBUTE-VALUE-STRING>
 <ATTRIBUTE-VALUE-STRING THE-VALUE="graf">
 <DEFINITION>
 <ATTRIBUTE-DEFINITION-STRING-REF>Author</ATTRIBUTE-DEFINITION-STRING-REF>
 </DEFINITION>
 </ATTRIBUTE-VALUE-STRING>
 </VALUES>
 </SPEC-OBJECT>
 </SPEC-OBJECTS>

 

The hierarchical structure is not part of the objects themselves in ReqIF, but an extra section. In this example, the hierarchy is derived from the numbering of the ID field, but we can use any algorithm (right now it is easily adjustable based on regular expressions). This results in the hierarchy structure.

<SPECIFICATIONS>
 <SPECIFICATION IDENTIFIER="Structure">
 <CHILDREN>
 <SPEC-HIERARCHY DESC="1">
 <CHILDREN>
 <SPEC-HIERARCHY DESC="1.1">
 <OBJECT>
 <SPEC-OBJECT-REF>_an1rwLONEeCkMvpBkQxRFw</SPEC-OBJECT-REF>
 </OBJECT>
 </SPEC-HIERARCHY>
 </CHILDREN>
 <OBJECT>
 <SPEC-OBJECT-REF>_anz2kbONEeCkMvpBkQxRFw</SPEC-OBJECT-REF>
 </OBJECT>
 </SPEC-HIERARCHY>
 </CHILDREN>
 </SPECIFICATION>
 </SPECIFICATIONS>
 </REQ-IF-CONTENT>
 </CORE-CONTENT>
</REQ-IF>

It turns out that our framework is a powerful tool to provide tools and functionality around ReqIF.

 Icon from: <A href=”http://www.vistaico.com”>VistaICO.com</A>

Links for “Next steps towards an integrated systems engineering platform”

If you attended my talk, here are the links of the topics adressed in the talk:

Eclipse: www.eclipse.org

Requirements / ReqIF: Contact Me directly

SysML/UML: http://www.eclipse.org/modeling/mdt/papyrus/

AUTOSAR: http://www.autosar.org/

Artop: http://www.artop.org/

Topcased: http://www.topcased.org/

OSEE: http://www.eclipse.org/osee/

Eclipse Automotive Industry Working Group: http://wiki.eclipse.org/Auto_IWG

OPEES: http://www.opees.org/

 

http://www.google.com/url?sa=t&source=web&cd=7&ved=0CDoQFjAG&url=http%3A%2F%2Fwww.stups.uni-duesseldorf.de%2Fpublications%2Fmbees2011.pdf&rct=j&q=reqif%20eclipse&ei=zgjSTdylCYz_-gaercnVCg&usg=AFQjCNFDv1PR48nzqceYQHvNT2NplqqMQg&cad=rja

Career as an engineer

At this year’s embedded world 2011 trade fair and conference I was invited to speak at a panel discussion titled “Do companies roll out a red carpet for engineers?”. The panel discussion was part of the embedded world “student day”, where the fair’s hosts actually organize buses from Germany’s universities to bring students to the trade fair. A large part of the students are engineers in electronics, but most of what has been said applies to graduates in computer science (focus on embedded) in Germany as well.

The main message from all participants (consulting companies, “body-leaser” and industry associations) is: Career opportunities are very good for young engineers right now in Germany, provided that graduates show mobility. Engineering jobs (and that IMHO applies to computer science / IT in embedded as well) are not evenly distributed in Germany and you might have to relocate, e.g. to Stuttgart or Munich (not half bad, I think). There is less job vacancies in eastern Germany with the exception of a few technology clusters.

Salaries for graduates were quoted at around 43KEUR , with variations depending on the area. If you find statistics on salaries for Germany, a good rule of thumb to find the local delta was mentioned: Look up the local rent index compared to the German average. The delta in percent is also an indication of the salary delta in percent (but that rule has its limits as well).

During the discussion the question was raised, if companies were willing to compromise, given that good qualifications seem harder to find. Most of the 5 participants seemed to agree that they were willing to compromise on technical qualifications, but not on soft skills. Those participants said that they would be willing to spend some time training technical stuff, but that they thought communication skills were very important for the recruiting. The participant from the labour leasing company seemed to have different point of view. One of the participants mentioned that small and medium companies are definitely worth applying at, since your application is more likely to be read by a human person and not filtered out by some algorithm matching your grades.

Regarding the itemis 4+1 work model (work 80%, learn 20%) I was asked if companies where willing to drastically increase salaries to find better qualifications. But as we all know, the motivational power of salary is limited. Major factors are self-realization, peer recognition and other such factors. Trying to create such a work environment (which is sometimes hard, if customers or customers’ procurement don’t see the added value easily) is an important factor. With that model, if you are so inclined, you can actually “build your personal brand” by blogging, tweeting and giving talks (talk to our automotive customers to learn how difficult it can be to get permission to talk at a conference, no chance about tweeting and blogging).

I am astonished, given the so-called crisis of 2009, how could perspectives for engineers seem right now.

Augmenting UML with textual DSLs

Funded by the BMBF

For the IMES research project we are investigating introducing more formal notations in the requirements specification of automotive software systems. We are currently in an early stage where we define the criteria that the notations should fulfill for several artifacts in the software development process. One of those stages will be the definition of more formal notations for functional nets. To show a prototyping scenario, I am using UML (classes with ports) as model elements for functions and use Papyrus and Xtext to show how to adapt the tool to specify timing constraints with formal notations.

The Papyrus team has been using the Xtext integration to create a nice user interface for UML modeling. My scenario differs a little bit in that I am not using Xtext to edit parts of the UML model (like transition labels, property definitions etc.) but I augment the UML with additional models. The screenshot shows a domain specific language for the definition of timing constraints of a Port, with full syntax highlighting, error detection. As you can see in the screenshot, the content assist is fully aware of the UML model, suggesting a list of ports that could be peers of the selected port.

Integration of the Xtext editor took about 4 hours, mostly related to the fact that there is a tutorial by the Papyrus team that is very good for the first half. But then I needed some serious debugging to find about the inner workings. But if you have done it once, integration of such a grammar is very fast.

Right now the timing constraints are simply stored as text in a UML-Comment that belongs to the port as plain text. This makes code generation and model validation a little inconvenient, but I already have an idea in my mind on how to use a resource set that would make life simpler.

All in all, it is great fun extending the Papyrus tool and impressive what nice features you can implement with Eclipse based tooling in very short time!

Link collection for the Requirements Talk

Funded by the BMBFIf you’ve come here you have probably attended one of my talks on ReqIF and domain specific languages. Here is a collection of links for further reading.

At Embbeded World we are at booth 10-529!

ReqIF/RIF

ReqIF/ OMG : http://www.omg.org/spec/ReqIF/

Automotive HIS – RIF: http://www.automotive-his.de/rif/doku.php?id=welcome

Prostep – RIF: http://www.prostep.org/de/projektgruppen/internationalization-of-the-requirements-interchange-format-intrif.html

Domain Specific Languages Tools

Xtext: http://www.eclipse.org/Xtext/

 

Textual Domain Specific Languages to be researched in VERDE / IMES

Natural Rule Language

OMG MARTE Profile spec (look at VSL for textual language) : http://www.omg.org/spec/MARTE/1.0/PDF/

Research Projects

VERDE – Research Project: http://www.itea-verde.org/

ProR – Research Project: http://www.pror.org/

IMES – Research Project (no website yet).

 

My related material:

Topcased Days, Toulouse – http://gforge.enseeiht.fr/docman/view.php/52/4277/A1-Itemis-HHUD.pdf
Article in ATZ-Elektronik: http://www.atzonline.de/Artikel/3/12285/Forschungsprojekt-Verde-–-Prozessuebergreifende-Werkzeugplattform-fuer-Embedded-Systems.html

Validating Xtext models with OCL

Installing the ocl editor

Installing the ocl editor

OCL and Xtext work together quite well on Eclipse: There are some OCL editors implemented with Xtext. But if you are familiar with OCL, could you also use it to write validations for Xtext models? Of course, since Xtext generates an .ecore for each grammar and you can use OCL constraints on an .ecore. How to do this is described in the Eclipse OCL help, but it is a little hard to find and if you are mainly an Xtext user you might not see all the necessary steps.

It is necessary to install the OCL editor and examples. Now, in your Xtext project, in the src-gen directory you will find the ecore that has been generated for your Xtext grammar. In the context menu, you will find a “open with OCLinEcore” menu entry that will open a textual editor.

In this editor, you can edit the OCL constraints for the model elements.

Editing OCL constraints
Editing OCL constraints

After that, you have to tweak the genmodel a bit, by opening it and setting the property Model/Operation Reflection to true and regenerating code from the genmodel. Caution: If you change your grammar and restart the .mwe2 work flow, the .ecore might be overwritten. So keep a copy of all your OCL constraints to be able to paste them back in.

Now start your Xtext editor and you will see the OCL constraint errors show up alongside with the normal Xtext errors:

OCL constraints in Xtext
OCL constraints in Xtext

Other useful things:

You can also open Xtext models with the builtin-tree like editor (Open with … <GrammarName> Model Editor).  Open the OCL console and you can type away OCL queries on the element selected in the tree view (which is your Xtext model).

Opening the OCL console
Opening the OCL console
1: Type the query, 2: query result

Xtext and existing .ecore: Making use of the Atlan Model zoo

Xtext is a great tool for defining your own languages and generating editors for that. A lot of the work involved is in the definition of the language. So obviously, if there is some available similar language or meta-model you might want to reuse that. With Xtext, you can use existing .ecore files and create a grammar and editor out of those.

The AtlanMod Meta-Model Zoos

One source to start looking for existing meta-models is the AtlanMod Meta-Model Zoo. It provides a variety of meta-models. The meta-models are originally in KM3 format, but they have an automatic conversion into other meta-meta-models, so you can download meta-models in UML or .ecore format. As an example, we will use the FiniteStateMachine.ecore from the AtlandMod Meta-Model Zoo.

The EMF Project

Adding the .ecore to your project

Adding the .ecore to your project

To start, simply download your .ecore from the website. Create an empty EMF Project in your Eclipse workspace and copy the .ecore into your “model” sub-directory. The Xtext project wizard does actually require a .genmodel, so we need to create this first. If you open your .ecore and have a look at the properties of the FSM and PrimitiveTypes package, you will see that the properties “Ns Prefix” and “Ns URI” are unset. We need to set these values to be able to create a genmodel.

In addition, the PrimitiveTypes have unset Instance Type Names and we need those for the genmodel too, so we set them (we will get rid of most of the Primitive Types package later, but at this stage we still need to keep it).

Setting the model properties
Setting the model properties

At this point in time, make sure that the ecore is saved and then create a genmodel for the .ecore. After the genmodel is created, create all necessary emf-projects.

Creating the Xtext Project

Use the Create New Project wizard and select “Xtext from existing ecore”. Add the FSM genmodel by clicking “add” and you are almost ready to go.

A small but important selection

There is a very important but small detail: In the dialog-box you have to select the model element that should be the root rule of your grammar. Usually this is not set to a good default, so you need to pick it by yourself (State Machine in our case). Create the project and find the default grammar.

Cleaning the Xtext Project

You will find some errors in the project. The first is a duplicate import. Since the AtlanMod .ecores have more than one root package, we need to do some tweaking here by adding the fragment here. If you are familiar with the URI syntax, you might have tried something like appending “#/FSM”.

Problem with import

However, this does not work for root packages. As Sebastian Zarnekow points out, the names don’t work on the root level, you have to use the numeric reference to the packages.

Import fixed

In addition there is a number of optimizations that you can do:

  • References in the grammar often use EString as the parser rule, you might want to replace the EStrings with the ID
  • The model zoo uses String for names etc., you might want to really check if you want Strings, replace them with the terminal STRING, or if you want IDs (replace them with ID)
  • Similar with Integers (replace by INT) and other datatypes. At this point, you might have replaced all data types by built-ins so that you might want to remove the PrimitiveTypes import and rules completely
  • The generated grammar is verbose. Often you would want the “name” rule of an element to just be an ID, not preceded by the keyword “name”. So you might want to remove some of the keywords (Have a look at the generated editor to see where the keywords are too annoying for you).

At this point, you are ready to go and try your new editor.

Reporting with EMF, Xpand and Excel

itemis is participating in several research projects and I am currently the itemis PM for the projects VERDE and IMES. Research projects involve some reporting to the sponsoring entities (german ministry, european union…). Necessary data for the reports is distributed over a number of data sources (project reporting tools, excel sheets) etc. I had started out with Excel-based reporting, but that is really annoying if reporting format, level of detail changes.

Initial Meta-Model. Layout won't win a design price.

Initial Meta-Model. Layout won't win a design price.

When looking for an alternative that allows for reporting in the required format, I couldn’t find any decent available tooling, so the option was DIY. The meta-model for the project data was easily setup as an ecore model (originally modelling with Papyrus UML). Data imports from .csv where written by providing an Xtext grammar for CSV and then transforming the data to the project meta-model by means of QVTO.

Then there is the reporting part. Obviously you cannot avoid Excel when it comes to reporting to project management and sponsoring entities. So the question is how to generate Excel?

The first approach was to generate a .ecore out of XMLs schema definition and use M2M transformations. This turned out to be hell. There is a lot of xsds that you have to download and then there is problems with missing attributes names. I gave up on this.

Then there is the .zip based format of recent office versions. Data is split among several XML files which cross-reference each other. That is also very tedious.

Generating SpreadsheetXML with Xpand
Generating SpreadsheetXML with Xpand

However, office still supports the .xml format from office 2003 (known as SpreadsheetML). SpreadsheetML is a nice option if you want to generate Excel documents. It contains the Excel sheet in one flat file. Find out how the table data is stored by creating a small document that looks like you would like it to and then create a Xpand template out of it.

There are a few caveats:

  • Style ids: Cells will have a “style” attribute in .xml. However, the same style may have different style ids in two different documents, depending on the order of creation. This takes some effort to get it right.
  • Rows, cells, etc. might have some information about their position. This needs to be removed in most cases
  • Each worksheet has information about the number of columns and rows in it. Make sure that you adjust this, otherwise your sheets will not load.
  • Debugging: If you .xml is wrong, Excel will complain and show you the path to a log file. Examine that to find out more information on the error. Information is brief, prepare for some searching.
Generated report (one worksheet of many)
Generated report (one worksheet of many)

But there is also a number of nice features:

  • You can set the page breaks. I have information in sheets, that should exactly print to several pages. The breaks are correctly generated, no manual tweaking required.
  • Layout of sheets can be easily changed by the generator.

Not everything can be created this way however. It is impossible to create SmartArt and diagrams.