Just recently I posted a small article with an overview on the Xtext scoping architecture. One of the things that took me a while to understand is the difference of global and local scoping. Several people I discussed this had their own ways to explain these. Here is my current way of explaining some types of scoping in Xtext.
Assume that we have a language that supports defining classes, much like in UML: A class can have a name, attributes and it can inherit from another class. And we want to support models that consist of several Xtext files.
Suppose we have the following two files:
A simple definition of two classes, with B inheriting from A. So, obviously we are having a reference from B to A (inheritance). In the grammar that could look like
|
|
Class: 'class' name=ID ('inherits' general=[Class|FQN])? |
and Xtext supports us by providing all the automatic linking when it sees “inherits”. But how does it know what to link? If all our models were restricted to be contained in one single file / resource, it would be easy: Just get the current resource and traverse all the contained model elements to find one that matches “A” and set up the link.
Global Scoping
But now, with models split over several files, what do we have to do:
- First, find all the model files candidates that could be relevant for our search
- Define a strategy to decide if a model file is really considered for traversal
- Traverse the model file and find all the candidates that could match our reference to “A”
We wouldn’t want to to this manually. Luckily, Xtext does it for you – actually, it does it a little different, since the approach above would be inefficient – traversing all the model files for each reference would be slow.
So, every time a file changes, Xtext collects the objects and the names that they are known by and puts this information into the index. So instead of traversing all models in all candidate files, Xtext just has to have a look if there is a matching entry in the index. Assuming that the classes above would be contained in a package called “P”. The index could contain:
| name |
Object |
| P |
Package “P” |
| P.A |
Class A |
| P.A.a |
Attribute b |
| P.B |
Class B |
| P.B.b |
Attribute b |
But would we want to see all the possible candidate elements in our reference (types are automatically checked). In most cases no. So Xtext provides several mechanisms to “restrict” the search. Please the the Xtext documentation for details:
- URI imports: You can have import statements, that actually reference the full URIs of the other files to be considered.
- Namespace import: You can define a namespace import. If you than provide something like “import P.*” in your grammar, the search will be restricted to those elements whose fully qualified name starts with “P.”
- Java class path based
There is additional ways to tweak and modify Xtext behavior and there is more data stored in the index (references) etc. But I hope this gives you the general idea. However, the story is not finished.
Local Scoping
Now suppose we want to have a second grammar, where we define instances of the classes and values for their attributes, just like this:
Since the classes are in a separate file, it is obvious that the link from the instance to its defining class is done in the way described above. But also the attributes values are linked to the attribute definition in the class model, so how is this done?
Obviously, we could also just use the index. But we have to restrict the possible references as well, since in instance “a”, a reference to attribute “b” would not be valid. So what we could do is to get the list of possible references from the index, traverse the class hierarchy and filter out all attributes that are not valid.
But, for finding all valid attributes, we are traversing all attributes in the class hierarchy anyway. So why not forget about the index, and just build up the list of possible references while we traverse the model? This is done by local scoping, which you can easily customize to modify the list of reference candidates. See the Xtext documentation on how to do that.
So, then why do we need the attributes in the index? Simple, we don’t. The list of possible attribute values is calculated by us, so we don’t need them there. For our own Xtext grammars, we can override the strategy of what is put into the index and what not. So we decide to not put attributes in the index. Xtext per default cannot know what you want to put in the index, so it puts in everything from your model that has a “name” attribute. It might be worthwhile customizing this behavior, since it could save us a lot of CPU and memory.
Summary
For me, it is useful to divide scoping in two different aspects:
- I can find out the possible candidates of a link by traversing references in my model. The traversal can easily span multiple resources/files, as long as the traversal can be done by standard EMF means.
- To find the possible candidates, I would have to think about a strategy on which model files to load etc. I cannot access these candidates by traversing the references that I already have, because I might not even reach those models –> global scoping.
