When opening a project in a newer version of MPS, all the automated migration scripts that come with that version of MPS will be executed to bring the project up to date with the languages and APIs. Sometimes additional manual steps might be needed or some special care must be taken by the user. This page lists such instructions for each past version of MPS.
Migration to MPS 2021.3
VM Options have been moved
In 2021.3 the IntelliJ platform changed the logic how vm options are collected. Previously vm options were taken either from the bin folder of the installation or from the config folder. When the user created custom vm options (Help | Edit custom vm options), the options from the bin folder were copied into the config folder and used by MPS together with the customized ones.
In 2021.3 the custom vm options of the previous release configs are not simply copied to be used with the new version of MPS. All the custom options that are the same as the default options in the bin folder are removed from the custom vm options. The startup scripts take into account both the default and the custom options when the IDE starts. However, this is currently not true for MPS. The MPS startup files mps.bat and mps.sh scripts are not adjusted as they should have been. So if the user has specified custom options, only those are used at startup, which causes MPS to fail.
To overcome the temporary problem, these default options should be manually added to the custom options:
Re-route references to util.jar in MPS.IDEA to MPS.Boot
On our way towards replacing huge single-point-of-entry stub modules (like MPS.Core) with slim modules that target narrow usage scenarios, we extracted the MPS.Boot solutiion. It holds classes related to the startup of MPS as well as a few IntelliJ stubs related to the IDE startup logic, which used to be in MPS.IDEA. The migration re-targets the references pointing to the corresponding stub models of the solution MPS.IDEA to the new stub solution.
Replace ToRemove and ScheduledForRemoval with Deprecated
The ToRemove and ScheduledForRemoval annotation both mark deprecated code. The standard Deprecated annotation will be used instead of those. The version parameter of the two annotations is migrated into the "since" parameted of Deprecated. The migration only works for instances of IBLDeprecatable.
Adopt asynchronous update mechanism for UI Actions
There’s a major change in IDEA Platform to update UI actions in an asynchronous manner. Data necessary to update action state is collected and consumed asynchronously and independently. There’s a promise of more responsive UI and general performance improvements. MPS started to adopt the change, and unless you have some hand-written action code or custom UI components that serve as DataProviders, you should not notice anything. Actions generated from j.m.lang.plugin are not affected, if you stick to MPS actions/UI abstractions, your code doesn’t need additional attention. However, if you deal explicitly with IDEA’s DataProvider/DataContext APIs (e.g. provide custom UI view implementing DataProvider), you might need to apply some extra migration efforts.
MPS 2021.3 still comes with asynchronous update turned off by default, mostly as a precaution, however, one can try asynchronous update mode by editing respective settings in the Registry.
Delayed model loading
There's experimental functionality to load models only when requested. MPS used to load all module’s models the moment the module becomes known (aka ‘attached’) to a repository. Now, publishing a module doesn’t force loading of its models, only when there’s a need to access models, MPS makes an attempt to load them. Generally, this happens quite soon after a module is published, the change is just a first step to make the model's lifecycle more independent of its module.
If you experience any issues with delayed loading of models, you can turn legacy behavior on with a ‘mps.models.force=true’ system property.
Notable API changes
UI: The NewSolutionDialog has been replaced with a generic NewModuleDialog
SModel: [openapi] SNode.setReference(SReferenceLink, SReference) has been deprecated, with few alternatives available.
SModel: [impl] SNode.toString and node.presentation operation
Make: changes in the runtime modules for few j.m.make.* languages. You might notice them if you write custom make facets.
BaseLanguage: Support for the "[package]Classifier" reference notation has been removed
Migration to MPS 2021.2
Replace implicit node.toString() with explicit smodel operation
node.presentation operation and a migration have been introduced to jetbrains.mps.lang.smodel. The migration replaces the usages of a node in string concatenation ("string" + node and node + "string") with the explicit node.presentation operation. At the moment, this operation delegates to the getPresentation() behavior method, which is what node.toString() used to do. Since MPS ceased to use the getPresentation() behavior method in the toString() method implementation, it is advised to use explicit node operations (like node.presentation, node.getPresentation() or any other tailored behavior) when you intend to show the node presentation to the end-user.
Split MPS.Core stub models into distinct modules
The stub models exposed through the MPS.Core module (lib/mps-core.jar, lib/mps-generator.jar and lib/mps-textgen.jar) have been separated into MPS.Generator and MPS.Textgen. This mostly impacts users who use the generator or the textgen APIs directly.
An automated migration merely updates references to use affected stubs to be used from the new modules. If it fails, the MPS.Generator and MPS.TextGen can be imported manually and then the classes from the proper modules must be selected in code using the code completion dialog.
For compatibility, MPS.Core still exposes the jars, but it will cease to expose them in 2021.3.
Plugin descriptor placement
The IntelliJ platform now loads plugins only if their plugin descriptor (the plugin.xml file) is located in a specific place in the plugin layout:
plugin_folder/lib/one_of_the_plugins.jar!/META-INF/plugin.xml Previously, the
META-INF/plugin.xml was allowed to be placed anywhere in the plugin folder.
Unless the plugin descriptor is mentioned explicitly in the plugin layout at the right location, an automatic transaction will generate it when the build script is generated. It will add the required jar file to the plugin layout and will make it hold hold the plugin descriptor. The code intended for adding during generation is displayed in a read-only section of the plugin layout: An intention called "Override Plugin Descriptor Layout" is available that allows the build script author to override the default behavior with her own - the intention generates the required part of the layout and enables it for manual editing: An inspection will report a warning whenever the plugin descriptor cannot be found in the layout definition.
Migration to MPS 2020.1
MPS 2020.1 comes with four migrations that will update your projects to the latest versions of jetbrains.mps.baseLanguage, jetbrains.mps.lang.quotationand jetbrains.mps.lang.editor:
UpdateSingleLineCommentToUseLinePerComment will update instances of SingleLineComment to use a different child to hold the textual description of the comment. The comment can now only hold a single line of text, so the migration has to split single line comments that spread across several lines.
MigrateTryStatement will update instances of TryCatchStatement and TryFinallyStatement to TryUniversalStatement, which allows for JDK 8 features, such as multiple exceptions in a single catch clause and with-resource capabilities.
LightQuotation_InitProperyExpression wraps property initialisers of light quotations into instances of NodeBuilderPropertyExpression.
Migrate_MergeNamedAndDefaultMenus will upside definitions of transformation and substitute menus to the new concepts. Transformation menus (TM) and Substitute menus (SM) were refactored so that Named and Default concept variants were merged into one concept each: TransformationMenu and SubstitutionMenu, respectively. The distinction between default and named variants is still there, but is now handled by the type property of each concept instead of having a separate root concept for them.
Update the VCS settings
The MPS merge driver has been improved. Make sure your merge driver as well as the VCS settings are up-to-date before merging conflicting branches. The VCS subsystem of MPS needs Git to be configured in a certain way in order to function properly. An outdated VCS configuration of a project is indicated to the user upon opening the project.
The user can trigger the "Update" action from the automatically displayed popup or from the VCS -> Install MPS VCS Add-ons menu.
When the installation completes, you can rely on the VCS system to be ready.
Recommended migration strategy
Merging two branches that both have been migrated to MPS 2020.1 separately is likely to give you headaches. Avoid it! Otherwise this will happen to you and your loved ones:
All single line comments that spread across several lines will be split into new instances of single line comments, which will be different nodes in each branch and will be in conflict.
New instances of TryUniversalStatement in both branches will be in conflict if there are additional changes inside this statement.
Light quotations that the migration enhances with the NodeBuilderPropertyExpression wrappers, will be in conflict.
All Transformation menus and Substitute menus will be in conflict and you will need to accept one or the other (root node's concept has changed and there are new child nodes in the menus). This is because the root nodes of Transform/Substitute menus changed their concepts and are in fact different instances. You can merge them one by one and in case the menu was not changed in either branch OR if the menu changed in just one branch you can use the Accept Yours or Accept Theirs strategy as appropriate.
This document overrides the information given in the Migration with branching documentation page and should be followed when migrating to MPS 2020.1.
The prefered scenario is:
Merge all the branches to master.
Migrate the master using the migration assistant.
Run the Migrate try statement enhancement script.
Start new branches off the migrated master.
Alternative migration strategy
If you can't help it and have to merge a not-migrated branch into a migrated master, you should follow the scenario described in this section:
Resolve merge conflicts
Execute re-runable migrations
Manually check the diff
Run the "Migrate try statement" enhancement script
Manually check the diff
Resolve merge conflicts
Conflicts in editor.mps file (jetbrains.mps.lang.editor)
This may happen if you try to merge a non-migrated branch to a migrated branch and your non-migrated branch contains some changes to the same menu that you have migrated in the other branch. In this case it's recommended to Accept Yours (keep the migrated menu) and apply the modifications done in the other branch manually.
Conflicts in SingleLineComment (jetbrains.mps.baseLanguage)
A merge conflict will be reported for all single-line comments that have been modified in the branch. The single-line comments that have not been changed in the branch will be taken from the master and so will have been migrated already. These can be auto-resolved using the appropriate tool button. The remaining conflicting changes have to be migrated manually, perhaps by accepting the changes from the branch (accept theirs). The not migrated instances of SingleLineComment will thus be re-inserted into the code. Special care must be taken for single-line comments spreading across multiple lines, as is in the last comment in the illustrating code above. Since the migration separated the additional lines into standalone instances of SingleLineComment, which are hanging below the original SingleLineComment instance, and since merging the changes from the branch will only affect the original (top) instance of SingleLineComment, the lines below will be duplicating the text of the comment. The duplicating trailing single-line comments can be deleted. Some of the lines, however, may hold changes to the text of the comment coming from the master branch. These must be manually copied into the single-line comment and only then the trailing comment can be deleted. This way you resolve all the conflicts and close the dialog. The code now, however, contains not-migrated instances of SingleLineComment coming from the branch.
Conflicts between not-migrated TryCatchStatement/TryFinallyStatement and already migrated TryUniversalStatement (jetbrains.mps.baseLanguage)
A merge conflict will be reported if there were changes inside the non-migrated or the migrated try statement. There are 3 possible cases:
changes were only inside the migrated try statement - This is the simplest case and there should be no conflict. Just accept changes if they were not accepted automatically.
changes were only inside not-migrated try statement - Here you should accept the changes from the not-migrated branch and ignore those from the migrated one (actually there are no meaningful changes to loose, just the migration that you'll repeat in the next step, anyway).
changes were inside the try statements in both branches - This is the most complicated case. Choose one of the branches (not-migrated or migrated) from which you'll accept the changes. We suggest to use the migrated branch but sometimes most of the changes are in not-migrated, in which case you should perhaps choose that one. Then accept the changes in try statements from the chosen branch and ignore the changes in try statements from the other branch. Finally manually copy/paste the changes from the other branch. You can conveniently do it directly in the 3-way merge dialog: copy the statements from the right or the left editor and then paste into the center one.
Conflicts between not-migrated and migrated light quotations
Depending on the changes made to either of the branches or both of them, handle the situation in a way similar to the previous case.
Execute re-runable migrations
Since the master after merge may contain code that has not been migrated, the migrations have to be re-run to upgrade the code to MPS 2020.1. Run the Execute Re-Runnable Migrations action from the menu to get that done easily.
Manually check the diff
Verify manually that the changes above did not introduce any errors into your code.
Run the 'Migrate try statement' enhancement script
Just like in the normal migration scenario, run the Migrate try statement enhancement script, to attempt to migrate automatically the cases that the migration skipped to avoid breaking code.
Manually check the diff again
Since the previous step could have broken some code, you should inspect the changes manually and correct them eventually.
You are done. The code can now be committed.
Migration to MPS 2019.2
Migration of structure's enumeration declarations
MPS 2019.2 features a new way of declaring structure enumerations. Therefore the existing enumeration declarations become deprecated by this feature, MPS provides a set of migration scripts to replace them as well as their usages with the new constructions. This article precisely describes how these migrations are processed.
Upgrading enumeration declaration
Firstly, the migration replaces the old enumeration declarations with the featured declarations. For each deprecated enumeration in a migrated language the migration creates a new enumeration using the featured concept. Also, the migration places the old enumeration declaration in a migration's particular attribute marking that this declaration is no longer operating and stays there only for documenting and as migration's auxiliary.
Since the new declaration holds different kinds of properties (means in general sense, not a concept's property), mapping old declaration's properties to the new ones might be non-trivial. Here we describe how the migration handles each property of the old enumeration declaration:
name of the old enumeration are trivially copied to the name to the new declaration;
"has default" and "default member" became just "default member";
"empty text", "member datatype" and "member identifier policy" are no longer supported;
For each old enumeration's member, the migration creates a member in the new declaration. Member's "external value" became "presentation" (that is the optional text line in 2nd column of new enumeration declaration). Member's "internal value" and "identifier" no longer are supported but they are used to choose a suitable name for new enumeration member.
Upgrading enumeration properties
For each migrated enumeration declaration, the migration replaces property declarations of the old enumeration type with a new property declaration of the new enumeration type. The migration places the old property declaration in a migration's attribute to handle the migration of its usages properly.
Migration of smodel and aspect languages' code
MPS 2019.2 provides concise experience working with the new structure's enumerations. Therefore the SModel language presents new enumeration's property values with "enummember<_>" type in smodel queries instead of raw values. The raw values were essential for old enumeration members but had lack of design-time support in smodel queries and therefore were dropped with the new enumerations and replaced with "enummember<_>" types. A new approach guides the language designer towards writing correct code with more restrictive types for property access operations and reduced boilerplate via providing elegant switch operation over enum members. However, this approach is not compatible wild the smodel code behaviour for old enumerations where the language designer was forced to use "raw value" of an enumeration member. Therefore, additional migration is required to update smodel queries to work with the new enumerations.
Migrations of smodel and aspects code are triggered when the code refers to enumeration properties or enumeration declarations that were replaced with the new alternatives by the structure aspect migrations. Basically, for each such reference, one of these migrations updates it so that it refers to the new counterpart. The migration also adjusts the code nearby to preserve code's semantics. There are several cases where such procedure might change code significantly, so we describe them in more detail below.
Node's property access operations for enumeration properties represents one such situation where the migration changes the code a lot, since the typing rules of these expressions are different between the old and the new enumerations. There are general rules which migration follows when handling smodel code:
If the enumeration declaration had a "boolean" member datatype, then smodel code semantics is restored by use of "is" checks on enumeration members.
If the enumeration declaration had "integer" member datatype and its members form ordered ordinal number sequence starting from 0 or 1, then the migration generates "indexOf" operations upon the enumeration's member list.
If the enumeration declaration had "string" member datatype, the migration could generate "name" or "presentation" operation on enumeration member, if the given names/presentations for new member declarations match their past internal values. Usually, this is the case unless the internal values do not satisfy naming constraints and don’t match member presentations.
Otherwise, the migration produces calls to special methods that emulate the semantics of an initial smodel code. The migration script creates such methods beforehand in a distinct model called "enumMigration" inside the enumeration-owning language.
The migration script applies similar transformations in all places where type mismatch might occur due to upgrades to some enumeration property. Here is a list of such places:
Node's property read and write accesses
Generator's property macro queries
Property constraint's queries
Editor's queries for transactional property cells
Property initializers for node builders (a.k.a. light quotations)
Property antiquotations in node quotations
Usages of property pattern variables in node patterns
As there are potentially lots of places where a transformations might be used, the migration script also detects patterns in the code where such transformation are not required or can be simplified in order to produce fewer changes to the user codebase.
Although MPS offers automatic migration for new enumeration, a user might end up doing manual migration of his/her code. For example, this may happen during resolving merge conflicts. Here we describe common scenarios of a manual migration.
If you encounter non-migrated enumeration or non-migrated enum property in a language on which MPS has already executed migrations consider to rerun migrations scripts via `Migration -> Migrations -> Execute Re-Runnable Migrations`.
Non-migrated smodel code can be handled with this action also. However, you also can update smodel code manually by replacing references on properties, enumerations or its members in smodel code from old declarations to new ones. After doing that, type errors might occur, so you have to resolve them manually as well.
Sometimes smodel code might depend on property not by directly referencing it but rather indirectly via property attribute. To migrate property attributes manually, you have to invoke "Migrate by hand" intention on the attribute and fix probable type errors afterwards.
Enumeration migration produces lots of additional attributes during the migration process. The migration framework requires such attributes for processing correctly further migrations on depending modules and projects. Although, language designer might not expect to have such depending modules outside his/her project and hence would like to remove migration attributes. This chapter describes how to do it and in which situations it is a safe change.
There are three kinds of artefacts that the migration process produces: migration attributes on new enumerations, migration attributes on enumeration properties and auxiliary methods in "enumMigration" model.
Attributes on the migrated enumerations provide data for correct migration of smodel code that refers to particular enumeration and of property instances in models written in the owning language. Generally, it means that if the language that owns such enumeration is published externally, migration attribute should be kept to perform the migration correctly. However, it's not the case for sandbox/test languages or languages in internal projects. For these scenarios, a language designer might legitimately want to drop enumeration's migration attributes by invoking "Drop Enumeration Migration's Attribute" intention on a migrated enumeration.
Attributes on migrated enumeration properties are required only for handling dependent smodel code. That means that if a language designer doesn't expect anyone to extend his/her language or maintain smodel code that operates with the enumeration property, then such attribute can be dropped safely. To make this, consider invoking "Drop Enum Property Migration's Attribute".
Likewise, auxiliary methods in "enumMigration" model are only needed for migrating smodel code. If the designer doesn't expect any smodel code operating with the migrated enumeration's members outside his/her project and has no occurrences inside the project, he/she might remove such methods manually.
Note that removing attributes is up to your choice. The MPS team, for example, schedules to perform automatic code clean up after a few releases.
Implementing automatic migrations for meta-languages
This chapter is mainly for MPS extensions' writers who implement either a new aspects or extend existing aspects (e.g. editor). Here we describe how they should update their extensions to tailor changes in MPS meta-languages.
This chapter describes the MPS metalanguages API for migrating enumeration data types and therefore contains lots of link to MPS source code that refer to either handy utility methods or examples for migrations. Open MPS 2019.2 application to be able to navigate by these links.
Generally speaking, if some MPS extension provides a concept that refers to a property declarations or a datatype declarations or depends on them by other means, the designer of this concept has to provide a migration to update the concept's instances. Otherwise, after applying the MPS enumeration migrations, nodes of this concept become to depend on outdated code, so a user becomes forced to fix them manually. The migration might differ between different concept so MPS can't provide a general solution for all possible extensions. Instead, MPS provides a set of utility functions that help extension's author to implement migration efficiently. Below we describe a few possible common scenarios.
If a concept has a reference link on a property declaration:
For such a concept, a language designer has to write a migration script that updates it on a new declaration if this property is of enumeration type that has been migrated. To do so, the extension's author might use the auxiliary method `EnumUsagesMigration#migratePropertyReference`. It accepts a node that has a reference for migrating and a reference link that targets PropertyDeclaration instances. The method returns a new target node of a given reference if it has been updated, or null otherwise. To use this auxiliary, the extension designer needs to import `jetbrains.mps.lang.structure.migration` model into the extension's migration aspect.
A canonical example here is a migration for editor cell properties.
If a concept has a reference link to a datatype declaration:
Similarly to the scenario above, a language designer has to implement a migration script to update such concept's nodes. He/she might use `EnumUsagesMigration#migrateEnumReference` to achieve this. The method accepts a node that has a reference for migrating and a reference link that targets to EnumerationDataTypeDeclaration_Old or its superconcepts. It returns a new target node for the given reference if it has been updated, or null otherwise.
Migration for “enummember<_>” instances might be an example of “migrateEnumReference” usage.
If a concept has a reference link to an enumeration's member declaration:
To migrate such concept, the language designer might use `EnumerationMemberDeclaration_Old#findReplacement()` behavior method to get an EnumerationMemberDeclaration instance that replaces old declaration. As `EnumerationMemberDeclaration_Old` and `EnumerationMemberDeclaration` concepts have no essential common superconcept, there is no option to reuse one reference link to refer to nodes of both concepts. Therefore the language designer has to consider introducing a new link in the concept and migrate from the old reference link to the new one. If a concept has the meaning of a smart reference, as an alternative, the language designer can introduce a new concept with the new link and migrate old concept's instances to new concept's instances.
Migration for deprecated “is” operation over enumeration properties can serve as an example.
If the typing rules of BL expressions or queries depend on a datatype:
Some extensions might cooperate with baseLanguage and smodel languages by introducing new expressions or queries. For such concepts, a language designer writes typing rules which might consider a baseLanguage's type of a referenced property or enumeration. MPS 2019.2 now provides a utility to evaluate a baseLanguage type for given structure's datatype - `jetbrains.mps.lang.smodel.typesystem.RulesUtil#datatypeBLType(node<DataTypeDeclaration>)`. This utility should be used in the typesystem aspect's rules to acquire proper property value's BL type by a given datatype.
If some concept is being migrated from the old enumerations to the new ones as in the first two described scenarios, and it has typing rules that manipulate with property value's BL type, then a node of this concept might end up having type errors even if this node was correct before the migration.
Lets return back to an example of migration of smodel code for the “WeekDays” enumeration described in “Migration of smodel and aspect languages' code” section. When the smodel migration updates a property reference in the property access operation, the BL type of such operation suddenly changes from `string` to `enummember<WeekDays>` and hence the smodel migration additionally wraps this property read with a call to the `name` operation that extracts the raw value from the enumeration member so that the overall expression behaves the same way as it did before the migration was applied. The smodel migration script does not actually implement the creation of such a call but delegates to an auxiliary class `jetbrains.mps.lang.smodel.migration.EnumExpressionsMigration` instead. This class provides a set of methods that the language designer can use to pass through similar problems in his/her extensions.
Particularly, it provides several methods:
`downgradeExpressionType(node<EnumerationDeclaration> enumeration, node<Expression> expression)` - transforms a given $expression$ of `enummember<$enumeration$>` type to evaluates it to a raw value of the originally evaluated enumeration member.
`upgradeExpressionType(node<EnumerationDeclaration> enumeration, node<Expression> expression)` - transforms a given $expression$ that evaluates to a raw value by means of the old declaration of $enumeration$ to evaluate it to an actual enumeration member that represents such a raw value.
`upgradeQueryReturnExpressions(node<EnumerationDeclaration> enumeration, node<IMethodLike> query)` - transforms a given $query$ body that evaluates to a raw value by means of the old declaration of $enumeration$ to evaluate it to an actual enumeration member that represents such araw value. Technically it collects all return clauses of the query and transforms them with `upgradeExpressionType` functionality.
`optimize()` - attempts to optimize the code around an expression that has been transformed with this the `EnumExpressionsMigration` instance. A general workflow with `EnumExpressionsMigration` follows these steps: it is instantiated at the beginning of a migration script, it uses the 3 methods described above to correct the user models and then the `optimize()` method is invoked to clean up the transformed code.
If a concept extends PropertyAttribute:
Even if the `PropertyAttribute` concept has no direct reference to `PropertyDeclaration`, the language designer might discover such instances through a deployed `SProperty` instance via `PropertyAttribute#getProperty()`. Since 2019.2, the `PropertyAttribute` concept provides a `#getPropertyDeclaration()` method that is aware of the enumeration migration, and for enumeration properties it provides the old property declaration if the attribute hasn't been migrated yet and the new property declaration if the migration has already been applied.
If a concept extends `PropertyAttribute` then there should be a migration that updates attributes of such a concept. The migration script has to invoke `EnumUsagesMigration#migrateEnumPropertyAttribute(node<PropertyAttribute>)` to update it. The method returns a new enumeration property declaration if the attribute is for the property of an enum type and the attribute hasn't been migrated yet. Otherwise, the method returns null.
The generator’s property macros is a good example of ‘PropertyAttribute#getPropertyDeclaration’ usage and property attribute migration.