MPS 2023.3 Help

Migration Guide

When opening a project in a newer version of MPS, all the automated migration scripts that come with that version of MPS will be executed to bring the project up to date with the languages and APIs. Sometimes additional manual steps might be needed or some special care must be taken by the user. This page lists such instructions for each past version of MPS.

Before you update your MPS version, it is advised to run the Pre-Update Check against your project . The Pre-Update Check action can be started from the Main Menu | Migration | Run Pre-Update Check menu item. You run it in the old MPS instance to verify that there are no unmigrated left-overs in the project that the new version of MPS may have issues with. Namely, it invokes all check() methods of all migrations scripts of the languages used in the project. The action will attempt to fix all probable problems of these cases:

  • New nodes written using old language versions were merged into a migrated branch from an unmigrated branch

  • Some nodes weren't fully migrated, typically because the migration gave up migrating them for being too complex and instead recommended a manual migration for these nodes, which then never happened.

Migration to MPS 2023.3

Clear return type from method node id in stub models

MPS stopped writing the return type as part of java stub node ids. A migration was added that removes the remaining cases of java stub ids that are still using the old persistence format containing a return type.

Do not use the ${module} macro at runtime

At runtime, the “${module}” macro for a deployed module no longer resolves to the module.jar file, but instead it resolves to module-src.jar. Generally, this macro shall not be in use during runtime, there are better ways to access runtime resources (e.g. through the module ClassLoader).

Stubs no longer exposed by MPS.Core

MPS.Core no longer exposes stub for gnu.trove and org.jdom.

ReloadableSModule will stop extending SModule

As an ongoing effort to improve the class-loading story we recommend an action item for you: Do not assume that ReloadableSModule extends SModule, these 2 interfaces are to get separated in one of the nnext versions.

Disable the new UI promotion banner in standalone IDEs

Since MPS now supports new UI, standalone IDEs, when first started, would prompt their users whether they want to enable new UI or not. Since most of the standalone IDEs may not be supporting all the new UI visual elements, it is advisable to disable this new banner so that the users cannot accidentally switch on the new UI.

The banner gets disabled by adding a property -Dexperimental.ui.used.once to the startup scripts that start the standalone IDE process (mps.sh or mps.bat). Newly generated IDEs will get this property set by the generator. Existing startup scripts have to be either modified manually or regenerated with the new version of MPS.

On the other hand, if you want to enable the banner for the users of your standalone IDE, remove the property from your build scripts manually.

Compatibility mode for legacy Java stub method references removed

MPS 2023.3 finally removed the compatibility mode for legacy Java stub method references with included return type. Since 2019.1 MPS has been using stub node ids without the return type. A migration has been included in MPS back then, but a few references could have slipped in due to merging of not-migrated older branches. In such cases, the project migration will be included in the migration assistant's list of migrations to apply to your project and, additionally, your project will not be able to compile until you migrate it.
You may also get some errors in the migrations pre-check because of the removed compatibility mode. Please ignore those and proceed with the migration.

Migration to MPS 2023.2

Use enhanced notion of extended/targeted languages

A migration in the j.m.lang.generator.plan language will update node<Plan> instances to use the new enhanced version of the Transform statement, the one with support for generator grouping, that reduces the languages that target the specified one (sort of “earlier than” priority rule, bound to a specific step) as well as those languages that extend the one specified. This functionality existed in parallel with the old limited transform, now we encourage you to move forward to the more sophisticated one.

Java Module Facet settings manage module classloading

We continue the effort started in MPS 2022.3 to improve module properties (stored in the .mpl/.msd/.mpst files) and structure them according to module facets (to address MPS-20726). In MPS 2023.3, the legacy module properties compileInMPS and pluginKind are no longer serialized in the descriptor file, MPS relies solely on the Java Module Facet settings to manage module classloading.

Restart MPS after applying the migrations

A restart might be necessary after running the migrations as modules with more than 1 owner are not reloaded automatically.

Persist Java-specific module settings under Java facet

In our ongoing effort to keep Java settings of a module under Java Module Facet, values for additional source paths and java libraries are now stored under the <facet> element in module descriptor persistence. The migration moves all values from the old location in ".msd/mpl (solution/sourcePath/source)" to "solution/facets/facet[@type="java"]/source" and from "solution/stubModelEntries/stubModelEntry" to "solution/facets/facet[@type="java"]/library". This change also allows us to keep more information about path the macro/variables used in the path specification, so that we try to preserve the original value in case when several path variables get resolved to the same path. This is the only visible change for the end-user (in the Module Properties dialog, we show the macro name and its original value from the descriptor as a tooltip in addition to the source/library path).

Replace dataflow runtime stub references MPS.Core to regular nodes

The runtime for the dataflow language used to be exposed through MPS.Core, although there’s a regular MPS module j.m.dataFlow.runtime available and there’s no real need to use stubs, not mentioning exposing then through MPS.Core. (Although there were such usages in MPS, we have eliminated these in our effort to keep the “core” independent of various specific languages and frameworks - like pattern, migration, refactoring, dataflow, etc). This migration basically re-routes any dataFlow.runtime stub references through MPS.Core to respective nodes in j.m.dataFlow.runtime.

Threading for actions updated

Migration for the lang.plugin actions to guess a proper update thread has been added. It chooses one of two options:

  • Background(ActionUpdateThread.BGT)

  • EDT

If you experience troubles accessing some keys in EDT, you may try to use the mps.actions.old_edt registry setting, which resorts to using the ActionUpdateThread.OLD_EDT value, set ‘update in background:’ to false and check if migration didn’t change one of these for your action.

Create modules with language and solution providers

Legacy code to create/configure modules (e.g. in AbstractModuleCreationDialog, AbstractModuleCreationSettings) is gone. Use LanguageProducer/SolutionProducer, instead.

Separate generator for language runtime integration code

The j.m.lang.structure language now contains a separate generator for the language runtime (aka ‘@descriptor’ model) integration code. This is our ongoing effort to improve the language aspect declarations.

New home for the TODO tool

The ToDo Tool moved to a dedicated module out of mps-workbench (aka j.m.ide module) and is now available as part of a newly introduced separate plugin jetbrains.mps.ide, along with some other tools and language IDE integrations (e.g. typesystem and module dependency actions). If your RCP re-uses these tools, please make sure to include the new plugin into your distribution.

Say goodbye to the Terminal window in your build scripts

The platform has been repackaged to fix a problem with the Terminal tool window on unix systems MPS-36027. This change entailed a change of build projects generated by the Build Wizard for RCP MPS-36088. Existing build projects for RCP, however, won't get this change automatically and need a manual intervention to properly package the Terminal tool window again.

The Safe Delete option for deleting models was removed from RCPs

The Safe Delete option is no longer available when deleting models in custom-built MPS-based RCPs (standalone IDE's), unless the mps-rcp plugin is added to the existing build scripts. The Build Project wizard adds mps-rcp as a dependency to all newly created build scripts for standalone ide scenarios. Landuage designers can choose to remove mps-rcp from the build scripts.

Update existing ant scripts when migrating from the console

The path.mps.ant.path classpath must be updated to refer to the util-8.jar file instead of util.jar.

Migration to MPS 2022.3

Java Module Facet settings

A dedicated project migration saves Java Module Facet settings in the module descriptors as explicit values rather than unrelated values scattered around the descriptor (e.g. compileInMps and pluginKind were attributes of a module, and external classloading was managed by an additional module facet).

Extract gnu.trove and org.jdom libraries

A project migration extracts gnu.trove and org.jdom libraries out of MPS.Core. There are 2 new stub modules for these libraries. The migration redirects references to their stub classes that used to go through MPS.Core right to the new modules. These modules can be used independently of MPS.Core, if necessary.

Make facets, TextGen and Generate moved

Make facets, TextGen and Generate, used to reside in a plugin aspect of the j.m.lang.core language and now got moved out into a dedicated module, j.m.make.facetsi>. You may notice updated Make Target names in your generated sources if you reference these facets. Although unlikely, you may need to adjust model imports in the model with your custom Make Facet in case you reference TextGen/Generate as a dependency.

JNA library update

The generic distribution of MPS now contains two versions of the native jna library, each held in a separate subfolder of the /lib/jna/ folder:

  • amd64 - for the Intel-based Mac computers

  • aarch64 - for the Apple hardware Mac computers

The build script wizard that creates build scripts for standalone MPS IDEs, already provides an additional MPS-macos-aarch64.zip layout that utilizes the aarch64 variant of the jna library, while the other three, previously existing, layouts have been updated to correctly include their corresponding jna libraries from the new location.
Update existing build scripts manually to accommodate for the change in location of the jna library as well as to adjust the set of binary files according to the new platform.

Migration to MPS 2022.2

Stop supporting the old format for enumeration values

Although the structure language's enumerations and their persistence format had changed many releases ago, the old format was still supported for compatibility reasons. This supports ended with MPS 2022.2, which only supports the new format. This may cause problems reading models in MPS 2022.2, if the models have not been migrated properly in the previouos versions of MPS. To fix this, re-run the migration "Update enumeration properties" that has been introduced intoo the structure language in MPS 2019.1. You have to run the migration in an older MPS before opening the project in MPS 2022.2.

MPS Kotlin language updates

Automatic migrations have been provided to reflect the recent improvements in the MPS Kotlin language:

  • Statement support has been made generic in the IStatementHolder interface; other concepts using statements on their own have been migrated to use this concept.

  • String literal and multiline literal have been merged into a single string literal concept.

  • Inheritance modifier (final, abstract, etc.) on function declarations is now mandatory because of the usage of the Inheritable interface; declarations will be migrated accordingly.

  • Support for deconstructing variables has been grouped under the IDeconstructingDeclarations interface; for statement and lambda multi-parameter are migrated to use it.

  • Concepts with receiver types no longer use the ReceiverType concept (which is an expression referring to a type) and hold the type directly.

  • Unsigned, Long and Integer literal concepts have been grouped under the IntegerLiteral concept (which uses boolean flags). Moreover, negative number literals are no longer supported and must be created using the unary minus concept.

Using JDK 17

MPS 2022.2 has been developed and tested with JDK 17, so it is recommended to use JDK 17 for running MPS as well as for running standalone IDEs built with MPS. There may potentially be some problems with the visibility of different classes that your code may be accessing using Java reflection API. You can solve the issues in the same way we solved it for the JDK classes when running MPS. Use the --add-opens JVM argument to add the respective packages:

--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED --add-opens=java.base/sun.security.util=ALL-UNNAMED --add-opens=java.desktop/java.awt=ALL-UNNAMED --add-opens=java.desktop/java.awt.dnd.peer=ALL-UNNAMED --add-opens=java.desktop/java.awt.event=ALL-UNNAMED --add-opens=java.desktop/java.awt.image=ALL-UNNAMED --add-opens=java.desktop/java.awt.peer=ALL-UNNAMED --add-opens=java.desktop/javax.swing=ALL-UNNAMED --add-opens=java.desktop/javax.swing.plaf.basic=ALL-UNNAMED --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED --add-opens=java.desktop/sun.awt.datatransfer=ALL-UNNAMED --add-opens=java.desktop/sun.awt.image=ALL-UNNAMED --add-opens=java.desktop/sun.awt.windows=ALL-UNNAMED --add-opens=java.desktop/sun.awt.X11=ALL-UNNAMED --add-opens=java.desktop/sun.awt=ALL-UNNAMED --add-opens=java.desktop/sun.font=ALL-UNNAMED --add-opens=java.desktop/sun.java2d=ALL-UNNAMED --add-opens=java.desktop/sun.swing=ALL-UNNAMED --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED --add-opens=jdk.jdi/com.sun.tools.jdi=ALL-UNNAMED --add-opens=java.desktop/com.apple.laf=ALL-UNNAMED --add-opens=java.desktop/com.apple.eawt=ALL-UNNAMED --add-opens=java.desktop/com.apple.eawt.event=ALL-UNNAMED

Changes in logging configuration

Due to abandoned log4j, bin/log.xml configuration is no longer in use. There’s no mechanism to migrate your settings from log.xml, you need to re-create them in log.properies or to use IDE actions to configure logging. MPS no longer relies on configuration files for default log configuration (done in code, instead), and you need to turn on the use of configuration file explicitly through command-line switches.
For a hand-written logging code, it’s recommended to replace log4j code with jetbrains.mps.logging.Logger.getLogger(Class<>) to minimize the changes. Users of jetbrains.mps.baseLanguage.logging language don’t need to bother, just regenerate the code with the new templates to get updated.

MPS.Core no longer exports unrelated stubs

MPS.Core no longer exports stubs available from MPS.Generator and MPS.Textgen. MPS.Generator and MPS.TextGen were introduced a year ago in 21.2, to keep dependencies focuse and to prevent MPS.Core growing into a monster hog with a lot of unrelated functionality. For a few releases, MPS.Core exported java stubs for the Generator and TextGen subsystems, along with designated MPS.Generator and MPS.TextGen stub solutions. In this release, we finalize the split and MPS.Core no longer exports stubs for these subsystems. There’s re-runnable migration to re-route references to MPS.Generator and MPS.TextGen stubs you may need to run it again in case your models still reference Generator/TextGen stubs through MPS.Core solution.

New api in lang.plugin to register shortcuts

You may notice changes in the generated code if you declare custom shortcut sets or override the existing shortcuts for your actions. This streamlined and simplifies the generated code.

False dependencies need to be corrected

The fact ‘deps.cp’ keeps a snapshot of actual module compile dependencies at transformation time may ruin hacks/workarounds when an unused dependency was added to a module/model to satisfy compilation-time (Java classpath) dependency. Instead, this dependency has to be introduced in a proper way (in majority of cases, it’s just a misconfigured runtime of an engaged language).

Changes to pathToFile in the Make facet

In the Make facet, the pathToFile(String,IFile) function parameter gone. It served few very specific cases and was unlikely you ever needed it. If you do, however, there has been a replacement, pathToFile(IFile, IFile), available for a few releases already.

Deputy concept in aspect declaration

EXPERIMENTAL: ‘Deputy’ concept in aspect declaration. You may notice these if you declare your own aspects. This is a work towards meaningful @descriptor models. MPS employs this for the ‘structure’ aspect now. If you feel this is the right move (or a wrong one), please share your opinion.

mpsCore import in build scripts (MPS-34554)

MPS used to re-export mpsCore project and the respective plug-in from mpsStandalone project. If the build project for your RCP used to bundle/import mpsCore, you need to add a direct project dependency to mpsCore for import to get resolved again.

An additional dependency needed for the build scripts

The lib/util_rt.jar file has to be added in the boot classpath of the MpsStartupScript roots of the existing projects that use the build language to create standalone IDEs. Otherwise, exception like this one may occur when you attempt to start your standalone IDE:

Exception in thread "main" java.lang.NoClassDefFoundError: com/intellij/openapi/util/SystemInfoRt at com.intellij.openapi.application.PathManager.getHomePath(PathManager.java:101) at com.intellij.openapi.application.PathManager.getHomePath(PathManager.java:68) at com.intellij.openapi.application.PathManager.getBinPath(PathManager.java:186) at jetbrains.mps.Launcher.main(Launcher.java:28)
BuildScriptMigration.png

VM options changed

The VM options used to start MPS IDEs have changed. The existing build scripts for standalone IDEs have to be updated manually to provide all the necessary VM options. An intention is in the works that will add all the missing depedencies. Until it is in place, the recommended way is to generate a new build script and copy the just created MpsStartupScript root node into your build script, or just use the list of options below:

# Common IntelliJ Platform options: -Xms256m -Xmx2048m -XX:ReservedCodeCacheSize=512m -XX:+UseG1GC -XX:SoftRefLRUPolicyMSPerMB=50 -XX:CICompilerCount=2 -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -XX:+IgnoreUnrecognizedVMOptions -ea -Dsun.io.useCanonCaches=false -Dsun.java2d.metal=true -Djdk.http.auth.tunneling.disabledSchemes="" -Djdk.attach.allowAttachSelf=true -Djdk.module.illegalAccess.silent=true -Dkotlinx.coroutines.debug=off -Dsun.tools.attach.tmp.only=true # Additional MPS options: -client -Dfile.encoding=UTF-8 -Dapple.awt.graphics.UseQuartz=true -Dide.mac.message.dialogs.as.sheets=false -Didea.invalidate.caches.invalidates.vfs=true -Didea.trust.disabled=false -Dfreeze.reporter.enabled=false -Didea.indices.psi.dependent.default=false # -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5071

Re-create run configurations

Existing run configurations will not work because the VM options required to start Java processes in MPS have changed. You can either create new run configurations, which will inherit updated VM options automatically. Or you can update the value in the run configuration's Virtual Machine Parameters field to

--add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED --add-opens=java.base/sun.security.util=ALL-UNNAMED --add-opens=java.desktop/com.apple.eawt=ALL-UNNAMED --add-opens=java.desktop/com.apple.eawt.event=ALL-UNNAMED --add-opens=java.desktop/com.apple.laf=ALL-UNNAMED --add-opens=java.desktop/java.awt=ALL-UNNAMED --add-opens=java.desktop/java.awt.dnd.peer=ALL-UNNAMED --add-opens=java.desktop/java.awt.event=ALL-UNNAMED --add-opens=java.desktop/java.awt.image=ALL-UNNAMED --add-opens=java.desktop/java.awt.peer=ALL-UNNAMED --add-opens=java.desktop/javax.swing=ALL-UNNAMED --add-opens=java.desktop/javax.swing.plaf.basic=ALL-UNNAMED --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED --add-opens=java.desktop/sun.awt.datatransfer=ALL-UNNAMED --add-opens=java.desktop/sun.awt.image=ALL-UNNAMED --add-opens=java.desktop/sun.awt.windows=ALL-UNNAMED --add-opens=java.desktop/sun.awt=ALL-UNNAMED --add-opens=java.desktop/sun.font=ALL-UNNAMED --add-opens=java.desktop/sun.java2d=ALL-UNNAMED --add-opens=java.desktop/sun.swing=ALL-UNNAMED --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED --add-opens=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED --add-opens=jdk.jdi/com.sun.tools.jdi=ALL-UNNAMED --add-opens=java.desktop/sun.awt.X11=ALL-UNNAMED

.

Editor hints allowed only from extended languages

In the previous versions, when defining editors and assigning editor hints to them, MPS tolerated editor hints from other language modules available through mere 'default' dependency. Now MPS only allows editors to use editor hints from languages explicitly "extended" by the language holding the editors.

Obsolete custom VM options need to be removed

If you installed MPS 2021.2 EAP 2 from the toolbox you may have ended up with three custom VM options set:

-Dsplash=true -DactionSystem.update.actions.async=false -DactionSystem.update.actions.async.ui=false

These options are now obsolete and you are highly recommended to remove them. Just choose "Edit Custom VM Options" from the "Help" menu and remove the lines manually.

Replace the deprecated needNoWriteAction property with accessMode

The node test cases got a new accessMode property. Previously, the node tests were executed wrapped in either runWithinCommand or runWithinRead methods of BaseTestBody. Now the tests are executed under runWithinCommand, runWithinRead or without any wrapping depending on the value of the accessMode property.
The ExecutionModeNodesTestCase migration removes the needsNoWriteAction deprecated property and sets the accessMode property to either command or nonedepending on the previous value of needsNoWriteAction.

Custom extensions of the test language and the node test cases that provide their own test methods should pay special attention in their generators to proper wrapping of the code of the custom test methods with locking. The runtime class BaseTestBody now provides two additional methods runWithinCommand and runWithinRead that should be used just like they are used in the switch_Test2Method template switch.

Migration to MPS 2021.3

Add log4j to the build script

Build scripts created by the Build Script Wizzard in earlier versions of MPS may have problems finding the log4j jar file in 2021.3.

MigrationBuild1.png

To fix this problem the lib/3rd-party-rt.jar must be added into the boot classpath section of the MpsStartupScript node of the build model.

MigrationBuild2.png

Re-route references to util.jar in MPS.IDEA to MPS.Boot

On our way towards replacing huge single-point-of-entry stub modules (like MPS.Core) with slim modules that target narrow usage scenarios, we extracted the MPS.Boot solutiion. It holds classes related to the startup of MPS as well as a few IntelliJ stubs related to the IDE startup logic, which used to be in MPS.IDEA. The migration re-targets the references pointing to the corresponding stub models of the solution MPS.IDEA to the new stub solution.

Replace ToRemove and ScheduledForRemoval with Deprecated

The ToRemove and ScheduledForRemoval annotation both mark deprecated code. The standard Deprecated annotation will be used insed of those. The version parameter of the two annotations is migrated into the "since" parameted of Deprecated. The migration only works for instances of IBLDeprecatable.

Adopt asynchronous update mechanism for UI Actions

There’s a major change in IDEA Platform to update UI actions in an asynchronous manner. Data necessary to update action state is collected and consumed asynchronously and independently. There’s a promise of more responsive UI and general performance improvements. MPS started to adopt the change, and unless you have some hand-written action code or custom UI components that serve as DataProviders, you should not notice anything. Actions generated from j.m.lang.plugin are not affected, if you stick to MPS actions/UI abstractions, your code doesn’t need additional attention. However, if you deal explicitly with IDEA’s DataProvider/DataContext APIs (e.g. provide custom UI view implementing DataProvider), you might need to apply some extra migration efforts.

Delayed model loading

There's experimental functionality to load models only when requested. MPS used to load all module’s models the moment the module becomes known (aka ‘attached’) to a repository. Now, publishing a module doesn’t force loading of its models, only when there’s a need to access models, MPS makes an attempt to load them. Generally, this happens quite soon after a module is published, the change is just a first step to make the model's lifecycle more independent of its module.

Notable API changes

  • UI: The NewSolutionDialog has been replaced with a generic NewModuleDialog

  • SModel: [openapi] SNode.setReference(SReferenceLink, SReference) has been deprecated, with few alternatives available.

  • SModel: [impl] SNode.toString and node.presentation operation

  • Make: changes in the runtime modules for few j.m.make.* languages. You might notice them if you write custom make facets.

  • BaseLanguage: Support for the "[package]Classifier" reference notation has been removed

Migration to MPS 2021.2

NodeTestCases need regeneration

When upgrading to 2021.2.6 all NodeTestCases have to be regenerated for them to run successfully. Otherwise you will be getting exceptions like

java.lang.reflect.InvocationTargetException at jdk.internal.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.intellij.execution.process.ProcessHandler$2.invoke(ProcessHandler.java:247) at com.sun.proxy.$Proxy32.onTextAvailable(Unknown Source) at com.intellij.execution.process.ProcessHandler.notifyTextAvailable(ProcessHandler.java:221) at jetbrains.mps.execution.configurations.implementation.plugin.plugin.FakeProcessHandler$BlockingReader.onTextAvailable(FakeProcessHandler.java:155) at com.intellij.util.io.BaseOutputReader.sendText(BaseOutputReader.java:205)

Replace implicit node.toString() with explicit smodel operation

A new node.presentation operation and a migration have been introduced to jetbrains.mps.lang.smodel. The migration replaces the usages of a node in string concatenation ("string" + node and node + "string") with the explicit node.presentation operation. At the moment, this operation delegates to the getPresentation() behavior method, which is what node.toString() used to do. Since MPS ceased to use the getPresentation() behavior method in the toString() method implementation, it is advised to use explicit node operations (like node.presentation, node.getPresentation() or any other tailored behavior) when you intend to show the node presentation to the end-user.

Split MPS.Core stub models into distinct modules

The stub models exposed through the MPS.Core module (lib/mps-core.jar, lib/mps-generator.jar and lib/mps-textgen.jar) have been separated into MPS.Generator and MPS.Textgen. This mostly impacts users who use the generator or the textgen APIs directly.

An automated migration merely updates references to use affected stubs to be used from the new modules. If it fails, the MPS.Generator and MPS.TextGen can be imported manually and then the classes from the proper modules must be selected in code using the code completion dialog.

For compatibility, MPS.Core still exposes the jars, but it will cease to expose them in 2021.3.

Plugin descriptor placement

The IntelliJ platform now loads plugins only if their plugin descriptor (the plugin.xml file) is located in a specific place in the plugin layout: plugin_folder/lib/one_of_the_plugins.jar!/META-INF/plugin.xml Previously, the META-INF/plugin.xml was allowed to be placed anywhere in the plugin folder.

Unless the plugin descriptor is mentioned explicitly in the plugin layout at the right location, an automatic transaction will generate it when the build script is generated. It will add the required jar file to the plugin layout and will make it hold hold the plugin descriptor. The code intended for adding during generation is displayed in a read-only section of the plugin layout:

PluginDescriptor1.png

An intention called "Override Plugin Descriptor Layout" is available that allows the build script author to override the default behavior with her own - the intention generates the required part of the layout and enables it for manual editing:

PluginDescriptor2.png

An inspection will report a warning whenever the plugin descriptor cannot be found in the layout definition.

PluginDescriptor3.png

Migration to MPS 2020.1

Improved core.text language

The jetbrains.mps.core.text language has been improved and is now being used for comments in BaseLanguage. The CommentPart and TextCommentPart concepts have been deprecated. If your language utilizes these deprecated concepts, consider using the concepts like Text, Line and Word from the jetbrains.mps.core.text language instead. You'll then need to provide migrations for the users of your language that will migrate the text held in TextCommentParts to be transformed to instances of Words and Lines. The easiest way to implement such migrations is to:

  • instantiate a Line

  • instantiate a Word

  • add the word to the line using Line.addTextElement()

  • set the Word.value property to the text of a TextCommentLine

  • call the Word.normalize() method to have the text of the word split properly into words, all belonging to the same line

The new migrations

MPS 2020.1 comes with four migrations that will update your projects to the latest versions of jetbrains.mps.baseLanguage, jetbrains.mps.lang.quotation and jetbrains.mps.lang.editor:

  • UpdateSingleLineCommentToUseLinePerComment will update instances of SingleLineComment to use a different child to hold the textual description of the comment. The comment can now only hold a single line of text, so the migration has to split single line comments that spread across several lines.

  • MigrateTryStatement will update instances of TryCatchStatement and TryFinallyStatement to TryUniversalStatement, which allows for JDK 8 features, such as multiple exceptions in a single catch clause and with-resource capabilities.

  • LightQuotation_InitProperyExpression wraps property initialisers of light quotations into instances of NodeBuilderPropertyExpression.

  • Migrate_MergeNamedAndDefaultMenus will upside definitions of transformation and substitute menus to the new concepts. Transformation menus (TM) and Substitute menus (SM) were refactored so that Named and Default concept variants were merged into one concept each: TransformationMenu and SubstitutionMenu, respectively. The distinction between default and named variants is still there, but is now handled by the type property of each concept instead of having a separate root concept for them.

All four migrations are re-runnable and so can be run repetitively if a merge brings non-migrated code into your branch.

Update the VCS settings

The MPS merge driver has been improved. Make sure your merge driver as well as the VCS settings are up-to-date before merging conflicting branches. The VCS subsystem of MPS needs Git to be configured in a certain way in order to function properly. An outdated VCS configuration of a project is indicated to the user upon opening the project. VCS -> Install MPS VCS Add-ons menu.

VCSAddons1.png
VCSAddonsInstall.png

When the installation completes, you can rely on the VCS system to be ready.

Recommended migration strategy

Merging two branches that both have been migrated to MPS 2020.1 separately is likely to give you headaches. Avoid it! Otherwise this will happen to you and your loved ones:

  • All single line comments that spread across several lines will be split into new instances of single line comments, which will be different nodes in each branch and will be in conflict.

  • New instances of TryUniversalStatement in both branches will be in conflict if there are additional changes inside this statement.

  • Light quotations that the migration enhances with the NodeBuilderPropertyExpression wrappers, will be in conflict.

  • All Transformation menus and Substitute menus will be in conflict and you will need to accept one or the other (root node's concept has changed and there are new child nodes in the menus). This is because the root nodes of Transform/Substitute menus changed their concepts and are in fact different instances. You can merge them one by one and in case the menu was not changed in either branch OR if the menu changed in just one branch you can use the Accept Yours or Accept Theirs strategy as appropriate.

We strongly recommend you to migrate all your branches to MPS 2020.1 in a single step.

The prefered scenario is:

  • Merge all the branches to master.

  • Migrate the master using the migration assistant.

  • Run the Migrate try statement enhancement script.

  • Start new branches off the migrated master.

The "Run the Migrate try statement enhancement script" step needs a bit of explanation. The normal migration of try statements works in a defensive way and avoids migrating in situations that can potentially lead to broken code. These are typically try statements in generator template fragments, macros, commented out nodes or inside quotations. To avoid the need to manually update the code that the migration skips (the not-migrated nodes are reported by the migration, so you will see a list of them) we provide an enhancement script called Migrate try statement to attempt to migrate those complicated cases. It may, however, lead to broken code, so manual verification of the result is needed.

mig23.png

Alternative migration strategy

If you can't help it and have to merge a not-migrated branch into a migrated master, you should follow the scenario described in this section:

  • Resolve merge conflicts

  • Execute re-runable migrations

  • Manually check the diff

  • Run the "Migrate try statement" enhancement script

  • Manually check the diff

  • Commit

We will discuss these topics in more detail now.

Resolve merge conflicts

Conflicts in editor.mps file (jetbrains.mps.lang.editor)

Conflicts in SingleLineComment (jetbrains.mps.baseLanguage)

blc3.png

The single-line comments that have not been changed in the branch will be taken from the master and so will have been migrated already. These can be auto-resolved using the appropriate tool button.

blc4.png

The remaining conflicting changes have to be migrated manually, perhaps by accepting the changes from the branch (accept theirs). The not migrated instances of SingleLineComment will thus be re-inserted into the code.

blc5.png

Special care must be taken for single-line comments spreading across multiple lines, as is in the last comment in the illustrating code above. Since the migration separated the additional lines into standalone instances of SingleLineComment, which are hanging below the original SingleLineComment instance, and since merging the changes from the branch will only affect the original (top) instance of SingleLineComment, the lines below will be duplicating the text of the comment.

blc6.png

The duplicating trailing single-line comments can be deleted. Some of the lines, however, may hold changes to the text of the comment coming from the main branch. These must be manually copied into the single-line comment and only then the trailing comment can be deleted.

blc7.png
blc8.png
blc9.png

This way you resolve all the conflicts and close the dialog. The code now, however, contains not-migrated instances of SingleLineComment coming from the branch.

blc10.png

Conflicts between not-migrated TryCatchStatement/TryFinallyStatement and already migrated TryUniversalStatement (jetbrains.mps.baseLanguage)

  • changes were only inside the migrated try statement - This is the simplest case and there should be no conflict. Just accept changes if they were not accepted automatically.

  • changes were only inside not-migrated try statement - Here you should accept the changes from the not-migrated branch and ignore those from the migrated one (actually there are no meaningful changes to loose, just the migration that you'll repeat in the next step, anyway).

  • changes were inside the try statements in both branches - This is the most complicated case. Choose one of the branches (not-migrated or migrated) from which you'll accept the changes. We suggest to use the migrated branch but sometimes most of the changes are in not-migrated, in which case you should perhaps choose that one. Then accept the changes in try statements from the chosen branch and ignore the changes in try statements from the other branch. Finally manually copy/paste the changes from the other branch. You can conveniently do it directly in the 3-way merge dialog: copy the statements from the right or the left editor and then paste into the center one.

Conflicts between not-migrated and migrated light quotations

Execute re-runable migrations

Since the master after merge may contain code that has not been migrated, the migrations have to be re-run to upgrade the code to MPS 2020.1. Run the Execute Re-Runnable Migrations action from the menu to get that done easily.

mig111.png

Manually check the diff

Verify manually that the changes above did not introduce any errors into your code.

Run the 'Migrate try statement' enhancement script

Just like in the normal migration scenario, run the Migrate try statement enhancement script, to attempt to migrate automatically the cases that the migration skipped to avoid breaking code.

mig23.png

Manually check the diff again

Since the previous step could have broken some code, you should inspect the changes manually and correct them eventually.

Commit

You are done. The code can now be committed.

Migration to MPS 2019.2

Migration of structure's enumeration declarations

MPS 2019.2 features a new way of declaring structure enumerations. Therefore the existing enumeration declarations become deprecated by this feature, MPS provides a set of migration scripts to replace them as well as their usages with the new constructions. This article precisely describes how these migrations are processed.

Upgrading enumeration declaration

Firstly, the migration replaces the old enumeration declarations with the featured declarations. For each deprecated enumeration in a migrated language the migration creates a new enumeration using the featured concept. Also, the migration places the old enumeration declaration in a migration's particular attribute marking that this declaration is no longer operating and stays there only for documenting and as migration's auxiliary.

Since the new declaration holds different kinds of properties (means in general sense, not a concept's property), mapping old declaration's properties to the new ones might be non-trivial. Here we describe how the migration handles each property of the old enumeration declaration:

  • name of the old enumeration are trivially copied to the name to the new declaration;

  • "has default" and "default member" became just "default member";

  • "empty text", "member datatype" and "member identifier policy" are no longer supported;

  • For each old enumeration's member, the migration creates a member in the new declaration. Member's "external value" became "presentation" (that is the optional text line in 2nd column of new enumeration declaration). Member's "internal value" and "identifier" no longer are supported but they are used to choose a suitable name for new enumeration member.

Upgrading enumeration properties

For each migrated enumeration declaration, the migration replaces property declarations of the old enumeration type with a new property declaration of the new enumeration type. The migration places the old property declaration in a migration's attribute to handle the migration of its usages properly.

Migration of smodel and aspect languages' code

MPS 2019.2 provides concise experience working with the new structure's enumerations. Therefore the SModel language presents new enumeration's property values with "enummember<_>" type in smodel queries instead of raw values. The raw values were essential for old enumeration members but had lack of design-time support in smodel queries and therefore were dropped with the new enumerations and replaced with "enummember<_>" types. A new approach guides the language designer towards writing correct code with more restrictive types for property access operations and reduced boilerplate via providing elegant switch operation over enum members. However, this approach is not compatible wild the smodel code behaviour for old enumerations where the language designer was forced to use "raw value" of an enumeration member. Therefore, additional migration is required to update smodel queries to work with the new enumerations.

Migrations of smodel and aspects code are triggered when the code refers to enumeration properties or enumeration declarations that were replaced with the new alternatives by the structure aspect migrations. Basically, for each such reference, one of these migrations updates it so that it refers to the new counterpart. The migration also adjusts the code nearby to preserve code's semantics. There are several cases where such procedure might change code significantly, so we describe them in more detail below.

Node's property access operations for enumeration properties represents one such situation where the migration changes the code a lot, since the typing rules of these expressions are different between the old and the new enumerations. There are general rules which migration follows when handling smodel code:

  • If the enumeration declaration had a "boolean" member datatype, then smodel code semantics is restored by use of "is" checks on enumeration members.

  • If the enumeration declaration had "integer" member datatype and its members form ordered ordinal number sequence starting from 0 or 1, then the migration generates "indexOf" operations upon the enumeration's member list.

  • If the enumeration declaration had "string" member datatype, the migration could generate "name" or "presentation" operation on enumeration member, if the given names/presentations for new member declarations match their past internal values. Usually, this is the case unless the internal values do not satisfy naming constraints and don’t match member presentations.

  • Otherwise, the migration produces calls to special methods that emulate the semantics of an initial smodel code. The migration script creates such methods beforehand in a distinct model called "enumMigration" inside the enumeration-owning language.

The migration script applies similar transformations in all places where type mismatch might occur due to upgrades to some enumeration property. Here is a list of such places:

  • Node's property read and write accesses

  • Generator's property macro queries

  • Property constraint's queries

  • Editor's queries for transactional property cells

  • Property initializers for node builders (a.k.a. light quotations)

  • Property antiquotations in node quotations

  • Usages of property pattern variables in node patterns

Another place where migration has to transform code significantly to keep original semantics is old "name" and "value" operations on enumeration member. The migration handles such places in a way similar to described above.

As there are potentially lots of places where a transformations might be used, the migration script also detects patterns in the code where such transformation are not required or can be simplified in order to produce fewer changes to the user codebase.

Manual migrations

Although MPS offers automatic migration for new enumeration, a user might end up doing manual migration of his/her code. For example, this may happen during resolving merge conflicts. Here we describe common scenarios of a manual migration.

If you encounter non-migrated enumeration or non-migrated enum property in a language on which MPS has already executed migrations consider to rerun migrations scripts via `Migration -> Migrations -> Execute Re-Runnable Migrations`.

Non-migrated smodel code can be handled with this action also. However, you also can update smodel code manually by replacing references on properties, enumerations or its members in smodel code from old declarations to new ones. After doing that, type errors might occur, so you have to resolve them manually as well.

Sometimes smodel code might depend on property not by directly referencing it but rather indirectly via property attribute. To migrate property attributes manually, you have to invoke "Migrate by hand" intention on the attribute and fix probable type errors afterwards.

Code clean-up

Enumeration migration produces lots of additional attributes during the migration process. The migration framework requires such attributes for processing correctly further migrations on depending modules and projects. Although, language designer might not expect to have such depending modules outside his/her project and hence would like to remove migration attributes. This chapter describes how to do it and in which situations it is a safe change.

There are three kinds of artefacts that the migration process produces: migration attributes on new enumerations, migration attributes on enumeration properties and auxiliary methods in "enumMigration" model.

Attributes on the migrated enumerations provide data for correct migration of smodel code that refers to particular enumeration and of property instances in models written in the owning language. Generally, it means that if the language that owns such enumeration is published externally, migration attribute should be kept to perform the migration correctly. However, it's not the case for sandbox/test languages or languages in internal projects. For these scenarios, a language designer might legitimately want to drop enumeration's migration attributes by invoking "Drop Enumeration Migration's Attribute" intention on a migrated enumeration.

Attributes on migrated enumeration properties are required only for handling dependent smodel code. That means that if a language designer doesn't expect anyone to extend his/her language or maintain smodel code that operates with the enumeration property, then such attribute can be dropped safely. To make this, consider invoking "Drop Enum Property Migration's Attribute".

Likewise, auxiliary methods in "enumMigration" model are only needed for migrating smodel code. If the designer doesn't expect any smodel code operating with the migrated enumeration's members outside his/her project and has no occurrences inside the project, he/she might remove such methods manually.

Note that removing attributes is up to your choice. The MPS team, for example, schedules to perform automatic code clean up after a few releases.

Implementing automatic migrations for meta-languages

This chapter is mainly for MPS extensions' writers who implement either a new aspects or extend existing aspects (e.g. editor). Here we describe how they should update their extensions to tailor changes in MPS meta-languages.

This chapter describes the MPS metalanguages API for migrating enumeration data types and therefore contains lots of link to MPS source code that refer to either handy utility methods or examples for migrations. Open MPS 2019.2 application to be able to navigate by these links.

Generally speaking, if some MPS extension provides a concept that refers to a property declarations or a datatype declarations or depends on them by other means, the designer of this concept has to provide a migration to update the concept's instances. Otherwise, after applying the MPS enumeration migrations, nodes of this concept become to depend on outdated code, so a user becomes forced to fix them manually. The migration might differ between different concept so MPS can't provide a general solution for all possible extensions. Instead, MPS provides a set of utility functions that help extension's author to implement migration efficiently. Below we describe a few possible common scenarios.

If a concept has a reference link on a property declaration:

A canonical example here is a migration for editor cell properties.

If a concept has a reference link to a datatype declaration:

Migration for “enummember<_>” instances might be an example of “migrateEnumReference” usage.

If a concept has a reference link to an enumeration's member declaration:

Migration for deprecated “is” operation over enumeration properties can serve as an example.

If the typing rules of BL expressions or queries depend on a datatype:

If some concept is being migrated from the old enumerations to the new ones as in the first two described scenarios, and it has typing rules that manipulate with property value's BL type, then a node of this concept might end up having type errors even if this node was correct before the migration.

Lets return back to an example of migration of smodel code for the “WeekDays” enumeration described in “Migration of smodel and aspect languages' code” section. When the smodel migration updates a property reference in the property access operation, the BL type of such operation suddenly changes from `string` to `enummember<WeekDays>` and hence the smodel migration additionally wraps this property read with a call to the `name` operation that extracts the raw value from the enumeration member so that the overall expression behaves the same way as it did before the migration was applied. The smodel migration script does not actually implement the creation of such a call but delegates to an auxiliary class `jetbrains.mps.lang.smodel.migration.EnumExpressionsMigration` instead. This class provides a set of methods that the language designer can use to pass through similar problems in his/her extensions.

Particularly, it provides several methods:

  • `downgradeExpressionType(node<EnumerationDeclaration> enumeration, node<Expression> expression)` - transforms a given $expression$ of `enummember<$enumeration$>` type to evaluates it to a raw value of the originally evaluated enumeration member.

  • `upgradeExpressionType(node<EnumerationDeclaration> enumeration, node<Expression> expression)` - transforms a given $expression$ that evaluates to a raw value by means of the old declaration of $enumeration$ to evaluate it to an actual enumeration member that represents such a raw value.

  • `upgradeQueryReturnExpressions(node<EnumerationDeclaration> enumeration, node<IMethodLike> query)` - transforms a given $query$ body that evaluates to a raw value by means of the old declaration of $enumeration$ to evaluate it to an actual enumeration member that represents such araw value. Technically it collects all return clauses of the query and transforms them with `upgradeExpressionType` functionality.

  • `optimize()` - attempts to optimize the code around an expression that has been transformed with this the `EnumExpressionsMigration` instance. A general workflow with `EnumExpressionsMigration` follows these steps: it is instantiated at the beginning of a migration script, it uses the 3 methods described above to correct the user models and then the `optimize()` method is invoked to clean up the transformed code.

Example: Migration script for property constraints highlights the usages of EnumExpressionsMigration utility.

If a concept extends PropertyAttribute:

If a concept extends `PropertyAttribute` then there should be a migration that updates attributes of such a concept. The migration script has to invoke `EnumUsagesMigration#migrateEnumPropertyAttribute(node<PropertyAttribute>)` to update it. The method returns a new enumeration property declaration if the attribute is for the property of an enum type and the attribute hasn't been migrated yet. Otherwise, the method returns null.

The generator’s property macros is a good example of ‘PropertyAttribute#getPropertyDeclaration’ usage and property attribute migration.

Last modified: 07 March 2024