Kotlin Project Model Design Documentation 1.0-master Help

Core notions of Kotlin Project Model

Module

Module- a piece of software that should be fully defined in one place 1 (you cannot define part of the module, publish it and define other parts later) and is the primary unit of dependency management.

Being distributable is the defining property of modules, so each module has a public ID. It’s worth mentioning that “distributable” inherently adds dependency on the specific medium of distribution (Maven, NPM, Space packages, etc.), thus a lot of details, like the format of that ID, or how uniqueness of that ID is ensured, are left upon the particular implementation.

Simple modules tend to be in 1:1 mapping with VCS repositories. More sophisticated configurations might see multiple modules living side-by-side in one VCS repo: e.g., kotlin -repo has at least several modules: kotlin-test, kotlin-gradle-plugin, kotlin-compiler-embeddable, etc. kotlinx.coroutines ship multiple modules as well

Module consists of a fragments, some of which are variants

Module fragment

Module fragment: a smallest unit of grouping .kt-sources together.

  • All sources in one module fragment are analyzed with one and the same Compiler Settings. Quick examples of compiler settings include (but are not limited to): dependencies on other module fragments, language version, enabled features, set of targets for which this module fragment will be compiled.

  • Each fragment has Kotlin Attributes. They are inferred as an intersection (given by lattices of attribute values) of attributes of all variants this fragment contributes to 1
    • Consequently, there’s an induced is-compatible relation on fragments as well

  • Each Kotlin declaration belongs to exactly one module fragment
    • Therefore, module consists of a disjoint union of fragments (w.r.t. module sources)

  • Pragmatically speaking, has a folder with sources
    • Potentially might have several folders, but we haven’t discussed that deeply

The need in uniform compiler settings is very pragmatic and is dictated by requirements of our toolchain, which needs a clear context for each file at the very least. This means that a fragment can be as small as one file, but not smaller ( one file can not contain several fragments), hence in the definition we say “grouping of .kt-sources”, not “grouping of Kotlin declarations”.

Module fragments might have three kinds of outgoing edges:

  • Module dependencies

  • Fragment dependencies

  • Refines dependencies

See Dependencies section for details

Variant

Variant- a special case of fragment, which, together with its refines-closure, designates a set of complete API exposed by module. “Complete” here means that all expects received their actuals 1,2.

Notes:

  • Each module has a set of its variants. Trivial module consists of a single variant (see more in Single-platform projects as a special case of multiplatform projects)

  • Each variant is contained in exactly one module

  • Represented as one or several Gradle variant in Gradle (we hide some non-interesting variants like api/implementation under the hood, hence 1:M mapping)

Dependencies

There are several kinds of dependencies in Kotlin Project Model.

Fragment dependencies

Fragment dependencies go from one module fragment to another and express that the start-fragment sees symbols of end-fragment.

Note that not all fragment dependencies are theoretically correct (intuitively, dependency from fragment with JVM-code to the fragment with JS-code makes no sense), so they should respect Kotlin Attributes and induced is-compatible relation on fragments themselves.

Currently, users can not declare fragment dependencies manually; instead they declare module dependencies which then expanded by our toolchain into fragment dependencies.

Refinement dependencies

Refinement dependencies (or “refines”-edges) go from one fragment to another in the same module1 and express that the start-fragment can provide actual s for expect s from end-fragment, as well as see all symbols ( including internals, see Internal visibility for further details).

Quite obviously, the start fragment should be more specific than the end one. This is as well given by the rules of matching Kotlin Attributes. Moreover, we’re limiting the refinement only to the platforms dimension of Kotlin Attributes, meaning that all other attribute values should be exactly the same (a.k.a you can’t have expects between debug and release) 2

We define "refines-closure" of a given fragment as a set of fragments, which a reachable from it via refines -edges. On the example below refines-closure of fragment js is shown (highlighted in red):

refines-closure

Module dependencies

They go from a Module Fragment to a Module. Module dependency from fragment F to module M means that F should see a suitable set of fragments from M.

The exact definition of the "suitable set of fragments" is given by the Module dependencies expansion algorithm. The simple intuition is that it is the largest set of sound API from the end-module. For example, if the F shares sources between JVM and JS, and M has JVM, JS and Native, then that set will include all fragments of M which participate both in JVM and JS, but won’t include those which are unique to Native, JVM or JS.

One can think of those dependencies as of dependency sugar: they are essential for user-facing dependency management because declaring all fragment dependencies is extremely verbose and error-prone, but mosts of low-level clients (IDE, resolution, etc.) need more fine-grained dependencies given by Fragment Dependencies

Also, it’s worth mentioning that module dependency is propagated to the refining fragments of the start-fragment. Indeed, in the example above, suppose F uses expect class E from M. Then, during compilation of the containing module to JVM classfiles, we’ll need a corresponding actual of E for JVM from M. In other words, the JVM variant in which F participates should receive a dependency on the respective JVM variant of M. The Module dependencies expansion algorithm ensures it.

Here’s the example of dependencies:

Example of dependenciess

Legend: rectangles are module fragments, black dashed arrow are refinements dependencies, coloured arrows are a module fragment dependencies. Names of fragments intuitively imply their platforms and corresponding attributes.

Kotlin Attributes

Kotlin attribute — meta-information about properties of the API.

  • Each attribute is a key-value pair, where key is an uniquely identified attribute class, while the value is one of attribute values defined by that class.

  • Attribute values form a complete lattice

  • There’s an is-compatible relation of attributes, which induces similar relation on variants

One of the most important attribute classes is Kotlin Platforms, which represent across which platforms the respective code is shared. Values are modelled as sets of platforms rather than single values, so, attribute class KotlinPlatform consists of “attribute values” like {jvm, js, linux}, {jvm}, {js, linux}, etc. The is-compatible relation is given by the subset-relation

Compiler settings

Compiler settings — the set of data which completely defines the compiler behaviour. Semantically, it consists of frontend settings (which define how the code is analyzed) and backend settings (define how the resulting binary is compiled/linked)

Compiler settings consist of:

  • Fragment and refines dependencies

  • analyzer settings like language version, enabled features, etc.

  • other compiler keys and arguments, like -Xjvm-default mode or -Xjsr305 mode
    • including compiler plugins and their arguments

  • notably, set of targets for which this module fragment will be compiled. Examples: {JVM 1.8, JS}, {Android, iOSx64, iOS ARM 64}, etc.

Compiler settings represent low-level abstraction, i.e. users are usually not supposed to form a complete pack of compiler settings manually. E.g., fragment dependencies are too error-prone to configure manually, and instead are inferred based on module dependencies

Compiler settings consistency

Compiler settings can’t be chosen entirely deliberately. If fragment A refines 1 fragment B, then, for example, new language feature can not be enabled in B but disabled in A. Indeed, A sees declarations from B which potentially need that language feature to be analyzed/compiled properly.

Therefore, we get an is-consistent relation on compiler settings. Basically, it's decomposed to is-consistent relation of values of each particular compiler setting. As this relation is essentially a partial order, it makes sense to speak about monotonic increase/decrease of compiler settings along the ordered array of fragments in a variant, from the most common one to the most specific one.

To define is-compatible relation, we divide all language settings into several kinds depending on how they restrict fragments which are in refines-relation:

  • DONT_CARE — presence of this settings doesn't affect other fragments in any way.
    • Example: any feature that changes how bodies are generated — deprecations, semantics changes (ProperForInArrayLoopRangeVariableAssignmentSemantic ), etc.

  • MONOTONIC — states of this setting are ordered, and we demand that for A refines B it is necessary that state of such setting in A <= (>=) state of such feature in B. In other words, state of this setting should be monotonic (increasing or decreasing, depending on feature) along any path formed by refines-edges:
    • Example: language version should monotonically non-increase along any path, otherwise child fragments may struggle to read parents' binaries.

    • Example: (actually, more general formulation of the previous point) if some LanguageFeature enables new language constructs (e.g. InlineClasses ), it should monotonically non-increase along any path. In other words, once such LanguageFeature is enabled, it must be enabled in any dependent fragment.

  • CONSTANT — we demand that for A refines B it is necessary that states of this setting both in A and B are equal. In other words, such settings must be the same along any path <=> it is a project-wide setting.
    • Example: if fragments use different states of JSR-305, then they can see binary artifacts differently, leading to various inconsistencies (to be honest, we haven't thought of exact counter-examples, but this configuration seems complex enough to be forbidden, at least until we find a good reason to allow it)

Because the relation forms a partial order, and thus is transitive, the check of correctness of compiler settings can be performed in O(E) complexity where E is an amount of refines-edges in a module.

Last modified: 24 June 2021