New Release of Essence#: Golden Rule (Alpha Build 11)

The ‘Golden Rule’ release introduces Traits into Essence#. The name ‘Golden Rule’ continues the theme of using Biblical names for the Alpha releases (‘trait’ others as you would have them ‘trait’ you.)

Executive Summary

Traits are composable units of behavior that can be used by classes–or by other Traits–without regard to the class inheritance hierarchy. Traits are used for the same purpose as multiple inheritance, but do not suffer from the same flaws.

The new release can be downloaded from the Essence# site on CodePlex (which gets you the installer.) Alternatively, the code for the run time system (written in C#) can be obtained from theEssence# CodePlex site using Git, and the code for the Essence# Standard Library can be obtained from GitHub.

Traits–The Conceptual Model

A trait is like a class in that it defines methods, but unlike a class in that it cannot have or create instances and cannot inherit from a superclass (nor from a ‘super trait.’) Also, the current implementations of Traits in Squeak, Pharo and Essence# do not allow traits to define or use instance variables–but that may change: Traits that can define and use instance variables very probably will be implemented in Essence# at some point in the future.

Traits in Essence# are also namespacesjust like classes are. And they can (and normally should) be defined in an Essence# class library–also just like classes. Unfortunately, as of this writing, the documentation on the CodePlex site has not yet been updated to describe all the changes brought about by the introduction of traits. So this blog post will be the initial salvo attacking that problem.

Basic usage: If you want a class (or a trait) to”import” the methods defined by a trait, then send the message uses: aTrait to that class (or to that trait), with the source trait (from which the methods will be imported) as the message argument. The uses: aTrait operation is transitive: Except as explained below, all methods defined by the source trait will be imported by the receiver, along will all methods imported by the source trait from any other traits it uses.

Methods defined locally by the importing class (or trait) will not be overridden by the import (uses:) operation. On the other hand, any methods inherited by the importing class from its superclass which have the same names as those imported will be overridden. In other words, methods imported from a trait have higher precedence than inherited methods, but have lower precedence than locally-defined methods.

It makes no difference whether the uses: aTrait message is sent to a class before or after the class defines its own methods locally: The end result will always be the same. However, sending the message uses: aTrait multiple times is not additive: Each such message send has the effect of completely undoing the effects of any previous sends to the same receiver. Of course, if the same value is used as the message argument, then the end result will be the same as having sent the message only once (with that same argument value.)

With one exception, methods imported from a trait should be indistinguishable from those implemented locally by the importing class or trait. The exception is that methods defined in a trait bind to non-local variables (e.g, class variables) based on the namespace environment of the trait that defines the methods, and not based on the namespace environment of the class or trait that uses (“imports”) those methods.

In contrast, messages sent to super in a method defined in a trait will act as though they were defined locally in the importing class–in other words, the semantics of a message sent to super depend on the class that imports methods that send such messages, and does not depend on the trait where methods that send messages to super are defined. The same is true of messages sent to self.

Trait combinatorial algebra: Semantic issues arise whenever a class (or a trait) uses two or more traits–issues that are analogous to those encountered when using multiple inheritance. Traits solve those problems by means of algebraic trait-combination expressions which disambiguate the semantics of combining two or more traits together–doing so in a way that does not degrade the robustness or the flexibility of the code, unlike multiple inheritance.

The trait combinatorial algebra has three operators: +, – and @, which are implemented in Essence# by the binary messages #+, #- and #@.

The “+” operator constructs a new trait that is the symmetric set difference of the two operands (the term ‘symmetric set difference’ will be fully explained below, but it often means the same things as the union of two sets.)

The “-” operator constructs a new trait from which one or more methods have been removed (“excluded”) with respect to the trait that is the operand on the left-hand side of the operator.

The “@” operator constructs a new trait in which one or more of the methods have been renamed (technically, aliased) with respect to the trait that is the operand on the left-hand side of the operator.

Fundamental concept: Regardless of the complexity of a trait combination expression, and regardless of the number of traits combined or of the trait usage graph of any traits involved in a trait combination expression, the result is always a single (usually anonymous) trait that is dynamically created by the combinatorial expression, and which contains all the methods of all the traits involved in the combinatorial expression–including those methods directly or indirectly used by the traits in the expression, and excluding only those methods which must be excluded as a function of the semantics of the combinatorial expression. In other words, the result of a trait combinatorial expression is a flattened set of the all the methods defined directly or indirectly by any trait that occurs in the expression, although some of the methods may be renamed or excluded, as required by the laws of the algebra. That means that the user of a trait defined by a trait combinatorial expression always sees itself as using a single trait, with a single, unified set of methods. It does not see an “inheritance hierarchy” or “usage graph,” and so does not (and cannot) behave differently based on how the traits it is using may be structured. The only thing that matters is the end result of the combinatorial expression, not its form or structure.

Combination operator: Although the result of applying the fundamental combination operator “+” to two traits is often simply a new trait that contains all the methods defined and/or imported by both operands, that is not always the case. The reason is because the semantics of the “+” operator applied to two operands is symmetric set difference, and not set union. If–and only if–each operand only contains methods whose selectors are unique to it, and which the other operand does not contain, then the result of the operation is the same as it would be for set union. Of course, that will often be the case. But when it is not, the result of the “+” operation will be a new trait that contains all those methods contained by only one of the operands, but excludes any methods contained by both operands. That’s the semantics of symmetric set difference.

Note, however, that the symmetric set difference is computed based on all the traits in a combinatorial expression, and not computed serially between each two operands. That means that the combination operator “+” is technically not really a ‘symmetric set difference’ operator. Its actual effect is to compute a new trait that does not contain any methods defined in more than one of the traits in the combinatorial expression.

The reason that the “+” operator has that effect is simply because a) that’s the only way to make the combinatorial expression algebra commutative–and thus order independent, and b) it’s the only way to make the resolution of such conflicts a specific and explicit responsibility of the users of traits, and to do so in a way that has a transitive closure that never goes more than one level deep.

The purpose of the other two operators is to specify how to deal with the fact that the “+” combination operator excludes any and all conflicting methods (those defined by two or more traits in the same combinatorial expression.)

Exclusion operator: One way to handle such conflicts is to use just one of the conflicting methods and discard the other. That’s done using the exclusion operator (“-“), as in the following example, which combines the traits TReadStream and TWriteStream, but excludes the method #on: defined by TReadStream from the resulting trait combination (perhaps so that the #on: method defined by TWriteStream won’t be excluded because it would conflict with the #on: method defined by TReadStream):

ReadWriteStream uses: TReadStream – #on: + TWriteStream.

Multiple methods can be excluded from the same trait by using an array of method selectors as the argument to the #- message, instead of using a stand-alone symbol:

ReadWriteStream uses: TReadStream – #(on: #position) + TWriteStream.

Aliasing operator: The other way to handle such conflicts is to rename one or more of the conflicting methods, as in the following example:

ReadWriteStream uses: TReadStream @ {on: -> from:. #position -> #readPosition} + TWriteStream.

The example aliases the #on: method defined by TReadStream to the name #from: and aliases the #position method defined by TReadStream to the name #readPosition–but only in the anonymous trait that is the result of the combinatorial expression. The aliasing has no effect on TReadStream, nor on any other users of TReadStream. The argument of the “@” message must be an array of associations, where the association key is the name of the method in the source trait, and the value of the association is the name of the method in the trait that results from the expression.

Defining Traits In A Class Library

The Essence# class library format has been extended to enable the definition of traits. Note that there is a strong convention that trait names should be prefixed by a capital T, although there is nothing that enforces that convention.

To define a trait in a class library, create a folder whose name is the name of the trait to be defined, and which is located in the namespace where the trait should be declared, and create a text file in that folder named “trait.configure” and/or one named “classTrait.configure.”  It is an error to also have a file in the same folder named any of “class.configure,” “configure.class,” “metaclass.configure” or “configure.metaclass.” To define the instance methods of the trait, include them in a file (in the same folder) named either “methods.instance” or “instance.methods.” To define the class methods of the trait, include them in a file (also in the same folder) named either “methods.class” or “class.methods.” The file format is identical to that used to define the methods of a class or metaclass (as described in the documentation for class libraries on the Essence# CodePlex site.)

You might consider adopting a convention that traits should be defined in namespaces dedicated to traits. I don’t know yet whether that is what I will do myself. We shall see.

Note: Traits as implemented in Essence# are strongly congruent–but not identical–to their implementation in Pharo. For more details, please see the release notes on CodePlex.


Are we there yet?

If you’re waiting for the next release, the current ETA is Monday 30 June.

Implementing Traits has required multiple major refactorings of the C# code. Its impact has been both broad and deep. Even the class library format has to be extended (and the Library Loader has be to extended accordingly.)

The Essence# implementation of Traits will mirror the functionality and API of Traits as implemented in Pharo. That’s partly out of respect for prior art, partly for inter-Smalltalk compatibility, but most importantly because Professor Ducasse got the formalism right.

I also took into account the Pharo team’s work on Traits that define instance variables, so although the initial release won’t have that feature, adding support for it in the future should not be all that hard. As an example of that, in the next release the compiler will be able to use one Behavior-like object for binding to instance variables but another one for binding to global variables–which is necessary in order to have Traits that define instance variables. It’s also necessary to support Traits that have their own namespaces–which the initial release will in fact support.

Did you know? Essence# supports both Squeak-style dynamic array literals and Amber-style Dictionary literals.

Essence# has supported both dynamic array literals and dictionary literals from its inception. Those were both working from the moment the system first successfully compiled and executed a “do it” earlier this year (8 March 2014.)

Here’s an example of an Amber-style dictionary literal:

#{'one' -> 1. 'two' -> 2. 'three' -> 3}
                [:key:value |
                                write: 'key = '; 
                                write: key; 
                                write: ' value = '; 
                                writeLine: value]

And here’s how to run the above code using the ES script running tool:

es -d “#{‘one’ -> 1. ‘two’ -> 2. ‘three’ -> 3} keysAndValuesDo: [:key:value | CLR.System.Console write: ‘key = ‘; write: key; write: ‘ value = ‘; writeLine: value]”

New Release of Essence#: Lot-4 (Alpha Build 10)

The Lot-4 release is a combination of bug fixes along with a substantial refactoring of (the C# implementation of) the classes ESBehavior and ESMethod, and also of the AbstractSyntaxTreeNode classes. The refactoring was for the purpose of supporting the implementation of Traits.

Other than the bug fixes, the only change visible to those not directly dealing with the C# code is that instances of Behavior no longer support the specification of instance variables for their instances. That makes Behavior instances more like those in Squeak, Pharo or VisualWorks.

The new release can be downloaded from the Essence# site on CodePlex (which gets you the installer.) Alternatively, the code for the run time system (written in C#) can be obtained from theEssence# CodePlex site using Git, and the code for the Essence# Standard Library can be obtained from GitHub.




The next release will introduce “Traits” into Essence#

Seminal paper on Traits: Traits: Composable Units of Behaviour.

Traits are even more useful, and more needed, in Essence# than they would be in a typical programming language because of Essence#’s primary mission: interoperability with other languages, and especially with the .Net Base Class Library.  That’s because the .Net BCL was designed and implemented for statically-typed languages, where it is often either impossible or greatly inconvenient to use a common inheritance hierarchy for conceptually similar constructs. Traits provide the right mechanism to rectify that flaw. They also make it easier to provide an identical API for both native Essence# objects and for those provided by the .Net BCL.

New Release of Essence#: Lot-3 (Alpha Build 9)

Lot-3 is very probably the most robust and stable release so far published. It also has the most functionality. Since it is now 3 and a half weeks since the initial public release on 26 May, now would be  a good time to step back and review our progress:

Since the initial release (“Eden,”) we’ve greatly enhanced the site’s documentation, greatly enhanced the functionality of the ES tool that runs ES scripts, created an installation program using Nullsoft’s NSIS installation program generator, enhanced the Visual Studio solution and project configuration/build files so that it’s relatively easy to compile the Essence# run time system for any combination of .Net framework and processor architecture (Essence# can be compiled for any .Net framework version from 3.5 onwards, and can be compiled either as a 32-bit or as a 64-bit application,) added the capability to define Exception classes and to raise and handle Exceptions (including native CLR exceptions,) added the classes Set and OrderedCollection to the Essence# Standard Library (even the initial release already had arrays, Dictionary and IdentityDictionary,) significantly enhanced the functionality of Characters, Strings and Symbols, added many fundamental methods to the abstract classes in the Collection hierarchy, added many classes and methods needed to make instances of the CLR Collection classes look and act like native Essence# collections, made key improvements/enhancements to inter-language interoperability, fixed many bugs, and substantially improved the usability and robustness of the system.

This release extends the ability to pass blocks as arguments to CLR methods: Previously, that would only work if the static type of the corresponding parameter was a delegate whose return type and parameter signature exactly matched that of the block passed in as an argument–meaning the delegate’s return type had to be Object, all of its parameters had to have Object as their static typing constraint, and the name of the delegate had to match (e.g, “System.Func<>”.) In other words, the odds that it would work weren’t all that good.

But as of the Lot-3 release, the only thing that matters is whether or not the delegate and the block have the same number of parameters, and whether or not there is some way to type convert between the actual type of each argument at run time and the static type of the delegate’s parameter (and the same applies to the delegate’s return type and the value actually returned by the block.) And the delegate name is now irrelevant, meaning that a block with a single parameter can be passed in as an argument when the corresponding parameter type is Predicate<T>. So the following code will now work:

| list |

list := OrderedCollection new.

list addAll: #(3 5 8 13).

list find: [:x | x \\ 2 = 0] “The message #find: will bind to the CLR method List<T>.Find(Predicate<T>); it has essentially the same semantics as #detect:”

The class OrderedCollection is an Essence# class that represents and defines the behavior in Essence# code (and only in Essence# code) of instances of the CLR type System.Collections.Generic.List<Object>. In spite of that, OrderedCollection instances will respond to all the standard messages in the standard manner, as though they were fully native Essence# objects. And by the way, the class OrderedCollection resides in the default Essence# namespace, instead of residing in the CLR.System.Collections.Generic namespace. To see how that’s done, browse the code of OrderedCollection in the Essence# Standard Library.

The new release can be downloaded from the Essence# site on CodePlex (which gets you the installer.) Alternatively, the code for the run time system (written in C#) can be obtained from the Essence# CodePlex site using Git, and the code for the Essence# Standard Library can be obtained from GitHub.

If you were looking for the installation file for the Lot-2 release….

Apparently, there was initially no downloadable installer attached to the CodePlex download page for the Lot-2 release. In spite of the fact that it took 5 minutes to “upload” it, and presented no ‘failed to upload’ notification.

The problem is now fixed: The installation program is now attached to the Download page on the Essence# CodePlex site.