Arrow 0.7 - A Quality of Life upgrade

After release 0.6 brought the merger of funktionale and Kategory, we received lots of feedback about the rougher edges of the library. Working on these changes meant completely breaking binary compatibility with older versions of the library, and not providing a great migration path other than going through your current code and fixing the changes. We understand this will disrupt your codebase in the short term, although it is for the better in the long term to help early adopters and future newcomers. We're thinking now about what release 1.0 will look like and how many library-wide changes needed to happen. We feel that we should make any breaking changes as early as possible.

With that out of the way, let's talk about the pain points we've addressed in this release:

• Type Theory oriented naming of some of the constructs, like ev(), -HK, or -Kind
• The magic around annotation processing and global typeclass lookup
• How to seamlessly use typeclasses in your existing code

Let's address each individually.

The Renamening

The names were originally taken by the authors due to familiarity with the theory, and we acknowledge they would limit adoption. To improve the current situation, several maintaners sat down and did multiple passes on renaming these concepts that sound alien to newcomers. We're happy to present the final result:

• We're suffixing existing class wrappers with K instead of KW. Our wrapper for List is ListK, Observable becomes ObservableK... you get the idea. To wrap any available type we provide global k() functions for each. Note that after release of Kotlin 1.3 we'll make these wrappers into inline data class so they'll have 0 overhead.

• A container or kind is any class that has one generic parameter and implements the empty interface Kind<F, A> for their respective For- tag class. So ListK<A> implements Kind<ForListK, A>. To cast from Kind<ForListK, A> to ListK<A> you have to use the function fix() that's available at a global scope.

• To refer to a container (a.k.a. kind), you'll use a tag class with the prefix For-. This way you will define a Monad<ForListK>, a Traverse<ForEither>, or even Async<ForObservableK>. Note that some typeclasses don't require using tag classes, for example Eq<ListK<A>>, or Show<ListK<A>>, although they're easy to spot once you fail to implement them.

• We provide a typealias for Kind<F, A> using the name of the container suffixed by -Of. So, a OptionOf<A> is an alias for Kind<ForOption, A> that you can safely cast to Option<A> using fix().

• The constructor for a single value pure is now called just, which aligns better with existing Kotlin libraries.

• MonadSuspend is now MonadDefer. The function suspend is going to be added to the Kotlin Standard Library, so we needed to rename to avoid conflicts.

Some examples in context:

sealed class Option<A>
: OptionOf<A> { ... }

fun fetchUser(ME: MonadError<ForOption, Throwable>, id: UserId)
: Option<User>

fun OptionOf<A>.ifNoneReturn(f: () -> Option<A>)
: Option<A> {
  val fixed: Option<A> = fix()
  return fixed.fold(f, ::Some)
}

When working with generic and abstract functions that use typeclasses it's usual to have -Of returns you need to fix(). Because of that, as a rule of thumb we recommend declaring extension functions on typealiases -Of because they will work with both Option and OptionOf. For return types you can always use Option<A>, 'cause math :D

val optionA: Option<Name> = Option.monad().binding {
  val company = lookupCompany("1").bind()
  company.name
}.fix()

val optionB = Option.monad().binding {
  lookupCompany("1").bind()
}.ifNoneReturn { Option.just(Company("Default")) }
.map { it.name }

Adios, global typeclass lookups

Ever since we started developing Arrow we wanted to emulate global lookup for typeclasses, the same way Haskell does it and Scala implements using implicits.

Raul concocted a smart solution using reified types and map lookups, similar to how other languages do it under the hood. Sadly, due to type erasure and other limitations the implementation was not safe. You could request an instance of a typeclass that couldn't exist and crash at runtime — this was not acceptable for the philosophy of the library. Another problem was that this lookup could not be used in places like interface methods or open classes. It also had issues keeping track of typeclasses in external libraries. Lastly, it was tied to the JVM, which would limit any future KotlinJS and KotlinNative versions of the lib. In short, it did not scale!

In version 0.7 we have removed all traces of global lookup. That includes all calls to the global lookup using a method with the typeclass name. So inlined reified functions like applicative(), monad(), monoid() and the others are just gone from the codebase. You can get individual typeclasses from the companion object of the datatype that implements them, like Option.monad() or Int.monoid().

So, what have we replaced global lookups with? As KEEP-87 is still under consideration by Jetbrains until proven by a proof of concept, it requires vast amounts of work in the compiler and the roadmap is still unknown. It'll be months, maybe even a year, until it is completed if it's only a handful of Arrow maintainers involved. It's been known in the community that the compiler is a largely undocumented, ad-hoc codebase, and we're just a few mavericks. Any help from experienced compiler engineers is heavily appreciated, so ask your friends!

Meanwhile, the Arrow maintainers decided for a transitional solution. Using scoped extension functions and some methods in the standard library, there is a solution that is both idiomatic and ergonomic. It is what's been named the Typeclassless technique, described in this blog series.

A classless rewrite

All typeclasses in Arrow have been rewritten to behave more like Kotlin DSLs. As you know, a typeclass is a set of functionalities defined for a type or a constructor For-. So, all methods inside a typeclass will have one of two shapes:

• Constructor: functions that create a new Kind<F, A> from a value, a function, an error... Examples are just, async, defer or binding.

• Extensions: they add new functionality to Kind<F, A> so they're now defined as extension functions.

All typeclass objects can now be run to access their corresponding functions and helpers.

fun fetchUser(ME: MonadError<ForOption, Throwable>, id: UserId)
: Option<User> = 
  ME.run {
    just(id)
      .flatMap(::validate)
      .flatMap { fetchForDatabase(it) }
      .handleError { User.NoUser }
  }

You can still call the constructors directly over the typeclass object, including access to Monad Comprehensions using binding.

fun fetchUser(ME: MonadError<ForOption, Throwable>, id: UserId)
: Option<User> =
  ME.bindingCatch {
    try {
      val validId = validate(id).bind()
      fetchForDatabase(validId).bind()
    } catch (t: Throwable) {
      ME.just(User.NoUser).bind()
    }
  }

These changes open the door to a refactor that makes every function an extension function over the typeclasses they require, effectively removing one or multiple parameter calls from every function.

fun <F> MonadError<F, Throwable>.fetchUserGeneric(id: UserId)
: Kind<F, User> = 
  bindingCatch {
      val validId = validate(id).bind()
      fetchForDatabase(validId).bind()
  }.handleError { User.NoUser }

fun <F> MonadError<F, Throwable>.fetchAllUsers(ids: List<UserId>)
: List<Kind<F, User>> = 
  ids.map { fetchUserGeneric(it) }

fun <F, TC> TC.requestAllUsers(id: List<UserId>)
: Kind<F, List<User>> 
where TC: MonadError<F, Throwable>, TC: Foldable<ForListK> = 
  fetchAllUsers(id).k().sequence(this)

So now you can request all users for any implementation of MonadError, but you don't have to commit to it until you're atop of you call chain!

fun MonadError<F, Throwable>.rauObject() = 
  object : MonadError<F, Throwable> by this,
            Foldable<ForListK> by ListK.foldable() {}


fun main(args: Array<String>) {
  Option.monadError().rauObject().requestAllUsers(args).fix()
  // Option<List<User>>

  IO.monadError().rauObject().requestAllUsers(args).fix()
  // IO<List<User>>

  ObservableK.monadError().rauObject().requestAllUsers(args).fix()
  // ObservableK<List<User>>
}

Mindblowing, eh? If you'd like to know more about the Typeclassless encoding and how to do Dependency Injection with it, you can read the blog series on pacoworks or the dependency injection page of the Arrow docs.

Generic Metaprogramming

One of the new modules, which is still experimental, is arrow-generic. This module focuses on code generation of tranformations between types with the same structure.

For the first release you can annotate any data class with @product to get a set of functions to allow lossless conversion between your data class, tuples, and heterogeneous lists.

@product
data class Account(val balance: Int, val available: Int) {
  companion object
}

Account(1000, 900).tupled()
// Tuple2(a=1000, b=900)
Tuple2(1000, 900).toAccount()
// Account(balance=1000, available=900)

Account(1000, 900).toHList()
// HCons(head=1000, tail=HCons(head=900, tail=arrow.generic.HNil@3b307f1c))

We use these transformations to enable new use cases, for example to automatically generate instances for some typeclasses like Eq, Monoid, and Show; and smart constructors using Applicative like shown below.

val maybeBalance: Option<Int> = Option(1000)
val maybeAvailable: Option<Int> = Option(900)

Option.applicative().mapToAccount(maybeBalance, maybeAvailable)
// Some(Account(balance=1000, available=900))

Looking forward to 0.8

While these are the major changes for version 0.7, there are still some things we would like to tackle on the next version. Some of the improvements we have in mind are:

• A refactor of binding to allow non-blocking operations
• More Optics! More Laws! More Typeclasses!
• Removing all remaining deprecated APIs
• Improving the codata stack (coreader, cofree, comonad comprehensions...)
• Making the arrow-recursion package stable

Meanwhile, you can contact us on Gitter and the #arrow channel in the Kotlinlang Slack. See you there!