Drako's Developer Blog

My thoughts on programming

Introduction to Coroutines – Part 2

Welcome back to my series on coroutines. Today we will have a look at the only two higher level functions we get from Kotlin directly: buildIterator and buildSequence. As one can be implemented calling the other (which is actually done in the original implementations) we will only look into buildIterator.

buildSequence is really just:

Here we also see, that coroutines can get a receiver, which is pretty important for our two functions.

Usage

First, let’s have a look at how these functions are used.

As we can see our receiver provides us with two more functions: yield and yieldAll. These suspend our coroutine until the next value from the sequence/iterator is queried. So all in all we get a lazily generated sequence/iterator. These two are actually pretty similar to generators in Python (which uses a yield keyword).

Here is another practical example with a potentially infinite generator:

Restrictions

Sadly yield and yieldAll are the only suspending functions we can call in our coroutine scope. This has two reasons:

  • Functions like delay shown in the previous article require the coroutine context to provide certain features.
  • SequenceBuilder (our receiver) is annotated with @RestrictsSuspension, which prevents us from calling any suspending functions other than the ones it provides.

The second one sounds like an arbitrary restriction, but is rather useful considering the first one. For example calling delay in a coroutine context which does not support it, would result in errors at quite confusing locations. Therefore just telling the developer, that the function cannot be called in this scope is way more helpful.

Actually we can also define custom suspending functions that can be called. These need to be extension functions to SequenceBuilder though which makes them inherit the restriction. Therefore they also may only call yield, yieldAll and other extension functions of SequenceBuilder.

Implementation

From the “Usage” section we can take how our API needs to look and behave. So let’s start with the sequence builder:

As we want the values in our built iterator to be lazily generated we will need some kind of state machine:

So the flow will be like this: Our iterator starts out “Not Ready”. When we call hasNext the first time, the coroutine would be resumed. If we then called yield, the state would switch to “Ready”, we would store the yielded value, and true would be returned by hasNext. A following call to next would then return the stored value and switch the state back to “Not Ready”. If we called hasNext and the coroutine reached its end the state would be switched to “Done” and hasNext would return false. Calling hasNext on a “Done” iterator will then always return false, while calling next will throw a NoSuchElementException.

So here is an implementation of the aforementioned concept:

As you can see it was possible to transfer the state machine model to code quite easily. The following implementation of buildIterator is pretty trivial and mostly equivalent to the original:

Working example

Let’s return to the examples from the start. There, the original functions from the Kotlin standard library are imported. Now we’ll see our own implementation in action:

See also

We saw that the functionalities of buildIterator and buildSequence are quite trivial. We also learned, that they provide an easy way to create lazily generated sequences/iterators. But we also saw their limitations in the context of more complex coroutine features like asynchronity and concurrency. As a more feature complete alternative kotlinx.coroutines provides us with produce, which returns a ReceiveChannel. Such a channel is basically a sequence on steroids.

NPEs in Kotlin

While Kotlin comes along with all its nice null-safety features like nullity encoded in the type system, smart casts, safe calls and the elvis operator, there are still situations where you might encounter a NullPointerException (NPE), some of which are pretty obvious, while others are very subtle and can give you real headaches. The goal is to prepare you, so you have a better guess at what might be the source of your current NPE.

The not-null assertion operator (!!)

The most obvious one, and actually the only one in pure Kotlin I could come up with, is forcing Kotlin to treat a nullable as if it was not.

The double exclamation mark (a.k.a. the not-null assertion operator) tells Kotlin to treat the value as non-nullable and if it actually is null, throws an NPE.
There are two situation to use this:

You know more than the compiler

This would for example happen if you are working with a container. Getting something out of a container could usually return null, as the container might not contain the requested item. But sometimes you control every part of that container, so you can be completely sure, the item is there. In that case the operator can be used safely.

You actually want an NPE if the value turns out to be null

You might be at a location in the code where you need a value and a null should really not appear there. You also can’t react to null in a reasonable way, so you force the NPE for some caller to handle the error.

Reading and writing platform types

Most NPEs will be results of interfacing the platform (e.g. JVM, JS, Android). While Kotlin supports a lot of annotations to determine whether parameters, fields or return types are nullable, this requires the used library to make use of these. If the nullity is not stated, Kotlin does not make any assumptions about it and uses so called platform types. In compiler output they look like Type! (notice the single exclamation mark). It is not possible to create variables in Kotlin with a platform type other than capturing the result of a platform function in a variable with automatically deduced type (i.e. without explicitly stating a type).

In the following examples I am talking about Java functions, but the same behaviour can be assumed for other platforms.

Return types

When calling a Java function returning a platform type, Kotlin will trust us, when we assign the value to a variable:

The danger in the first assignment comes from the fact, that it is not obvious, that an NPE might be thrown. Therefore I prefer to do a forced conversion in such cases:

Parameters

When calling a Java function taking a platform type, Kotlin will also trust us on whatever we pass in.

Sadly we don’t have any way to make it more obvious, that the second version might throw an NPE.

Calling a Java function passing in the result of another Java function

Now this is a really subtle one. Especcially, because this one can throw an NPE even though the Java code would deal with a null just fine.

Why does this happen? Kotlin tries to prevent propagation of undesired null values. In Java you might face the problem, that a null occurred at some location where it should not have. Yet it is not checked our accessed there. Instead, it is passed to one function, then to another. Several levels down the call stack we then access it without checking and BOOM! we have an NPE at a location not even remotely close to the location where the faulty null came to life. Kotlin perfectly prevents that in itself, because the type always carries the nullity information. When interfacing with Java it atleast tries to help us by preventing the propagation of null.

So assuming passIntToJava supports null, how do we tell that to Kotlin? We need to capture the result of getIntFromJava in a temporary value of which we state the nullity explicitly:

Introduction to Coroutines – Part 1

Writing concurrent or asynchronous code can be quite difficult, reading it even more. But does it have to be that way? Let me introduce you to coroutines, a new and still experimental feature which simplifies your concurrent code by providing you with a nice syntax and high level abstractions that prevent misuse.

In this series of articles I want to provide you with some in depth knowledge about coroutines and how their fancy high level abstractions work. Some of the concepts might be a bit difficult to grasp at the start, which is completely normal, and you will eventually have a moment, where your mind is blown and it all becomes clear to you.

All the code snippets in this post are executable and can even be modified. So feel free to experiment around while reading through the article. The snippets utilize the embeddable Kotlin playground, which sadly does not support streaming output. Therefore running the snippets will not continuously display their respective output. Instead, you would wait for the full runtime of the snippet and then see the full output of the code at once.

Experimental status

Coroutines have existed in Kotlin since version 1.1 and are still experimental in 1.2.*. But don’t let that “experimental” scare you. Kotlin’s backwards compatibility guarantee also holds for them and by now they became pretty stable in terms of feature additions.

What is a coroutine?

A coroutine is a suspendable (a.k.a. resumable) function. For this to work the function needs to remember where it was suspended, so it can later be resumed at that point. Also suspending is something that coroutines actively do. That means a currently executed coroutine might figure that it needs to wait for something to be available. In that case it not only suspends itself, but also tells someone else when it wants to be resumed. This means we usually need some kind of runtime environment which takes care of resuming currently suspended coroutines when their time has come.

A very simple coroutine might look like this:

The first thing we notice is the suspend keyword. This makes the function a coroutine. Coroutines cannot be called directly by normal functions. Why is that? When a function calls another, it expects for the function to return a result, when it is done. However, a coroutine might not get executed to the end. You could even discard a suspended coroutine without ever running it to the end. So we actually need some code that would take care of running the coroutine and also handling the different states it might get into.

Other than that we might also notice, that despite the function being suspended at the delay (which means some other code might get executed during that time on this thread or another one), it reads like synchronous code. We don’t need to deal with promises and later explicitely waiting for them, nor do we have to care for manually switching execution contexts. Instead, we have nice syntactical sugar and a comfortable supporting library (kotlinx.coroutines) with high level abstractions.

For the rest of this and also for future posts, we will actually have a look at the inner workings to get a better understanding for them, as well as on the high level abstractions provided for us.

Running a coroutine

In its simplest form the runtime environment could look like this:
(You may also hit the plus above the snippet to see the rest of it.)

Now this extremely simple example only works because neither of the two coroutines (the lambda and the sayHello function) actually suspends. So our coroutines run perfectly synchronously, which means, when startCoroutine returns, we already have our result.

Resuming coroutines

For suspension to work we need several things:

  • Some kind of queue where we can store currently suspended coroutines.
  • Some runner which would execute said coroutines until all of them are finished.

Now how do we do this? A coroutine can suspend itself via the suspendCoroutine function. When a coroutine is suspended, we get a so called continuation, which is the current state of the coroutine. We can call resume on it, to continue the execution of the coroutine, or resumeExceptionally to inject an exception. To have our coroutine resumed at a later point in time we need to register that continuation in the queue mentioned above, so the runtime environment can resume it later on.

Let’s have a look at a very basic implementation for this:

When you run the example, the printlns should make it clear, at which points the coroutines are suspended and when they are resumed.

Conditional execution

To close this article let’s get back to the starting example. To implement the delay function we would need to support some kind of conditional execution. This means, we want to be able to suspend our coroutine and tell the runtime, that we want to be resumed, when some condition becomes true. In the case of delay we want to be resumed, after the given time has elapsed. Other use cases would be waiting for data to be available from some network communication, or reacting to some external signals.

Now this implementation is not really resource friendly as we basically do busy waiting, i.e. we are constantly running code checking whether we can continue any of our coroutines. This could be improved by registering for certain events and then having the program sleep until one of the events happens and the associated coroutine can be resumed. We will have a look into this in follow up posts, so stay tuned.

See also

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close