Safer Exceptions in Scala 3

I mentioned in What’s Changed Since Scala 3.0.0 that Scala 3.1 introduced an experimental feature for safer exceptions. Let’s explore it in more detail.

Ornament, © 2021, Dean Wampler

Java’s checked exceptions were an attempt to express in method signatures the possibility that an exception might be thrown. While laudable in theory, in practice the requirement to add throws clauses everywhere or handle the exception “quickly” became burdensome, leading most developers to avoid using checked exceptions. Unchecked exceptions, which are RuntimeException and its subclasses and Error and its subclasses, became the norm in most Java code.

I think part of the problem was unprincipled use of exceptions in software designs, but I’ll come back to that below.

Using unchecked exceptions, which is basically how Scala has treated all exceptions, eliminated the tedium, but did nothing to make exceptions part of the type signature for methods. This is a big practical hole, as far as program correctness is concerned.

Instead, the Scala community has encouraged handling “exceptional” cases monadically. For example, Map.get(key) returns an Option[V], instead of a V (for a value of type V). None is returned if a value doesn’t exist for the key, instead of throwing an exception.

More generally, Either[E,V] has been used as a return type with the convention that returning Left[E] holds an error E, such as an exception, while Right[V] returns a successful value of type V. Try[V] works similarly.

Among other benefits, the monadic approach treats errors as first class values using the same method return (i.e., stack popping) infrastructure used for normal return values. Discontinuous stack jumps used for exception throwing are avoided. The caller must examine the Option, Either, or Try to determine success or failure and take the correct action.

Still, you can argue there are times when exceptions are useful for jumping out of a low-level method to some higher-level method that knows what to do if an exception occurs.

Scala 3.1 introduced an experimental feature to add declaring of checked exceptions back into Scala, but use a different mechanism to ensure they are handled.

Let’s walk through an example. First consider the following code:

This simple program takes a list of file names and prints their sizes in bytes, or throws an exception for each one that doesn’t exist.

We need the @experimental annotation on any type or individual method that uses an experimental feature, like safer exceptions. We enable the language feature with the import on line 5.

Note the declaration of openExistingFile, which has the comment // <1>. We can add a throws clause for this method that will throw a Java IOException if the specified file doesn’t exist. The return type explicitly indicates an IOException might be thrown.

The feature is actually implemented as an implicit capability, along with some syntactic sugar provided by the compiler.

The term capability is used in the following sense, the method is made capable of throwing an IOException. It is given this capability through an implicit (or using) parameter. In fact, the following two declarations are equivalent:

def openExistingFile(fileName: String): File throws IOException
def openExistingFile(fileName: String)(using CanThrow[IOException]): File

CanThrow is a new class in the library. If a given instance of it for IOException is in scope for calls to openExistingFile, then the capability is provided for the method to throw this exception. The compiler provides the syntactic sugar allowing us to write File throws IOException instead of a regular given argument list.

I picked the checked exception IOException for this example, because this feature only works with JVM checked exceptions. It is not intended to be used with unchecked exceptions.

As currently written, there is no given instance in scope, so compiling this code produces the following error:

[error] -- Error: .../SaferExceptions.scala:13:41
[error] 13 |    val file = openExistingFile(fileName)
[error]    |                                         ^
[error]    |The capability to throw exception java.io.IOException is missing.
[error]    |The capability can be provided by one of the following:
[error]    | - A using clause `(using CanThrow[java.io.IOException])`
[error]    | - A `throws` clause in a result type such as `X throws java.io.IOException`
[error]    | - an enclosing `try` that catches java.io.IOException
[error]    |
[error]    |The following import might fix the problem:
[error]    |
[error]    |  import unsafeExceptions.canThrowAny
[error]    |
[error] one error found

When introducing this feature incrementally, the easiest fix is the last one suggested, import unsafeExceptions.canThrowAny. This import provides a given that works for all checked exceptions. It is intended as a temporary fix, until you add the handlers, such as the suggested try/catch wrapper.

If you add the import and run it with arguments for a file that exists (say build.sbt) and one that doesn’t (badfile), you get this output (using sbt with Scala 3.1.0; see also the final callout below):

sbt> runMain ...SaferExceptions build.sbt badfile
[info] running (fork) ...SaferExceptions build.sbt badfile
[info] file build.sbt (.../build.sbt) has 4434 bytes.
[error] Exception in thread "main" java.io.IOException: badfile doesn't exist!
[error]   at progscala3.rounding.saferexceptions.SaferExceptions$package$.openExistingFile(SaferExceptions.scala:30)
[error]   at progscala3.rounding.saferexceptions.SaferExceptions$.main$$anonfun$1(SaferExceptions.scala:21)
[error]   at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error]   at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error]   at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
[error]   at ...SaferExceptions$.main(SaferExceptions.scala:24)
[error]   at ...SaferExceptions.main(SaferExceptions.scala)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / runMain) Nonzero exit code returned from runner: 1

We see that build.sbt is 4434 bytes long and we get a familiar stack trace for the unhandled exception thrown for badfile.

Doing the right thing™, adding atry/catch clause, looks like this:

The compiler generates a given CanThrow[IOException] for us, because of the catch clause with a case ioe: IOException. If you run this version, you get the following, more user-friendly output:

sbt> runMain ...SaferExceptions build.sbt badfile
[info] compiling 1 Scala source to .../target/scala-3.1.1-RC2/classes ...
[info] running (fork) ...SaferExceptions build.sbt badfile
[info] file build.sbt (.../build.sbt) has 4434 bytes.
[info] file badfile: IOException caught: badfile doesn't exist!

Incidentally, if you remove the throws IOException clause from the declaration of openExistingFile, you get the following compilation error:

[error] -- Error: .../SaferExceptions.scala:43:35
[error] 43 |    if file.exists() == false then throw new IOException(s"$fileName doesn't exist!")
[error]    |                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[error]    |The capability to throw exception java.io.IOException is missing.
[error]    |The capability can be provided by one of the following:
[error]    | - A using clause `(using CanThrow[java.io.IOException])`
[error]    | - A `throws` clause in a result type such as `X throws java.io.IOException`
[error]    | - an enclosing `try` that catches java.io.IOException
[error]    |
[error]    |The following import might fix the problem:
[error]    |
[error]    |  import unsafeExceptions.canThrowAny
[error]    |
[error] one error found

Looks familar. Now that we have enabled saferExceptions, the compiler won’t allow the method to throw a checked IOException unless we use the escape hatch import unsafeExceptions.canThrowAny or we add either the throws IOException or ausing CanThrow[IOException] argument list to the method signature.

Finally, what about nested calls?

If you try to declare wrapper without the throws IOException, it won’t compile. Hence, safer exceptions does not change the Java-style requirement that each method in the call chain must either declare it throws the exception or it must handle checked exceptions thrown by methods it calls.

What this feature does provide is the ability to declare checked exception can be thrown and it leverages the implicit mechanism to enable this capability, a mechanism that might be extended to other effects.

Are We Using Exceptions the Way that We Should?

So, we still don’t have a way to say something like, “this low level method I’m writing might throw a checked exception, but I want all methods above me in the stack to be oblivious to this possibility, except for some high-level method capable of handling the error, which I promise to provide…”

There are really two things wrong with this “quote”. First, it is impossible for the Scala compiler to enforce my promise, because the compiler doesn’t have the global program scope available. How could it, when we build libraries and other incremental deliveries?

I think the real problem, though, is the first part of this quote, where I give myself permission to do risky operations at a low level that might throw such an exception.

Could it be that the “pain” of checked exceptions is really a signal to us that our design is sloppy?

I’ve seen many large code bases where low-level code makes calls to external APIs (file systems, databases, cluster services, etc.), validates that some string can be parsed into an integer, etc., etc. There is little discipline about where such logic belongs and where it doesn’t. I claim this is the real problem that needs solving!

We need to discipline ourselves to write modules that only perform “risky” operations near their boundaries, at the beginning of the work they are asked to do, not deep in the call stack. Once you go below a few levels, the data required should be complete, transformed, validated, etc., so it’s nearly impossible to throw an exception (excepting OOMs and catastrophes of that sort).

This discipline would also force us to write “flatter” code by removing unnecessary layers.

A Few More Details and Where to Go for More Information

If a method can throw more than one checked exception, use a union type:

def m(stuff: Seq[Thing]): OtherThing throws E1 | E2 | E3

More details are described in this scala-lang.org page. Note that it currently contains an error; it uses an earlier syntax OtherThing canThrow E, where now throws is used.

The research work behind this feature is described in this paper (ACM citation). The first half is worth reading yourself for details I didn’t cover. The second half dives into potential improvements to Scala’s type system to address limitations of the current solution using the existing type sytem. It is more advanced reading…

This feature is too new to be discussed in Programming Scala, Third Edition, but I have added an example to the book’s code repo here.