FP Complete


On September 28 we released 3.1 of the FP Complete public services. We’ve discussed this a bit already, but in this blog post, we want to share some of the more technical details of what we’ve added.

Laundry list of features

We’ve added a whole bunch of new features to FP Haskell Center. Here are some of the most notable:

Keep reading for a drill down of some of this list. And for some lower-level, GHC-facing bits of our work, jump ahead to the “faster” sections.

Open Publish model

As announced previously, we’ve decided to make FP Haskell Center even more open. Since our 3.1 release, all IDE features are turned on for all users, with and without a paid account.

The new FPHC terms & conditions no longer restrict the free community version to non commercial use. Even anonymous users can access the full spread of features in the IDE, from type checking to executable generation, and even Git repositories (though you’ll definitely want to log in to be able to use that properly).

There are two distinctions between our community and paid licenses now:

Additionaly, there are now no restrictions on how you license your published code. This means that although you give others the right to view your published code, they receive no implicit license to distribute your code or use it in derived works. You can still explicitly license your code under an open source license if you choose.

Autorun active code samples

Based on feedback (mainly from Edward Kmett), there’s a new directive in the code fence syntax in School of Haskell posts, which will auto-run the code snippet when the page loads. This will be especially nice for showing graphical web pages or to begin an interactive process. The code is written as:

``` haskell active autorun
main = print "Hello, World!"
```

School of Haskell improvements

The School of Haskell now supports adding Disqus comments to your own tutorials (yet again suggested by Ed).

To enable this, go into your account settings under “Profile” and you can choose:

☑ Use site default (currently: no comments)

☐ Disable comments

☐ Use FP Complete’s Disqus account

☐ Use your own Disqus account

Disqus account ID: [ ]

To use your own Disqus account, hit the “Use your own Disqus account” checkbox and put your account id in the box. Then go into your Disqus site settings under the “Advanced” tab and add fpcomplete.com under the “Trusted Domains” text box.

GHC 7.8, new libraries

We’ve updated our default compiler version to GHC 7.8. We still provide a 7.4 environment, but it is considered deprecated, and we recommend users upgrade to 7.8. In addition, as usual, we’ve updated our library set to the newest versions of upstream packages, and will begin releasing unstable package snapshots next month.

Defer type errors

Together with GHC 7.8 come some new compiler features. For the most part, these are simply available to you without any IDE support. We added one nifty feature though: You can now defer type errors using the checkbox in the Messages tab:

☑ Enable suggestions ☑ Enable warnings ☑ Defer type errors

This will change type errors into warnings which are instead thrown at runtime. A very useful way to keep getting type information when you have a simple type error elsewhere in the code.

This plays in very nicely with the new Type Holes feature. Now you can program Python in Haskell!

Improved root & filtering

In the “Root & Filtering” tab in Settings, there is a way to change the root directory of your project, this is similar to doing a cd in your terminal. This is useful if you have several projects in one repository and you want to just compile the modules within one directory, rather than everything in the whole repository.

You can also ignore certain files using the blacklist input, e.g. src/Main.hs will hide that file from the list. This can be useful for hiding things from compilation.

This feature is still considered experimental.

Faster

You should notice a definite improvement in responsive in the IDE. We’ve actually implemented many different changes to make this happen. Our network communication layer, for example, has less overhead involved, by removing some unnecessary parsing from our reverse proxy machines. We’ve also refactored quite a bit of our async architecture to provide better concurrency for backend actions.

However, the biggest change comes to how we interact with GHC. We use GHC for a few different things in the IDE:

Those first two bullets dovetail together rather nicely, though the second can take more time than the first. However, the third point doesn’t play so nicely with the other two. The issue is that, while GHC is running the bytecode, it can’t continue to compile new code. And this is definitely something we want to allow. For example, a common use case it running a web application in the IDE, while continuing to hack on it while it’s running in the background.

Before the 3.1 release, our solution was to have three copies of GHC running for each project: one to do a quick typecheck, one to extract type information, and one for running code. This meant you would get error messages quickly, but couldn’t always get type information right away. It also meant your project required much more memory, which overall slowed things down.

Our new architecture involves just a single GHC process running for each project. We load code into this process, and it typechecks, extracts information, and generates bytecode, all at the same time. The trick is what happens when you press run? We would like to use the bytecode already available in the current GHC process, without tying that process up on just running the bytecode.

The solution we ended up at is calling forkProcess. However, it wasn’t quite as simple as that. To quote forkProcess‘s documentation:

forkProcess comes with a giant warning: since any other running threads are not copied into the child process, it’s easy to go wrong: e.g. by accessing some shared resource that was held by another thread in the parent.

Sure enough, we ran into exactly this bug almost immediately. And fortunately, we have some very good news to report: it’s fixed! You can see more information in GHC trac tickets 9295 and 9296 (and check Phabricator for some great reaction gifs). These changes have been merged upstream into GHC. Since writing those patches, we’ve stress tested our IDE significantly, and have no longer been able to reproduce any forkProcess bugs, which looks incredibly encouraging.

Note, though, that there are still some issues around forkProcess. If you’re planning on using it in your own code, you should be aware that you can still run into problems, especially with multithreaded code.

One other change we made was switching to a dynamically linked GHC process. We’d wanted to do this for a while due to improved memory usage. Ultimately, we had to make the switch due to a bug in C static initializers. Now we get the nice benefit that separate processes on the same machine can share memory space for their dynamic libraries.

Subscribe to our blog via email

Email subscriptions come from our Atom feed and are handled by Blogtrottr. You will only receive notifications of blog posts, and can unsubscribe any time.