In previous posts, I hinted that someday I would discuss concurrent evaluation of functions. That time has finally come. Also, I've made the switch to Java 1.7 at home, so don't be shocked by some of the syntax here (like multi-catch or try-with-resources).
Using that, it's quite easy to dispatch evaluation of a
Function0 to an
ExecutorService using a helper class:
This is a pretty powerful (but simple) pattern, since a
Function0 can be built up at runtime based on successive lazy applications of functions. (Effectively, the
Function0 type is a form of thunk.) Once you have several work units of decent size, you can dispatch them to background threads for evaluation, and then pass the resulting
Function0s (which are probably still running) to other lazy functions. Once you need a result (e.g. for output), you invoke
get() and (if necessary), your thread will block until the required background tasks have finished.
Let's see this in action:
These test cases download the two previous posts on this blog, counts their bytes (using a
foldLeft), and output the sum of the byte counts, along with the time taken to compute the result. The only difference is that
testConcurrentPageDownload kicks off evaluation of the two calls to
getPageBytes on an
ExecutorService. On my computer, the serial test typically takes about 1100ms, while the concurrent test takes about 600ms, suggesting that it takes about 500ms to download each page (they're pretty similar in size) and 100ms to count the bytes (since we're not using a particularly efficient data structure).
That's probably enough for now. I've got some code written that extends
Function0 to include asynchronous evaluation of callbacks when the result is ready. Unfortunately, I still haven't developed a good example that makes it look useful yet. Once that's ready, though, I'll write it up. Stay tuned.