18 May 2012

Scala Stream Tail Evaluation

There was an unanswered question from our discussion of Scala Laziness last night: Why do we need all that code for managing the evaluation of the tail of a Stream?

package scala.collection.immutable
 
object Stream {
  ... 

  final class Cons[+A](hd: A, tl: => Stream[A]) 
      extends Stream[A] { 
    override def isEmpty = false 
    override def head = hd 
    @volatile private[this] var tlVal: Stream[A] = _ 
    def tailDefined: Boolean = tlVal ne null
    override def tail: Stream[A] = {
      if (!tailDefined)
        synchronized {
          if (!tailDefined) tlVal = tl
        }

      tlVal
    }
  }
}

It seemed like passing the tail parameter using call-by-name might be sufficient. Here’s my first attempt at trying that out:

abstract class MyStream1[+A] {
  def isEmpty: Boolean
  def head: A
  def tail: MyStream1[A]
}

object MyStream1 {
  object Empty extends MyStream1[Nothing] {
    override def isEmpty = true
    override def head = { throw new Exception("Bad!") }
    override def tail = { throw new Exception("Bad!") }
  }

  class Cons[A](head: A, tail: => MyStream1[A])
      extends MyStream1[A] {
    override def isEmpty = false
  }
}

Uh-oh, the compiler isn’t happy with this definition of Cons:

class Cons needs to be abstract, since:
method tail in class MyStream1 of type => MyStream1[A] is not defined
method head in class MyStream1 of type => A is not defined

This Cons doesn’t implement head and tail as specified in the parent class. It looks like putting them in the constructor argument list isn’t enough. So let’s try this:

abstract class MyStream2[+A] {
  def isEmpty: Boolean
  def head: A
  def tail: MyStream2[A]
}

object MyStream2 {
  object Empty extends MyStream2[Nothing] {
    override def isEmpty = true
    override def head = { throw new Exception("Bad!") }
    override def tail = { throw new Exception("Bad!") }
  }

  class Cons[A](hd: A, tl: => MyStream2[A])
      extends MyStream2[A] {
    override def isEmpty = false
    override val head = hd
    override lazy val tail = tl
  }
}

OK, that compiles. Let’s try running this:

object Foo2 {
  def from(n: Int): MyStream2[Int] = {
    println("from(" + n + ")")
    new MyStream2.Cons(n, from(n + 1))
  }

  def main(args: Array[String]) {
    val ints = from(0)

    println(ints.head)
    println(ints.tail.head)
    println(ints.tail.tail.head)

    println(ints.head)
    println(ints.tail.head)
    println(ints.tail.tail.head)
  }
}

Here’s what we get:

from(0)
0
from(1)
1
from(2)
2
0
1
2

Cool, that seems to work. We see the side effect of the node evaluation the first time we visit each node, but not afterwards.

So we need the tail passed as call-by-name to delay its evaluation, and we need to override head and tail to match the definition of the abstract parent class. I used lazy val for the tail, so that it isn’t evaluated right away when the node is constructed. Try changing that to just val:

...
from(1538)
from(1539)
from(1540)
Exception in thread "main" java.lang.StackOverflowError

Oops! Now as soon as you create a node, the tail is evaluated, and its tail is evaluated, and its tail…

So why doesn’t the Stream implementation in the Scala standard library use lazy val for the tail? I suspect it’s an optimization because I don’t think lazy val would have any threading issues. Or it could be that the tailDefined attribute is useful elsewhere in the class.

26 April 2012

Recreational Programming

The number one sign of a professional programmer is a commitment to continued learning. That's probably true of many if not most professions these days. And a learning programmer, besides doing his/her day gig, probably has one or more side projects going at any given time.

I learned about Project Euler a little over a year ago. I had started learning Python at about the same time, and dove into the Project Euler problems as a means of learning Python. In retrospect, Python and Project Euler are a particularly good fit. Some of Python's language features, such as its arbitrary-precision integers, its generators, its iterator and numerical analysis libraries and its dynamic typing, make it especially well suited to cranking out solutions to these problems. Project Euler has proven freakishly addictive, and it is worthy of a separate, lengthy post sometime soon.

Python was my 2011 language, and Clojure is my language of 2012. The Chicago Clojure meetup recently held a fortuitous newbie-night, where they introduced us to 4clojure, a site with Clojure exercises that you can complete entirely within the browser. For each problem, 4clojure checks your code against a number of tests and tracks your progress. It starts with tiny baby steps, and introduces a new concept with each problem. I think it's a great way to get started. At this week's meetup, someone asked me what I'll do next with it. It definitely helps to have a goal. I may go back and redo some of the Project Euler problems in Clojure, or I may need to find another modest side project and commit to finishing it in Clojure.

Even more recently, someone introduced me to the Haskell School of Music. Another interesting language, Haskell, taught in the context of music and math. How perfect is that? It's so tempting to dive into that with abandon, but I've already committed to Clojure for the year. Maybe I can combine the two somehow, possibly using a Clojure library like Overtone.

30 January 2012

Regular Expressions

Regular expressions can be remarkably useful, but they can also be quite fickle. I use regular expressions in my code that scrapes the John Pizzarelli site to produce the Radio Deluxe RSS feed. There was one extra space in the HTML for the current episode that my pattern wasn't expecting, so that's why the 7 January episode hasn't been in the feed. It's fixed now.

03 January 2012

Blacklisted

Holy crap. I was writing an email to some colleagues earlier today about a filter feature in our application. I wrote something like, "Should we use a blacklist or a whitelist?" These are common terms in application development. Filters in various contexts are often implemented as either a blacklist (a list of "bad" values, which are then blocked) or a whitelist (only the permissible values are in the list, and all others values are blocked).

Then it hit me like a frozen fish on the side of my head. These terms are really, fantastically, politically incorrect. Black is bad, white is good. Holy crap. It had never occurred to me.

I changed the email to read "'good' list" and "'bad' list". I'll go with those terms until I find a better alternative. Unfortunately, blacklist is pervasive today.

Am I overreacting? I was well into adulthood when I discovered that the phrase "that sucks" was once vulgar. In my youth, it may have had some spice, but it was never vulgar to me. By adulthood it was bland. I say it all the time and I'll keep saying it. I'm confident that most IT professionals use the term blacklist without the slightest thought of its potential for offensiveness, as did I before today.

02 January 2012

Le Freak

My favorite present at Christmas this year was the Nile Rodgers bio Le Freak. I tore through it, and I'm usually a very slow reader. Any fan of pop music from the late 70's - early 80's will be interested in the details behind the production of several landmark recordings of the time: Bowie's Let's Dance, Madonna's Like a Virgin, and of course Chic. While I read, I listened to some of these albums with a fresh understanding. What I found even more interesting was the story of Rodgers' childhood and early career. It reminded me of the story of George Washington Carver -- there were any number of points at which one could not expect this person to survive at all, much less succeed.

The details become more sparse after the mid 80's -- has the statute of limitations not yet run out on those transgressions? -- but there is a moving account of Bernard Edwards' tragic demise. Meanwhile, Rodgers continues the story in real time on his blog, which is also fascinating.

01 January 2012

2012

Resolution #1: Write more.

If you're wondering why the Radio Deluxe feed hasn't been updated in a while, it's because they haven't updated the Radio Deluxe page since the Thanksgiving show.

I really hope they haven't stopped updating it because of the feed. I love the show, and they don't air it on any Chicago stations. I'd like to listen to it at work, and I can't stream it there. I buy the music of artists that I hear on the show.

We were in Germany in November. I listened to the show on the flight back home and I loved being able to do that. Someday, perhaps soon, streaming one's entertainment of choice on a trans-Atlantic flight will be commonplace, but not yet.