Overview

A musing about my enjoyment of Scheme and having to work a little harder.

Unfamiliar with Scheme?

Side note for those who aren’t familiar with Scheme:

Scheme is an umbrella term for many languages. Most of them try and follow one of the official standards like R5RS, but some go off on their own and explore completely different paths like Racket. Schemes are considered dialects of Lisp, but Lisp too is a term for a collection of languages. I’ll use the terms somewhat interchangeably below.

Please also see the bit at the end before complaining about the parentheses to me.

Old Cycles & New Learnings

I’ve been coding professionally for just shy of 30 years now. All of the professional work has been web development.

At its core web development is taking form submissions, sticking them in a database, taking the stuff in the database and showing it on a web page to read, or a web page with a form to edit. That’s it. I don’t care if you’re using Rails, or Django, or Phoenix, or some homebrew creation. The back end is essentially the same. It’s not until you start approaching Google scale that what you do, or how the infrastructure supports that begins to change.

Most of us don’t deal with requests at that scale. Most of us are coding minor variations on the same thing again and again. I suspect that it’s the same for back-end developers in any vertical, be it mobile games, or radar systems. Even if you switch languages it’s the same stuff with slightly different syntax most of the time.

I suspect that’s why so many of us work on hobby projects, or open source libraries even though they’ll eat up our free time, and are unlikely to ever make us money. Those of us who love programming, love the problem solving, and figuring out how to express an idea with in a different set of constraints.

Batteries Not Included

A programming language that has “Batteries Included” is one that has a large / versitile “Standard Library”. Languages like Ruby could be said to come with not only “batteries”, but a spare generator, & solar panels. It’s got a web server, JSON parser, HTTP support, and lots more.

Scheme’s aren’t like that. Some Scheme’s - like Racket and Chicken Scheme - have a thriving ecosystem of third party libraries, but with the exception of the SRFIs¹ those libraries are frequently written in ways that make it as difficult, or awkward to use them as to write your own implementation.

In the end, you end up needing to write a lot of your own “batteries”, and that means thinking about “simple” things that you’ve been using for years, but probably never thought about before.

Forced Thought

This is where Scheme comes in for me. Scheme is considered a multi-paridimatic language. It’s not explicitly functional, and there are various Object Orientation libraries that have been written for them. It’ll let you write code in a very OOP-like fashion, but everything seems to work a bit more smoothly if you embrace a more Lispy approach.

For most of us that means a different approach. For example, by default Scheme doesn’t have an method to break a loop or return early from a function, at least not like most languages². This isn’t a bug, or an oversight. You just don’t need them when you start thinking in a Lispy Way.

Lisp, and by extension Scheme, has a historical mystique to it. It’s a fundamental part of the history of machine learning and programming language creation.

I learned it because - at the time - I hadn’t done any functional programming and I wanted try something that would give me new challenges & force me to learn new ways to solve problems. Common Lisp³, & Scheme gave me that. Those learnings helped give me new solutions to problems in my day job too.

But I’ve been finding real joy - and frustration - in having to implement little things that we take for granted in other languages. For example, yesterday I needed to extract the first, or last N elements of an array. In Ruby I could do something simple like this.

long_array  = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
array_start = long_array[0,3]  # => [1, 2, 3]
array_end   = long_array[-3..] # => [8, 9, 10]

Scheme has a lot of list manipulation functions, but it’s also missing a lot of things we take for granted, so when I didn’t find a function to do this (see below), I figured I had to write my own. This is frustrating when you’re trying to get something done quickly, but it can also be joyful in that it makes you stop and think about simple things. “🤔 how do I efficiently select a subset of a list?”

My solution uses tail recursion, and Scheme’s ability to define a function within a function which lets me expose a wrapper function (first-n) to the world that doesn’t force the users to deal with passing in an accumulator, and doesn’t add a second top-level method to this module that future me will have to think about separately. It also doesn’t use any loop / iteration functionality. It’s radically different than how I’d solve the same problem in Ruby, but I think it’s also a pretty Lispy solution, that runs quickly.

  ; returns the first n elements of the list
  (define (first-n number a-list)
    (define (inner-first-n number inner-list accumulator)
      (if (or  (eq? (length accumulator) number)
               (null? inner-list))
          accumulator
          (inner-first-n number
                         (cdr inner-list)
                         (append accumulator (list (car inner-list))))))

    (if (> (length a-list) number)
        (inner-first-n number a-list '())
        a-list))

Scheme’s got a lot of oddly named functions, so occasionally you’ll not find a built-in one just because you don’t know what keyword to search for. In this case, I didn’t know the built-in methods for this behavior were called take and take-right, and I’d never heard anyone refer to that behavior as “taking”.

Even when I found those, I had to modify things, because those methods don’t behave well when you pass in a list that’s shorter than the number you’re trying to take, and Scheme doesn’t have built-in try/catch exception handling like most languages⁴. There are libraries to do it of course, but the point is that even when there is a “battery” - built in or 3rd party - you frequently need to futz with it.

  ; returns the first n elements of the list
  ; or the entire list if its length is < number
  (define (first-n number a-list)
    (if (<= (length a-list) number)
        a-list
        (take a-list number)))

Thinking about this as I write it, this sounds terrible. Admittedly, sometimes it is damn annoying having to write so many wee low-level functions.

I’ve got a module that just a bunch of list manipulation functions to let me easily implement common tasks that aren’t built in, like sorting a list of strings. Every one of those represents a little bit of learning, a problem that I’d never have to solve in my day job.

Each one of those is also written in a way that matches my brain. That too is one of the arguments for, and against, Lisps. Codebases, it is argued, tend to end up looking like DSLs where everything is a function tailored to the specific purpose of the app. You can end up with a codebase that makes weirdly specific things incredibly easy, but also requires a longer spin-up time for new team members because so much of it is written using custom functions instead of generic “batteries” from the language.

Parting thoughts

When starting a new project, ask yourself if you want to get something written quickly, or if you want to learn new things. If you want to learn new things, maybe checkout a language like Scheme.

If you’re interested in learning a Scheme I’d highly recommend you check out Racket. It’s got a ton of 3rd party libraries so you don’t have to write everything from scratch, and the documentation of them tends to be very high quality. Chicken Scheme’s pretty good too, with a good ecosystem of 3rd party libraries⁵ but I find the documentation of most libraries to be simultaneously comprehensive, maddening, and nearly useless. For example, every function listed with a sentence or two describing what it does, but each description is based on a pile of domain knowledge that people new to the library, or new to the language don’t have.

Post Script About The Parentheses

Whinging about Lisp’s parenthesis is a lot like being fatphobic. It’s claiming something is “bad” or “lesser” because you don’t like how it looks without any real understanding of why that particular person / thing is the way it is.

The only people who whinge about Lisp’s parentheses are people who don’t code Lisp. Not only that, but Scheme is so damn flexible that there are multiple implementations that don’t use parentheses. Wisp (Whitespace to Lisp: SRFI-119), for example, uses indentation levels like Python. The Readable Lisp S-expressions Project uses parentheses, indentation, and curlies and resembles C.

So it’s not just “yucking someone’s yum” because you don’t like how it looks, it’s also really ignorant because you don’t even need to use them.

To me, Lisp’s parentheses are little hugs that keep my codey thoughts safe and secure. They help Lisp to be a homoiconic language, and every decent editor makes them easy to work with, and helpful. So stop trying to get rid of my hugs!