This post is part of our Bookshelf series organized by the Data Science R&D department at Civis Analytics. In this series, Civis data scientists share links to interesting software tools, blog posts, scientific articles, and other things that they have read about recently, along with a little commentary about why these things are worth checking out. Are you reading anything interesting? We’d love to hear from you on Twitter.
This blog post looks at performance on standard tasks in various #AI fields over time to try to characterize the impact of deep learning. The author “moderately confident(ly)” states that deep learning has been associated with “discontinuous jumps” in image and speech recognition performance but more modest improvements in fields such as natural language processing. This meta-review is limited by the very small and imperfect set of standard evaluations across AI-related research fields, but it’s interesting to consider which AI problems are getting close to human performance and which AI problems may eternally be “just around the corner.”
A Mind at Play is a thoughtful, comprehensive biography of Claude Shannon, a pioneer of the information age. My impression is that Shannon isn’t super well-known outside of particular fields like cryptography, so here’s a super brief summary: Shannon grew up in a small town in Michigan, wrote a Master’s thesis about Boolean logic and circuit design that serves as a foundation for modern computing, started the field of Communication Theory (a.k.a., Information Theory), defined the term “bit” as a unit of information, helped develop key technologies for digital audio, made theoretical contributions to cryptography, developed early artificial intelligence… and was an avid juggler and unicyclist! See also Gertner’s The Idea Factory about Shannon’s long-time employer, Bell Labs, or Gleick’s The Information.
New programmers often write duplicate code all over the place, making a difficult-to-maintain mess. A less well-appreciated phenomenon, it seems, is that not-so-new programmers learn that duplication is bad and try to abstract everything away so that no code will ever need to be duplicated. Unfortunately, this can also often lead to a difficult-to-maintain mess, though of a different kind, since the abstractions may not be the right ones for the next use case. The post linked above offers some clear thinking about this issue. There are lots of good heuristics like DRY and the Rule of Three. “Duplication is far cheaper than the wrong abstraction” is probably another one to keep in mind.
arXiv is a great place to find PDFs of the latest and greatest scientific research papers, particularly around AI, computational linguistics, machine learning, etc., but reading on a PDF can be a pain, particularly on mobile devices. The site linked above renders arXiv papers as web pages for your reading pleasure.
Nowadays, there are a number of options for implementing neural network / deep learning algorithms, but not so long ago, theano was pretty much the only game in town, unless you wanted to learn Lua. It greatly facilitated the implementation of complex statistical models via automatic differentiation, such that users didn’t have to explicitly implement complicated derivative functions, and it also enabled fast execution on GPUs. Other, more recent tools such as TensorFlow also have these features, but theano laid a lot of the groundwork. The post linked above is a few weeks old, but seemed worth calling attention to. It’s a bit sad to see development stop, but the theano creators can be proud of helping push the state-of-the-art forward for all of us.