[Valid Atom 1.0] This is a valid Atom 1.0 feed.


This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.


  1. <?xml version="1.0" encoding="utf-8"?>
  2. <feed xmlns="">
  3. <title>saladwithsteve</title>
  4. <link href="" rel="self"/>
  5. <link href="" rel="alternate"/>
  6. <updated>2015-02-10T18:09:20+00:00</updated>
  7. <id></id>
  8. <author>
  9.   <name>Steve Jenson</name>
  10.   <email>[email protected]</email>
  11. </author>
  12. <entry>
  13.   <title>Building F# on Mac OS X</title>
  14.   <link href=""/>
  15.   <updated>2010-07-22T00:00:00+00:00</updated>
  16.   <id></id>
  17.   <content type="html">&lt;p&gt;I&amp;#8217;m attending the Emerging Languages Conference here in Portland,
  18. OR, where I ran into Joe Pamer on the F# team. I&amp;#8217;ve been following F#
  19. for a while but have had trouble running it on the Mac.&lt;/p&gt;
  20. &lt;p&gt;Joe says his team at Microsoft runs into these issues often with users
  21. and was kind enough to walk me through the steps to get F# running on
  22. the mac.&lt;/p&gt;
  23. &lt;p&gt;First install Mono, there&amp;#8217;s a &lt;a href=&quot;;&gt;dmg&lt;/a&gt;&lt;/p&gt;
  24. &lt;p&gt;1. Download the &lt;a href=&quot;;&gt;zip file&lt;/a&gt;
  25. 2. Run &lt;code&gt;;/code&gt;&lt;/p&gt;
  26. &lt;p&gt;This will complain about not finding &lt;code&gt;dpkg-buildpackage&lt;/code&gt; but the dll will be signed with the mono key and be ready to use.&lt;/p&gt;
  27. &lt;p&gt;&lt;tt&gt;
  28. Assembly bin/FSharp.Core.dll signed. &lt;br /&gt;
  29. ./ line 19: dpkg-buildpackage: command not found
  30. &lt;/tt&gt;&lt;/p&gt;
  31. &lt;p&gt;Now run &lt;code&gt;gacutil&lt;/code&gt; which will install the assembly into mono&amp;#8217;s library path.&lt;/p&gt;
  32. &lt;p&gt;&lt;tt&gt;
  33. sudo gacutil -i ./fsharp/bin/FSharp.Core.dll
  34. &lt;/tt&gt;&lt;/p&gt;
  35. &lt;p&gt;And now you&amp;#8217;re ready to write some programs! There are some known
  36. bugs around how it interacts with mono winforms (I think that&amp;#8217;s a
  37. thing, it&amp;#8217;s greek to me) so we&amp;#8217;ll turn that off.&lt;/p&gt;
  38. &lt;p&gt;Now let&amp;#8217;s fire up the repl, run an expression, and exit.&lt;/p&gt;
  39. &lt;p&gt;&lt;tt&gt;
  40. $ mono fsharp/bin/fsi.exe &lt;del&gt;-gui&lt;/del&gt; &lt;br /&gt;
  41. Microsoft&amp;#174; F# 2.0 Interactive build;br /&gt;
  42. Copyright &amp;#169; Microsoft Corporation. All Rights Reserved. &lt;br /&gt;
  43. For help type #help;; &lt;br /&gt;
  44. &amp;gt; printf &amp;#8220;hi\n&amp;#8221;;; &lt;br /&gt;
  45. hi &lt;br /&gt;
  46. val it : unit = () &lt;br /&gt;
  47. &amp;gt; #q;;&lt;br /&gt;
  48. &lt;br /&gt;
  49. - Exit&amp;#8230;&lt;br /&gt;
  50. &lt;/tt&gt;&lt;/p&gt;
  51. &lt;p&gt;Joe says that they are working on getting DMGs up for Mac users &lt;span class=&quot;caps&quot;&gt;ASAP&lt;/span&gt;. (within days)&lt;/p&gt;</content>
  52.   <author>
  53.     <name>Steve Jenson</name>
  54.     <uri></uri>
  55.   </author>
  56. </entry>
  57. <entry>
  58.   <title>Hello Again!</title>
  59.   <link href=""/>
  60.   <updated>2010-05-21T00:00:00+00:00</updated>
  61.   <id></id>
  62.   <content type="html">&lt;h1&gt;Hello Again!&lt;/h1&gt;
  63. &lt;p&gt;I&amp;#8217;ve changed some things around here.&lt;/p&gt;
  64. &lt;p&gt;This site is now generated with
  65. &lt;a href=&quot;;&gt;Jekyll&lt;/a&gt;&lt;/p&gt;
  66. &lt;p&gt;Switching from Blogger wasn&amp;#8217;t an easy decision but I felt like it was
  67. time to jump start things and make it fun to write and tinker here
  68. again.&lt;/p&gt;
  69. &lt;p&gt;Something that didn&amp;#8217;t make the conversion were all of the old
  70. comments. Jekyll doesn&amp;#8217;t support comments. I considered porting them
  71. over statically but I had already sat on the conversion for 6 months
  72. and was ready to move on. See, I actually switched back in December
  73. but had never actually updated this version of the site.&lt;/p&gt;
  74. &lt;p&gt;People left many wonderful comments. In the future, I&amp;#8217;d like to
  75. encourage you to simply email me. My email address is in the &amp;#8216;About
  76. Me&amp;#8217; page linked at the top. If you&amp;#8217;d like me to post your response,
  77. let me know.&lt;/p&gt;
  78. &lt;p&gt;Oh, you can also see a mirror of all my content on the blog&amp;#8217;s &lt;a href=&quot;;&gt;github
  79. repo&lt;/a&gt;&lt;/p&gt;</content>
  80.   <author>
  81.     <name>Steve Jenson</name>
  82.     <uri></uri>
  83.   </author>
  84. </entry>
  85. <entry>
  86.   <title>Higher Order Select in Ruby</title>
  87.   <link href=""/>
  88.   <updated>2009-12-13T00:00:00+00:00</updated>
  89.   <id></id>
  90.   <content type="html">&lt;h1&gt;Higher Order Select in Ruby&lt;/h1&gt;
  91. &lt;p&gt;When I was converting my blog to jekyll, I had to write a nested
  92. select in Ruby and found it to be a little painful.&lt;/p&gt;
  93. &lt;p&gt;In Ruby:&lt;/p&gt;
  94. &lt;div class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-ruby&quot; data-lang=&quot;ruby&quot;&gt;&lt;span class=&quot;c1&quot;&gt;# For a given feed, return all the entries that&lt;/span&gt;
  95. &lt;span class=&quot;c1&quot;&gt;# are blog posts rather than templates, etc.&lt;/span&gt;
  96. &lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;posts&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;feed&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
  97.  &lt;span class=&quot;c1&quot;&gt;# The post must have a category of type #post&lt;/span&gt;
  98.  &lt;span class=&quot;n&quot;&gt;feed&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;entries&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;select&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;entry&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;
  99.    &lt;span class=&quot;o&quot;&gt;!&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;entry&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;categories&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;select&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;c&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;
  100.      &lt;span class=&quot;n&quot;&gt;c&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;term&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&amp;#39;;#39;&lt;/span&gt;
  101.    &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;empty?&lt;/span&gt;
  102.  &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
  103. &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;In Scala, things are a little simpler thanks to anaphora: the ability
  104. to easily reference earlier state implicitly. That&amp;#8217;s what the _ means
  105. in this context.&lt;/p&gt;
  106. &lt;div class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-scala&quot; data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;n&quot;&gt;feed&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;entries&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;filter&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(!&lt;/span&gt;&lt;span class=&quot;k&quot;&gt;_&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;categories&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;filter&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;k&quot;&gt;_&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;term&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&amp;quot;;quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;isEmpty&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Imagine the following sentence in English: &amp;#8220;Steve used Steve&amp;#8217;s keys to
  107. start Steve&amp;#8217;s car&amp;#8221;. Most programming languages insist on that level of
  108. verbosity.&lt;/p&gt;
  109. &lt;p&gt;Raganwald has a nice &lt;a href=&quot;;&gt;piece&lt;/a&gt; exploring Anaphora in Ruby.&lt;/p&gt;</content>
  110.   <author>
  111.     <name>Steve Jenson</name>
  112.     <uri></uri>
  113.   </author>
  114. </entry>
  115. <entry>
  116.   <title>Functional Refactoring</title>
  117.   <link href=""/>
  118.   <updated>2009-07-21T00:00:00+00:00</updated>
  119.   <id></id>
  120.   <content type="html">&lt;p&gt;
  121.    Recently, while working on some code, I noticed that I have been making the same transform on lots of functional code in Scala. I think it's clearly a Refactoring and should have a name.
  122.    &lt;/p&gt;
  124.    &lt;p&gt;
  125.    &lt;h4&gt;&quot;Replace conditional inside fold with filter&quot;&lt;/h4&gt;
  126.    &lt;/p&gt;
  128.    &lt;p&gt;
  129.    The idea is that if you're folding over items in a list and manipulating them based on a conditional, it might be clearer if you pull out the conditional and filter the list on that conditional first.
  130.    &lt;/p&gt;
  131.    &lt;p&gt;
  132.    Here is a contrived example borrowed from a fake test suite. Let's say I had code with a conditional inside of a fold:
  133.    &lt;/p&gt;
  134.    &lt;p&gt;
  135.    &lt;div class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-scala&quot; data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;n&quot;&gt;sampleUsers&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;values&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;foreach&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;u&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;=&amp;gt;&lt;/span&gt;
  136.      &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;u&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;id&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;{&lt;/span&gt;
  137.        &lt;span class=&quot;nc&quot;&gt;Authentication&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;u&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;id&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;u&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;hash&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;passwordToken&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;mustEqual&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;u&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;passwordToken&lt;/span&gt;
  138.      &lt;span class=&quot;o&quot;&gt;}&lt;/span&gt;
  139.    &lt;span class=&quot;o&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  140.    &lt;/p&gt;
  141.    &lt;p&gt;
  142.    Let's apply this refactoring to make the test case clearer.
  143.    &lt;/p&gt;
  144.    &lt;p&gt;
  145.      &lt;div class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-scala&quot; data-lang=&quot;scala&quot;&gt;&lt;span class=&quot;n&quot;&gt;sampleUsers&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;values&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;filter&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;u&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;=&amp;gt;&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;u&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;id&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;foreach&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;u&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;=&amp;gt;&lt;/span&gt;
  146.      &lt;span class=&quot;nc&quot;&gt;Authentication&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;u&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;id&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;u&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;hash&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;passwordToken&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;mustEqual&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;u&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;passwordToken&lt;/span&gt;
  147.    &lt;span class=&quot;o&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
  148.    &lt;/p&gt;
  149.    &lt;p&gt;
  150.    This works with multiple inner conditionals as well. You can fit them into one filter or chain multiple filters together.
  151.    &lt;/p&gt;
  152.    &lt;p&gt;
  153.    Even though this is a simple refactoring, it seems like a worthwhile place to start cataloging.
  154.    &lt;/p&gt;
  155. </content>
  156.   <author>
  157.     <name>Steve Jenson</name>
  158.     <uri></uri>
  159.   </author>
  160. </entry>
  161. <entry>
  162.   <title>More Varnish Thoughts from the Author</title>
  163.   <link href=""/>
  164.   <updated>2008-12-09T00:00:00+00:00</updated>
  165.   <id></id>
  166.   <content type="html">&lt;a href=&quot;;&gt;Poul-Henning Kamp&lt;/a&gt;, the lead author of Varnish, was kind enough to respond to my blog post.
  168.    &lt;tt&gt;
  169.    &lt;p&gt;
  170.    Hi Steve,
  171.    &lt;/p&gt;
  172.    &lt;p&gt;
  173.    I read your comments about Varnish and thought I'd better explain
  174.    myself, feel free to post this if you want.
  175.    &lt;/p&gt;
  176.    &lt;p&gt;
  177.    The reason why we don't multiplex threads in Varnish is that it has
  178.    not become a priority for us yet, and I am not convinced that we
  179.    will ever see it become a priority at the cost it bears.
  180.    &lt;/p&gt;
  181.    &lt;p&gt;
  182.    A common misconception is that varnish parks a thread waiting for
  183.    the client to send the next request.  We do not, that would be
  184.    horribly stupid.  Sessions are multiplexed when idle.
  185.    &lt;/p&gt;
  186.    &lt;p&gt;
  187.    The Varnish worker thread takes over when the request is completely
  188.    read and leaves the session again when the answer is delivered to
  189.    the network stack (and no further input is pending).
  190.    &lt;/p&gt;
  191.    &lt;p&gt;
  192.    For a cache hit, this takes less than 10 microseconds, and we are
  193.    still shaving that number.
  194.    &lt;/p&gt;
  195.    &lt;p&gt;
  196.    The problem about multiplexing is that it is horribly expensive,
  197.    both in system calls, cache pressure and complexity.
  198.    &lt;/p&gt;
  199.    &lt;p&gt;
  200.    If we had a system call that was called &quot;write these bytes, but
  201.    don't block longer than N seconds&quot;, multiplexing could be sensibly
  202.    implemented.
  203.    &lt;/p&gt;
  204.    &lt;p&gt;
  205.    But with POSIX mostly being a historical society, it will cost you
  206.    around 7 system calls, some surprisingly expensive, to even find
  207.    out if multiplexing is necessary in the first place.
  208.    &lt;/p&gt;
  209.    &lt;p&gt;
  210.    In comparison, today Varnish delivers a cache-hit in 7 system calls
  211.    *total* of which a few are technically only there for debugging.
  212.    &lt;/p&gt;
  213.    &lt;p&gt;
  214.    The next good reason is that we have no really fiddled with the
  215.    sendbuffer sizes yet, but obviously, if you sendbuffer can swallow
  216.    the read, the thread does not wait.
  217.    &lt;/p&gt;
  218.    &lt;p&gt;
  219.    And if that fails, a thread blocked in a write(2) system call is
  220.    quite cheap to have around.
  221.    &lt;/p&gt;
  222.    &lt;p&gt;
  223.    It uses a little RAM, but it's not that much, and we can probably
  224.    tune it down quite a bit.
  225.    &lt;/p&gt;
  226.    &lt;p&gt;
  227.    The scheduler does not bother more with the thread than it has to,
  228.    and when it does, the VM hardware system is not kickstarted every
  229.    time we cross the user/kernel barrier.
  230.    &lt;/p&gt;
  231.    &lt;p&gt;
  232.    Without getting into spectroscoping comparisons between apples and
  233.    oranges, a thread just lounging around, waiting for the network
  234.    stack to do its thing, is much, much cheaper than a thread which
  235.    does a lot of system calls and fd-table walking, only to perform
  236.    a few and small(-ish) writes every time the network stack wakes up.
  237.    &lt;/p&gt;
  238.    &lt;p&gt;
  239.    And the final reason why it may never become necessary to multiplex
  240.    threads, is that servers are cheap.
  241.    &lt;/p&gt;
  242.    &lt;p&gt;
  243.    But if we get to the point where we need multiplexing, we will do
  244.    it.
  245.    &lt;/p&gt;
  246.    &lt;p&gt;
  247.    But I like the old design principles from the X11 project: we will
  248.    not do it, until we have a server that doesn't work without it.
  249.    &lt;/p&gt;
  250.    &lt;p&gt;
  251.    But if you are in the business of delivering ISOs to 56k modems
  252.    then yes, Varnish is probably not for you.
  253.    &lt;/p&gt;
  254.    &lt;p&gt;
  255.    Poul-Henning
  256.    &lt;/p&gt;
  257.    &lt;/tt&gt;
  258. </content>
  259.   <author>
  260.     <name>Steve Jenson</name>
  261.     <uri></uri>
  262.   </author>
  263. </entry>
  264. <entry>
  265.   <title>Thoughts on Varnish</title>
  266.   <link href=""/>
  267.   <updated>2008-12-07T00:00:00+00:00</updated>
  268.   <id></id>
  269.   <content type="html">&lt;p&gt;&lt;a href=&quot;;&gt;Varnish&lt;/a&gt; is getting a lot of attention these days around the internet, and with good reason, it&amp;#8217;s a nicely written and speedy cache, and has a nice &lt;span class=&quot;caps&quot;&gt;DSL&lt;/span&gt; for caching. It has great features like hot reloading of cache rules and &lt;span class=&quot;caps&quot;&gt;ESI&lt;/span&gt;.&lt;/p&gt;
  272.    &lt;p&gt;One thing that&amp;#8217;s really surprised me, though, is that Varnish uses one thread per connection. Most network programs designed for high number of connection don&amp;#8217;t use one thread per connection anymore as it has serious drawbacks.&lt;/p&gt;
  275.    &lt;p&gt;With slow clients, many of your threads are spending a lot of time doing nothing but blocking in &lt;tt&gt;write()&lt;/tt&gt;. In all internet consumer apps, I believe, slow clients make up the majority of your connections. But even though the threads are doing nothing, the OS still has memory and scheduling overhead in dealing with them. You find yourself with an artificially low ceiling on the amount of users you can service with a single machine.&lt;/p&gt;
  277.    &lt;p&gt;What makes a client slow, though? Both speed and latency. Cell phones, 56k modems, and users on high speed links but not geographically close to your data center can all be classified as &amp;#8216;slow&amp;#8217;.&lt;/p&gt;
  280.    &lt;p&gt;One design that is more appropriate for dealing with the slow client problem uses a pool of worker
  281.    threads or processes behind the scene and
  282.    &lt;a href=&quot;;&gt;epoll&lt;/a&gt; /
  283.    &lt;a href=&quot;;&gt;kqueue&lt;/a&gt; / &lt;a href=&quot;;&gt;event
  284.    ports&lt;/a&gt; handling slow clients and
  285.    telling the pool of workers that a socket is ready with a change notification. Your cost is still
  286.    correlated with growth but at a much lower rate and the number of users you can service will
  287.    dramatically increase.&lt;/p&gt;
  289.    &lt;p&gt;So why does Varnish use this older, more troublesome model? Probably because most services aren&amp;#8217;t
  290.    going to notice the bottleneck; They simply don&amp;#8217;t have enough concurrent connections to worry about
  291.    using a few extra machines. If you&amp;#8217;re never saturated a load balancer or firewall, you&amp;#8217;ve probably
  292.    never had to seriously consider the &lt;a href=&quot;;&gt;C10k&lt;/a&gt; issues involved.&lt;/p&gt;
  295.    &lt;p&gt;Also, unfortunately, the way most people write load tests is that they are only testing the All Fast
  296.    Clients scenario and not a mix of fast clients and slow clients. I&amp;#8217;m guilty of this, too.&lt;/p&gt;
  299.    &lt;p&gt;My executive summary: Varnish is a nice piece of software, and I hope they spend the time to make it
  300.    useful for larger sites as well as smaller ones.&lt;/p&gt;
  301. </content>
  302.   <author>
  303.     <name>Steve Jenson</name>
  304.     <uri></uri>
  305.   </author>
  306. </entry>
  307. <entry>
  308.   <title>More RPMs means faster access times. No news there.</title>
  309.   <link href=""/>
  310.   <updated>2008-11-18T00:00:00+00:00</updated>
  311.   <id></id>
  312.   <content type="html">&lt;p&gt;
  313.    When I upgraded my home laptop from a 2-year old MacBook Pro to one of the newly released unibody models, I decided to upgrade from a 5400 RPM drive to a 7200 RPM drive. I ran some &lt;a href=&quot;;&gt;bonnie-64&lt;/a&gt; benchmarks and noticed a 40% improvement in random seeks/sec and some other impressive numbers. It's helped make my weekend hacking much more pleasant.
  314.    &lt;/p&gt;
  315.    &lt;p&gt;
  316.    Here are the old numbers:
  317.    &lt;script src=&quot;;&gt;&lt;/script&gt;
  318.    &lt;/p&gt;
  319.    &lt;p&gt;
  320.    and the new numbers:
  321.    &lt;script src=&quot;;&gt;&lt;/script&gt;
  322.    &lt;/p&gt;
  323.    &lt;p&gt;
  324.    &lt;strong&gt;Bottom line: Recommended&lt;/strong&gt;
  325.    &lt;/p&gt;
  326. </content>
  327.   <author>
  328.     <name>Steve Jenson</name>
  329.     <uri></uri>
  330.   </author>
  331. </entry>
  332. <entry>
  333.   <title>Amazon as Political Pulpit</title>
  334.   <link href=""/>
  335.   <updated>2008-09-13T00:00:00+00:00</updated>
  336.   <id></id>
  337.   <content type="html">&lt;p&gt;
  338.    Check out the &lt;a href=&quot;;showViewpoints=1&quot;&gt;Amazon reviews for Spore&lt;/a&gt;, the new Will Wright game. 2,016 of the 2216 reviews gave it 1 star for the heavy-handed DRM. I use Amazon reviews heavily and I've never seen protest brought to the reviews at this scale before. Let's see if people take notice.
  339.    &lt;/p&gt;
  340.    &lt;p&gt;
  341.    &lt;a href=&quot;;&gt;TorrentFreak&lt;/a&gt; points out the following:
  342.    &lt;blockquote&gt;DRM doesn’t stop people from pirating a game, on the contrary. It only hurts legitimate customers since the DRM is removed from the pirate version.&quot;&lt;/blockquote&gt;
  343.    &lt;/p&gt;
  344.    &lt;p&gt;There are lots of philosophical reasons not to buy things with DRM. For me, the practical reason wins out. I'd rather deal with having to store music CDs and not play some potentially awesome games than deal with losing my data due to short-sighted DRM.
  345.    &lt;/p&gt;
  346. </content>
  347.   <author>
  348.     <name>Steve Jenson</name>
  349.     <uri></uri>
  350.   </author>
  351. </entry>
  352. <entry>
  353.   <title>Alex's Scala Talk at C4[2]</title>
  354.   <link href=""/>
  355.   <updated>2008-09-12T00:00:00+00:00</updated>
  356.   <id></id>
  357.   <content type="html">&lt;a href=&quot;;&gt;al3x&lt;/a&gt; gave a &lt;a href=&quot;;&gt;talk on Scala&lt;/a&gt; recently at &lt;a href=&quot;;&gt;C4[2]&lt;/a&gt; and it hits a lot of the high points as to why we're using Scala at Twitter to build back-end services. I've been programming in Scala seriously for about a year and a half now and it's positively ruined me for plain Java.
  358. </content>
  359.   <author>
  360.     <name>Steve Jenson</name>
  361.     <uri></uri>
  362.   </author>
  363. </entry>
  364. <entry>
  365.   <title>So I don't forget it.</title>
  366.   <link href=""/>
  367.   <updated>2008-07-30T00:00:00+00:00</updated>
  368.   <id></id>
  369.   <content type="html">A cheesy generic tcp proxy I found cruising the webs built out of &lt;code&gt;netcat&lt;/code&gt;, &lt;code&gt;fifo&lt;/code&gt;s, and &lt;code&gt;tee&lt;/code&gt;:
  371.    &lt;p&gt;
  372.    &lt;code&gt;
  373.    $ mkfifo backpipe&lt;br /&gt;
  374.    $ sudo nc -l 80 0&amp;lt;backpipe | tee -a inflow | nc localhost 8080| tee -a outflow 1&amp;gt;backpipe
  375.    &lt;/code&gt;
  376.    &lt;/p&gt;
  377.    This way you can also look at &lt;code&gt;inflow&lt;/code&gt; and &lt;code&gt;outflow&lt;/code&gt; to see what the actual contents were of the transaction.
  378. </content>
  379.   <author>
  380.     <name>Steve Jenson</name>
  381.     <uri></uri>
  382.   </author>
  383. </entry>
  384. <entry>
  385.   <title>Simulating Byzantine failure with SIGSTOP</title>
  386.   <link href=""/>
  387.   <updated>2008-06-30T00:00:00+00:00</updated>
  388.   <id></id>
  389.   <content type="html">&lt;p&gt;
  390.    If your service relies on connecting to an internal network server and
  391.    that server isn't accepting connections, your client will obviously throw an
  392.    error. This happens often enough that you probably already check for this and do the right thing in your various projects. But what if the server is accepting connections but never
  393.    returning any data? This failure case is rare but very deadly.  Chet
  394.    mentioned that you could simulate this using &lt;tt&gt;SIGSTOP&lt;/tt&gt; so I
  395.    decided to whip up an experiment with &lt;tt&gt;memcached&lt;/tt&gt; as my victim.
  396.    &lt;/p&gt;
  397.    &lt;p&gt;
  398.    &lt;pre&gt;
  399.    [email protected]:~$ ps auxww |grep memcache
  400.    stevej    3451  0.0  0.0   2928  1872 pts/0    T    01:21   0:00 memcached -vv -p 11211
  401.    [email protected]:~$ kill -stop 3451
  402.    &lt;/pre&gt;
  403.    &lt;/p&gt;
  404.    &lt;p&gt;
  405.    In another terminal:
  406.    &lt;/p&gt;
  407.    &lt;p&gt;
  408.    &lt;pre&gt;
  409.    [email protected]:~$ irb
  410.    irb(main):001:0&amp;gt; require 'rubygems'
  411.    =&amp;gt; true
  412.    irb(main):002:0&amp;gt; require 'memcache'
  413.    =&amp;gt; true
  414.    irb(main):003:0&amp;gt; CACHE = &quot;localhost:11211&quot;
  415.    =&amp;gt; &amp;lt;MemCache: 1 servers, 1 buckets, ns: nil, ro: false&amp;gt;
  416.    irb(main):004:0&amp;gt; CACHE.get(&quot;foo&quot;)
  417.    &lt;/pre&gt;
  418.    &lt;/p&gt;
  419.    &lt;p&gt;
  420.    The client library happily hung for several hours while I did other
  421.    things. How can a process that's suspended not timeout incoming
  422.    connections?  Well, it's the kernel that services network requests and
  423.    the process itself is only reading the buffers. If you want proof,
  424.    look at this tcpdump output. Remember, the process has already been
  425.    suspended by the time I ran tcpdump here.
  426.    &lt;/p&gt;
  427.    &lt;p&gt;
  428.    &lt;pre&gt;
  429.    [email protected]:~$ sudo tcpdump -i lo port 11211
  430.    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  431.    listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
  432.    18:02:40.576255 IP localhost.48124 &amp;gt; localhost.11211: F 2018798159:2018798159(0) ack 2012359105 win 257 &amp;lt;nop,nop,timestamp 15455978 14281170&amp;gt;
  433.    18:02:40.577037 IP localhost.11211 &amp;gt; localhost.48124: . ack 1 win 256 &amp;lt;nop,nop,timestamp 15455979 15455978&amp;gt;
  434.    18:03:19.037410 IP localhost.35662 &amp;gt; localhost.11211: S 2731273926:2731273926(0) win 32792 &amp;lt;mss 16396,sackOK,timestamp 15465593 0,nop,wscale 7&amp;gt;
  435.    18:03:19.037435 IP localhost.11211 &amp;gt; localhost.35662: S 2723119696:2723119696(0) ack 2731273927 win 32768 &amp;lt;mss 16396,sackOK,timestamp 15465593 15465593,nop,wscale 7&amp;gt;
  436.    18:03:19.037449 IP localhost.35662 &amp;gt; localhost.11211: . ack 1 win 257 &amp;lt;nop,nop,timestamp 15465593 15465593&amp;gt;
  437.    18:03:19.037768 IP localhost.35662 &amp;gt; localhost.11211: P 1:10(9) ack 1 win 257 &amp;lt;nop,nop,timestamp 15465593 15465593&amp;gt;
  438.    18:03:19.037776 IP localhost.11211 &amp;gt; localhost.35662: . ack 10 win 256 &amp;lt;nop,nop,timestamp 15465593 15465593&amp;gt;
  439.    &lt;/pre&gt;
  440.    &lt;/p&gt;
  441.    &lt;p&gt;
  442.    So a connect timeout wouldn't help here, you need a recv timeout or
  443.    something else. Restarting your client process won't help at all,
  444.    it'll simply get stuck in the same place. In Ruby, the easiest thing
  445.    to do is to use the Timeout module. Sadly, it only has second
  446.    granularity but that's a lot better than hanging for several
  447.    hours. You can also set use Socket#setsockopt with a recv timeout if
  448.    you need finer grained timeout resolution.
  449.    &lt;/p&gt;
  450.    &lt;p&gt;
  451.    &lt;pre&gt;
  452.    [email protected]:~$ irb
  453.    irb(main):001:0&amp;gt; require 'rubygems'
  454.    =&amp;gt; true
  455.    irb(main):002:0&amp;gt; require 'memcache'
  456.    =&amp;gt; true
  457.    irb(main):003:0&amp;gt; CACHE = &quot;localhost:11211&quot;
  458.    =&amp;gt; &amp;lt;MemCache: 1 servers, 1 buckets, ns: nil, ro: false&amp;gt;
  459.    irb(main):004:0&amp;gt; require 'timeout'
  460.    =&amp;gt; false
  461.    irb(main):005:0&amp;gt; foo = Timeout::timeout(1) do
  462.    irb(main):006:1*   CACHE.get(&quot;foo&quot;)
  463.    irb(main):007:1&amp;gt; end
  464.    /usr/lib/ruby/1.8/timeout.rb:54:in `cache_get': execution expired (Timeout::Error)
  465.    from /usr/lib/ruby/gems/1.8/gems/memcache-client-1.5.0/lib/memcache.rb:209:in `get'
  466.    from (irb):6:in `irb_binding'
  467.    from /usr/lib/ruby/1.8/timeout.rb:56:in `timeout'
  468.    from (irb):5:in `irb_binding'
  469.    from /usr/lib/ruby/1.8/irb/workspace.rb:52:in `irb_binding'
  470.    from /usr/lib/ruby/1.8/irb/workspace.rb:52
  471.    &lt;/pre&gt;
  472.    &lt;/p&gt;
  473.    &lt;p&gt;
  474.    Do you want to guess what happened when I sent &lt;tt&gt;SIGCONT&lt;/tt&gt; to memcache? My client processes, even the ones that had been hanging for hours, immediately returned with the expected data.
  475.    &lt;/p&gt;
  476.    &lt;p&gt;
  477.    The obvious thing to do is to write a new MemCache subclass decorating
  478.    all the calls to &lt;tt&gt;get&lt;/tt&gt;, &lt;tt&gt;put&lt;/tt&gt;, &lt;tt&gt;get_multi&lt;/tt&gt;, etc
  479.    with safer versions. Don't naively trust that the expected data made
  480.    it to the cache.
  481.    &lt;/p&gt;
  482. </content>
  483.   <author>
  484.     <name>Steve Jenson</name>
  485.     <uri></uri>
  486.   </author>
  487. </entry>
  488. <entry>
  489.   <title>More on Twitter!</title>
  490.   <link href=""/>
  491.   <updated>2008-05-22T00:00:00+00:00</updated>
  492.   <id></id>
  493.   <content type="html">Al3x wrote up a nice blog post &lt;a href=&quot;;&gt;talking about the future of twitter.&lt;/a&gt;
  494. </content>
  495.   <author>
  496.     <name>Steve Jenson</name>
  497.     <uri></uri>
  498.   </author>
  499. </entry>
  500. <entry>
  501.   <title>How work is going?</title>
  502.   <link href=""/>
  503.   <updated>2008-05-22T00:00:00+00:00</updated>
  504.   <id></id>
  505.   <content type="html">&lt;p&gt;
  506.    Since I started working at Twitter last month, I put up a standard work disclaimer along the side. It &lt;strong&gt;always&lt;/strong&gt; applies.
  507.    &lt;/p&gt;
  508.    &lt;p&gt;
  509.    Jack posted on the company blog: &lt;a href=&quot;;&gt;I have this graph up on my screen all the time. It should be flat. This week has been rough.&lt;/a&gt;
  510.    &lt;/p&gt;
  511.    &lt;p&gt;
  512.    So we have open job postings for something called a &lt;a href=&quot;;&gt;Systems Engineer&lt;/a&gt;, which is what I do at Twitter. Systems Engineering means building systems where graphs like that stay flat and where downtime means it was either planned or making sure that particular problem won't happen again (if it can be avoided: typical engineering trade-offs apply).
  513.    &lt;/p&gt;
  514.    &lt;p&gt;
  515.    Our problems are really interesting, I think. Lots of users, lots of connections, lots of messages flowing through the system, lots of endpoints, and lots of details to keep straight. All of this needs to be turned into a cohesive system that's simple to reason about and to run in order for me to consider my job a success. It's a tall order but it's what I signed up to do. I've been watching Twitter for a long time (I'm user #150) so I walked into things with my eyes wide open.
  516.    &lt;/p&gt;
  517.    &lt;p&gt;
  518.    If you've been reading this blog for a while, you know that I'm more interested in engineering than hacking together a site. Thinking and then doing. Measuring and then reasoning. Making guesses and then testing them. There's a natural tension between cowboying around and Analysis Paralysis and you have to learn to walk that tightrope if you want to succeed and I think at Twitter, we work pretty hard to Do the Right Thing.
  519.    &lt;/p&gt;
  520.    &lt;p&gt;
  521.    I'm writing this quick post because we're looking for great people who are interested in engineering big systems and in helping to make Twitter the utility-class company we see ourselves as needing to be. If you think you either have the skills or can learn them, please send us your resume to [email protected]
  522.    &lt;/p&gt;
  523. </content>
  524.   <author>
  525.     <name>Steve Jenson</name>
  526.     <uri></uri>
  527.   </author>
  528. </entry>
  529. <entry>
  530.   <title>Fun DTrace script</title>
  531.   <link href=""/>
  532.   <updated>2008-05-20T00:00:00+00:00</updated>
  533.   <id></id>
  534.   <content type="html">&lt;pre&gt;
  535.    &lt;code&gt;
  536.    #!/usr/sbin/dtrace -s
  538.    syscall:::entry
  539.    /pid == $1/
  540.    {
  541.    @sys[probefunc, ustack()] = count();
  542.    }
  544.    END {
  545.    trunc(@sys, 2);
  546.    }
  547.    &lt;/code&gt;
  548.    &lt;/pre&gt;
  550.    Tells you the 2 most often called system call/stack trace pair. Running it against firefox 3 beta while using Google Reader shows:
  552.    &lt;pre&gt;
  553.    &lt;code&gt;
  554.    $ sudo ./syscalldist.d 240
  555.    dtrace: script './syscalldist.d' matched 428 probes
  556.    ^C
  557.    CPU     ID                    FUNCTION:NAME
  558.    1      2                             :END
  560.    munmap                                            
  561.    libSystem.B.dylib`munmap$UNIX2003+0xa
  562.    libSystem.B.dylib`free+0x6a
  563.    CoreGraphics`CGEventCreateFromDataAndSource+0xbce
  564.    CoreGraphics`CGSDecodeEventRecord+0x6a
  565.    CoreGraphics`CGSDispatchDatagramsFromStream+0x28f
  566.    CoreGraphics`snarfEvents+0x12a
  567.    CoreGraphics`CGSGetNextEventRecordInternal+0x9f
  568.    CoreGraphics`CGEventCreateNextEvent+0x2c
  569.    HIToolbox`PullEventsFromWindowServerOnConnection(unsigned int, unsigned char)+0x58
  570.    CoreFoundation`__CFMachPortPerform+0x75
  571.    CoreFoundation`CFRunLoopRunSpecific+0xf51
  572.    CoreFoundation`CFRunLoopRunInMode+0x58
  573.    HIToolbox`RunCurrentEventLoopInMode+0x11b
  574.    HIToolbox`ReceiveNextEventCommon+0x176
  575.    HIToolbox`BlockUntilNextEventMatchingListInMode+0x6a
  576.    AppKit`_DPSNextEvent+0x291
  577.    AppKit`-[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]+0x80
  578.    AppKit`-[NSApplication run]+0x31b
  579.    XUL`JSD_GetValueForObject+0xad6ce
  580.    XUL`XRE_GetFileFromPath+0x61c563
  581.    961
  582.    mmap                                              
  583.    libSystem.B.dylib`mmap+0xa
  584.    libSystem.B.dylib`large_and_huge_malloc+0xcb
  585.    libSystem.B.dylib`szone_malloc+0x1cf
  586.    libSystem.B.dylib`malloc_zone_malloc+0x51
  587.    libSystem.B.dylib`malloc+0x37
  588.    CoreGraphics`CGEventCreateFromDataAndSource+0x15e
  589.    CoreGraphics`CGSDecodeEventRecord+0x6a
  590.    CoreGraphics`CGSDispatchDatagramsFromStream+0x28f
  591.    CoreGraphics`snarfEvents+0x12a
  592.    CoreGraphics`CGSGetNextEventRecordInternal+0x9f
  593.    CoreGraphics`CGEventCreateNextEvent+0x2c
  594.    HIToolbox`PullEventsFromWindowServerOnConnection(unsigned int, unsigned char)+0x58
  595.    CoreFoundation`__CFMachPortPerform+0x75
  596.    CoreFoundation`CFRunLoopRunSpecific+0xf51
  597.    CoreFoundation`CFRunLoopRunInMode+0x58
  598.    HIToolbox`RunCurrentEventLoopInMode+0x11b
  599.    HIToolbox`ReceiveNextEventCommon+0x176
  600.    HIToolbox`BlockUntilNextEventMatchingListInMode+0x6a
  601.    AppKit`_DPSNextEvent+0x291
  602.    AppKit`-[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]+0x80
  603.    997
  604.    &lt;/pre&gt;
  605.    &lt;/code&gt;
  606.    Thrilling, I know!
  607. </content>
  608.   <author>
  609.     <name>Steve Jenson</name>
  610.     <uri></uri>
  611.   </author>
  612. </entry>
  613. <entry>
  614.   <title>DTrace for Java 6 on Leopard</title>
  615.   <link href=""/>
  616.   <updated>2008-05-19T00:00:00+00:00</updated>
  617.   <id></id>
  618.   <content type="html">&lt;p&gt;
  619.    When Java 6 for Leopard was released a few weeks ago, one thing that nobody seemed to notice was that Java now had DTrace probes on par with Java on Solaris.
  620.    &lt;/p&gt;
  621.    &lt;p&gt;
  622.    What you expect is there:
  623.    &lt;ul&gt;
  624.    &lt;li&gt;&lt;a href=&quot;;&gt;DTrace Topics: Java&lt;/a&gt;&lt;/li&gt;
  625.    &lt;li&gt;&lt;a href=&quot;;&gt;DTrace Probes in HotSpot VM&lt;/a&gt;&lt;/li&gt;
  626.    &lt;/ul&gt;
  627.    &lt;/p&gt;
  628.    &lt;p&gt;
  629.    With one exception: &lt;code&gt;jstack&lt;/code&gt; doesn't appear to work. &lt;code&gt;ustack&lt;/code&gt; works fine.
  630.    &lt;/p&gt;
  632.    &lt;p&gt;
  633.    &lt;pre&gt;
  634.    &lt;code&gt;
  635.    $ sudo dtrace -x jstackstrsize=2048 -n 'syscall::read:entry /execname == &quot;java&quot;/ { jstack(); }'
  637.    dtrace: description 'syscall::read:entry ' matched 1 probe
  638.    CPU     ID                    FUNCTION:NAME
  639.    3  17600                       read:entry
  641.    2  17600                       read:entry
  643.    3  17600                       read:entry
  645.    3  17600                       read:entry
  647.    2  17600                       read:entry
  649.    2  17600                       read:entry
  651.    2  17600                       read:entry
  653.    2  17600                       read:entry
  654.    &lt;/code&gt;
  655.    &lt;/pre&gt;
  656.    &lt;/p&gt;
  658.    &lt;p&gt;
  659.    There should be java stack traces under each read:entry line. (This is true even with &lt;code&gt;-XX:+ExtendedDTraceProbes&lt;/code&gt; enabled)
  660.    &lt;/p&gt;
  662.    &lt;p&gt;
  663.    I used robey's scarling for my guinea pig and had a lot of fun poking around at it with dtrace.
  664.    &lt;/p&gt;
  665. </content>
  666.   <author>
  667.     <name>Steve Jenson</name>
  668.     <uri></uri>
  669.   </author>
  670. </entry>
  671. <entry>
  672.   <title>John McCarthy has a good sense of humor</title>
  673.   <link href=""/>
  674.   <updated>2008-05-11T00:00:00+00:00</updated>
  675.   <id></id>
  676.   <content type="html">From an informal talk he gave at Stanford recently that was written up in &lt;a href=&quot;;&gt;Hacker News&lt;/a&gt;:
  678.    &lt;blockquote&gt;
  679.    &lt;pre&gt;
  680.    Q. Can computers know?
  682.    A. This is largely a question of definition. If a camera looked at a table, we could
  683.    say it &quot;knows&quot; that there are four containers of liquid on the table (which was true).
  685.    Q. Is there any definition of &quot;know&quot; in which computers cannot succeed?
  687.    A. Well, I suppose the biblical sense.
  689.    Q. Ha, well, what makes you think that?
  691.    A. They don't satisfy the necessary axioms (laughter)
  692.    &lt;/pre&gt;
  693.    &lt;/blockquote&gt;
  694. </content>
  695.   <author>
  696.     <name>Steve Jenson</name>
  697.     <uri></uri>
  698.   </author>
  699. </entry>
  700. <entry>
  701.   <title>What are you doing?</title>
  702.   <link href=""/>
  703.   <updated>2008-04-22T00:00:00+00:00</updated>
  704.   <id></id>
  705.   <content type="html">reading @&lt;a href=&quot;;&gt;biz&lt;/a&gt; out me as a Twitter employee.
  706. </content>
  707.   <author>
  708.     <name>Steve Jenson</name>
  709.     <uri></uri>
  710.   </author>
  711. </entry>
  712. <entry>
  713.   <title>curious what delicious is saying about something?</title>
  714.   <link href=""/>
  715.   <updated>2008-04-15T00:00:00+00:00</updated>
  716.   <id></id>
  717.   <content type="html">&lt;a href=&quot;javascript:location.href=';url='+encodeURIComponent(location.href)&quot;&gt;Here's a bookmarklet that checks the current URL you're visiting with;/a&gt;.
  718. </content>
  719.   <author>
  720.     <name>Steve Jenson</name>
  721.     <uri></uri>
  722.   </author>
  723. </entry>
  724. <entry>
  725.   <title>My first Thrift app</title>
  726.   <link href=""/>
  727.   <updated>2008-04-13T00:00:00+00:00</updated>
  728.   <id></id>
  729.   <content type="html">&lt;p&gt;
  730.    When you find yourself working on big systems, a useful technique is
  731.    to decompose it into services. Moving from a big monolithic server to
  732.    a bunch of separate services can be a big challenge but if you had
  733.    foresight, many of your services were already decoupled in your system
  734.    from day 1 even though you were deploying it monolithicly.
  735.    &lt;/p&gt;
  736.    &lt;p&gt;
  737.    A common technique for decomposing services is using RPC. At Google, we
  738.    used protocol buffers, which were briefly descibed in the &lt;a href=&quot;;&gt;Sawzall&lt;/a&gt;
  739.    paper.
  740.    &lt;/p&gt;
  741.    &lt;p&gt;
  742.    Basically, you describe your data and the interface that process the
  743.    data in a language-independent format (a DDL, essentially) and use code
  744.    generators to turn that DDL into set of objects in your target
  745.    langauge that can create and send those structures over the wire. This
  746.    makes it easy to write servers in one language and clients in another
  747.    and the generated code deals with serialization.
  748.    &lt;/p&gt;
  749.    &lt;p&gt;
  750.    I found that using a DDL to describe your code and services was really
  751.    nice. When building a new service, you could simply reference your DDL
  752.    in the design doc and have a meanginful discussion about the service
  753.    without getting into the details of how it would be written until you had
  754.    the semantics nailed down.
  755.    &lt;/p&gt;
  757.    &lt;p&gt;
  758.    Facebook, as they were growing, decided to move to a homegrown binary
  759.    RPC mechanism similar to protocol buffers called &lt;a href=&quot;;&gt;Thrift&lt;/a&gt;.
  760.    &lt;/p&gt;
  762.    &lt;p&gt;
  763.    Let's say I wanted to write a simple service that would tell the
  764.    client what time it was on the server. Here would be the DDL file
  765.    describing both the data and the service plus a little extra to help
  766.    out the generated code files.
  767.    &lt;/p&gt;
  768.    &lt;pre name=&quot;code&quot; class=&quot;thrift&quot;&gt;
  769.    # time.thrift
  770.    namespace java tserver.gen
  771.    namespace ruby TServer.Gen
  773.    typedef i64 Timestamp
  775.    service TimeServer {
  776.    // Simply returns the current time.
  777.    Timestamp time()
  778.    }
  779.    &lt;/pre&gt;
  781.    &lt;p&gt;
  782.    After running &lt;tt&gt;thrift --gen java --gen rb time.thrift&lt;/tt&gt; on the
  783.    file, I'd have an interface and server that I could implement in Java
  784.    and a client that I could use in Ruby.
  785.    &lt;/p&gt;
  786.    &lt;p&gt;
  787.    Based on the generated java code, I could write a short server in Scala:
  788.    &lt;/p&gt;
  790.    &lt;pre name=&quot;code&quot; class=&quot;scala&quot;&gt;
  791.    &lt;code&gt;
  792.    package tserver
  794.    import tserver.gen._
  795.    import com.facebook.thrift.TException
  796.    import com.facebook.thrift.TProcessor
  797.    import com.facebook.thrift.TProcessorFactory
  798.    import com.facebook.thrift.protocol.TProtocol
  799.    import com.facebook.thrift.protocol.TProtocolFactory
  800.    import com.facebook.thrift.transport.TServerTransport
  801.    import com.facebook.thrift.transport.TServerSocket
  802.    import com.facebook.thrift.transport.TTransport
  803.    import com.facebook.thrift.transport.TTransportFactory
  804.    import com.facebook.thrift.transport.TTransportException
  805.    import com.facebook.thrift.server.TServer
  806.    import com.facebook.thrift.server.TThreadPoolServer
  807.    import com.facebook.thrift.protocol.TBinaryProtocol
  809.    /**
  810.    * TimeServer.time returns the current time according to the server.
  811.    */
  812.    class TimeServer extends TimeServer.Iface {
  813.    override def time: Long = {
  814.    val now = System.currentTimeMillis
  815.    println(&quot;somebody just asked me what time it is: &quot; + now)
  816.    now
  817.    }
  818.    }
  820.    object SimpleServer extends Application {
  821.    try {
  822.    val serverTransport = new TServerSocket(7911)
  823.    val processor = new TimeServer.Processor(new TimeServer())
  824.    val protFactory = new TBinaryProtocol.Factory(true, true)
  825.    val server = new TThreadPoolServer(processor, serverTransport,
  826.    protFactory)
  828.    println(&quot;starting server&quot;)
  829.    server.serve();    
  830.    } catch {
  831.    case x: Exception =&amp;gt; x.printStackTrace();
  832.    }
  833.    }
  834.    &lt;/code&gt;
  835.    &lt;/pre&gt;
  837.    &lt;p&gt;
  838.    (Geez, most of that space was taken up in my obsessive need to separate
  839.    out all my imports. You can thank Google for that bit of OCD.)
  840.    &lt;/p&gt;
  841.    &lt;p&gt;
  842.    The client is even shorter:
  843.    &lt;/p&gt;
  844.    &lt;pre name=&quot;code&quot; class=&quot;ruby&quot;&gt;
  845.    &lt;code&gt;
  846.    #!/usr/bin/ruby
  847.    $:.push('~/thrift/lib/rb/lib')
  848.    $:.push('../gen-rb')
  850.    require 'thrift/transport/tsocket'
  851.    require 'thrift/protocol/tbinaryprotocol'
  852.    require 'TimeServer'
  854.    transport =;localhost&quot;, 7911))
  855.    protocol =
  856.    client =
  860.    puts &quot;I wonder what time it is. Let's ask!&quot;
  861.    puts client.time()
  862.    &lt;/code&gt;
  863.    &lt;/pre&gt;
  864.    &lt;p&gt;
  865.    The ruby client took about 20ms to get an answer from the Scala server.
  866.    &lt;/p&gt;
  867.    &lt;p&gt;
  868.    Thrift advantages:&lt;br /&gt;
  869.    &lt;ul&gt;
  870.    &lt;li&gt;Pipelined connections means you spend less time in connection setup/teardown and TCP likes longer-lived connections.&lt;/li&gt;
  871.    &lt;li&gt;Asynchronous requests. Asynchronous replies would be nice too but would be trickier to use.&lt;/li&gt;
  872.    &lt;li&gt;Binary representation is much more efficient to transmit and process than, say, XML.&lt;/li&gt;
  873.    &lt;/ul&gt;
  874.    &lt;/p&gt;
  875.    &lt;p&gt;
  876.    Thrift drawbacks: &lt;br /&gt;
  877.    &lt;ul&gt;
  878.    &lt;li&gt;Integrating generated source into your build system can be tricky.
  879.    Typically, you rarely have to regenerate your stubs but debugging generated code can be a huge pain.&lt;/li&gt;
  880.    &lt;li&gt;It's Java server should move away from ServerSocket to NIO for increased throughput.
  881.    That's probably not more than a week's work as long as the existing code isn't too tightly coupled.&lt;/li&gt;
  882.    &lt;li&gt;Currently it doesn't build cleanly on the Mac. I did some work and got it working but I don't think it's used
  883.    extensively on the Mac so if that's your primary platform, you should be prepared to send them patches from time to time.&lt;/li&gt;
  884.    &lt;/ul&gt;
  885.    &lt;/p&gt;
  886.    &lt;p&gt;
  887.    If you're looking to move towards decoupled services, Thrift is worth a hard look.
  888.    &lt;/p&gt;
  889.    &lt;p&gt;
  890.    Here's a tarball with my &lt;a href=&quot;;&gt;time
  891.    server&lt;/a&gt;. It contains all the generated code as well as
  892.    libthrift.jar and a Makefile to run the example server.
  893.    &lt;/p&gt;
  894. </content>
  895.   <author>
  896.     <name>Steve Jenson</name>
  897.     <uri></uri>
  898.   </author>
  899. </entry>
  900. <entry>
  901.   <title>GVN and gold</title>
  902.   <link href=""/>
  903.   <updated>2008-04-07T00:00:00+00:00</updated>
  904.   <id></id>
  905.   <content type="html">&lt;p&gt;
  906.    Two things popped up on my radar recently:
  907.    &lt;/p&gt;
  908.    &lt;p&gt;
  909.    &lt;a href=&quot;;&gt;gvn&lt;/a&gt;, Google's wrappers around Subversion to help them work in their code-review heavy workflow. Even if you're not into code reviews, &lt;code&gt;tkdiff&lt;/code&gt; integration is a nice improvement over &lt;code&gt;colordiff&lt;/code&gt; or &lt;code&gt;FileMerge&lt;/code&gt;.
  910.    &lt;/p&gt;
  911.    &lt;p&gt;
  912.    &lt;a href=&quot;;&gt;gold&lt;/a&gt;, a new ELF linker built with giant binaries in mind. When you're building 900MB+ static binaries routinely, linking speed matters. gold claims to be at least 5x faster currently. Even if you have a massive distcc cluster, linking is still serial. One of gold's future design goals is to be concurrent and that would be pretty awesome. Imagine how fast I could link with a concurrent linker on my 8-core Mac Pro! Not that using an ELF linker under Leopard helps much since OS X uses Mach-O binaries but hey, there's always cross-compiling.
  913.    &lt;/p&gt;
  914.    &lt;p&gt;
  915.    BTW, Ian Lance Taylor, the author of gold, has &lt;a href=&quot;;&gt;an excellent series of blog articles on linkers.&lt;/a&gt;
  916.    &lt;/p&gt;
  917. </content>
  918.   <author>
  919.     <name>Steve Jenson</name>
  920.     <uri></uri>
  921.   </author>
  922. </entry>
  923. </feed>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

  1. Download the "valid Atom 1.0" banner.

  2. Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)

  3. Add this HTML to your page (change the image src attribute if necessary):

If you would like to create a text link instead, here is the URL you can use:

Copyright © 2002-9 Sam Ruby, Mark Pilgrim, Joseph Walton, and Phil Ringnalda