Congratulations!

[Valid RSS] This is a valid RSS feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

Source: https://labs.magnet.me/feed.xml

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  3.  <channel>
  4.    <title>Magnet.me | Labs</title>
  5.    <description>On this blog we share our thoughts and experiences.</description>
  6.    <link>https://labs.magnet.me/</link>
  7.    <atom:link href="https://labs.magnet.me/feed.xml" rel="self" type="application/rss+xml" />
  8.    <pubDate>Thu, 19 Jul 2018 14:10:22 +0000</pubDate>
  9.    <lastBuildDate>Thu, 19 Jul 2018 14:10:22 +0000</lastBuildDate>
  10.    <generator>Jekyll v3.8.1</generator>
  11.  
  12.      <item>
  13.        <title>Async await in AngularJS</title>
  14.        <description>&lt;p class=&quot;intro&quot;&gt;
  15. ES7 introduces &lt;a href=&quot;http://tc39.github.io/ecmascript-asyncawait/&quot;&gt;async functions&lt;/a&gt;, which allow you to write asynchronous code with a synchronous syntax. Unfortunately this doesn't work well with AngularJS: awaited code will run outside of Angular's digest loop, and therefore it won't trigger watchers or view updates.
  16. &lt;/p&gt;
  17.  
  18. &lt;!--more--&gt;
  19.  
  20. &lt;h1 id=&quot;the-problem&quot;&gt;The problem&lt;/h1&gt;
  21. &lt;p&gt;ES7 introduces async functions that allow to write async code with a synchronous syntax. This looks like this:&lt;/p&gt;
  22.  
  23. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-javascript&quot; data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;k&quot;&gt;async&lt;/span&gt; &lt;span class=&quot;kd&quot;&gt;function&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  24. &lt;span class=&quot;kr&quot;&gt;await&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;doSomethingAsync&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;();&lt;/span&gt;
  25. &lt;span class=&quot;nx&quot;&gt;console&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;Something is done!&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;);&lt;/span&gt;
  26. &lt;span class=&quot;kr&quot;&gt;await&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;doSomethingElseAsync&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;();&lt;/span&gt;
  27. &lt;span class=&quot;nx&quot;&gt;console&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;log&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;Something else is also done!&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;);&lt;/span&gt;
  28. &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  29.  
  30. &lt;p&gt;This works because &lt;code class=&quot;highlighter-rouge&quot;&gt;await&lt;/code&gt; expects a promise as arguments, and it waits until the promise is resolved before continuing.&lt;/p&gt;
  31.  
  32. &lt;p&gt;Unfortunately this doesn’t work well in AngularJS: awaited code is not run in the digest loop, and therefore we need to put &lt;code class=&quot;highlighter-rouge&quot;&gt;$scope.$apply&lt;/code&gt; calls all over the place. This results in code like this:&lt;/p&gt;
  33.  
  34. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-javascript&quot; data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;k&quot;&gt;async&lt;/span&gt; &lt;span class=&quot;kd&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;initialize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  35. &lt;span class=&quot;nx&quot;&gt;$scope&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;user&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kr&quot;&gt;await&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;users&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;getActiveUser&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;();&lt;/span&gt;
  36. &lt;span class=&quot;nx&quot;&gt;$scope&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;$apply&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;();&lt;/span&gt;
  37. &lt;span class=&quot;nx&quot;&gt;$scope&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;messages&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kr&quot;&gt;await&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;conversations&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;getMessagesFor&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;$scope&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;user&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;);&lt;/span&gt;
  38. &lt;span class=&quot;nx&quot;&gt;$scope&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;$apply&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;();&lt;/span&gt;
  39. &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  40.  
  41. &lt;p&gt;Obviously this isn’t ideal, it would be better if we could omit the &lt;code class=&quot;highlighter-rouge&quot;&gt;$scope.$apply&lt;/code&gt; calls.&lt;/p&gt;
  42.  
  43. &lt;h1 id=&quot;meet-async&quot;&gt;Meet $async&lt;/h1&gt;
  44. &lt;p&gt;Fortunately there is a way out. Thanks to the &lt;a href=&quot;https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*&quot;&gt;generator functions&lt;/a&gt; introduced in ES6 we can write our own async engine that &lt;em&gt;does&lt;/em&gt; work well with AngularJS. At Magnet.me we’ve created the &lt;a href=&quot;https://github.com/Magnetme/ng-async&quot;&gt;$async&lt;/a&gt; module which does exactly that. With &lt;code class=&quot;highlighter-rouge&quot;&gt;$async&lt;/code&gt; we can rewrite the initialize function like this:&lt;/p&gt;
  45.  
  46. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-javascript&quot; data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;kd&quot;&gt;const&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;initialize&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;$async&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;kd&quot;&gt;function&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  47. &lt;span class=&quot;nx&quot;&gt;$scope&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;user&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;yield&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;users&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;getActiveUser&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;();&lt;/span&gt;
  48. &lt;span class=&quot;nx&quot;&gt;$scope&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;messages&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;yield&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;conversations&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;getMessagesFor&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;$scope&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;user&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;);&lt;/span&gt;
  49. &lt;span class=&quot;p&quot;&gt;});&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  50.  
  51. &lt;p&gt;Instead of an async function we now use a generator function passed to &lt;code class=&quot;highlighter-rouge&quot;&gt;$async&lt;/code&gt;, and instead of &lt;code class=&quot;highlighter-rouge&quot;&gt;await&lt;/code&gt;ing a promise we now &lt;code class=&quot;highlighter-rouge&quot;&gt;yield&lt;/code&gt; a promise.&lt;/p&gt;
  52.  
  53. &lt;h1 id=&quot;how-it-works&quot;&gt;How it works&lt;/h1&gt;
  54. &lt;p&gt;Generator functions do not run to completion when they are invoked. Instead they return an iterator that can be iterated over manually. This iterator will stop and “return” at every call to &lt;code class=&quot;highlighter-rouge&quot;&gt;yield&lt;/code&gt;. For example, the following generator function generates all positive integers:&lt;/p&gt;
  55.  
  56. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-javascript&quot; data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;kd&quot;&gt;function&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;*&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;integersGenerator&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  57. &lt;span class=&quot;kd&quot;&gt;let&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;i&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  58. &lt;span class=&quot;k&quot;&gt;while&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  59. &lt;span class=&quot;k&quot;&gt;yield&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;i&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  60. &lt;span class=&quot;nx&quot;&gt;i&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;++&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  61. &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
  62. &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
  63.  
  64. &lt;span class=&quot;kd&quot;&gt;const&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;integersIterator&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;integersGenerator&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;();&lt;/span&gt;
  65. &lt;span class=&quot;nx&quot;&gt;integersIterator&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;().&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;value&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;c1&quot;&gt;//1&lt;/span&gt;
  66. &lt;span class=&quot;nx&quot;&gt;integersIterator&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;().&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;value&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;c1&quot;&gt;//2&lt;/span&gt;
  67. &lt;span class=&quot;nx&quot;&gt;integersIterator&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;().&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;value&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;c1&quot;&gt;//3&lt;/span&gt;
  68. &lt;span class=&quot;c1&quot;&gt;//etc.&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  69.  
  70. &lt;p&gt;The &lt;code class=&quot;highlighter-rouge&quot;&gt;$async&lt;/code&gt; module makes use of the ability to pause generator functions to emulate async functions. It expects a generator that will generate promises, and it will chain all these promises together correctly. The result is that each &lt;code class=&quot;highlighter-rouge&quot;&gt;yield&lt;/code&gt; call will wait for its promise to be resolved before continuing. The core of &lt;code class=&quot;highlighter-rouge&quot;&gt;$async&lt;/code&gt; looks something like this:&lt;/p&gt;
  71.  
  72. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-javascript&quot; data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;c1&quot;&gt;//This function iterates asynchronous over an iterator&lt;/span&gt;
  73. &lt;span class=&quot;kd&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;iterateAsync&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;iterator&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  74. &lt;span class=&quot;c1&quot;&gt;//First run until the first yield. &lt;/span&gt;
  75. &lt;span class=&quot;c1&quot;&gt;//`.value` should now contain the yielded promise&lt;/span&gt;
  76. &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;iterator&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;next&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;().&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;value&lt;/span&gt;
  77. &lt;span class=&quot;c1&quot;&gt;//Then we wait until the promise has resolved, &lt;/span&gt;
  78. &lt;span class=&quot;c1&quot;&gt;//and recursively iterate again&lt;/span&gt;
  79. &lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;then&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(()&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&amp;gt;&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;iterateAsync&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;iterator&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;));&lt;/span&gt;
  80. &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
  81.  
  82. &lt;span class=&quot;c1&quot;&gt;//usage:&lt;/span&gt;
  83. &lt;span class=&quot;nx&quot;&gt;iterateAsync&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;kd&quot;&gt;function&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  84. &lt;span class=&quot;k&quot;&gt;yield&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;somePromise&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  85. &lt;span class=&quot;k&quot;&gt;yield&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;someOtherPromise&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  86. &lt;span class=&quot;c1&quot;&gt;//etc.&lt;/span&gt;
  87. &lt;span class=&quot;p&quot;&gt;});&lt;/span&gt;
  88.  
  89. &lt;span class=&quot;c1&quot;&gt;//This is effectively the same as this:&lt;/span&gt;
  90. &lt;span class=&quot;nx&quot;&gt;somePromise&lt;/span&gt;
  91. &lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;then&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(()&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&amp;gt;&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;someOtherPromise&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;())&lt;/span&gt;
  92. &lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;then&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;cm&quot;&gt;/*etc*/&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;);&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  93.  
  94. &lt;p&gt;Now that we have our own async engine it is trivial to make it work properly with Angular: we can add the call to &lt;code class=&quot;highlighter-rouge&quot;&gt;$scope.$apply&lt;/code&gt; in the &lt;code class=&quot;highlighter-rouge&quot;&gt;iterateAsync&lt;/code&gt; function and there it is: an async function that works with AngularJS!&lt;/p&gt;
  95.  
  96. &lt;p&gt;The entire module is only a couple of lines long, if you’re interested in all the details then check it out &lt;a href=&quot;https://github.com/Magnetme/ng-async/blob/master/src/index.js&quot;&gt;here&lt;/a&gt;!&lt;/p&gt;
  97.  
  98. &lt;h1 id=&quot;inspiration&quot;&gt;Inspiration&lt;/h1&gt;
  99. &lt;p&gt;To give credit where credit’s due, the idea of using generators to implement aysnc functions isn’t new. Libraries that do so exist for a long time, with the most well-known being &lt;a href=&quot;https://www.npmjs.com/package/co&quot;&gt;co&lt;/a&gt;. &lt;code class=&quot;highlighter-rouge&quot;&gt;$async&lt;/code&gt; is mostly inspired by &lt;code class=&quot;highlighter-rouge&quot;&gt;co&lt;/code&gt;, but more lightweight and Angular-flavored.&lt;/p&gt;
  100. </description>
  101.        <pubDate>Mon, 16 Nov 2015 18:00:00 +0000</pubDate>
  102.        <link>https://labs.magnet.me/nerds/2015/11/16/async-await-in-angularjs.html</link>
  103.        <guid isPermaLink="true">https://labs.magnet.me/nerds/2015/11/16/async-await-in-angularjs.html</guid>
  104.  
  105.        <category>AngularJS</category>
  106.  
  107.        <category>JavaScript</category>
  108.  
  109.  
  110.        <category>Nerds</category>
  111.  
  112.      </item>
  113.  
  114.      <item>
  115.        <title>Configuration management with Consul</title>
  116.        <description>&lt;p class=&quot;intro&quot;&gt;Microservices are a hot topic, and there are many challenges you'll face when adopting this way of building software. In this blogpost we'll tackle the challenge of configuration. In particular how to manage the configuration of many different services located in a cluster where services are running in different datacenters, on a multitude of host machines or containers, and even potentially with different roles. Using &lt;a href=&quot;http://github.com/Magnetme/consultant&quot;&gt;Consultant&lt;/a&gt; - &lt;i&gt;a newly open-sourced Magnet.me project&lt;/i&gt; - we'll show you how to load your service's configuration with zero effort and subscribe to updates.&lt;/p&gt;
  117.  
  118. &lt;!--more--&gt;
  119.  
  120. &lt;h2 id=&quot;consul&quot;&gt;Consul&lt;/h2&gt;
  121.  
  122. &lt;p&gt;We’ve been experimenting with &lt;a href=&quot;https://consul.io&quot;&gt;Consul&lt;/a&gt; lately to manage service discovery, routing, and configuration for our (micro)services. Consul is a service developed by &lt;a href=&quot;https://hashicorp.com&quot;&gt;HashiCorp&lt;/a&gt; which you can run in your cluster. Consul uses &lt;a href=&quot;https://serfdom.io&quot;&gt;Serf&lt;/a&gt; to gossip between multiple running instances of Consul to share the location and state of services running in the cluster. In addition to this, Consul also provides a key/value store which allows you to store configuration or meta-data for the operation of your services. Consul is also datacenter aware, meaning that the you can not only search for the locations of a particular service in the entire cluster, but also limit that search to a particular datacenter to avoid long roundtrips. We’ll save the service discovery awesome-ness for another blogpost. Let’s focus on configuration!&lt;/p&gt;
  123.  
  124. &lt;h2 id=&quot;configuration&quot;&gt;Configuration&lt;/h2&gt;
  125.  
  126. &lt;p&gt;Configuration is an integral part of many services. If your service needs to talk to a database, it’s likely you’ll want to store the username, password, database name, and perhaps even the location of the database server if you’re not using service discovery. These settings are typically static, and don’t change while the service is running. There are also settings which are more dynamic and can change while a service is running, like settings regarding the number of workers for a certain task. All these settings are typically stored inside one or more configuration files which the service reads on startup. If the service supports dynamic configurations, typically the service watches these configuration files for changes, and reloads the configuration from disk when a change is detected.&lt;/p&gt;
  127.  
  128. &lt;p&gt;The problem is managing all this configuration for every instance of the service in the entire cluster. For instance, what do you need to do if you want to alter the config of all service instances in a particular datacenter, running on a particular host, or with a particular role? With microservices this can be a bit of challenge as it’s likely the service instances might be running just about anywhere in your cluster. The solution to this problem is to make your configuration available from a service such as Consul, and make your services look their own configuration up depending on their location and role in the cluster.&lt;/p&gt;
  129.  
  130. &lt;h2 id=&quot;integrating-consultant-with-a-service&quot;&gt;Integrating Consultant with a service&lt;/h2&gt;
  131.  
  132. &lt;p&gt;Assuming you’ve already got Consul setup for your cluster, let’s look at how we can integrate Consultant into our JVM-based services, and let them retrieve their own configuration from Consul. Consultant was designed to be extremely simple to use, and to integrate with existing services. Below you’ll see how we can create a Consultant object for our news service, and retrieve a &lt;code class=&quot;highlighter-rouge&quot;&gt;Properties&lt;/code&gt; object from it:&lt;/p&gt;
  133.  
  134. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-java&quot; data-lang=&quot;java&quot;&gt;&lt;span class=&quot;n&quot;&gt;Consultant&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;consultant&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Consultant&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;builder&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;()&lt;/span&gt;
  135.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;identifyAs&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;news&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;
  136.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;build&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;();&lt;/span&gt;
  137.  
  138. &lt;span class=&quot;n&quot;&gt;Properties&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;config&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;consultant&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;getProperties&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;();&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  139.  
  140. &lt;p&gt;It’s highly recommended to provide environment variables specifying the datacenter, host, and role of this particular service instance. In case of our news service we’d want to specify the following environment variables: &lt;code class=&quot;highlighter-rouge&quot;&gt;SERVICE_DC=eu-central&lt;/code&gt; to let it know it’s running in our EU datacenter, and &lt;code class=&quot;highlighter-rouge&quot;&gt;SERVICE_HOST=host-12.magnet.me&lt;/code&gt; to let it know that it’s running on a specific host.&lt;/p&gt;
  141.  
  142. &lt;p&gt;Consultant will instantly retrieve the configuration applicable for this news service running in that particular datacenter and on that particular host.&lt;/p&gt;
  143.  
  144. &lt;h2 id=&quot;subscribing-to-changes&quot;&gt;Subscribing to changes&lt;/h2&gt;
  145.  
  146. &lt;p&gt;But that’s not all Consultant does. Should we change any of settings in Consul’s key/value store, Consultant is immediately notified of this, and will update the &lt;code class=&quot;highlighter-rouge&quot;&gt;Properties&lt;/code&gt; object. Should you wish to validate any configuration changes, then you can do that too:&lt;/p&gt;
  147.  
  148. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-java&quot; data-lang=&quot;java&quot;&gt;&lt;span class=&quot;n&quot;&gt;Consultant&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;consultant&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Consultant&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;builder&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;()&lt;/span&gt;
  149.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;identifyAs&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;news&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;
  150.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;validateConfigWith&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;((&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&amp;gt;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;{&lt;/span&gt;
  151.      &lt;span class=&quot;n&quot;&gt;Preconditions&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;checkNotNull&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;getProperty&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;database.username&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;));&lt;/span&gt;
  152.      &lt;span class=&quot;n&quot;&gt;Preconditions&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;checkNotNull&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;getProperty&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;database.password&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;));&lt;/span&gt;
  153.      &lt;span class=&quot;n&quot;&gt;Preconditions&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;checkNotNull&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;getProperty&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;database.url&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;));&lt;/span&gt;
  154.   &lt;span class=&quot;o&quot;&gt;})&lt;/span&gt;
  155.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;build&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;();&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  156.  
  157. &lt;p&gt;Should the validation fail, the &lt;code class=&quot;highlighter-rouge&quot;&gt;Properties&lt;/code&gt; object will not be updated. This means that your service remains unaware of these changes in its configuration. Should you wish to be notified of when the service’s configuration is successfully changed you can also subscribe to these changes:&lt;/p&gt;
  158.  
  159. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-java&quot; data-lang=&quot;java&quot;&gt;&lt;span class=&quot;n&quot;&gt;Consultant&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;consultant&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Consultant&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;builder&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;()&lt;/span&gt;
  160.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;identifyAs&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;news&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;
  161.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;onValidConfig&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;((&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&amp;gt;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;{&lt;/span&gt;
  162.      &lt;span class=&quot;n&quot;&gt;System&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;out&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;println&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Yay, we've got a new config!&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;);&lt;/span&gt;
  163.   &lt;span class=&quot;o&quot;&gt;})&lt;/span&gt;
  164.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;build&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;();&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  165.  
  166. &lt;p&gt;Or if you’d like to be notified of a particular setting being changed:&lt;/p&gt;
  167.  
  168. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-java&quot; data-lang=&quot;java&quot;&gt;&lt;span class=&quot;n&quot;&gt;Consultant&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;consultant&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Consultant&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;builder&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;()&lt;/span&gt;
  169.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;identifyAs&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;news&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;
  170.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;onSettingUpdate&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;database.password&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;key&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;oldValue&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;newValue&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;-&amp;gt;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;{&lt;/span&gt;
  171.      &lt;span class=&quot;n&quot;&gt;System&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;out&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;println&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;Hey the db password changed to: &quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;newValue&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;&quot;!&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;);&lt;/span&gt;
  172.   &lt;span class=&quot;o&quot;&gt;})&lt;/span&gt;
  173.   &lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;na&quot;&gt;build&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;();&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  174.  
  175. &lt;p&gt;These listeners will be called every time Consultant is able to retrieve a configuration from Consul’s key/value store which passes the validation (if you’ve specified any validation checks).&lt;/p&gt;
  176.  
  177. &lt;h2 id=&quot;storing-the-config-in-consuls-keyvalue-store&quot;&gt;Storing the config in Consul’s key/value store&lt;/h2&gt;
  178.  
  179. &lt;p&gt;Now that we can have our services retrieve their configuration from Consul’s key/value store, it’s time to see how we can store our settings in there. The easiest way to do this, is to manually add the configuration through Consul’s web UI. Simply by adding a new entry with a key matching the pattern: &lt;code class=&quot;highlighter-rouge&quot;&gt;config/&amp;lt;service_name&amp;gt;/&amp;lt;setting_name&amp;gt;&lt;/code&gt; you can define a setting with that particular name, for that particular service.&lt;/p&gt;
  180.  
  181. &lt;p&gt;If you wish to define a setting which overrides the default value for a particular datacenter, host, or instance you can do so by defining a selector block. In that case the key should follow one of these patterns:&lt;/p&gt;
  182.  
  183. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-java&quot; data-lang=&quot;java&quot;&gt;&lt;span class=&quot;n&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;/&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;service_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;/[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;dc&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;datacenter_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;].&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;setting_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;
  184. &lt;span class=&quot;n&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;/&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;service_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;/[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;host&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;host_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;].&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;setting_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;
  185. &lt;span class=&quot;n&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;/&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;service_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;/[&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;instance&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;instance_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;/&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;role&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;].&amp;lt;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;setting_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  186.  
  187. &lt;p&gt;Note that you can even mix and match these to define even more specific groups by for instance specifying a selector such as: &lt;code class=&quot;highlighter-rouge&quot;&gt;[dc=eu-central,host=app-12.magnet.me]&lt;/code&gt;.&lt;/p&gt;
  188.  
  189. &lt;p&gt;&lt;img src=&quot;/images/posts/consultant-configuration-management-with-consul/consul-web-ui.png&quot; alt=&quot;Consul's web UI&quot; /&gt;&lt;/p&gt;
  190.  
  191. &lt;h2 id=&quot;summary&quot;&gt;Summary&lt;/h2&gt;
  192.  
  193. &lt;p&gt;Using Consul and Consultant, Magnet.me is able to have each service retrieve its own configuration from Consul. By subscribing to changes in that configuration, we are able to alter the behaviour of our services at runtime, tweak the performance of our services at runtime, and even run the same configuration in development environments as we do in production environments. We hope you’ve enjoyed this blogpost, and that you’ll give Consul and Consultant a try! Consultant is available from &lt;a href=&quot;http://search.maven.org/#search%7Cga%7C1%7Cme.magnet.consultant&quot;&gt;Maven’s Central Repository&lt;/a&gt;!&lt;/p&gt;
  194.  
  195. </description>
  196.        <pubDate>Mon, 26 Oct 2015 12:00:00 +0000</pubDate>
  197.        <link>https://labs.magnet.me/nerds/2015/10/26/consultant-configuration-management-with-consul.html</link>
  198.        <guid isPermaLink="true">https://labs.magnet.me/nerds/2015/10/26/consultant-configuration-management-with-consul.html</guid>
  199.  
  200.        <category>Configuration</category>
  201.  
  202.        <category>Tutorials</category>
  203.  
  204.        <category>Consul</category>
  205.  
  206.  
  207.        <category>Nerds</category>
  208.  
  209.      </item>
  210.  
  211.      <item>
  212.        <title>Trying out InfluxDB and Grafana</title>
  213.        <description>&lt;p class=&quot;intro&quot;&gt;At Magnet.me we try to measure and log everything. We use it to get a better understanding of our customers and our software. We already had Kibana and Logstash in place, but we were still missing some functionality to cover all our metric needs. We ended up trying InfluxDB and Grafana as our main tooling. In this blog post, we'll tell you how to try it out yourself, and why you should give it a shot.&lt;/p&gt;
  214.  
  215. &lt;!--more--&gt;
  216.  
  217. &lt;h2 id=&quot;why-measure&quot;&gt;Why measure&lt;/h2&gt;
  218.  
  219. &lt;p&gt;There are tons of &lt;a href=&quot;https://codeascraft.com/2010/12/08/track-every-release/&quot;&gt;great&lt;/a&gt; &lt;a href=&quot;https://vwo.com/blog/multi-armed-bandit-algorithm/&quot;&gt;blogs&lt;/a&gt; about measuring so I’ll just say a few words about our aproach.&lt;/p&gt;
  220.  
  221. &lt;p&gt;At Magnet.me we use &lt;a href=&quot;http://en.wikipedia.org/wiki/OKR&quot;&gt;OKRs&lt;/a&gt;. In short, it means we set quarterly objectives and we validate if we achieve them by setting &lt;em&gt;measurable&lt;/em&gt; goals for ourselves. This implies that we need to be able to measure everything we build. Often this concerns how users use our product.&lt;/p&gt;
  222.  
  223. &lt;p&gt;For example, when we try to get costumers to respond to a message 15% more often, we might choose to change the email we send them, or change user they way the reply button looks on our website. To measure the result of these changes we first need to know how many users used to go to the reply page, and from where. Then we try new versions of the email, add buttons to reply on the website, or change the buttons we already have. We then measure the difference. If we achieve our goal, we move on to the next goal. If we fail, we adjust our strategy. By doing this rapidly in a loop until we reach our goal, or decide it can’t be reached, it is easy for us to keep our focus and not falsly assume something is fixed because we put some effort in.&lt;/p&gt;
  224.  
  225. &lt;p&gt;To achieve this, we need a bunch of metric tools. One that:&lt;/p&gt;
  226.  
  227. &lt;ol&gt;
  228.  &lt;li&gt;measures how users navigate through our site. We use Google Analytics for this.&lt;/li&gt;
  229.  &lt;li&gt;allows us to A/B test. We also use Google Analytics for this if it is a test happening in the browser.&lt;/li&gt;
  230.  &lt;li&gt;tracks clicks in the email to the desired state, including the time it took the user to respond.&lt;/li&gt;
  231. &lt;/ol&gt;
  232.  
  233. &lt;p&gt;For the last job, logging the time it took to go to the state, we did not have a great solution yet. We used to just log it and then query it with Kibana, but that had a couple of downsides. Kibana 3 (at the time) could only show one dashboard per index. Also, you cannot annotate graphs in Kibana, so marking when you launched an experiment is not possible.&lt;/p&gt;
  234.  
  235. &lt;h2 id=&quot;why-influxdb&quot;&gt;Why InfluxDB&lt;/h2&gt;
  236.  
  237. &lt;p&gt;When we started to look for a metric database, you soon find out you need a &lt;a href=&quot;http://en.wikipedia.org/wiki/Time_series_database&quot;&gt;Time series database&lt;/a&gt;. It seems like most developers are using the &lt;a href=&quot;https://www.digitalocean.com/community/tutorials/how-to-configure-statsd-to-collect-arbitrary-stats-for-graphite-on-ubuntu-14-04&quot;&gt;StatsD + Graphite&lt;/a&gt; stack for this. However, I never liked the UI of Graphite so we decided to use &lt;a href=&quot;http://grafana.org/&quot;&gt;Grafana&lt;/a&gt; instead. On the Grafana site we ran into a great tool called &lt;a href=&quot;http://influxdb.com/docs/v0.8/introduction/overview.html&quot;&gt;InfluxDB&lt;/a&gt;. InfluxDB has a couple of design goals I like in comparison to the design of StatsD:&lt;/p&gt;
  238.  
  239. &lt;ol&gt;
  240.  &lt;li&gt;It’s horizontally scalable&lt;/li&gt;
  241.  &lt;li&gt;Besides metrics, it also stores events&lt;/li&gt;
  242.  &lt;li&gt;It stores on disk &lt;em&gt;and&lt;/em&gt; in-memory.&lt;/li&gt;
  243.  &lt;li&gt;Expand storage by adding servers to a cluster. Should make node replacement quick and easy.&lt;/li&gt;
  244.  &lt;li&gt;Automatically compute common queries continuously in the background.&lt;/li&gt;
  245. &lt;/ol&gt;
  246.  
  247. &lt;h1 id=&quot;setting-it-up&quot;&gt;Setting it up&lt;/h1&gt;
  248.  
  249. &lt;p&gt;Assuming you have Docker (or boot2docker if you’re on a mac) installed, we kick off by starting up an InfluxDB instance:&lt;/p&gt;
  250.  
  251. &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker run -d -p 8083:8083 -p 8086:8086 -e PRE_CREATE_DB=&quot;db1&quot; --name=influxdb tutum/influxdb
  252. &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
  253.  
  254. &lt;p&gt;This will fire up the admin dashboard on port &lt;code class=&quot;highlighter-rouge&quot;&gt;8083&lt;/code&gt; and the API on &lt;code class=&quot;highlighter-rouge&quot;&gt;8086&lt;/code&gt;. We set it up to pre-create the database &lt;code class=&quot;highlighter-rouge&quot;&gt;db1&lt;/code&gt; for testing purposes. Open &lt;a href=&quot;&quot;&gt;http://localhost:8083&lt;/a&gt; or on mac &lt;code class=&quot;highlighter-rouge&quot;&gt;open http://$(boot2docker ip):8083&lt;/code&gt;. Login with username and password &lt;code class=&quot;highlighter-rouge&quot;&gt;root&lt;/code&gt;. You don’t have to populate the database. You should see our &lt;code class=&quot;highlighter-rouge&quot;&gt;db1&lt;/code&gt; in the list. Good, now we’re ready to send some data!&lt;/p&gt;
  255.  
  256. &lt;p&gt;At this point, I recommend following the steps in the &lt;a href=&quot;http://influxdb.com/docs/v0.8/introduction/getting_started.html&quot;&gt;InfluxDB Getting Started Guide&lt;/a&gt;. They have a piece of JavaScript you can run to insert a bunch of data points. Take your time to go through the introduction and insert that data. Go on. I’ll wait.&lt;/p&gt;
  257.  
  258. &lt;h2 id=&quot;grafana&quot;&gt;Grafana&lt;/h2&gt;
  259.  
  260. &lt;p&gt;Now that you’ve played around with Influx a little it’s time to sprinkle a little of that Grafana magic on top. Pop back into your terminal and fire up a grafana instance:&lt;/p&gt;
  261.  
  262. &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker run -d -p 80:3000 --link=influxdb:influxdb grafana/grafana:2.0.2
  263. &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;
  264.  
  265. &lt;p&gt;Then open &lt;a href=&quot;&quot;&gt;http://localhost&lt;/a&gt; or on mac &lt;code class=&quot;highlighter-rouge&quot;&gt;open http://$(boot2docker ip)&lt;/code&gt;. Login using using username and password “admin”, click the Grafana logo on the top-left, and go to “Data Sources”, “Add New”. Here we’ll fill
  266. in our influx database. Because we linked the containers using &lt;code class=&quot;highlighter-rouge&quot;&gt;--link=influxdb:influxdb&lt;/code&gt; the Grafana instance (which proxies the HTTP requests) can reach influx at &lt;a href=&quot;&quot;&gt;http://influxdb:8086&lt;/a&gt; (the port for the API). We’ve set-up the influx database with username/password “root” and our database is called “db1”. Let’s just call the datasource “Influx”.&lt;/p&gt;
  267.  
  268. &lt;p&gt;&lt;img src=&quot;/images/posts/measure-all-the-things/add-datasource.png&quot; alt=&quot;adding a datasource&quot; /&gt;&lt;/p&gt;
  269.  
  270. &lt;p&gt;Now that we have a datasource, let’s create a new dashboard from the dashboard menu. Then click the little green button on the left and press “Add Panel” &amp;gt; “Graph”. Grafana will show a fake graph to give you an impression, but we want our data from influx. Click on the title of the graph and press ‘edit’. Select “Influx” as our data source. In the &lt;a href=&quot;http://influxdb.com/docs/v0.8/introduction/getting_started.html&quot;&gt;InfluxDB Getting Started Guide&lt;/a&gt; you have created two series: &lt;code class=&quot;highlighter-rouge&quot;&gt;cpu_idle&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;customer_events&lt;/code&gt;. Let’s use the former one for this graph. If you’ve set it up correctly you should now see your first Grafana chart!&lt;/p&gt;
  271.  
  272. &lt;p&gt;&lt;img src=&quot;/images/posts/measure-all-the-things/first-graph.png&quot; alt=&quot;Your first graph&quot; /&gt;&lt;/p&gt;
  273.  
  274. &lt;h3 id=&quot;final-remarks&quot;&gt;Final remarks&lt;/h3&gt;
  275.  
  276. &lt;ul&gt;
  277.  &lt;li&gt;In production, we found out that Raft doesn’t like docker changing ids all the time. To circumvent this, overwrite the config to set a steady id using the &lt;code class=&quot;highlighter-rouge&quot;&gt;hostname&lt;/code&gt; variable.&lt;/li&gt;
  278.  &lt;li&gt;We used Influx 0.8 and Grafana 2.0.2&lt;/li&gt;
  279. &lt;/ul&gt;
  280. </description>
  281.        <pubDate>Sun, 17 May 2015 10:01:24 +0000</pubDate>
  282.        <link>https://labs.magnet.me/nerds/2015/05/17/trying-out-influx-grafana.html</link>
  283.        <guid isPermaLink="true">https://labs.magnet.me/nerds/2015/05/17/trying-out-influx-grafana.html</guid>
  284.  
  285.        <category>Metrics</category>
  286.  
  287.        <category>Tutorials</category>
  288.  
  289.  
  290.        <category>Nerds</category>
  291.  
  292.      </item>
  293.  
  294.      <item>
  295.        <title>Importing Google contacts with javascript</title>
  296.        <description>&lt;p class=&quot;intro&quot;&gt;To enhance the user experience for inviting your friends to take a look Magnet.me, we decided to include a feature to import contacts from your Google account. There are multiple ways to approach this, including a client-side only approach with the use of just javascript. This should be an easy process to set up, however in practice this was not the case as we ran into some problems. This blogpost will describe the problems we experienced and provide solutions to these problems.&lt;/p&gt;
  297.  
  298. &lt;!--more--&gt;
  299.  
  300. &lt;h1 id=&quot;problems&quot;&gt;Problems&lt;/h1&gt;
  301.  
  302. &lt;h2 id=&quot;google-api-explorer&quot;&gt;Google API explorer&lt;/h2&gt;
  303. &lt;p&gt;Google has APIs for multiple purposes. For example, I could use a Google API to create short urls (like &lt;a href=&quot;http://goo.gl/Mp8EjW&quot;&gt;http://goo.gl/Mp8EjW)&lt;/a&gt;) or communicate with Google Plus. Google has set up an &lt;a href=&quot;https://developers.google.com/apis-explorer/#p/&quot;&gt;API explorer webpage&lt;/a&gt; where every developer can find information about any Google API. Although this is nice, I could not find the API I was looking for. In this case, it seems that information about Google contacts is missing, even though an API for managing contacts does exist.&lt;/p&gt;
  304.  
  305. &lt;h2 id=&quot;google-apps-documentation&quot;&gt;Google Apps documentation&lt;/h2&gt;
  306. &lt;p&gt;After some more searching I stumbled upon &lt;a href=&quot;https://developers.google.com/google-apps/contacts/v3/&quot;&gt;this documentation page&lt;/a&gt; about the contacts API. This page has detailed information about how to use this API which is great. However, any information on how to use this API with just javascript is lacking. After more looking around I found a &lt;a href=&quot;https://developers.google.com/google-apps/contacts/v1/developers_guide_js&quot;&gt;page which does describe on how to do this with javascript&lt;/a&gt; but this version is deprecated.&lt;/p&gt;
  307.  
  308. &lt;h2 id=&quot;google-javascript-api-library&quot;&gt;Google javascript API library&lt;/h2&gt;
  309. &lt;p&gt;Google has a nice &lt;a href=&quot;https://developers.google.com/api-client-library/javascript/&quot;&gt;javascript library&lt;/a&gt; for using many Google APIs. The basic flow on using this library is as follows:&lt;/p&gt;
  310.  
  311. &lt;ol&gt;
  312.  &lt;li&gt;Load the library&lt;/li&gt;
  313.  &lt;li&gt;Set the API key&lt;/li&gt;
  314.  &lt;li&gt;Set the scopes&lt;/li&gt;
  315.  &lt;li&gt;Set the client ID&lt;/li&gt;
  316.  &lt;li&gt;Call an authentication method to get an access token&lt;/li&gt;
  317.  &lt;li&gt;Load an API with an http request&lt;/li&gt;
  318.  &lt;li&gt;Call the API&lt;/li&gt;
  319. &lt;/ol&gt;
  320.  
  321. &lt;p&gt;In the first 4 steps I did not run into any issues. However, when trying to load the contacts API, the server responded that it did not exist.&lt;/p&gt;
  322.  
  323. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-javascript&quot; data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;nx&quot;&gt;gapi&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;client&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;load&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;'contacts'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;'v3'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;);&lt;/span&gt;
  324.  
  325. &lt;span class=&quot;cm&quot;&gt;/*
  326. which results in the following error:
  327.  
  328. GET https://content.googleapis.com/discovery/v1/apis/contacts/v3/rest?fields=rootUrl%2CservicePath%2Cresources%2Cparameters%2Cmethods&amp;amp;pp=0 404 (Not Found)
  329. */&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  330.  
  331. &lt;p&gt;Loading any other API that was listed in the API explorer was no problem though as loading the urlshortener did not result in an error.&lt;/p&gt;
  332.  
  333. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-javascript&quot; data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;nx&quot;&gt;gapi&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;client&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;load&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;'urlshortener'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;'v1'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  334.  
  335. &lt;p&gt;This made me think that the javascript library for the contacts just did not exist. After more searching I found out that it indeed does not exist in this Stack Overflow issue &lt;a href=&quot;http://stackoverflow.com/questions/20017487/getting-google-contacts-with-javascript&quot;&gt;http://stackoverflow.com/questions/20017487/getting-google-contacts-with-javascript&lt;/a&gt;.&lt;/p&gt;
  336.  
  337. &lt;h1 id=&quot;solution&quot;&gt;Solution&lt;/h1&gt;
  338.  
  339. &lt;h2 id=&quot;approach&quot;&gt;Approach&lt;/h2&gt;
  340. &lt;p&gt;The approach to our solution, which is also described in the Stack Overflow issue, turns out to be pretty simple. When you have an access token, it is possible to call any API with a simple http request, eliminating the need of a specific JavaScript library that wraps this process. The process of retrieving an access token remains the same:&lt;/p&gt;
  341.  
  342. &lt;p&gt;1. Load the library&lt;br /&gt;
  343. 2. Set the API key&lt;br /&gt;
  344. 3. Set the scopes&lt;br /&gt;
  345. 4. Set the client ID&lt;br /&gt;
  346. 5. Call an authentication method to get an access token&lt;br /&gt;&lt;/p&gt;
  347.  
  348. &lt;p&gt;Now, instead of using the &lt;code class=&quot;highlighter-rouge&quot;&gt;gapi&lt;/code&gt; library, use this access token to call the API.&lt;/p&gt;
  349.  
  350. &lt;p&gt;6. Call the API using an http request with the access token retrieved&lt;/p&gt;
  351.  
  352. &lt;p&gt;For example, to retrieve the contacts of our user (I used the &lt;code class=&quot;highlighter-rouge&quot;&gt;get&lt;/code&gt; method from jQuery):&lt;/p&gt;
  353.  
  354. &lt;figure class=&quot;highlight&quot;&gt;&lt;pre&gt;&lt;code class=&quot;language-javascript&quot; data-lang=&quot;javascript&quot;&gt; &lt;span class=&quot;nx&quot;&gt;$&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;kd&quot;&gt;get&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;https://www.google.com/m8/feeds/contacts/default/thin?alt=json&amp;amp;access_token=&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;nx&quot;&gt;authResult&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nx&quot;&gt;access_token&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;+&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&amp;amp;max-results=700&amp;amp;v=3.0&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
  355. &lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;
  356.  
  357. &lt;p&gt;7. Process the results to serve the information you need.&lt;/p&gt;
  358.  
  359. &lt;p&gt;I do recommend to use the library whenever possible as this will reduce the amount of boilerplate code you have to write.&lt;/p&gt;
  360.  
  361. &lt;h2 id=&quot;code-example&quot;&gt;Code example&lt;/h2&gt;
  362. &lt;p&gt;As a result, this code snippet will provide you with the contacts of a user. To serve this file, I used a simple python command to start an http server in the correct directory with:&lt;/p&gt;
  363.  
  364. &lt;p&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;
  365. python -m SimpleHTTPServer
  366. &lt;/code&gt;&lt;/p&gt;
  367.  
  368. &lt;script src=&quot;https://gist.github.com/wouter-willems/574a70cf4bc2aee1b2f7.js&quot;&gt;&lt;/script&gt;
  369.  
  370. &lt;p&gt;When running this example, check the console’s log to see the result.
  371. For &lt;a href=&quot;https://magnet.me&quot;&gt;Magnet.me&lt;/a&gt; we changed this to an implementation with AngularJS but the approach remains the same.&lt;/p&gt;
  372. </description>
  373.        <pubDate>Mon, 11 May 2015 18:26:00 +0000</pubDate>
  374.        <link>https://labs.magnet.me/nerds/2015/05/11/importing-google-contacts-with-javascript.html</link>
  375.        <guid isPermaLink="true">https://labs.magnet.me/nerds/2015/05/11/importing-google-contacts-with-javascript.html</guid>
  376.  
  377.        <category>Tutorials</category>
  378.  
  379.  
  380.        <category>Nerds</category>
  381.  
  382.      </item>
  383.  
  384.  </channel>
  385. </rss>
  386.  

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

  1. Download the "valid RSS" banner.

  2. Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)

  3. Add this HTML to your page (change the image src attribute if necessary):

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=https%3A//labs.magnet.me/feed.xml

Copyright © 2002-9 Sam Ruby, Mark Pilgrim, Joseph Walton, and Phil Ringnalda