Congratulations!

[Valid RSS] This is a valid RSS feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

Source: http://sodoherty.com/feed/

  1. <?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
  2. xmlns:content="http://purl.org/rss/1.0/modules/content/"
  3. xmlns:wfw="http://wellformedweb.org/CommentAPI/"
  4. xmlns:dc="http://purl.org/dc/elements/1.1/"
  5. xmlns:atom="http://www.w3.org/2005/Atom"
  6. xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
  7. xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
  8. >
  9.  
  10. <channel>
  11. <title>Talk to me</title>
  12. <atom:link href="https://sodoherty.ai/feed/" rel="self" type="application/rss+xml" />
  13. <link>https://sodoherty.ai</link>
  14. <description>A blog about Chat bots (mainly Watson), and life in general.</description>
  15. <lastBuildDate>Sun, 03 Dec 2017 16:05:32 +0000</lastBuildDate>
  16. <language>en-US</language>
  17. <sy:updatePeriod>hourly</sy:updatePeriod>
  18. <sy:updateFrequency>1</sy:updateFrequency>
  19. <site xmlns="com-wordpress:feed-additions:1">31007449</site> <item>
  20. <title>Testing your intents</title>
  21. <link>https://sodoherty.ai/2017/12/03/testing-your-intents/</link>
  22. <comments>https://sodoherty.ai/2017/12/03/testing-your-intents/#respond</comments>
  23. <pubDate>Sun, 03 Dec 2017 16:05:32 +0000</pubDate>
  24. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  25. <category><![CDATA[Python]]></category>
  26. <category><![CDATA[watson-conversation]]></category>
  27.  
  28. <guid isPermaLink="false">http://sodoherty.ai/?p=2909</guid>
  29. <description><![CDATA[So this really only helps if you are doing a large number of intents, and you have not used entities as your primary method of determining intent. First lets talk about perceived accuracy, and what this is trying to solve. Perceived accuracy is where someone will type in a few questions they know the answer [&#8230;]]]></description>
  30. <content:encoded><![CDATA[<p>So this really only helps if you are doing a large number of intents, and you have not used entities as your primary method of determining intent.</p>
  31. <p>First lets talk about perceived accuracy, and what this is trying to solve. Perceived accuracy is where someone will type in a few questions they know the answer to. Then depending on their manual test they perceive the system to be working or failing.</p>
  32. <p>It puts the person training the system into a false sense of how it is performing.</p>
  33. <p>If you have done the <a href="https://www.watson-academy.info" target="_blank" rel="noopener">Watson Academy</a> training for Conversation you will hear it mention K-fold testing. For this blog post, I&#8217;m going to skip the details as <a href="https://sodoherty.ai/2016/10/16/the-road-to-good-intentions/" target="_blank" rel="noopener">I briefly mentioned before</a>.</p>
  34. <blockquote><p><strong>K-fold cross validation</strong> : You split your training set into random segments (K). Use one set to test and the rest to train. You then work your way through all of them. This method will test everything, but will be extremely time-consuming. Also you need to pick a good size for K so that you can test correctly.</p></blockquote>
  35. <p>K-Fold works well by itself if you have a large training set that has come from a real world representative users. You will find this rarely happens. So you should use in conjunction with a blind.</p>
  36. <p>Previously I didn&#8217;t cover how you actually do the test. So with that, here is the notebook giving a demonstration:</p>
  37. <p>View the code on <a href="https://gist.github.com/sod1684/e6b4a82f4a8b5789b9f8cb3033badf02">Gist</a>.</p>
  38. <p>&nbsp;</p>
  39. <p>&nbsp;</p>
  40. ]]></content:encoded>
  41. <wfw:commentRss>https://sodoherty.ai/2017/12/03/testing-your-intents/feed/</wfw:commentRss>
  42. <slash:comments>0</slash:comments>
  43. <post-id xmlns="com-wordpress:feed-additions:1">2909</post-id> </item>
  44. <item>
  45. <title>Pushing My Buttons</title>
  46. <link>https://sodoherty.ai/2017/10/24/pushing-my-buttons/</link>
  47. <comments>https://sodoherty.ai/2017/10/24/pushing-my-buttons/#respond</comments>
  48. <pubDate>Tue, 24 Oct 2017 15:50:38 +0000</pubDate>
  49. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  50. <category><![CDATA[tips]]></category>
  51. <category><![CDATA[watson-conversation]]></category>
  52.  
  53. <guid isPermaLink="false">http://sodoherty.ai/?p=2797</guid>
  54. <description><![CDATA[So this is a long time pet peeve, but recently I have seen a load of these in succession. I am sure a lot of people who know me are going to read this and think &#8220;He&#8217;s talking about me&#8221;. Truth is there is no one person I am pointing my finger at. Let me [&#8230;]]]></description>
  55. <content:encoded><![CDATA[<p>So this is a long time pet peeve, but recently I have seen a load of these in succession. I am sure a lot of people who know me are going to read this and think &#8220;He&#8217;s talking about me&#8221;. Truth is there is no one person I am pointing my finger at.</p>
  56. <p>Let me start with what triggered this post. Have a look at this screen shot. There are three things wrong with it, although one of the reasons is not visible, but you can guess.</p>
  57. <p><img data-attachment-id="2804" data-permalink="https://sodoherty.ai/2017/10/24/pushing-my-buttons/pooruxdesign/" data-orig-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/10/pooruxdesign.png?fit=712%2C454&amp;ssl=1" data-orig-size="712,454" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="poorUXdesign" data-image-description="" data-medium-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/10/pooruxdesign.png?fit=300%2C191&amp;ssl=1" data-large-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/10/pooruxdesign.png?fit=712%2C454&amp;ssl=1" class="  wp-image-2804 aligncenter" src="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/10/pooruxdesign.png?resize=336%2C200&#038;ssl=1" alt="poorUXdesign" width="336" height="200" data-recalc-dims="1" /></p>
  58. <p>So disclosure, this is a competitors chat bot, it is also a common pattern I have seen on that chat bot. But I have also seen people do this with Watson Conversation.</p>
  59. <p style="text-align:center;"><em><strong>Did you guess the issues? </strong></em></p>
  60. <p><strong>Issue 1:</strong> Never ask the end user did you answer them correctly or not. If your system is well trained, and tested then you are going to know if it answered well or not.</p>
  61. <p>Those who think of a rebuttal to this, imagine you rang a customer support person and they asked you &#8220;Did I answer you correctly&#8221; every time they gave an answer? What would your action be? More than likely you would ask to speak to someone who does know what they are talking about.</p>
  62. <p>If you really need to get feedback, make it subtle, or ask for a survey at the end.</p>
  63. <p><strong>Issue 2: </strong>BUTTONS. I don&#8217;t know who started this button trend, but it has to die. You are not building a cognitive conversational system. You are building an application. You don&#8217;t need an AI for buttons, any average developer can build you a button based &#8220;<a href="https://en.wikipedia.org/wiki/Choose_Your_Own_Adventure" target="_blank" rel="noopener">Choose your own adventure</a>&#8220;.</p>
  64. <p><strong>Issue 3: </strong>Not visible on the image is that you are stuck until you click on yes or no. You couldn&#8217;t say yes or no, or &#8220;I am not sure&#8221;. For that matter I have seen cases where the answer is poorly written and the person would take the wrong answer as right, so what happens then?  For that matter selecting yes or no does nothing to progress the conversation.</p>
  65. <p>So what is the root cause in all this?  From what I have seen normally it is one thing.</p>
  66. <h2 style="text-align:center;"><strong>Developers</strong>.</h2>
  67. <p>Because older chat bots required a developer to build, it has sort of progressed along those lines for some time. In fact some chat bot companies tout the fact that it is developer orientated, and in some cases only offer code based systems.</p>
  68. <p>I&#8217;ve also gotten to listen to some developers tell me how Watson Conversation sucks (because &#8220;tensor flow&#8221;), or they could write better. I normally tell them to try.</p>
  69. <p>Realistically to make a good chat bot, the developer is generally far down the food chain in that creation. Watson conversation is targeted at your non-technical person.</p>
  70. <p>Heres a little graphic to help.</p>
  71. <p><img data-attachment-id="2839" data-permalink="https://sodoherty.ai/2017/10/24/pushing-my-buttons/beep-beep-robot/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/10/beep-beep-robot2.png?fit=830%2C778&amp;ssl=1" data-orig-size="830,778" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="beep beep robot" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/10/beep-beep-robot2.png?fit=300%2C281&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/10/beep-beep-robot2.png?fit=756%2C709&amp;ssl=1" class=" size-full wp-image-2839 alignnone" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/10/beep-beep-robot2.png?resize=756%2C709&#038;ssl=1" alt="beep beep robot" width="756" height="709" data-recalc-dims="1" /></p>
  72. <p style="text-align:left;">Now your chances of getting all these is hard, but the people you do get should have some skills in these areas. Let&#8217;s expand on each one.</p>
  73. <h2>Business Analyst</h2>
  74. <p>By far the most important, certainly at the start of the project.</p>
  75. <p>Most failed chat bot projects are because someone who knows the business hasn&#8217;t objectively looked at what it is you are trying to solve, and if it is even worth the time.</p>
  76. <p>By the same token, I have seen two business analysts create a conversational bot that on the face of it looked simple, but they could show that it saved over a million euros a year. All built in a day and a half. Because they knew the business and where to get the data.</p>
  77. <h2>Conversational Copywriter</h2>
  78. <p>Normally even getting a copywriter makes a huge difference, but one with actual conversational experience makes the solution shine. It&#8217;s the difference between something clinical, and something your end user can make an emotional attachment to.</p>
  79. <h2>Behavioural Scientist</h2>
  80. <p>Another thing I see all the time. You get an issue in the chat conversation that requires some complexity to solve. So you have your developer telling you how they can build something custom and complex to solve the issue (probably includes tensor flow somewhere in all of it).</p>
  81. <p>Your behavioural expert on the other hand will suggest changing the message you tell the end user. It&#8217;s really that simple, but often missed by people without experience in this area.</p>
  82. <h2>Subject Matter Expert (SME)</h2>
  83. <p>To be fair, at least on projects I&#8217;ve seen there is normally an SME there. But there are still different levels of SMEs. For example your expert in the material, may not be the expert that deals with the customer.</p>
  84. <p>But it is dangerous to think that just because you have a manual you can reference, that you are capable to building a system that can answer questions as if it is an SME.</p>
  85. <h2>Data Scientist</h2>
  86. <p>While you might not need a full blown one, all good conversational solutions are data driven. In what people ask, behaviours exhibited and needs met. Having someone able to sift through the existing data and make sense of it, helps make a good system.</p>
  87. <p>Also almost every engagement I&#8217;ve been on, people will tell you what they think the end user will say or do. But often it is never the case, and the data shows this.</p>
  88. <h2>UI/UX</h2>
  89. <p>What the conversational copywriter does for the engaging conversation, the UI/UX does for the system. If you are using existing channels like Facebook, Skype, Messenger, Slack, etc.. then you probably don&#8217;t need to worry as much. But it&#8217;s still possible to create something that can upset the user without good UX experience.</p>
  90. <p>It&#8217;s also a broad skill area. For example, UX for Web is very different to Mobile, IVR, and Robots.</p>
  91. <h2>Machine Learning</h2>
  92. <p>Watson conversation abstracts the ML layer from the end user. You only need to know how to cluster questions correctly. But knowing how to do K-Fold cross validation, or the importance of blind sets helps in training the system well.</p>
  93. <p>It also helps if your developers have at least a basic understanding of machine learning.</p>
  94. <p>I often see non-ML developers trying to fix clusters with comments like. &#8220;It used this keyword 3 times, so that&#8217;s why it picked this over that&#8221;, which is not how it works at all.</p>
  95. <p>It also prevents your developers (if they code the bot) to create something that is entity heavy. Non-ML Developers seem to like entities, as they can wrap their head around them. Fixed keywords, regex, all makes sense to a developer, but in the long run make the system unmaintainable (basically defeats the purpose of using Watson conversation).</p>
  96. <h2>Natural Language Processing (NLP)</h2>
  97. <p>I&#8217;ve made this the smallest. There was a time, certainly with the early versions of Watson you needed these skills. Not so much anymore. Still, it&#8217;s good to understand the basics of NLP, certainly for entities.</p>
  98. <h2>Developer</h2>
  99. <p>In the scheme of things, there will always be a place for the developer.</p>
  100. <p>You have UI development, application layer, back-end and integration, automation, testing, and so on.</p>
  101. <p>Just development skills alone will not help you in building something that the end user can feel a connection to.</p>
  102. <h3><em><strong>&#8230; and please, stop using buttons. </strong></em></h3>
  103. ]]></content:encoded>
  104. <wfw:commentRss>https://sodoherty.ai/2017/10/24/pushing-my-buttons/feed/</wfw:commentRss>
  105. <slash:comments>0</slash:comments>
  106. <post-id xmlns="com-wordpress:feed-additions:1">2797</post-id> </item>
  107. <item>
  108. <title>Chihuahua or Muffin, revisited.</title>
  109. <link>https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/</link>
  110. <comments>https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/#respond</comments>
  111. <pubDate>Thu, 28 Sep 2017 13:25:23 +0000</pubDate>
  112. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  113. <category><![CDATA[visual recognition]]></category>
  114. <category><![CDATA[watson]]></category>
  115.  
  116. <guid isPermaLink="false">http://sodoherty.ai/?p=2732</guid>
  117. <description><![CDATA[I just finished reading Maria Yao&#8216;s article &#8220;Chihuahua OR Muffin? Searching For The Best Computer Vision API&#8220;. It&#8217;s a fun read, but I felt it didn&#8217;t really show off the power of Watson Visual Recognition. For the demo in the article, the general classifier was being used. One of the main advantages of Watson Visual [&#8230;]]]></description>
  118. <content:encoded><![CDATA[<p>I just finished reading <a href="https://twitter.com/thinkmariya" target="_blank" rel="noopener">Maria Yao</a>&#8216;s article <em>&#8220;<a href="https://www.topbots.com/chihuahua-muffin-searching-best-computer-vision-api/" target="_blank" rel="noopener">Chihuahua OR Muffin? Searching For The Best Computer Vision API</a>&#8220;</em>. It&#8217;s a fun read, but I felt it didn&#8217;t really show off the power of Watson Visual Recognition.</p>
  119. <p>For the demo in the article, the general classifier was being used.</p>
  120. <p>One of the main advantages of Watson Visual recognition is that you can create your own custom classifiers. It is very simple too.</p>
  121. <h3>First, you need data.</h3>
  122. <p>Using Marias article I pulled the <a href="http://image-net.org/synset?wnid=n02085620" target="_blank" rel="noopener">Chihuahua</a> and <a href="http://image-net.org/synset?wnid=n07690273" target="_blank" rel="noopener">Muffin</a> pictures from ImageNet.</p>
  123. <p>Like most data, it tends to need a bit of cleaning. So I deleted any images below 14KB in size. The reason for this was the majority at that size were just corrupted. I also went through and deleted any images which were adverts or &#8220;this image is no longer there&#8221; banners.</p>
  124. <p>Overall that was 500 images deleted totally. It still left 3,000 images to play with.</p>
  125. <p>Next I created a <a href="https://console.bluemix.net/catalog/services/visual-recognition?env_id=ibm:yp:eu-gb" target="_blank" rel="noopener">Visual Recognition service</a>. For this I created the free version. This limited me to 250 events a day. So I had to lower my training sets to 100 pictures from each set.</p>
  126. <p>I took a random 100 from each. I didn&#8217;t examine the photos at all, but here are a few to give you an idea of how the images look.</p>
  127. <p><img data-attachment-id="2759" data-permalink="https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/example_training/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/example_training.png?fit=750%2C375&amp;ssl=1" data-orig-size="750,375" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="example_training" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/example_training.png?fit=300%2C150&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/example_training.png?fit=750%2C375&amp;ssl=1" class="alignnone size-full wp-image-2759" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/example_training.png?resize=750%2C375&#038;ssl=1" alt="example_training" width="750" height="375" data-recalc-dims="1" /></p>
  128. <p>As you can see, no thought put into worrying about other items in the picture.</p>
  129. <p>I zipped the images up, and created a classifier like so.</p>
  130. <p><img data-attachment-id="2764" data-permalink="https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/classifier/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/classifier.png?fit=1042%2C468&amp;ssl=1" data-orig-size="1042,468" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="classifier" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/classifier.png?fit=300%2C135&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/classifier.png?fit=756%2C340&amp;ssl=1" class="alignnone size-full wp-image-2764" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/classifier.png?resize=756%2C340&#038;ssl=1" alt="classifier" width="756" height="340" data-recalc-dims="1" /></p>
  131. <p>Then I clicked create, and waited a little over 10 minutes for it to analyse the pictures.</p>
  132. <p>Once the classifier was finished training, I was ready to test. As some may not be aware, Visual Recognition also offers a food classifier. So for my first two tests I tried my classifier, General and Food.</p>
  133. <p><img data-attachment-id="2768" data-permalink="https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/test1/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/test1.png?fit=1010%2C699&amp;ssl=1" data-orig-size="1010,699" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="test1" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/test1.png?fit=300%2C208&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/test1.png?fit=756%2C523&amp;ssl=1" class="alignnone size-full wp-image-2768" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/test1.png?resize=756%2C523&#038;ssl=1" alt="test1" width="756" height="523" data-recalc-dims="1" /></p>
  134. <p>So you can see the red bar on the classifier I made. This is more because I only gave 100 examples. As you give more training examples, it&#8217;s confidence increases. But you can see that the difference between Muffin and Chihuahua is clear.</p>
  135. <p>You can also see the food classifier got it as well.</p>
  136. <p>What about the Chihuahua?</p>
  137. <p><img data-attachment-id="2776" data-permalink="https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/test2/" data-orig-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/09/test2.png?fit=1017%2C760&amp;ssl=1" data-orig-size="1017,760" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="test2" data-image-description="" data-medium-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/09/test2.png?fit=300%2C224&amp;ssl=1" data-large-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/09/test2.png?fit=756%2C565&amp;ssl=1" class="alignnone size-full wp-image-2776" src="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/09/test2.png?resize=756%2C565&#038;ssl=1" alt="test2" width="756" height="565" data-recalc-dims="1" /></p>
  138. <p>As you can see, all three do quite well on classification. But what about the original pictures which look similar? I ran those through and ended up with this.</p>
  139. <p><img data-attachment-id="2781" data-permalink="https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/results2/" data-orig-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/09/results2.png?fit=592%2C826&amp;ssl=1" data-orig-size="592,826" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="results2" data-image-description="" data-medium-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/09/results2.png?fit=215%2C300&amp;ssl=1" data-large-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/09/results2.png?fit=592%2C826&amp;ssl=1" class="alignnone size-full wp-image-2781" src="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/09/results2.png?resize=592%2C826&#038;ssl=1" alt="results2" width="592" height="826" data-recalc-dims="1" /></p>
  140. <p>As you can see it got them all right! None of these were used to train against.</p>
  141. <p>As demos go though this is simple and fun. But with good classified images, it can be scary accurate for proper real world use cases.</p>
  142. <p>Having said that, I did have one failure. Testing the samples on Maria&#8217;s page, it was able to understand the cookie monster muffin and the man holding the chihuahua.</p>
  143. <p>But the muffin in the plastic bag with the Chihuahua it could not get. I tried cropping out the dog, but it still failed but with a lower confidence. I suspect this is a combination of training and a bad quality photo.</p>
  144. <p><img data-attachment-id="2790" data-permalink="https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/screen-shot-2017-09-28-at-17-40-17/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-28-at-17-40-17.png?fit=292%2C432&amp;ssl=1" data-orig-size="292,432" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="Screen Shot 2017-09-28 at 17.40.17" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-28-at-17-40-17.png?fit=203%2C300&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-28-at-17-40-17.png?fit=292%2C432&amp;ssl=1" class="alignnone size-full wp-image-2790 aligncenter" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-28-at-17-40-17.png?resize=292%2C432&#038;ssl=1" alt="Screen Shot 2017-09-28 at 17.40.17" width="292" height="432" data-recalc-dims="1" /></p>
  145. ]]></content:encoded>
  146. <wfw:commentRss>https://sodoherty.ai/2017/09/28/chihuahua-or-muffin-revisited/feed/</wfw:commentRss>
  147. <slash:comments>0</slash:comments>
  148. <post-id xmlns="com-wordpress:feed-additions:1">2732</post-id> </item>
  149. <item>
  150. <title>I have no confidence in Entities.</title>
  151. <link>https://sodoherty.ai/2017/09/10/i-have-no-confidence-in-entities/</link>
  152. <comments>https://sodoherty.ai/2017/09/10/i-have-no-confidence-in-entities/#respond</comments>
  153. <pubDate>Sun, 10 Sep 2017 15:51:35 +0000</pubDate>
  154. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  155. <category><![CDATA[watson]]></category>
  156. <category><![CDATA[watson-conversation]]></category>
  157.  
  158. <guid isPermaLink="false">http://sodoherty.ai/?p=2479</guid>
  159. <description><![CDATA[I have something I need to confess. I have a personal hatred of Entities. At least in their current form. There is a difference between deterministic and probabilisitic programming, that a lot of developers new to Watson find it hard to switch to. Entities bring them back to that warm place of normal development. For [&#8230;]]]></description>
  160. <content:encoded><![CDATA[<p>I have something I need to confess. I have a personal hatred of Entities. At least in their current form.</p>
  161. <p>There is a difference between deterministic and probabilisitic programming, that a lot of developers new to Watson find it hard to switch to. Entities bring them back to that warm place of normal development.</p>
  162. <p>For example, you are tasked with creating a learning system for selling <strong>Cats</strong>, <strong>Dogs</strong>, and <strong>Fishes</strong>. Collecting questions you get this:</p>
  163. <ul>
  164. <li>I want to get a kitten</li>
  165. <li>I want to buy a cat</li>
  166. <li>Can I get a calico cat?</li>
  167. <li>I want to get a siamese cat</li>
  168. <li>Please may I have a kitty?</li>
  169. <li>My wife loves kittens. I want to get her one as a present.</li>
  170. <li>I want to buy a dog</li>
  171. <li>Can I get a puppy?</li>
  172. <li>I would like to purchase a puppy</li>
  173. <li>Please may I have a dog?</li>
  174. <li>Sell me a puppy</li>
  175. <li>I would love to get a hound for my wife.</li>
  176. <li>I want to buy a fish</li>
  177. <li>Can I get a fish?</li>
  178. <li>I want to purchase some fishes</li>
  179. <li>I love fishies</li>
  180. <li>I want a goldfish</li>
  181. </ul>
  182. <p>The first instinct is to create a single intent of <strong>#PURCHASE_ANIMAL</strong> and the create entities for the cats, dogs and fishes. Because it&#8217;s easier to wrap your head around entities, then it is to wonder how Watson will respond.</p>
  183. <p>So you end up with something like this:</p>
  184. <p><img data-attachment-id="2638" data-permalink="https://sodoherty.ai/2017/09/10/i-have-no-confidence-in-entities/20170910entities/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entities.png?fit=739%2C216&amp;ssl=1" data-orig-size="739,216" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="20170910entities" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entities.png?fit=300%2C88&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entities.png?fit=739%2C216&amp;ssl=1" class="alignnone size-full wp-image-2638" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entities.png?resize=739%2C216&#038;ssl=1" alt="20170910entities" width="739" height="216" data-recalc-dims="1" /></p>
  185. <p>Wow! So easy! Let&#8217;s set up our dialog. To make it easier, lets use a slot.</p>
  186. <p><img data-attachment-id="2641" data-permalink="https://sodoherty.ai/2017/09/10/i-have-no-confidence-in-entities/20170910entitiesa/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesa.png?fit=781%2C275&amp;ssl=1" data-orig-size="781,275" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="20170910entitiesA" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesa.png?fit=300%2C106&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesa.png?fit=756%2C266&amp;ssl=1" class="alignnone size-full wp-image-2641" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesa.png?resize=756%2C266&#038;ssl=1" alt="20170910entitiesA" width="756" height="266" data-recalc-dims="1" /></p>
  187. <p>In under a minute, I have created a system that can help someone pick an animal to buy. You even test it and it works perfectly.</p>
  188. <h1 style="text-align:center;"><strong>IT&#8217;S A TRAP!</strong></h1>
  189. <p>First the biggest red flag with this is you have now turned your conversation into a deterministic system.</p>
  190. <p>Still doing cross validation to test your intents? Give up, it&#8217;s pointless.</p>
  191. <p>You can break it by just typing something like <em><strong>&#8220;I want to buy a bulldog&#8221;</strong></em>. You are stuck into an endless loop.</p>
  192. <p>The easiest solution is to tell the person what to type, or link/button it. But it doesn&#8217;t exhibit intelligence (and I hate buttons more than I hate entities <img src="https://s.w.org/images/core/emoji/2.3/72x72/1f642.png" alt="🙂" class="wp-smiley" style="height: 1em; max-height: 1em;" /> ).</p>
  193. <p>The other option is to add &#8220;<strong>bulldog</strong>&#8221; to the <strong>@Animals:Dog</strong> entity. But when you go down that rabbit hole, you could realistically add the following.</p>
  194. <ul>
  195. <li>500+ types of breeds.</li>
  196. <li>Common misspellings of those words.</li>
  197. <li>plurals of each breed.</li>
  198. <li>slang, variations and nicknames of those animals.</li>
  199. </ul>
  200. <p>You are easily into the thousands of keywords to match, and all it takes is one person to make a typo you don&#8217;t have in the list and it still won&#8217;t work.</p>
  201. <h2>Using entities in a probabilistic way.</h2>
  202. <p>All is not lost! You can still use entities, and keep your system intelligent. First we break up the intents into the types of animals like so:</p>
  203. <p><img data-attachment-id="2672" data-permalink="https://sodoherty.ai/2017/09/10/i-have-no-confidence-in-entities/20170910entitiesb/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesb.png?fit=241%2C263&amp;ssl=1" data-orig-size="241,263" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="20170910entitiesB" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesb.png?fit=241%2C263&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesb.png?fit=241%2C263&amp;ssl=1" class="alignnone size-full wp-image-2672" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesb.png?resize=241%2C263&#038;ssl=1" alt="20170910entitiesB" width="241" height="263" data-recalc-dims="1" /></p>
  204. <p>So now if I type <em><strong>&#8220;I want to buy a bulldog&#8221;</strong></em> I get <strong>#PurchaseDog</strong> with <strong>68%</strong> confidence. Which is great, as I didn&#8217;t even train it on that word.</p>
  205. <p>So next I try <em><strong>&#8220;I want to buy a pet&#8221;</strong></em> and I get <strong>#PurchaseCat</strong> with <strong>55%</strong> confidence.</p>
  206. <p><img data-attachment-id="2724" data-permalink="https://sodoherty.ai/2017/09/10/i-have-no-confidence-in-entities/20170910entitiescat/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiescat1.png?fit=400%2C400&amp;ssl=1" data-orig-size="400,400" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="20170910entitiesCat" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiescat1.png?fit=300%2C300&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiescat1.png?fit=400%2C400&amp;ssl=1" class="size-full wp-image-2724 aligncenter" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiescat1.png?resize=400%2C400&#038;ssl=1" alt="20170910entitiesCat" width="400" height="400" data-recalc-dims="1" /></p>
  207. <p>Hmm, great for cat lovers but we want conversation to be not so sure about this.</p>
  208. <p>So we create the entities as before for Cat, Dog, Fish. You can use the same values.</p>
  209. <p>Next before you check intents, add a node with the following condition.</p>
  210. <p><img data-attachment-id="2687" data-permalink="https://sodoherty.ai/2017/09/10/i-have-no-confidence-in-entities/20170910entitiesc/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesc.png?fit=505%2C82&amp;ssl=1" data-orig-size="505,82" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="20170910entitiesC" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesc.png?fit=300%2C49&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesc.png?fit=505%2C82&amp;ssl=1" class="alignnone size-full wp-image-2687" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/20170910entitiesc.png?resize=505%2C82&#038;ssl=1" alt="20170910entitiesC" width="505" height="82" data-recalc-dims="1" /></p>
  211. <p>This basically ensures that irrelevant hasn&#8217;t been hit, and then checks if the animal entities have not been mentioned.</p>
  212. <p>Then in your JSON response you add the following code.</p>
  213. <pre>{
  214.    "context": {
  215.    "adjust_confidence": "&lt;? intents[0].confidence = intents[0].confidence - 0.36 ?&gt;"
  216.    },
  217.    "output": {
  218.        "text": {
  219.            "values": [],
  220.            "selection_policy": "sequential"
  221.        }
  222.    }
  223. }</pre>
  224. <p>The important part is the <strong>&#8220;adjust_confidence&#8221;</strong> context variable. This will lower the first intents confidence by 0.36 (36%).</p>
  225. <p>We set the node to jump to the next node in line, so it can check the intents.</p>
  226. <p>Now we get &#8220;I don&#8217;t understand&#8221; for the pet question. Bulldog still works as it doesn&#8217;t fall below the 20%.</p>
  227. <h3>Demo Details.</h3>
  228. <p>I used 36% for the demo, but this will vary in other projects. Also if your confidence level is too high, you can pick a smaller value, and then have another check for a lower bound. In other words, set your conversation to ignore any intent with a confidence lower then 30%, and then set your adjustment confidence to -10%.</p>
  229. <h2>Advantages</h2>
  230. <p>Using this approach, you don&#8217;t need to worry as much with training your entities, only your intents. This allows you to build a probabilistic model which isn&#8217;t impacted unless it is unsure to begin with.</p>
  231. <p>I have supplied a <a title="EntitiesAndConfidence" href="https://sodoherty.ai/wp-content/uploads/2017/09/entitiesandconfidence1.zip">Sample Conversation</a> which demos above.</p>
  232. ]]></content:encoded>
  233. <wfw:commentRss>https://sodoherty.ai/2017/09/10/i-have-no-confidence-in-entities/feed/</wfw:commentRss>
  234. <slash:comments>0</slash:comments>
  235. <post-id xmlns="com-wordpress:feed-additions:1">2479</post-id> </item>
  236. <item>
  237. <title>Manufacturing Intent</title>
  238. <link>https://sodoherty.ai/2017/09/09/manufacturing-intent/</link>
  239. <comments>https://sodoherty.ai/2017/09/09/manufacturing-intent/#respond</comments>
  240. <pubDate>Sat, 09 Sep 2017 15:10:49 +0000</pubDate>
  241. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  242. <category><![CDATA[spacy]]></category>
  243. <category><![CDATA[watson]]></category>
  244.  
  245. <guid isPermaLink="false">http://sodoherty.ai/?p=2466</guid>
  246. <description><![CDATA[Let me start this article with a warning:  Manufacturing questions causes more problems than it solves. Sure the documentation, and many videos say the reverse. But they tend to give examples that have a narrow scope. Take the car demo for example. It works because there is a common domain language that everyone who uses [&#8230;]]]></description>
  247. <content:encoded><![CDATA[<p>Let me start this article with a warning:  <strong>Manufacturing questions causes more problems than it solves</strong>.</p>
  248. <p>Sure the documentation, and many videos say the reverse. But they tend to give examples that have a narrow scope.</p>
  249. <p>Take the <a href="https://conversation-demo.mybluemix.net">car demo</a> for example. It works because there is a common domain language that everyone who uses a car knows. For someone who has never seen a car before, won&#8217;t understand what a &#8220;window wiper&#8221; is, but they may say something like &#8220;I can&#8217;t see out the window, because it is raining&#8221;.</p>
  250. <p>This is why when building your conversation system, it is important to get questions from people who actually use the system, but don&#8217;t know the content. They tend not to know how to ask a question to get the answer.</p>
  251. <p>But there are times when it can&#8217;t be avoided. For example, you might be creating a system that has no end users yet. In this case, manufacturing questions can help bootstrap the system.</p>
  252. <p>There are some things to be aware of.</p>
  253. <h2>Manual creation.</h2>
  254. <p>This is actually very hard, even for the experienced. Here are the things you need to be aware of.</p>
  255. <h3>Your education and culture will shape what you write.</h3>
  256. <p>You can&#8217;t avoid it. Even if you are aware of this, you will fall back into the same patterns as you progress through creating questions. It&#8217;s not easy to see until you have a large sample. Sorting can give you a quick glance, while a <a href="https://en.wikipedia.org/wiki/Bag-of-words_model">bag of words</a> makes it more evident.</p>
  257. <h3>If you know the content, you will write what you know.</h3>
  258. <p>Again, having knowledge of the systems answers will have you writing domain language in the questions. You will use terms that define the system, rather than describing what it does.</p>
  259. <h3>If you don&#8217;t know the content, use user stories.</h3>
  260. <p>If you manage to get someone who could be a representative user, be careful in how you ask them to write questions. If they don&#8217;t fully understand what you ask, they will use terms as keywords, rather than their underlying meaning.</p>
  261. <p>Let&#8217;s compare two user stories:</p>
  262. <ul>
  263. <li>&#8220;Ask questions about using the window wipers.&#8221;</li>
  264. <li>&#8220;It is raining outside while you are driving, and it is getting harder to see. How might you ask the car to help you?&#8221;</li>
  265. </ul>
  266. <p>With the first example, you will find that people will use &#8220;window wipers&#8221;, &#8220;wipers&#8221;, &#8220;window&#8221; frequently. Most of the questions will be about switching it on/off.</p>
  267. <p>With the second example, you may end up seeing questions like this.</p>
  268. <ul>
  269. <li>Switch on the windshield wipers.</li>
  270. <li>Activate the windscreen wipers.</li>
  271. <li>Is my rain sensor activated?</li>
  272. <li>Please clear the windows.</li>
  273. </ul>
  274. <h3>Your final application will shape the questions as well.</h3>
  275. <p>If you have your question creation team working on a desktop machine, they are going to create questions which won&#8217;t be the same as someone typing on mobile, or talking to a robot.</p>
  276. <h3>The internet can be your friend.</h3>
  277. <p>Looking for similar questions online in forums can help you in seeing terms that people may use. For example, all these mean the same thing: &#8220;NCT&#8221;, &#8220;MOT&#8221;, &#8220;Smog Test&#8221;, &#8220;RWC&#8221;, &#8220;WoF&#8221;, &#8220;COF&#8221;.</p>
  278. <p>But those are meaningless to people if they are in different countries.</p>
  279. <h2>Automated Creation</h2>
  280. <p>A lot of what I have seen in automation tends not to fare much better. If it did, we wouldn&#8217;t need people to write questions. <img src="https://s.w.org/images/core/emoji/2.3/72x72/1f642.png" alt="🙂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p>
  281. <p>One technique is to try and create a few questions from existing questions. Again I should stress, this is generally a bad idea, and this example doesn&#8217;t work well, but might give you something to build on.</p>
  282. <p>Take this example.</p>
  283. <ul>
  284. <li>Can my child purchase a puppy?</li>
  285. <li>Are children allowed to buy dogs?</li>
  286. </ul>
  287. <p>From a manual view we can see the intent is to allow minors to buy. Going over to the code.</p>
  288. <p>For this I am using <a href="https://spacy.io" target="_blank" rel="noopener">Spacy</a>, which can check the similarity of each word against each other. For example.</p>
  289. <pre>import spacy
  290. nlp = spacy.load('en')
  291.  
  292. dog = nlp(u'dog')
  293. puppy = nlp(u'puppy')
  294. print( dog.similarity(puppy))
  295. </pre>
  296. <p>Will output: 0.760806754875</p>
  297. <p>The higher the number, the closer the words to each other. By setting a threshold on the value, you can reduce it to the important words. Setting a threshold of 0.7, we get:</p>
  298. <p><img data-attachment-id="2564" data-permalink="https://sodoherty.ai/2017/09/09/manufacturing-intent/screen-shot-2017-09-09-at-18-31-18/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-09-at-18-31-18.png?fit=860%2C258&amp;ssl=1" data-orig-size="860,258" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="Screen Shot 2017-09-09 at 18.31.18" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-09-at-18-31-18.png?fit=300%2C90&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-09-at-18-31-18.png?fit=756%2C227&amp;ssl=1" class="alignnone size-full wp-image-2564" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-09-at-18-31-18.png?resize=756%2C227&#038;ssl=1" alt="Screen Shot 2017-09-09 at 18.31.18" width="756" height="227" data-recalc-dims="1" /></p>
  299. <p>Playing with larger questions you will find that certain parts of speech (POS) are more noise. So you can drop the following to remove possible noise.</p>
  300. <ul>
  301. <li>DET = determiner</li>
  302. <li>PART = particle</li>
  303. <li>CCONJ = conjunction</li>
  304. <li>ADP = adposition</li>
  305. </ul>
  306. <p>Now that you have reduced it to the main terms, you can build a synonym off of these, like so:</p>
  307. <pre>dog = nlp('dog')
  308. word = nlp.vocab[dog.text]
  309. sym = sorted(word.vocab,
  310.             key=lambda w: word.similarity(w),
  311.             reverse=True
  312. )
  313. print('\n'.join([w.orth_ for w in sym[:10]]))</pre>
  314. <p>&nbsp;</p>
  315. <p>Which will print out the following:</p>
  316. <ul>
  317. <li>dog</li>
  318. <li>DOG</li>
  319. <li>Dog</li>
  320. <li>dogs</li>
  321. <li>DOGS</li>
  322. <li>Dogs</li>
  323. <li>puppy</li>
  324. <li>Puppy</li>
  325. <li>PUPPY</li>
  326. <li>pet</li>
  327. </ul>
  328. <p>As you can see, a lot of repetition. You can remove duplicates. Also be wary to set the upper bound of the <strong>sym</strong> object when reading.</p>
  329. <p>So after you generate a group of sample synonyms, you end up with something like this.</p>
  330. <p><img data-attachment-id="2590" data-permalink="https://sodoherty.ai/2017/09/09/manufacturing-intent/screen-shot-2017-09-09-at-18-50-07/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-09-at-18-50-07.png?fit=1936%2C254&amp;ssl=1" data-orig-size="1936,254" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="Screen Shot 2017-09-09 at 18.50.07" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-09-at-18-50-07.png?fit=300%2C39&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-09-at-18-50-07.png?fit=756%2C99&amp;ssl=1" class="alignnone size-full wp-image-2590" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/09/screen-shot-2017-09-09-at-18-50-07.png?resize=756%2C99&#038;ssl=1" alt="Screen Shot 2017-09-09 at 18.50.07" width="756" height="99" data-recalc-dims="1" /></p>
  331. <p>Now it&#8217;s a simple matter of just generating a random set of questions from this table. You end up with something like:</p>
  332. <ul>
  333. <li>need children purchasing dogs</li>
  334. <li>can kids buying puppy</li>
  335. <li>may child purchases dog</li>
  336. <li>will children purchase dogs</li>
  337. <li>will kids buy pets</li>
  338. <li>make children cheap puppies</li>
  339. <li>need children purchase puppies</li>
  340. </ul>
  341. <p>As you can see, they are pretty bad. Not because of the word salad, but that you have a very narrow scope of what can be answered. But it can give you enough to have your intent trigger.</p>
  342. <p>You can also mitigate this by creating a tensor of a number of similar questions, n-grams instead of single words, a custom domain dictionary and increasing your dictionary terms.</p>
  343. <p>At the end of the day though, they are still going to be manufactured.</p>
  344. ]]></content:encoded>
  345. <wfw:commentRss>https://sodoherty.ai/2017/09/09/manufacturing-intent/feed/</wfw:commentRss>
  346. <slash:comments>0</slash:comments>
  347. <post-id xmlns="com-wordpress:feed-additions:1">2466</post-id> </item>
  348. <item>
  349. <title>Anaphora? I hardly knew her.</title>
  350. <link>https://sodoherty.ai/2017/07/15/anaphora-i-hardly-knew-her/</link>
  351. <comments>https://sodoherty.ai/2017/07/15/anaphora-i-hardly-knew-her/#respond</comments>
  352. <pubDate>Sat, 15 Jul 2017 06:21:45 +0000</pubDate>
  353. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  354. <category><![CDATA[tips]]></category>
  355. <category><![CDATA[watson-conversation]]></category>
  356.  
  357. <guid isPermaLink="false">http://sodoherty.ai/?p=1929</guid>
  358. <description><![CDATA[One of common requests for conversation is being able to understand the running topic of a conversation. For example: USER: Can I feed my goldfish peas? WATSON: Goldfish love peas, but make sure to remove the shells! USER: Should I boil them first? The second response &#8220;them&#8221; is called an &#8220;anaphora&#8221;. The &#8220;them&#8221; refers to the [&#8230;]]]></description>
  359. <content:encoded><![CDATA[<p><span id="selectionBoundary_1498076682606_46799825361031044" class="rangySelectionBoundary" style="line-height:0;display:none;"></span>One of common requests for conversation is being able to understand the running topic of a conversation.</p>
  360. <p>For example:</p>
  361. <blockquote>
  362. <p style="text-align:left;"><strong>USER</strong>: Can I feed my goldfish peas?</p>
  363. <p style="text-align:left;"><strong>WATSON: </strong>Goldfish love peas, but make sure to remove the shells!</p>
  364. <p style="text-align:left;"><strong>USER</strong>: Should I boil them first?</p>
  365. </blockquote>
  366. <p>The second response &#8220;them&#8221; is called an &#8220;anaphora&#8221;. The &#8220;them&#8221; refers to the peas. So you can&#8217;t answer the question without first knowing the previous question.</p>
  367. <p>On the face of it, it looks easy. But you have &#8220;goldfish&#8221;, &#8216;peas&#8221;, &#8216;shells&#8221; which could potentially be the reference, and no one wants to boil their goldfish!</p>
  368. <p>So the tricky part is determining the topic. There are a number of ways to approach this.</p>
  369. <h2>Entities</h2>
  370. <p>The most obvious way is to determine what entity the person mentioned, and store that for use later. This works well if the user actually mentions an entity to work with. However in a general conversation, the subject of the conversation may not always be by the person who asks the question.</p>
  371. <h2>Intents</h2>
  372. <p>When asking a question and determining the intent, it may not always be that an entity can be involved. So this has limited help in this regards.</p>
  373. <p>That said, there are certain cases where intents have been used with a context in mind. So it can be easily done by creating a suffix to the intent. For example:</p>
  374. <pre>    #FEEDING_FISH_e_Peas</pre>
  375. <p>In this case we believe that peas is a common entity that has a relationship to the intent of Feeding Fish. For coding convention we use &#8220;_e_&#8221; to denote that the following piece of the intent name is an entity identifier.</p>
  376. <p>At the application layer, you can do a regex on the intent name &#8220;<strong>_e_(.*?)$</strong>&#8221; for the group 1 result. If it is not blank, store it in a context variable.</p>
  377. <p>&nbsp;</p>
  378. <h2>Regular Expressions</h2>
  379. <p>Like before, you can use regular expressions to capture an earlier pattern to store it at a later point.</p>
  380. <p>One way to approach this is have a gateway node that activates before working through the intent tree. Something like this:</p>
  381. <p><img data-attachment-id="2409" data-permalink="https://sodoherty.ai/2017/07/15/anaphora-i-hardly-knew-her/example1507/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example15071.png?fit=238%2C234&amp;ssl=1" data-orig-size="238,234" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="example1507" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example15071.png?fit=238%2C234&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example15071.png?fit=238%2C234&amp;ssl=1" class="alignnone size-full wp-image-2409 aligncenter" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example15071.png?resize=238%2C234&#038;ssl=1" alt="example1507" width="238" height="234" data-recalc-dims="1" /></p>
  382. <p>The downside to this is that there is a level of complexity to maintain in a complex regular expression.</p>
  383. <p>You can make at least maintaining a little easier by setting the primary condition check as &#8220;true&#8221; and then individual checks in the node itself.</p>
  384. <p><img data-attachment-id="2416" data-permalink="https://sodoherty.ai/2017/07/15/anaphora-i-hardly-knew-her/example1507a/" data-orig-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507a.png?fit=640%2C623&amp;ssl=1" data-orig-size="640,623" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="example1507a" data-image-description="" data-medium-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507a.png?fit=300%2C292&amp;ssl=1" data-large-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507a.png?fit=640%2C623&amp;ssl=1" class="  wp-image-2416 aligncenter" src="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507a.png?resize=403%2C378&#038;ssl=1" alt="example1507a" width="403" height="378" data-recalc-dims="1" /></p>
  385. <h2>Answer Units</h2>
  386. <p>An answer unit is the text response you give back to the end user. Once you have responded with an answer, you have created a lot of context within that answer that the user may follow up on. For example:</p>
  387. <p><img data-attachment-id="2421" data-permalink="https://sodoherty.ai/2017/07/15/anaphora-i-hardly-knew-her/example1507b/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507b.png?fit=569%2C205&amp;ssl=1" data-orig-size="569,205" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="example1507b" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507b.png?fit=300%2C108&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507b.png?fit=569%2C205&amp;ssl=1" class="alignnone size-full wp-image-2421 aligncenter" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507b.png?resize=569%2C205&#038;ssl=1" alt="example1507b" width="569" height="205" data-recalc-dims="1" /></p>
  388. <p>Even with the context markers of the answer, the end user may never pick up on them. So it is very important to craft your answer that will drive the user to the context you have selected.</p>
  389. <h2>NLU</h2>
  390. <p>The last option is to pass the questions through <a href="https://www.ibm.com/watson/developercloud/natural-language-understanding.html" target="_blank" rel="noopener">NLU</a>. This should be able to give you the key terms and phrases to store as context. As well as create knowledge graph information.</p>
  391. <h1>I have the context. Now what?</h1>
  392. <p>When the user gives a question that does not have context, you will normally get back low confidence intents, or irrelevant response.</p>
  393. <p>If you are using Intent based context, you can check the returning intents for a similar context to what you have stored. This also allows you to discard unrelated intents. The results from this are not always stellar, but offer a cheaper one time call.</p>
  394. <p>The other option you can take is to preload the question that was asked and send it back. For example:</p>
  395. <pre>  PEAS !! Can I boil them first?</pre>
  396. <p>You can use the !! as a marker that your question is trying to determine context. Handy if you need to review the logs later.</p>
  397. <h1>As time passes&#8230;</h1>
  398. <p>So as the conversation presses on, what the person is talking about can move away from the original context, but it may still remain the dominant. One solution is to build a weighted context list.</p>
  399. <p>For example:</p>
  400. <pre>"entity_list" : "peas, food, fish"</pre>
  401. <p>In this case we maintain the last three context found. As a new context is found, it uses LIFO to maintain the list. Of course this means more API calls, which can cost money.</p>
  402. <h3>Lowering calls on the tree.</h3>
  403. <p>Another option in this to create a poor mans knowledge graph. Let&#8217;s say the last two context were &#8220;bowl&#8221; and &#8220;peas&#8221;. Rather then creating multiple context nodes, you can build a tree which can be passed back to the application layer.</p>
  404. <pre>"entity" : "peas-&gt;food-&gt;care-&gt;fish"
  405. ...
  406. "entity" : "bowl-&gt;care-&gt;fish"</pre>
  407. <p>You can use something like <a href="http://tinkerpop.apache.org" target="_blank" rel="noopener">Tinkerpop</a> to create a knowledge graph (IBM Graph in Bluemix is based on this).<br />
  408. <img data-attachment-id="2433" data-permalink="https://sodoherty.ai/2017/07/15/anaphora-i-hardly-knew-her/example1507c/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507c.png?fit=303%2C209&amp;ssl=1" data-orig-size="303,209" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="example1507c" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507c.png?fit=300%2C207&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507c.png?fit=303%2C209&amp;ssl=1" class="alignnone size-full wp-image-2433 aligncenter" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507c.png?resize=303%2C209&#038;ssl=1" alt="example1507c" width="303" height="209" data-recalc-dims="1" /></p>
  409. <p>Now when a low confidence question is found, you can use &#8220;bowl&#8221;, &#8220;peas&#8221; to disambiguate, or use &#8220;care&#8221; as the common entity to find the answer.</p>
  410. <h2>Talk&#8230; like.. a&#8230; millennial&#8230;</h2>
  411. <p>One more common form of anaphora that you have to deal with, is how people talk on instant messaging systems. The question is often split across multiple lines.</p>
  412. <p><img data-attachment-id="2457" data-permalink="https://sodoherty.ai/2017/07/15/anaphora-i-hardly-knew-her/example1507d/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507d.png?fit=145%2C136&amp;ssl=1" data-orig-size="145,136" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="example1507d" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507d.png?fit=145%2C136&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507d.png?fit=145%2C136&amp;ssl=1" class="alignnone size-full wp-image-2457 aligncenter" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/example1507d.png?resize=145%2C136&#038;ssl=1" alt="example1507d" width="145" height="136" data-recalc-dims="1" /></p>
  413. <p>Normal conversation systems take one entry, and give one response. So this just wreaks their AI head. Because not only do you need to know where the real question stops, but where the next one starts.</p>
  414. <p>One way to approach this is capture the average timing mechanism between each entry of the user. You can do this by passing the timestamps from the client to the backend. The backend can then build an average of how the user talks. This needs to be done at the application layer.</p>
  415. <p>Sadly no samples this time, but it should give you some insights into how context is worked with a conversation system.</p>
  416. ]]></content:encoded>
  417. <wfw:commentRss>https://sodoherty.ai/2017/07/15/anaphora-i-hardly-knew-her/feed/</wfw:commentRss>
  418. <slash:comments>0</slash:comments>
  419. <post-id xmlns="com-wordpress:feed-additions:1">1929</post-id> </item>
  420. <item>
  421. <title>Removing the confusion in intents.</title>
  422. <link>https://sodoherty.ai/2017/07/07/removing-the-confusion-in-intents/</link>
  423. <comments>https://sodoherty.ai/2017/07/07/removing-the-confusion-in-intents/#respond</comments>
  424. <pubDate>Fri, 07 Jul 2017 14:59:01 +0000</pubDate>
  425. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  426. <category><![CDATA[Python]]></category>
  427. <category><![CDATA[watson]]></category>
  428. <category><![CDATA[watson-conversation]]></category>
  429.  
  430. <guid isPermaLink="false">http://sodoherty.ai/?p=2347</guid>
  431. <description><![CDATA[While the complexity of building Conversation has reduced for non-developers, one of the areas that people can sometimes struggle is training intents. When trying to determine how the system performs, it is important to use tried and true methods of validation. If you go by someone just interacting with the system you can end up [&#8230;]]]></description>
  432. <content:encoded><![CDATA[<p>While the complexity of building Conversation has reduced for non-developers, one of the areas that people can sometimes struggle is training intents.</p>
  433. <p>When trying to determine how the system performs, it is important to use tried and true methods of validation. If you go by someone just interacting with the system you can end up with what is called &#8220;perceived accuracy&#8221;. A person may ask three to four questions, and three may fail. Their perception becomes that the system is broken.</p>
  434. <p>Using cross validation allows you to give a better feel of how the system is working. As will a blind / test set. But knowing how the system performs and trying to trying to interpret the results is where it takes practise.</p>
  435. <p>Take this example test report. Each line is where a question is asked, and it determines what the answer should be, versus what answer came back. The X denotes where an answer failed.</p>
  436. <p><img data-attachment-id="2358" data-permalink="https://sodoherty.ai/2017/07/07/removing-the-confusion-in-intents/report_0707/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/07/report_0707.png?fit=454%2C244&amp;ssl=1" data-orig-size="454,244" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="report_0707" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/07/report_0707.png?fit=300%2C161&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/07/report_0707.png?fit=454%2C244&amp;ssl=1" class="alignnone size-full wp-image-2358 aligncenter" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/07/report_0707.png?resize=454%2C244&#038;ssl=1" alt="report_0707" width="454" height="244" data-recalc-dims="1" /></p>
  437. <p>Unless you are used to doing this analysis full time, it is very hard to see the bigger picture. For example, is DOG_HEALTH the issue, or is it LITTER_HEALTH + BREEDER_GUARANTEE?</p>
  438. <p>You have to manually analyse each of these clusters and determine what changes are required to be made.</p>
  439. <p>Thankfully <a href="http://scikit-learn.org" target="_blank" rel="noopener">Sci-Kit</a> makes your life easier with being able to create a confusion matrix. With this you can see how each intent performs against each other. So you end up with something like this:</p>
  440. <p><img data-attachment-id="2368" data-permalink="https://sodoherty.ai/2017/07/07/removing-the-confusion-in-intents/confusion_matrix/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/07/confusion_matrix.png?fit=1350%2C1415&amp;ssl=1" data-orig-size="1350,1415" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="confusion_matrix" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/07/confusion_matrix.png?fit=286%2C300&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/07/confusion_matrix.png?fit=756%2C792&amp;ssl=1" class="alignnone size-full wp-image-2368 aligncenter" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/07/confusion_matrix.png?resize=756%2C792&#038;ssl=1" alt="confusion_matrix" width="756" height="792" data-recalc-dims="1" /></p>
  441. <p>So now you can quickly see what Intents are getting confused with others. So you focus on those to improve your accuracy better. In the example above, DOG_HEALTH and BREEDER_INFORMATION offer the best areas to investigate.</p>
  442. <p>I&#8217;ve created a <a title="ConversationConfusionMatrix" href="https://sodoherty.ai/wp-content/uploads/2017/07/conversationconfusionmatrix.zip">Sample Notebook</a> which demonstrates the above, so you can modify to test your own conversations training.</p>
  443. <p>&nbsp;</p>
  444. ]]></content:encoded>
  445. <wfw:commentRss>https://sodoherty.ai/2017/07/07/removing-the-confusion-in-intents/feed/</wfw:commentRss>
  446. <slash:comments>0</slash:comments>
  447. <post-id xmlns="com-wordpress:feed-additions:1">2347</post-id> </item>
  448. <item>
  449. <title>To see the world in a grain of sand&#8230;</title>
  450. <link>https://sodoherty.ai/2017/07/07/to-see-the-world-in-a-grain-of-sand/</link>
  451. <comments>https://sodoherty.ai/2017/07/07/to-see-the-world-in-a-grain-of-sand/#respond</comments>
  452. <pubDate>Fri, 07 Jul 2017 14:04:21 +0000</pubDate>
  453. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  454. <category><![CDATA[Uncategorized]]></category>
  455.  
  456. <guid isPermaLink="false">http://sodoherty.ai/?p=2317</guid>
  457. <description><![CDATA[Just a short blog update to inform everyone that I have moved over to a new role in IBM. I now work in IBM Dubai as a Technical Solutions Manager, assisting with the somewhat recently announced ai Lab. For me in a sense it means more (relative) free time, as a lot of my time [&#8230;]]]></description>
  458. <content:encoded><![CDATA[<p>Just a short blog update to inform everyone that I have moved over to a new role in IBM. I now work in IBM Dubai as a Technical Solutions Manager, assisting with the somewhat recently announced <a href="http://www.smartdubai.ae/story0328.php" target="_blank" rel="noopener">ai Lab</a>.</p>
  459. <p>For me in a sense it means more (relative) free time, as a lot of my time was devoted to travelling previously. But I am just getting ramped up, so apologies again if I still maintain my one post a month. <img src="https://s.w.org/images/core/emoji/2.3/72x72/1f642.png" alt="🙂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p>
  460. <p>On the plus side, I will be playing with a more wider spectrum of Watson + AI related technologies, and will discuss here when I get the chance.</p>
  461. <p>In the meantime here are some blogs I recommend.</p>
  462. <ul>
  463. <li><a href="http://marcnehme.com/marcnehme/blog/" target="_blank" rel="noopener">Marc Neheme</a></li>
  464. <li><a href="https://medium.com/@snrubnomis" target="_blank" rel="noopener">Simon Burns</a></li>
  465. <li><a href="http://heidloff.net" target="_blank" rel="noopener">Niklas Heidioff</a></li>
  466. <li><a href="https://medium.com/@5agado" target="_blank" rel="noopener">Alex Martinelli</a></li>
  467. <li>Dave&#8217;s blogs (always fun!)  <a href="http://liveatthewitchtrials.blogspot.com" target="_blank" rel="noopener">Blog1</a>, <a href="http://www.meversusanpost.com" target="_blank" rel="noopener">Blog2</a>.</li>
  468. </ul>
  469. <p>I would also recommend to check out the <a href="https://chatbotsmagazine.com/100-bot-people-to-watch-botwatch-aa3fac1ef06" target="_blank" rel="noopener">Top 100 people to watch</a> list.</p>
  470. <p>&nbsp;</p>
  471. <p>&nbsp;</p>
  472. ]]></content:encoded>
  473. <wfw:commentRss>https://sodoherty.ai/2017/07/07/to-see-the-world-in-a-grain-of-sand/feed/</wfw:commentRss>
  474. <slash:comments>0</slash:comments>
  475. <post-id xmlns="com-wordpress:feed-additions:1">2317</post-id> </item>
  476. <item>
  477. <title>Watson Conversation just got turned up to 11.</title>
  478. <link>https://sodoherty.ai/2017/06/23/watson-conversation-just-got-turned-up-to-11/</link>
  479. <comments>https://sodoherty.ai/2017/06/23/watson-conversation-just-got-turned-up-to-11/#comments</comments>
  480. <pubDate>Fri, 23 Jun 2017 06:38:20 +0000</pubDate>
  481. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  482. <category><![CDATA[watson]]></category>
  483. <category><![CDATA[watson-conversation]]></category>
  484.  
  485. <guid isPermaLink="false">http://sodoherty.ai/?p=2253</guid>
  486. <description><![CDATA[I just wanted to start with why my blog doesn&#8217;t get updated that often. Life is hectic. With continual travel, and a number of other things going on, the chance to be able to sit down for peace and quiet is rare. That may change one way or another soon. I try to avoid other [&#8230;]]]></description>
  487. <content:encoded><![CDATA[<p>I just wanted to start with why my blog doesn&#8217;t get updated that often.</p>
  488. <ol>
  489. <li>Life is hectic. With continual travel, and a number of other things going on, the chance to be able to sit down for peace and quiet is rare. That may change one way or another soon.</li>
  490. <li>I try to avoid other peoples thunder on a range of issues. For example I recommend keeping an eye on <a href="https://medium.com/@snrubnomis/gathering-information-with-ibm-watson-conversation-e23887ccbe3d" target="_blank" rel="noopener">Simon Burns blog</a> in relation to the new features.</li>
  491. <li>My role continues to evolve, at the moment that is one of &#8220;Expert Services&#8221;, where people like myself go around the world to help our customers run the show themselves. This probably warrants it&#8217;s own page, but it is a balance between what is added value to that service, and what everyone should know. (I&#8217;m in the latter camp, but I got to eat too).</li>
  492. </ol>
  493. <p>The biggest reason for lack of updates? <strong>Development keep changing things!</strong> I&#8217;m not the only one that suffers from this. I read a fantastic design patterns document earlier this year by someone in GBS, which went mostly out of date a week or two after I got it.</p>
  494. <p>So it has been the same situation with the latest update to conversation. It is a total game changer. To give you an example of how awesome it is, here is an image of a sample &#8220;I want to buy a dog&#8221; dialog flow.</p>
  495. <p><img data-attachment-id="2277" data-permalink="https://sodoherty.ai/2017/06/23/watson-conversation-just-got-turned-up-to-11/pre-slots/" data-orig-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/pre-slots.png?fit=448%2C1246&amp;ssl=1" data-orig-size="448,1246" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="pre-slots" data-image-description="" data-medium-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/pre-slots.png?fit=108%2C300&amp;ssl=1" data-large-file="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/pre-slots.png?fit=368%2C1024&amp;ssl=1" class="alignnone size-full wp-image-2277 aligncenter" src="https://i0.wp.com/sodoherty.ai/wp-content/uploads/2017/06/pre-slots.png?resize=448%2C1246&#038;ssl=1" alt="pre-slots" width="448" height="1246" data-recalc-dims="1" /></p>
  496. <p>Now compare that to the new Slots code. (btw, you may have noticed the UI looks cooler too).</p>
  497. <p><img data-attachment-id="2281" data-permalink="https://sodoherty.ai/2017/06/23/watson-conversation-just-got-turned-up-to-11/slots/" data-orig-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/slots.png?fit=550%2C253&amp;ssl=1" data-orig-size="550,253" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="slots" data-image-description="" data-medium-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/slots.png?fit=300%2C138&amp;ssl=1" data-large-file="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/slots.png?fit=550%2C253&amp;ssl=1" class=" size-full wp-image-2281 aligncenter" src="https://i1.wp.com/sodoherty.ai/wp-content/uploads/2017/06/slots.png?resize=550%2C253&#038;ssl=1" alt="slots" width="550" height="253" data-recalc-dims="1" /></p>
  498. <p>The same functionality took a fraction of the time, and has even more complexity of understanding than the previous version. That single node looks something like this:</p>
  499. <p><img data-attachment-id="2291" data-permalink="https://sodoherty.ai/2017/06/23/watson-conversation-just-got-turned-up-to-11/slots2/" data-orig-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/06/slots2.png?fit=401%2C394&amp;ssl=1" data-orig-size="401,394" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="slots2" data-image-description="" data-medium-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/06/slots2.png?fit=300%2C295&amp;ssl=1" data-large-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/06/slots2.png?fit=401%2C394&amp;ssl=1" class="alignnone size-full wp-image-2291 aligncenter" src="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/06/slots2.png?resize=401%2C394&#038;ssl=1" alt="slots2" width="401" height="394" data-recalc-dims="1" /></p>
  500. <p>That&#8217;s all for the moment, better if you play with it yourself. I have some free time shortly, and I will be posting some outstanding stuff.</p>
  501. <p>&nbsp;</p>
  502. ]]></content:encoded>
  503. <wfw:commentRss>https://sodoherty.ai/2017/06/23/watson-conversation-just-got-turned-up-to-11/feed/</wfw:commentRss>
  504. <slash:comments>1</slash:comments>
  505. <post-id xmlns="com-wordpress:feed-additions:1">2253</post-id> </item>
  506. <item>
  507. <title>I love Pandas!</title>
  508. <link>https://sodoherty.ai/2017/04/19/i-love-pandas/</link>
  509. <comments>https://sodoherty.ai/2017/04/19/i-love-pandas/#comments</comments>
  510. <pubDate>Wed, 19 Apr 2017 09:55:41 +0000</pubDate>
  511. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  512. <category><![CDATA[Python]]></category>
  513. <category><![CDATA[watson-conversation]]></category>
  514.  
  515. <guid isPermaLink="false">http://sodoherty.ai/?p=2116</guid>
  516. <description><![CDATA[Not the bamboo eating kind (but they are cute too), Python Pandas! But first&#8230; Conversation has a new feature! Logging!  You can now download your logs from your conversation workspace into a JSON format. So I thought I&#8217;d take this moment to introduce Pandas. Some people love the &#8220;Improve&#8221; UI, but personally I like being able to [&#8230;]]]></description>
  517. <content:encoded><![CDATA[<p>Not the bamboo eating kind (but they are cute too), <a href="http://pandas.pydata.org">Python Pandas</a>!</p>
  518. <p>But first&#8230; Conversation has a new feature!</p>
  519. <h2 style="text-align:center;"><strong>Logging! </strong></h2>
  520. <p>You can now download your logs from your conversation workspace into a JSON format. So I thought I&#8217;d take this moment to introduce Pandas. Some people love the &#8220;Improve&#8221; UI, but personally I like being able to easily mold the data to what I need.</p>
  521. <p>First, if you are new to Python, I strongly recommend getting a Python Notebook like <a href="https://jupyter.org" target="_blank">Jupyter</a> set up or use <a href="http://datascience.ibm.com">IBM Data Science Experience</a>. It makes learning so much easier, and you build your applications like actual documentation.</p>
  522. <p>I have a <a title="AnalysingConversationLogs" href="https://sodoherty.ai/wp-content/uploads/2017/04/analysingconversationlogs.zip">notebook</a> created so you can play along.</p>
  523. <h2>Making a connection</h2>
  524. <p>As the feature is just out, the SDK&#8217;s don&#8217;t have the API for it, so I will be using requests library.</p>
  525. <pre>url='https://gateway.watsonplatform.net/conversation/api/v1/workspaces/WORKSPACE_ID/logs?version=2017-04-21'
  526. basic_auth = HTTPBasicAuth(ctx.get('username'), ctx.get('password'))
  527. response = requests.get(url=url, auth=basic_auth)
  528. j = json.loads(response.text)</pre>
  529. <p>So we have the whole log now sitting in <code>j</code> but we want to make a dataframe. Before we do that however, let&#8217;s talk about log analysis and the fields you need. There are three areas we want to analyse in logs.</p>
  530. <p><strong>Quantitive</strong> &#8211; These are fixed metrics, like number of users, response times, common intents, etc.</p>
  531. <p><strong>Qualitative</strong> &#8211; This is analysing how the end user is speaking, and how the system interpreted and responded. Some examples would be where the answer returned may give the wrong impression to the end user, or users ask things out of expected areas.</p>
  532. <p><strong>Debugging</strong> &#8211; This is really looking for coding issues with your conversation tree.</p>
  533. <p>So on to the fields that cover these areas. These are all contained in <code>j['response']</code>.</p>
  534. <table style="margin:0 auto;font-size:9pt;max-width:100%;">
  535. <tbody>
  536. <tr>
  537. <td><b>Field</b></td>
  538. <td><b>Usage</b></td>
  539. <td width="60%"><b>Description</b></td>
  540. </tr>
  541. <tr>
  542. <td>input.text</td>
  543. <td>Qualitative</td>
  544. <td>This is what the user or the application typed in.</td>
  545. </tr>
  546. <tr>
  547. <td>intents[]</td>
  548. <td>Qualitative</td>
  549. <td>This tells you the primary intent for the users question. You should capture the intent and confidence into columns. If the value is [] then means it was irrelevant.</td>
  550. </tr>
  551. <tr>
  552. <td>entities[]</td>
  553. <td>Quantitive</td>
  554. <td>The entities found in relation to the call. With this and intents though, it&#8217;s important to understand that the application can override these values.</td>
  555. </tr>
  556. <tr>
  557. <td>output.text[]</td>
  558. <td>Qualitative</td>
  559. <td>This is the response shown to the user (or application).</td>
  560. </tr>
  561. <tr>
  562. <td>output.log_messages</td>
  563. <td>Debugging</td>
  564. <td>Capturing this field is handy to look for coding issues within your conversation tree. SPEL errors show up here if they happen.</td>
  565. </tr>
  566. <tr>
  567. <td>output.nodes_visited</td>
  568. <td>Debugging<br />
  569. Qualitive</td>
  570. <td>This can be used to see how a progression through a tree happens</td>
  571. </tr>
  572. <tr>
  573. <td>context.conversation_id</td>
  574. <td>All</td>
  575. <td>Use this to group users conversation together. In some solutions however, one pass calls are sometimes done mid conversation. So if you do this, you need to factor that in.</td>
  576. </tr>
  577. <tr>
  578. <td>context.system.branch_exited</td>
  579. <td>Debugging</td>
  580. <td>This tells you if your conversation left a branch and returned to root.</td>
  581. </tr>
  582. <tr>
  583. <td>context.system.branch_exited_reason</td>
  584. <td>Debugging</td>
  585. <td>If branch.exited is true then this will tell the why. completed means that the branch found a matching node, and finished. fallback means that it could not find a matching node, so it jumps back to root to find the match.</td>
  586. </tr>
  587. <tr>
  588. <td>context.???</td>
  589. <td>All</td>
  590. <td>You may have context variables you want to capture. You can either do these individually, or code to remove conversation objects and grab what remains</td>
  591. </tr>
  592. <tr>
  593. <td>request_timestamp</td>
  594. <td>Quantitive<br />
  595. Qualitative</td>
  596. <td>When conversation received the users response.</td>
  597. </tr>
  598. <tr>
  599. <td>response_timestamp</td>
  600. <td>Quantitive<br />
  601. Qualitative</td>
  602. <td>When conversation responded to the user. You can do a delta to see if there are conversation performance issues, but generally keep one of the timestamp fields for analysis.</td>
  603. </tr>
  604. </tbody>
  605. </table>
  606. <p>&nbsp;</p>
  607. <p>So we create a row array, and fill it with dict objects of the columns we want to capture. For clarity of the blog post, the sample code below</p>
  608. <pre>import pandas as pd
  609. rows = []
  610.  
  611. # for object in Json Logs array.
  612. for o in j['logs']:
  613.    row = {}
  614.    # Let's shorthand the response object.
  615.    r = o['response']
  616.    row['conversation_id'] = r['context']['conversation_id']
  617.    # We need to check the fields exist before we read them.
  618.    if 'text' in r['input']: row['Input'] = r['input']['text']
  619.    if 'text' in r['output']:row['Output'] = ' '.join(r['output']['text'])
  620.    # Again we need to check it is not an Irrelevant response.
  621.    if len(r['intents']) &gt; 0:
  622.        row['Confidence'] = r['intents'][0]['confidence']
  623.        row['Intent'] = r['intents'][0]['intent']
  624.  
  625.    rows.append(row)
  626.  
  627. # Build the dataframe.
  628. df = pd.DataFrame(rows,columns=['conversation_id','Input','Output','Intent','Confidence'])
  629. df = df.fillna('')
  630.  
  631. # Display the dataframe.
  632. df</pre>
  633. <p>When this is run, all going well you end up with something like this:</p>
  634. <p><img data-attachment-id="2183" data-permalink="https://sodoherty.ai/2017/04/19/i-love-pandas/report1-1804/" data-orig-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report1-1804.png?fit=997%2C185&amp;ssl=1" data-orig-size="997,185" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="report1-1804" data-image-description="" data-medium-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report1-1804.png?fit=300%2C56&amp;ssl=1" data-large-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report1-1804.png?fit=756%2C140&amp;ssl=1" class="alignnone size-full wp-image-2183" src="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report1-1804.png?resize=756%2C140&#038;ssl=1" alt="report1-1804" width="756" height="140" data-recalc-dims="1" /></p>
  635. <p>The notebook has a better report, and is also sorted so it is actually readable.</p>
  636. <p><img data-attachment-id="2186" data-permalink="https://sodoherty.ai/2017/04/19/i-love-pandas/report2-1804/" data-orig-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report2-1804.png?fit=1002%2C236&amp;ssl=1" data-orig-size="1002,236" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="report2-1804" data-image-description="" data-medium-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report2-1804.png?fit=300%2C71&amp;ssl=1" data-large-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report2-1804.png?fit=756%2C178&amp;ssl=1" class="alignnone size-full wp-image-2186" src="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report2-1804.png?resize=756%2C178&#038;ssl=1" alt="report2-1804" width="756" height="178" data-recalc-dims="1" /></p>
  637. <p>Once you have everything you need in the dataframe, you can manipulate it very fast and easy. For example, let&#8217;s say you want to get a count of the intents found.</p>
  638. <pre># Get the counts.
  639. q_df = df.groupby('Intent').count()
  640.  
  641. # Remove all fields except conversation_id and intents.
  642. q_df = q_df.drop(['request TS', 'response TS', 'User Input', 'Output', 'Confidence', 'Exit Reason', 'Logging'],axis=1)
  643.  
  644. # Rename the conversation_id field to "Count".
  645. q_df.columns = ['Count']
  646.  
  647. # Sort and display.
  648. q_df = q_df.sort_values(['Count'], ascending=[False])
  649. q_df</pre>
  650. <p>This creates this:</p>
  651. <p><img data-attachment-id="2192" data-permalink="https://sodoherty.ai/2017/04/19/i-love-pandas/report3-1804/" data-orig-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report3-1804.png?fit=249%2C459&amp;ssl=1" data-orig-size="249,459" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="report3-1804" data-image-description="" data-medium-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report3-1804.png?fit=163%2C300&amp;ssl=1" data-large-file="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report3-1804.png?fit=249%2C459&amp;ssl=1" class="alignnone size-full wp-image-2192" src="https://i2.wp.com/sodoherty.ai/wp-content/uploads/2017/04/report3-1804.png?resize=249%2C459&#038;ssl=1" alt="report3-1804" width="249" height="459" data-recalc-dims="1" /></p>
  652. <p>The Jupyter notebook also allows for visualisation of data as well. Although I haven&#8217;t put any in the sample notebook.</p>
  653. ]]></content:encoded>
  654. <wfw:commentRss>https://sodoherty.ai/2017/04/19/i-love-pandas/feed/</wfw:commentRss>
  655. <slash:comments>6</slash:comments>
  656. <post-id xmlns="com-wordpress:feed-additions:1">2116</post-id> </item>
  657. </channel>
  658. </rss>
  659.  

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

  1. Download the "valid RSS" banner.

  2. Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)

  3. Add this HTML to your page (change the image src attribute if necessary):

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//sodoherty.com/feed/

Copyright © 2002-9 Sam Ruby, Mark Pilgrim, Joseph Walton, and Phil Ringnalda