Congratulations!

[Valid RSS] This is a valid RSS feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

Source: http://sodoherty.com/feed/

  1. <?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
  2. xmlns:content="http://purl.org/rss/1.0/modules/content/"
  3. xmlns:wfw="http://wellformedweb.org/CommentAPI/"
  4. xmlns:dc="http://purl.org/dc/elements/1.1/"
  5. xmlns:atom="http://www.w3.org/2005/Atom"
  6. xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
  7. xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
  8. xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
  9. >
  10.  
  11. <channel>
  12. <title>Talk to me</title>
  13. <atom:link href="https://sodoherty.ai/feed/" rel="self" type="application/rss+xml" />
  14. <link>https://sodoherty.ai</link>
  15. <description>A blog about Chat bots (mainly Watson), and life in general.</description>
  16. <lastBuildDate>Thu, 27 Apr 2017 14:00:38 +0000</lastBuildDate>
  17. <language>en</language>
  18. <sy:updatePeriod>hourly</sy:updatePeriod>
  19. <sy:updateFrequency>1</sy:updateFrequency>
  20. <generator>http://wordpress.com/</generator>
  21. <cloud domain='sodoherty.ai' port='80' path='/?rsscloud=notify' registerProcedure='' protocol='http-post' />
  22. <image>
  23. <url>https://s2.wp.com/i/buttonw-com.png</url>
  24. <title>Talk to me</title>
  25. <link>https://sodoherty.ai</link>
  26. </image>
  27. <atom:link rel="search" type="application/opensearchdescription+xml" href="https://sodoherty.ai/osd.xml" title="Talk to me" />
  28. <atom:link rel='hub' href='https://sodoherty.ai/?pushpress=hub'/>
  29. <item>
  30. <title>I love Pandas!</title>
  31. <link>https://sodoherty.ai/2017/04/19/i-love-pandas/</link>
  32. <comments>https://sodoherty.ai/2017/04/19/i-love-pandas/#comments</comments>
  33. <pubDate>Wed, 19 Apr 2017 09:55:41 +0000</pubDate>
  34. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  35. <category><![CDATA[Python]]></category>
  36. <category><![CDATA[watson-conversation]]></category>
  37.  
  38. <guid isPermaLink="false">http://sodoherty.ai/?p=2116</guid>
  39. <description><![CDATA[Not the bamboo eating kind (but they are cute too), Python Pandas! But first&#8230; Conversation has a new feature! Logging!  You can now download your logs from your conversation workspace into a JSON format. So I thought I&#8217;d take this moment to introduce Pandas. Some people love the &#8220;Improve&#8221; UI, but personally I like being able to [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=2116&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  40. <content:encoded><![CDATA[<p>Not the bamboo eating kind (but they are cute too), <a href="http://pandas.pydata.org">Python Pandas</a>!</p>
  41. <p>But first&#8230; Conversation has a new feature!</p>
  42. <h2 style="text-align:center;"><strong>Logging! </strong></h2>
  43. <p>You can now download your logs from your conversation workspace into a JSON format. So I thought I&#8217;d take this moment to introduce Pandas. Some people love the &#8220;Improve&#8221; UI, but personally I like being able to easily mold the data to what I need.</p>
  44. <p>First, if you are new to Python, I strongly recommend getting a Python Notebook like <a href="https://jupyter.org" target="_blank">Jupyter</a> set up or use <a href="http://datascience.ibm.com">IBM Data Science Experience</a>. It makes learning so much easier, and you build your applications like actual documentation.</p>
  45. <p>I have a <a title="AnalysingConversationLogs" href="https://sodohertydotcom.files.wordpress.com/2017/04/analysingconversationlogs.zip">notebook</a> created so you can play along.</p>
  46. <h2>Making a connection</h2>
  47. <p>As the feature is just out, the SDK&#8217;s don&#8217;t have the API for it, so I will be using requests library.</p>
  48. <pre>url='https://gateway.watsonplatform.net/conversation/api/v1/workspaces/WORKSPACE_ID/logs?version=2017-04-21'
  49. basic_auth = HTTPBasicAuth(ctx.get('username'), ctx.get('password'))
  50. response = requests.get(url=url, auth=basic_auth)
  51. j = json.loads(response.text)</pre>
  52. <p>So we have the whole log now sitting in <code>j</code> but we want to make a dataframe. Before we do that however, let&#8217;s talk about log analysis and the fields you need. There are three areas we want to analyse in logs.</p>
  53. <p><strong>Quantitive</strong> &#8211; These are fixed metrics, like number of users, response times, common intents, etc.</p>
  54. <p><strong>Qualitative</strong> &#8211; This is analysing how the end user is speaking, and how the system interpreted and responded. Some examples would be where the answer returned may give the wrong impression to the end user, or users ask things out of expected areas.</p>
  55. <p><strong>Debugging</strong> &#8211; This is really looking for coding issues with your conversation tree.</p>
  56. <p>So on to the fields that cover these areas. These are all contained in <code>j['response']</code>.</p>
  57. <table style="margin:0 auto;font-size:9pt;max-width:100%;">
  58. <tbody>
  59. <tr>
  60. <td><b>Field</b></td>
  61. <td><b>Usage</b></td>
  62. <td width="60%"><b>Description</b></td>
  63. </tr>
  64. <tr>
  65. <td>input.text</td>
  66. <td>Qualitative</td>
  67. <td>This is what the user or the application typed in.</td>
  68. </tr>
  69. <tr>
  70. <td>intents[]</td>
  71. <td>Qualitative</td>
  72. <td>This tells you the primary intent for the users question. You should capture the intent and confidence into columns. If the value is [] then means it was irrelevant.</td>
  73. </tr>
  74. <tr>
  75. <td>entities[]</td>
  76. <td>Quantitive</td>
  77. <td>The entities found in relation to the call. With this and intents though, it&#8217;s important to understand that the application can override these values.</td>
  78. </tr>
  79. <tr>
  80. <td>output.text[]</td>
  81. <td>Qualitative</td>
  82. <td>This is the response shown to the user (or application).</td>
  83. </tr>
  84. <tr>
  85. <td>output.log_messages</td>
  86. <td>Debugging</td>
  87. <td>Capturing this field is handy to look for coding issues within your conversation tree. SPEL errors show up here if they happen.</td>
  88. </tr>
  89. <tr>
  90. <td>output.nodes_visited</td>
  91. <td>Debugging<br />
  92. Qualitive</td>
  93. <td>This can be used to see how a progression through a tree happens</td>
  94. </tr>
  95. <tr>
  96. <td>context.conversation_id</td>
  97. <td>All</td>
  98. <td>Use this to group users conversation together. In some solutions however, one pass calls are sometimes done mid conversation. So if you do this, you need to factor that in.</td>
  99. </tr>
  100. <tr>
  101. <td>context.system.branch_exited</td>
  102. <td>Debugging</td>
  103. <td>This tells you if your conversation left a branch and returned to root.</td>
  104. </tr>
  105. <tr>
  106. <td>context.system.branch_exited_reason</td>
  107. <td>Debugging</td>
  108. <td>If branch.exited is true then this will tell the why. completed means that the branch found a matching node, and finished. fallback means that it could not find a matching node, so it jumps back to root to find the match.</td>
  109. </tr>
  110. <tr>
  111. <td>context.???</td>
  112. <td>All</td>
  113. <td>You may have context variables you want to capture. You can either do these individually, or code to remove conversation objects and grab what remains</td>
  114. </tr>
  115. <tr>
  116. <td>request_timestamp</td>
  117. <td>Quantitive<br />
  118. Qualitative</td>
  119. <td>When conversation received the users response.</td>
  120. </tr>
  121. <tr>
  122. <td>response_timestamp</td>
  123. <td>Quantitive<br />
  124. Qualitative</td>
  125. <td>When conversation responded to the user. You can do a delta to see if there are conversation performance issues, but generally keep one of the timestamp fields for analysis.</td>
  126. </tr>
  127. </tbody>
  128. </table>
  129. <p>&nbsp;</p>
  130. <p>So we create a row array, and fill it with dict objects of the columns we want to capture. For clarity of the blog post, the sample code below</p>
  131. <pre>import pandas as pd
  132. rows = []
  133.  
  134. # for object in Json Logs array.
  135. for o in j['logs']:
  136.    row = {}
  137.    # Let's shorthand the response object.
  138.    r = o['response']
  139.    row['conversation_id'] = r['context']['conversation_id']
  140.    # We need to check the fields exist before we read them.
  141.    if 'text' in r['input']: row['Input'] = r['input']['text']
  142.    if 'text' in r['output']:row['Output'] = ' '.join(r['output']['text'])
  143.    # Again we need to check it is not an Irrelevant response.
  144.    if len(r['intents']) &gt; 0:
  145.        row['Confidence'] = r['intents'][0]['confidence']
  146.        row['Intent'] = r['intents'][0]['intent']
  147.  
  148.    rows.append(row)
  149.  
  150. # Build the dataframe.
  151. df = pd.DataFrame(rows,columns=['conversation_id','Input','Output','Intent','Confidence'])
  152. df = df.fillna('')
  153.  
  154. # Display the dataframe.
  155. df</pre>
  156. <p>When this is run, all going well you end up with something like this:</p>
  157. <p><img data-attachment-id="2183" data-permalink="https://sodoherty.ai/2017/04/19/i-love-pandas/report1-1804/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png?w=756" data-orig-size="997,185" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="report1-1804" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png?w=756?w=756" class="alignnone size-full wp-image-2183" src="https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png?w=756" alt="report1-1804" srcset="https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png?w=756 756w, https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png?w=300 300w, https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png?w=768 768w, https://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png 997w" sizes="(max-width: 756px) 100vw, 756px"   /></p>
  158. <p>The notebook has a better report, and is also sorted so it is actually readable.</p>
  159. <p><img data-attachment-id="2186" data-permalink="https://sodoherty.ai/2017/04/19/i-love-pandas/report2-1804/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png?w=756" data-orig-size="1002,236" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="report2-1804" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png?w=756?w=756" class="alignnone size-full wp-image-2186" src="https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png?w=756" alt="report2-1804" srcset="https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png?w=756 756w, https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png?w=300 300w, https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png?w=768 768w, https://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png 1002w" sizes="(max-width: 756px) 100vw, 756px"   /></p>
  160. <p>Once you have everything you need in the dataframe, you can manipulate it very fast and easy. For example, let&#8217;s say you want to get a count of the intents found.</p>
  161. <pre># Get the counts.
  162. q_df = df.groupby('Intent').count()
  163.  
  164. # Remove all fields except conversation_id and intents.
  165. q_df = q_df.drop(['request TS', 'response TS', 'User Input', 'Output', 'Confidence', 'Exit Reason', 'Logging'],axis=1)
  166.  
  167. # Rename the conversation_id field to "Count".
  168. q_df.columns = ['Count']
  169.  
  170. # Sort and display.
  171. q_df = q_df.sort_values(['Count'], ascending=[False])
  172. q_df</pre>
  173. <p>This creates this:</p>
  174. <p><img data-attachment-id="2192" data-permalink="https://sodoherty.ai/2017/04/19/i-love-pandas/report3-1804/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/04/report3-1804.png?w=756" data-orig-size="249,459" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="report3-1804" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/04/report3-1804.png?w=756?w=163" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/04/report3-1804.png?w=756?w=249" class="alignnone size-full wp-image-2192" src="https://sodohertydotcom.files.wordpress.com/2017/04/report3-1804.png?w=756" alt="report3-1804" srcset="https://sodohertydotcom.files.wordpress.com/2017/04/report3-1804.png 249w, https://sodohertydotcom.files.wordpress.com/2017/04/report3-1804.png?w=81 81w" sizes="(max-width: 249px) 100vw, 249px"   /></p>
  175. <p>The Jupyter notebook also allows for visualisation of data as well. Although I haven&#8217;t put any in the sample notebook.</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/2116/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/2116/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=2116&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  176. <wfw:commentRss>https://sodoherty.ai/2017/04/19/i-love-pandas/feed/</wfw:commentRss>
  177. <slash:comments>5</slash:comments>
  178. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  179. <media:title type="html">simonodoherty</media:title>
  180. </media:content>
  181.  
  182. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/report1-1804.png" medium="image">
  183. <media:title type="html">report1-1804</media:title>
  184. </media:content>
  185.  
  186. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/report2-1804.png" medium="image">
  187. <media:title type="html">report2-1804</media:title>
  188. </media:content>
  189.  
  190. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/report3-1804.png" medium="image">
  191. <media:title type="html">report3-1804</media:title>
  192. </media:content>
  193. </item>
  194. <item>
  195. <title>I have a dream&#8230;</title>
  196. <link>https://sodoherty.ai/2017/04/18/i-have-a-dream/</link>
  197. <comments>https://sodoherty.ai/2017/04/18/i-have-a-dream/#respond</comments>
  198. <pubDate>Tue, 18 Apr 2017 16:30:09 +0000</pubDate>
  199. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  200. <category><![CDATA[watson]]></category>
  201. <category><![CDATA[watson-text-to-speech]]></category>
  202.  
  203. <guid isPermaLink="false">http://sodoherty.ai/?p=1966</guid>
  204. <description><![CDATA[Following on from Speech to Text, let&#8217;s jump over to Text to Speech. Similar to conversation, what can make or break the system is the tone and personality you build into the system. Developers tend to think about the coding, and not the user experience so much. To give an example, let&#8217;s take a piece [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1966&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  205. <content:encoded><![CDATA[<p>Following on from Speech to Text, let&#8217;s jump over to Text to Speech. Similar to conversation, what can make or break the system is the tone and personality you build into the system.</p>
  206. <p>Developers tend to think about the coding, and not the user experience so much.</p>
  207. <p>To give an example, let&#8217;s take a piece of a <a href="http://www.americanrhetoric.com/speeches/mlkihaveadream.htm">very famous speech from MLK</a>. Small sample so it doesn&#8217;t take all day:</p>
  208. <blockquote><p>I still have a dream. It is a dream deeply rooted in the American dream.</p>
  209. <p>I have a dream that one day this nation will rise up and live out the true meaning of its creed: &#8220;We hold these truths to be self-evident, that all men are created equal.&#8221;</p></blockquote>
  210. <p>Let&#8217;s listen to Watson as it directly translates.</p>
  211. <!--[if lt IE 9]><script>document.createElement('audio');</script><![endif]-->
  212. <audio class="wp-audio-shortcode" id="audio-1966-1" preload="none" style="width: 100%;" controls="controls"><source type="audio/wav" src="https://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream.wav?_=1" /><a href="https://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream.wav">https://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream.wav</a></audio>
  213. <p>It sounds like how I act when I am reading a script. <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f642.png" alt="🙂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p>
  214. <p>Now lets listen to MLK.<br />
  215. <audio class="wp-audio-shortcode" id="audio-1966-2" preload="none" style="width: 100%;" controls="controls"><source type="audio/wav" src="https://sodohertydotcom.files.wordpress.com/2017/04/mlk-ihaveadream.wav?_=2" /><a href="https://sodohertydotcom.files.wordpress.com/2017/04/mlk-ihaveadream.wav">https://sodohertydotcom.files.wordpress.com/2017/04/mlk-ihaveadream.wav</a></audio></p>
  216. <p>You can feel the emotion behind it. The pauses and emphasis adds more meaning to it. Thankfully Watson supports <a href="https://www.ibm.com/watson/developercloud/doc/text-to-speech/SSML.html">SSML</a>, which allows you to mimic the speech.</p>
  217. <p>For this example I only used two tags. The first was <strong>&lt;parsody&gt;</strong> which allows Watson to have the same speaking speed as MLK. The other tag was<strong> &lt;break&gt;</strong> which allows me to make those dramatic pauses.</p>
  218. <p>Using <a href="http://www.audacityteam.org" target="_blank">Audacity</a> I was able to put the generated speech against the MLK speech. Then selecting the pause areas, I can quickly see the pause lengths.</p>
  219. <p><img data-attachment-id="2098" data-permalink="https://sodoherty.ai/2017/04/18/i-have-a-dream/audicity/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/04/audicity.png?w=756" data-orig-size="549,224" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="audicity" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/04/audicity.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/04/audicity.png?w=756?w=549" class="alignnone size-full wp-image-2098" src="https://sodohertydotcom.files.wordpress.com/2017/04/audicity.png?w=756" alt="audicity" srcset="https://sodohertydotcom.files.wordpress.com/2017/04/audicity.png 549w, https://sodohertydotcom.files.wordpress.com/2017/04/audicity.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/04/audicity.png?w=300 300w" sizes="(max-width: 549px) 100vw, 549px"   /></p>
  220. <p>I finally ended up with this:<br />
  221. <audio class="wp-audio-shortcode" id="audio-1966-3" preload="none" style="width: 100%;" controls="controls"><source type="audio/wav" src="https://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream-markedup.wav?_=3" /><a href="https://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream-markedup.wav">https://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream-markedup.wav</a></audio></p>
  222. <p>Audacity also allows you to overlay audio, to get a feel to how it would sound if there were crowds listening.<br />
  223. <audio class="wp-audio-shortcode" id="audio-1966-4" preload="none" style="width: 100%;" controls="controls"><source type="audio/wav" src="https://sodohertydotcom.files.wordpress.com/2017/04/combined-mlk-watson.wav?_=4" /><a href="https://sodohertydotcom.files.wordpress.com/2017/04/combined-mlk-watson.wav">https://sodohertydotcom.files.wordpress.com/2017/04/combined-mlk-watson.wav</a></audio></p>
  224. <p>The final script ends up like this:</p>
  225. <pre>&lt;prosody rate="x-slow"&gt;I still have a dream.&lt;/prosody&gt;
  226. &lt;break time="1660ms"&gt;&lt;/break&gt;
  227. &lt;prosody rate="slow"&gt;It is a dream deeply rooted in the American dream.&lt;/prosody&gt;
  228. &lt;break time="500ms"&gt;&lt;/break&gt;
  229. &lt;prosody rate="slow"&gt;I have a dream&lt;/prosody&gt;
  230. &lt;break time="1490ms"&gt;&lt;/break&gt;
  231. &lt;prosody rate="x-slow"&gt;that one day&lt;/prosody&gt;
  232. &lt;break time="1480ms"&gt;&lt;/break&gt;
  233. &lt;prosody rate="slow"&gt;this nation &lt;prosody rate="x-slow"&gt;will &lt;/prosody&gt;ryeyes up&lt;/prosody&gt;
  234. &lt;break time="1798ms"&gt;&lt;/break&gt;
  235. &lt;prosody rate="slow"&gt;and live out the true meaning of its creed:&lt;/prosody&gt;
  236. &lt;break time="362ms"&gt;&lt;/break&gt;
  237. &lt;prosody rate="slow"&gt;"We hold these truths to be self-evident,&lt;/prosody&gt;
  238. &lt;break time="594ms"&gt;&lt;/break&gt;
  239. &lt;prosody rate="slow"&gt;that all men are created equal."&lt;/prosody&gt;</pre>
  240. <p>I have zipped up all the files for download, just in case you are having issues running the audio.</p>
  241. <p style="padding-left:30px;"><a title="IHaveADream" href="https://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream.zip">IHaveADream.zip</a></p>
  242. <p>In closing, if you plan to build a conversational system that speaks to the end user, you also need skills in talking to people, just not being able to write.</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1966/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1966/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1966&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  243. <wfw:commentRss>https://sodoherty.ai/2017/04/18/i-have-a-dream/feed/</wfw:commentRss>
  244. <slash:comments>0</slash:comments>
  245. <enclosure url="http://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream.wav" length="651662" type="audio/wav" />
  246. <enclosure url="http://sodohertydotcom.files.wordpress.com/2017/04/mlk-ihaveadream.wav" length="1179392" type="audio/wav" />
  247. <enclosure url="http://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream-markedup.wav" length="1204250" type="audio/wav" />
  248. <enclosure url="http://sodohertydotcom.files.wordpress.com/2017/04/combined-mlk-watson.wav" length="1179392" type="audio/wav" />
  249. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  250. <media:title type="html">simonodoherty</media:title>
  251. </media:content>
  252.  
  253. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/audicity.png" medium="image">
  254. <media:title type="html">audicity</media:title>
  255. </media:content>
  256.  
  257. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream.wav" medium="audio" />
  258.  
  259. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/mlk-ihaveadream.wav" medium="audio" />
  260.  
  261. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/ihaveadream-markedup.wav" medium="audio" />
  262.  
  263. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/combined-mlk-watson.wav" medium="audio" />
  264. </item>
  265. <item>
  266. <title>Speech to Text and Conversation</title>
  267. <link>https://sodoherty.ai/2017/04/18/speech-to-text-and-conversation/</link>
  268. <comments>https://sodoherty.ai/2017/04/18/speech-to-text-and-conversation/#respond</comments>
  269. <pubDate>Tue, 18 Apr 2017 12:57:49 +0000</pubDate>
  270. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  271. <category><![CDATA[tips]]></category>
  272. <category><![CDATA[watson]]></category>
  273. <category><![CDATA[watson-speech-to-text]]></category>
  274.  
  275. <guid isPermaLink="false">http://sodoherty.ai/?p=1967</guid>
  276. <description><![CDATA[I thought I would take a moment to play with Speech to Text and a utility that was released a few months ago. The Speech to Text Utils allows you to train S2T using your existing conversational system. To give a quick demo, I got my son to ask about buying a puppy. I set up [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1967&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  277. <content:encoded><![CDATA[<p>I thought I would take a moment to play with Speech to Text and a utility that was released a few months ago.</p>
  278. <p>The <a href="https://github.com/watson-developer-cloud/speech-to-text-utils" target="_blank">Speech to Text Utils</a> allows you to train S2T using your existing conversational system. To give a quick demo, I got my son to ask about buying a puppy.</p>
  279. <audio class="wp-audio-shortcode" id="audio-1967-5" preload="none" style="width: 100%;" controls="controls"><source type="audio/wav" src="https://sodohertydotcom.files.wordpress.com/2017/04/p4u-example1.wav?_=5" /><a href="https://sodohertydotcom.files.wordpress.com/2017/04/p4u-example1.wav">https://sodohertydotcom.files.wordpress.com/2017/04/p4u-example1.wav</a></audio>
  280. <p>I set up some quick Python code to print out results:</p>
  281. <pre>import json
  282. from watson_developer_cloud import SpeechToTextV1
  283.  
  284. # ctx is Service credentials copied from S2T Service.
  285.  
  286. s2t = SpeechToTextV1(
  287. username=ctx.get('username'),
  288. password=ctx.get('password')
  289. )
  290.  
  291. def wav(filename, **kwargs):
  292.  with open(filename,'rb') as wav:
  293.    response = s2t.recognize(wav, content_type='audio/wav', **kwargs)
  294.  
  295. if len(response['results']) &gt; 0:
  296.  return response['results'][0]['alternatives'][0]['transcript']
  297. else:
  298.  return '???';</pre>
  299. <p>So testing the audio with the following code:</p>
  300. <pre>wav_file = 'p4u-example1.wav'
  301. print('Broadband: {}'.format(wav(wav_file)))
  302. print('NarrowBand: {}'.format(wav(wav_file,model='en-US_NarrowbandModel')))</pre>
  303. <p>Gets these results:</p>
  304. <pre>Broadband: can I get a puppy
  305. NarrowBand: can I get a puppy</pre>
  306. <p>Of course the recording is crystal clear, which is why such a good result. So I added some ambient noises from <a href="https://www.soundjay.com/ambient-sounds-2.html" target="_blank">SoundJay</a> to the background. So now it sounds like it is in a subway.</p>
  307. <audio class="wp-audio-shortcode" id="audio-1967-6" preload="none" style="width: 100%;" controls="controls"><source type="audio/wav" src="https://sodohertydotcom.files.wordpress.com/2017/04/p4u-example2.wav?_=6" /><a href="https://sodohertydotcom.files.wordpress.com/2017/04/p4u-example2.wav">https://sodohertydotcom.files.wordpress.com/2017/04/p4u-example2.wav</a></audio>
  308. <p>Running the code above again get&#8217;s these results.</p>
  309. <pre>Broadband: Greg it appropriate
  310. Narrowband: can I get a phone</pre>
  311. <p>Ouch!</p>
  312. <h3>Utils to the rescue!</h3>
  313. <p>So the purpose of asking about a puppy is that I have a <a title="puppies4you" href="https://sodohertydotcom.files.wordpress.com/2017/04/puppies4you.zip">sample conversation</a> system that is about buying a dog. Using that conversation file I did the following.</p>
  314. <p>1: Installed <a href="https://github.com/watson-developer-cloud/speech-to-text-utils" target="_blank">Speech to Text Utils</a>.</p>
  315. <p>2: Before you begin you need to set up the connection to your S2T service (using service credentials).</p>
  316. <pre><code>watson-speech-to-text-utils set-credentials</code></pre>
  317. <p>It will walk you through the username and password.</p>
  318. <p>3: Once that was set up, I then tell it to create a customisation.</p>
  319. <pre>watson-speech-to-text-utils corpus-from-workspace puppies4you.json</pre>
  320. <p>You need to map to a particular model. For testing, I attached it to en-US_NarrowbandModel and en-US_BroadbandModel.</p>
  321. <p>4: Once it was run, I get the ID numbers for the customisations.</p>
  322. <pre>watson-speech-to-text-utils customization-list</pre>
  323. <p>Once I have the ID&#8217;s I try the audio again:</p>
  324. <pre>wav_file='p4u-example2.wav'
  325. print('Broadband: {}'.format(wav(wav_file,customization_id='beeebd80-2420-11e7-8f1c-176db802f8de',timestamps=True)))
  326. print('Narrowband: {}'.format(wav(wav_file,model='en-US_NarrowbandModel',customization_id='a9f80490-241b-11e7-8f1c-176db802f8de')))</pre>
  327. <p>This outputs:</p>
  328. <pre>Broadband: can I get a puppy
  329. Narrowband: can I get a phone</pre>
  330. <p>So the broadband now works. Narrowband is likely the quality is too poor to work with. There is also more <a href="http://www.soapboxlabs.com">specialised language models for children</a> done by others to cope with this.</p>
  331. <h3>One swallow does not make a summer.</h3>
  332. <p>So this is one example, of one phrase. Really for testing, you should test the whole model. From a demonstration from development, it was able to increase a S2T model accuracy from around 50% to over 80%.</p>
  333. <p>&nbsp;</p>
  334. <p>&nbsp;</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1967/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1967/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1967&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  335. <wfw:commentRss>https://sodoherty.ai/2017/04/18/speech-to-text-and-conversation/feed/</wfw:commentRss>
  336. <slash:comments>0</slash:comments>
  337. <enclosure url="http://sodohertydotcom.files.wordpress.com/2017/04/p4u-example1.wav" length="243756" type="audio/wav" />
  338. <enclosure url="http://sodohertydotcom.files.wordpress.com/2017/04/p4u-example2.wav" length="243850" type="audio/wav" />
  339. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  340. <media:title type="html">simonodoherty</media:title>
  341. </media:content>
  342.  
  343. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/p4u-example1.wav" medium="audio" />
  344.  
  345. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/p4u-example2.wav" medium="audio" />
  346. </item>
  347. <item>
  348. <title>Watson V3 Certification</title>
  349. <link>https://sodoherty.ai/2017/04/04/watson-v3-certification/</link>
  350. <comments>https://sodoherty.ai/2017/04/04/watson-v3-certification/#respond</comments>
  351. <pubDate>Tue, 04 Apr 2017 07:16:13 +0000</pubDate>
  352. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  353. <category><![CDATA[certification]]></category>
  354. <category><![CDATA[watson]]></category>
  355.  
  356. <guid isPermaLink="false">http://sodoherty.ai/?p=1898</guid>
  357. <description><![CDATA[So I got my Watson V3 Certification a week or so ago, and the badge just arrived yesterday. I sat the mock exam without studying and passed. So I thought I&#8217;d try the real exam, and passed that too. Overall if you have been working in the Watson group for 3+ years, where your job [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1898&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  358. <content:encoded><![CDATA[<p><a href="https://www.youracclaim.com/badges/ff8e531d-6987-41a3-b5e8-1554e6e47c29/public_url"><img data-attachment-id="1901" data-permalink="https://sodoherty.ai/2017/04/04/watson-v3-certification/ibm-certified-application-developer-watson-v3-certification/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/04/ibm-certified-application-developer-watson-v3-certification.png?w=756" data-orig-size="125,125" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="ibm-certified-application-developer-watson-v3-certification" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/04/ibm-certified-application-developer-watson-v3-certification.png?w=756?w=125" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/04/ibm-certified-application-developer-watson-v3-certification.png?w=756?w=125" class=" size-full wp-image-1901 alignleft" src="https://sodohertydotcom.files.wordpress.com/2017/04/ibm-certified-application-developer-watson-v3-certification.png?w=756" alt="ibm-certified-application-developer-watson-v3-certification"   /></a>So I got my Watson V3 Certification a week or so ago, and the badge just arrived yesterday.</p>
  359. <p>I sat the mock exam without studying and passed. So I thought I&#8217;d try the real exam, and passed that too.</p>
  360. <p>Overall if you have been working in the Watson group for 3+ years, where your job role is to have medium to expert knowledge of all (non-Health) Watson products, then you are probably going to find the exam OK to pass.</p>
  361. <p>For people who haven&#8217;t, it&#8217;s not going to be easy. I strongly recommend following the <a href="http://www-03.ibm.com/certify/content/studyguides/study_guide_c7020-230.pdf">study guide</a> on the <a href="http://www-03.ibm.com/certify/tests/eduC7020-230.shtml">test preparation</a> certification page if you plan to get this.</p>
  362. <p>My only quibbles on the exam is that the technology changes a lot.</p>
  363. <p>For example, all the design patterns for coding conversation before December last year are not that relevant any more, and will likely change again soon. (Which is part reason for lack of updates on the blog, the other being laziness <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f642.png" alt="🙂" class="wp-smiley" style="height: 1em; max-height: 1em;" /> )</p>
  364. <p>So you need to know the current active technologies even if they are going away. Plus there will probably be a V4 exam in 6 months or so time.</p>
  365. <p>I&#8217;d also like to see more focused certifications for some parts of the Watson Developer Cloud. For example, being an expert at Discovery Service, doesn&#8217;t make you an expert of Conversation and vise-versa.</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1898/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1898/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1898&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  366. <wfw:commentRss>https://sodoherty.ai/2017/04/04/watson-v3-certification/feed/</wfw:commentRss>
  367. <slash:comments>0</slash:comments>
  368. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  369. <media:title type="html">simonodoherty</media:title>
  370. </media:content>
  371.  
  372. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/04/ibm-certified-application-developer-watson-v3-certification.png" medium="image">
  373. <media:title type="html">ibm-certified-application-developer-watson-v3-certification</media:title>
  374. </media:content>
  375. </item>
  376. <item>
  377. <title>Watson in the black and white room.</title>
  378. <link>https://sodoherty.ai/2017/02/10/watson-in-the-black-and-white-room/</link>
  379. <comments>https://sodoherty.ai/2017/02/10/watson-in-the-black-and-white-room/#comments</comments>
  380. <pubDate>Fri, 10 Feb 2017 20:32:54 +0000</pubDate>
  381. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  382. <category><![CDATA[watson-conversation]]></category>
  383.  
  384. <guid isPermaLink="false">http://sodoherty.ai/?p=1841</guid>
  385. <description><![CDATA[Let&#8217;s talk about the recent changes of how Watson determines it&#8217;s confidence. It seems to be a hot topic at the moment, and probably not best understood. &#160; Before:  Imagine that you are Watson, you are in a room with no doors or windows. You have learned everything about the world from Wikipedia. There is [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1841&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  386. <content:encoded><![CDATA[<p>Let&#8217;s talk about the recent changes of how Watson determines it&#8217;s confidence. It seems to be a hot topic at the moment, and probably not best understood.</p>
  387. <p>&nbsp;</p>
  388. <h2><strong>Before: </strong></h2>
  389. <p>Imagine that you are Watson, you are in a room with no doors or windows. You have learned everything about the world from Wikipedia. There is two objects, a cube and a pyramid in front of you.</p>
  390. <p>Now if someone tells you a question, you can use Wikipedia to try and figure out what the answer is, but you can only point to one of the two objects in the room. There is no other answer.</p>
  391. <p>So they may ask &#8220;Which one is an Orange?&#8221;. You may think that a cube is similar to the Discovery Cube in Orange county. You can also see that a food pyramid has an orange in it. Neither is a direct fit, but you only have two answers.</p>
  392. <p>So you respond: <em>&#8220;I am 51% sure that it is this pyramid&#8221;</em></p>
  393. <h2>After:</h2>
  394. <p>Now you are in the same room, but this time there is a window that shows you the outside world.</p>
  395. <p>You are asked the same question. You still come to the same conclusion, but because you can see the outside world you know that the answer is not in the room.</p>
  396. <p>This time you respond: <em>&#8220;I am confident that neither of these objects are an Orange&#8221;</em></p>
  397. <h2>But what about the lower confidence?</h2>
  398. <p>The first thing you notice is that the confidence is not as high as before. This in itself is not a bad thing. It is the relationship of the answer to the others found. For example:</p>
  399. <p><img data-attachment-id="1775" data-permalink="https://sodoherty.ai/2017/02/06/compound-questions/conv060217-2/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=756" data-orig-size="361,262" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv060217-2" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=756?w=361" class=" size-full wp-image-1775 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=756" alt="conv060217-2" srcset="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png 361w, https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=300 300w" sizes="(max-width: 361px) 100vw, 361px"   /></p>
  400. <p>You can see in this example the first answer is 72%, while the next one is 70%. So it is either a compound question, or you need training to differentiate between the two intents that are close together. In the previous version you could not see this.</p>
  401. <p>The main point to take from this, the confidence hasn&#8217;t actually changed. You are just finally seeing the real confidence.</p>
  402. <h2>How does this impact me?</h2>
  403. <p>First Watson would always ignore an intent if the confidence is &lt;0.2. But how the confidences were previously determined, it was rare that you would hit this condition.</p>
  404. <p>Now this is possible.</p>
  405. <p>Also if you have written conditions to determine the real confidence boundary (<a href="https://sodoherty.ai/2016/09/05/handling-low-confidence-answers-in-conversation/">detailed here</a>), you need to determine the correct boundaries.</p>
  406. <p>Lastly if no intent is matched, the you get an empty intents list.</p>
  407. <h2>In closing</h2>
  408. <p>Although the new feature is considerably better, always test before you deploy!</p>
  409. <p><em>As for the title reference: </em></p>
  410. <div class="jetpack-video-wrapper"><iframe class='youtube-player' type='text/html' width='756' height='456' src='https://www.youtube.com/embed/mGYmiQkah4o?version=3&#038;rel=1&#038;fs=1&#038;autohide=2&#038;showsearch=0&#038;showinfo=1&#038;iv_load_policy=1&#038;wmode=transparent' allowfullscreen='true' style='border:0;'></iframe></div><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1841/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1841/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1841&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  411. <wfw:commentRss>https://sodoherty.ai/2017/02/10/watson-in-the-black-and-white-room/feed/</wfw:commentRss>
  412. <slash:comments>1</slash:comments>
  413. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  414. <media:title type="html">simonodoherty</media:title>
  415. </media:content>
  416.  
  417. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png" medium="image">
  418. <media:title type="html">conv060217-2</media:title>
  419. </media:content>
  420. </item>
  421. <item>
  422. <title>Compound Questions</title>
  423. <link>https://sodoherty.ai/2017/02/06/compound-questions/</link>
  424. <comments>https://sodoherty.ai/2017/02/06/compound-questions/#respond</comments>
  425. <pubDate>Mon, 06 Feb 2017 21:05:33 +0000</pubDate>
  426. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  427. <category><![CDATA[Python]]></category>
  428. <category><![CDATA[watson-conversation]]></category>
  429.  
  430. <guid isPermaLink="false">http://sodoherty.com/?p=1762</guid>
  431. <description><![CDATA[One problem that is tricky to solve is if a user has asked two questions. Previously some solutions were&#160;to look for conjunctions (&#8220;and&#8221;) or question marks. Then try to guess if it is a question. But you could end up with a question like &#8220;Has my dog been around other dogs and other people?&#8221;. This [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1762&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  432. <content:encoded><![CDATA[<p>One problem that is tricky to solve is if a user has asked two questions. Previously some solutions were&nbsp;to look for conjunctions (&#8220;and&#8221;) or question marks. Then try to guess if it is a question.</p>
  433. <p>But you could end up with a question like <em>&#8220;Has my dog been around other dogs and other people?&#8221;</em>. This is clearly one question.</p>
  434. <p>With the new conversation feature of &#8220;Absolute Confidences&#8221;, it is now possible to detect this. Earlier versions of conversation would have all intents would add up to 1.0.</p>
  435. <p>Now each confidence has it&#8217;s own value. Taking the earlier example, if we map the confidences to a chart, we get:</p>
  436. <p><img data-attachment-id="1768" data-permalink="https://sodoherty.ai/2017/02/06/compound-questions/conv060217-1/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-1.png?w=756" data-orig-size="361,262" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv060217-1" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-1.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-1.png?w=756?w=361" class="size-full wp-image-1768 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-1.png?w=756" alt="conv060217-1" srcset="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-1.png 361w, https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-1.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-1.png?w=300 300w" sizes="(max-width: 361px) 100vw, 361px"  ></p>
  437. <p>Visually we can see that the first and second intent are not related. The next sentence&nbsp;<em>&#8220;Has my dog been around other dogs and is it certified?&#8221;</em> is two questions. When we chart this we see:</p>
  438. <p><img data-attachment-id="1775" data-permalink="https://sodoherty.ai/2017/02/06/compound-questions/conv060217-2/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=756" data-orig-size="361,262" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv060217-2" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=756?w=361" class="size-full wp-image-1775 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=756" alt="conv060217-2" srcset="https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png 361w, https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png?w=300 300w" sizes="(max-width: 361px) 100vw, 361px"  ></p>
  439. <p>Very easy to see that there are two questions. So how to do it in your code?</p>
  440. <p>You can use a clustering technique called K-means. This will cluster your data into sets of &#8216;K&#8217;. In this case we have &#8220;important intents&#8221; and &#8220;unimportant intents&#8221;. Two groups, means K = 2.</p>
  441. <p>For this demonstration I am going to use Python, but K-means exists in a number of languages. I have a sample of the full code, and example conversation workspace. So for this I will only show code snippets.</p>
  442. <h3>Walkthrough</h3>
  443. <p>Conversation request&nbsp;needs to set <strong>alternate_intents</strong> to <strong>true</strong>. So that you can get access to the top 10 intents.</p>
  444. <p>Once you get your response back, convert your confidence list into an array.</p>
  445. <pre>intent_confidences = list(o['confidence'] for o in response['intents'])</pre>
  446. <p>Next the main method will return True if it thinks it is a compound question. It requires numpy + scipy.</p>
  447. <pre>def compoundQuestion(intents):
  448. &nbsp; &nbsp; v = np.array(intents)
  449. &nbsp; &nbsp; codebook, _ = kmeans(v,2)
  450. &nbsp; &nbsp; ci, _ = vq(v,codebook)
  451.  
  452. &nbsp; &nbsp; # We want to make everything in the top bucket to have a value of 1.
  453. &nbsp; &nbsp; if ci[0] == 0: ci = 1-ci
  454. &nbsp; &nbsp; if sum(ci) == 2: return True
  455. &nbsp; &nbsp; return False</pre>
  456. <p>The first three lines will take the array of confidences and generate two centroids. A centroid is the mean of each cluster found. It will then group each of the confidences into one of the two centroids.</p>
  457. <p>Once it runs <strong>ci&nbsp;</strong>will look something like this: <strong>[ 0, 0, 1, 1, 1, 1, 1, 1, 1, 1 ]</strong> . This however can be the reverse.</p>
  458. <p>The first value is the first intent. So if the first value is 0 we invert the array and then add up all the values:</p>
  459. <p><strong>[ 1, 1, 0, 0, 0, 0, 0, 0, 0, 0 ] =&gt; 2&nbsp;</strong></p>
  460. <p>If we get a value of 2, then the first two intents are related to the question that was entered. Any other value, then we only have one question, or potentially more than two important intents.</p>
  461. <p>Example output from the code:</p>
  462. <pre><strong>Has my dog been around other dogs and other people?</strong>
  463. &gt; Single intent: DOG_SOCIALISATION (0.9876400232315063)
  464.  
  465. <strong>Has my dog been around others dogs and is it certified?</strong>
  466. &gt; This might be a compound question. Intent 1: DOG_SOCIALISATION (0.7363447546958923). Intent 2: DOG_CERTIFICATION (0.6973928809165955).
  467.  
  468. <strong>Has my dog been around other dogs? Has it been around other people?</strong>
  469. &gt; Single intent: DOG_SOCIALISATION (0.992318868637085)
  470.  
  471. <strong>Do I need to get shots for the puppy and deworm it?</strong>
  472. &gt; This might be a compound question. Intent 1: DOG_VACCINATIONS (0.832768440246582). Intent 2: DOG_DEWORMING (0.49955931305885315).</pre>
  473. <p>Of course you still need to write code to take action on both intents, but this might make it a bit easier to handle compound questions.</p>
  474. <p>Here is the <a title="compoundquestionexample" href="https://sodohertydotcom.files.wordpress.com/2017/02/compoundquestionexample.zip">sample code and workspace</a>.</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1762/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1762/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1762&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  475. <wfw:commentRss>https://sodoherty.ai/2017/02/06/compound-questions/feed/</wfw:commentRss>
  476. <slash:comments>0</slash:comments>
  477. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  478. <media:title type="html">simonodoherty</media:title>
  479. </media:content>
  480.  
  481. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/02/conv060217-1.png" medium="image">
  482. <media:title type="html">conv060217-1</media:title>
  483. </media:content>
  484.  
  485. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/02/conv060217-2.png" medium="image">
  486. <media:title type="html">conv060217-2</media:title>
  487. </media:content>
  488. </item>
  489. <item>
  490. <title>Improving your Intents with Entities.</title>
  491. <link>https://sodoherty.ai/2017/01/16/improving-your-intents-with-entities/</link>
  492. <comments>https://sodoherty.ai/2017/01/16/improving-your-intents-with-entities/#respond</comments>
  493. <pubDate>Mon, 16 Jan 2017 08:07:37 +0000</pubDate>
  494. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  495. <category><![CDATA[tips]]></category>
  496. <category><![CDATA[watson-conversation]]></category>
  497.  
  498. <guid isPermaLink="false">http://sodoherty.com/?p=1709</guid>
  499. <description><![CDATA[You might notice that when you update your entities that Conversation says &#8220;Watson is training on your recent changes&#8221;. What is happening is that Intents and Entities work together in the NLU engine. So it is possible to build entities that can be referenced within your intents. Something similar to how Dialog entities. Work. For [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1709&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  500. <content:encoded><![CDATA[<p>You might notice that when you update your entities that Conversation says &#8220;Watson is training on your recent changes&#8221;. What is happening is that Intents and Entities work together in the NLU engine.</p>
  501. <p>So it is possible to build entities that can be referenced within your intents. Something similar to how Dialog entities. Work.</p>
  502. <p>For this example I am going to use two entities.</p>
  503. <ul>
  504. <li>ENTITY_FOODSTUFF</li>
  505. <li>ENTITY_PETS</li>
  506. </ul>
  507. <p>In my training questions I create the following example.</p>
  508. <p><img data-attachment-id="1719" data-permalink="https://sodoherty.ai/2017/01/16/improving-your-intents-with-entities/conv-150118-1/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-1.png?w=756" data-orig-size="380,415" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv-150118-1" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-1.png?w=756?w=275" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-1.png?w=756?w=380" class="alignnone size-full wp-image-1719 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-1.png?w=756" alt="conv-150118-1" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-1.png 380w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-1.png?w=137 137w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-1.png?w=275 275w" sizes="(max-width: 380px) 100vw, 380px"   /></p>
  509. <p>The #FoodStore question list is exactly the same, only the entity name is changed.</p>
  510. <p>Next up create your entities. It doesn&#8217;t matter what the entity itself is called, only that if has one value that mentions the entity identifiers above. I have @Entity set to the same as the value for clarity.</p>
  511. <p><img data-attachment-id="1726" data-permalink="https://sodoherty.ai/2017/01/16/improving-your-intents-with-entities/conv-150118-2/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-2.png?w=756" data-orig-size="536,319" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv-150118-2" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-2.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-2.png?w=756?w=536" class="alignnone size-full wp-image-1726 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-2.png?w=756" alt="conv-150118-2" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-2.png 536w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-2.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-2.png?w=300 300w" sizes="(max-width: 536px) 100vw, 536px"   /></p>
  512. <p><img data-attachment-id="1725" data-permalink="https://sodoherty.ai/2017/01/16/improving-your-intents-with-entities/conv-150118-3/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-3.png?w=756" data-orig-size="524,283" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv-150118-3" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-3.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-3.png?w=756?w=524" class="alignnone size-full wp-image-1725 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-3.png?w=756" alt="conv-150118-3" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-3.png 524w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-3.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-3.png?w=300 300w" sizes="(max-width: 524px) 100vw, 524px"   /></p>
  513. <p>&nbsp;</p>
  514. <p>&#8220;What is the point?&#8221; you might ask? Well you will notice that both entities have a value of &#8220;fish&#8221;.</p>
  515. <p>When I ask &#8220;I want to get a fish&#8221; I get the following back.</p>
  516. <ul>
  517. <li>FoodStore confidence: 0.5947581078492985</li>
  518. <li>Petshop confidence: 0.4052418921507014</li>
  519. </ul>
  520. <p>So Watson is not sure, as both intents could be the right answer. This is what you would expect.</p>
  521. <p>Now after we delete the &#8220;fish&#8221; value from both entities, I then add the same training question &#8220;I want a fish&#8221; to both intents. After Watson has trained and I ask &#8220;I want to get a fish&#8221;, you get the following back.</p>
  522. <ul>
  523. <li>Petshop confidence: 0.9754140796608233</li>
  524. <li>FoodStore confidence: 0.02458592033917674</li>
  525. </ul>
  526. <p>Oh dear, now it appears to be more confident then it should be. So entities can help in making questions ambiguous if training is not helping.</p>
  527. <p>This is not without it&#8217;s limitations.</p>
  528. <p>Entities are fixed keywords, and the intents will treat them as such. So while it will find &#8220;fish&#8221; in our example, it won&#8217;t recognise &#8220;fishes&#8221; unless it&#8217;s explicitly stated in the entity.</p>
  529. <p>Another thing to be wary of is that all entities are used in the intents. So if a question mentioned &#8220;toast&#8221;, then @ENTITY_FOODSTUFF becomes a candidate in trying to determine which intent is correct.</p>
  530. <p>The last thing to be aware of is that training questions take priority over entities when it comes to determining what is correct.</p>
  531. <p>If we were to add a training question &#8220;I want fishes&#8221; to the first example. Then ask the earlier question, you would find that foodstore now takes priority. If we add &#8220;I want fishes&#8221; to both intents and ask the question &#8220;I want to get a fish&#8221;, you will get the same results as if the entities never had the word &#8220;fish&#8221; in it.</p>
  532. <p>This can be handy for forcing common spelling mistakes that may not be picked up, or clearly defined domain keywords a user may enter (eg. product ID)</p>
  533. <p>&nbsp;</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1709/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1709/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1709&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  534. <wfw:commentRss>https://sodoherty.ai/2017/01/16/improving-your-intents-with-entities/feed/</wfw:commentRss>
  535. <slash:comments>0</slash:comments>
  536. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  537. <media:title type="html">simonodoherty</media:title>
  538. </media:content>
  539.  
  540. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-1.png" medium="image">
  541. <media:title type="html">conv-150118-1</media:title>
  542. </media:content>
  543.  
  544. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-2.png" medium="image">
  545. <media:title type="html">conv-150118-2</media:title>
  546. </media:content>
  547.  
  548. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/conv-150118-3.png" medium="image">
  549. <media:title type="html">conv-150118-3</media:title>
  550. </media:content>
  551. </item>
  552. <item>
  553. <title>Prioritizing Intents</title>
  554. <link>https://sodoherty.ai/2017/01/15/prioritizing-intents/</link>
  555. <comments>https://sodoherty.ai/2017/01/15/prioritizing-intents/#comments</comments>
  556. <pubDate>Sun, 15 Jan 2017 13:53:01 +0000</pubDate>
  557. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  558. <category><![CDATA[tips]]></category>
  559. <category><![CDATA[watson-conversation]]></category>
  560.  
  561. <guid isPermaLink="false">http://sodoherty.com/?p=1600</guid>
  562. <description><![CDATA[A common question that comes up is how to handle where the end user makes two utterances, but you only want to take action on one. The most common being someone saying hello, versus saying hello with a question. You would want the question to take priority. It&#8217;s very easy to do. You just do [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1600&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  563. <content:encoded><![CDATA[<p>A common question that comes up is how to handle where the end user makes two utterances, but you only want to take action on one.</p>
  564. <p>The most common being someone saying hello, versus saying hello with a question. You would want the question to take priority.</p>
  565. <p>It&#8217;s very easy to do. You just do the following:</p>
  566. <p><img data-attachment-id="1838" data-permalink="https://sodoherty.ai/2017/01/15/prioritizing-intents/conv-150117-0/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=756" data-orig-size="1237,1006" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv-150117-0" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=756?w=756" class="alignnone size-full wp-image-1838" src="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=756" alt="conv-150117-0" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=756 756w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=300 300w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=768 768w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png?w=1024 1024w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png 1237w" sizes="(max-width: 756px) 100vw, 756px"   /></p>
  567. <ol>
  568. <li>Create your first node with a condition of <strong>True</strong> and create your priority intents under this. Set your top node to jump to the first in that branch.</li>
  569. <li>Create your second node which handles greetings.</li>
  570. <li>Add a <strong>True</strong> node at the end of your important intents, and let it jump to greeting condition.</li>
  571. </ol>
  572. <p>And that&#8217;s it! But that is the old style conversation way. Just before the new year a new version of conversation was released that makes this so much more simple.</p>
  573. <p><img data-attachment-id="1837" data-permalink="https://sodoherty.ai/2017/01/15/prioritizing-intents/conv-150117-1-2/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-11.png?w=756" data-orig-size="384,764" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv-150117-1" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-11.png?w=756?w=151" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-11.png?w=756?w=384" class=" size-full wp-image-1837 alignleft" src="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-11.png?w=756" alt="conv-150117-1" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-11.png?w=75 75w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-11.png 384w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-11.png?w=151 151w" sizes="(max-width: 384px) 100vw, 384px"   /></p>
  574. <p>The magic is in the first node.</p>
  575. <p><img data-attachment-id="1836" data-permalink="https://sodoherty.ai/2017/01/15/prioritizing-intents/conv-150117-2-2/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-21.png?w=756" data-orig-size="713,546" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="conv-150117-2" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-21.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-21.png?w=756?w=713" class="alignnone size-full wp-image-1836" src="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-21.png?w=756" alt="conv-150117-2" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-21.png 713w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-21.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-21.png?w=300 300w" sizes="(max-width: 713px) 100vw, 713px"   /></p>
  576. <p>Here we check to ensure that a greeting hasn&#8217;t been mentioned, then check each important intent.</p>
  577. <p>With this method you don&#8217;t need any complex branches or jumping around. One important thing is to ensure that your less important intents do not have any training data that may cause it to pick it over the important intents.</p>
  578. <p><a title="prioritizingintents" href="https://sodohertydotcom.files.wordpress.com/2017/01/prioritizingintents1.zip">Sample workspaces available</a>.</p>
  579. <p>&nbsp;</p>
  580. <p>&nbsp;</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1600/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1600/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1600&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  581. <wfw:commentRss>https://sodoherty.ai/2017/01/15/prioritizing-intents/feed/</wfw:commentRss>
  582. <slash:comments>3</slash:comments>
  583. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  584. <media:title type="html">simonodoherty</media:title>
  585. </media:content>
  586.  
  587. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-0.png" medium="image">
  588. <media:title type="html">conv-150117-0</media:title>
  589. </media:content>
  590.  
  591. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-11.png" medium="image">
  592. <media:title type="html">conv-150117-1</media:title>
  593. </media:content>
  594.  
  595. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/conv-150117-21.png" medium="image">
  596. <media:title type="html">conv-150117-2</media:title>
  597. </media:content>
  598. </item>
  599. <item>
  600. <title>Conversing in three dimensions.</title>
  601. <link>https://sodoherty.ai/2017/01/01/conversing-in-three-dimensions/</link>
  602. <comments>https://sodoherty.ai/2017/01/01/conversing-in-three-dimensions/#comments</comments>
  603. <pubDate>Sun, 01 Jan 2017 10:02:14 +0000</pubDate>
  604. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  605. <category><![CDATA[tips]]></category>
  606. <category><![CDATA[watson-conversation]]></category>
  607.  
  608. <guid isPermaLink="false">http://sodoherty.com/?p=1497</guid>
  609. <description><![CDATA[There is one feature of Conversation that many people don&#8217;t even factor in when creating a conversational system. Let&#8217;s take the standard plan to spell it out. Unlimited API queries/month Up to 20 workspaces Up to 2000 intents Shared public cloud Yep, you have 20 workspaces to play with! Most people starting off just use [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1497&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  610. <content:encoded><![CDATA[<p>There is one feature of Conversation that many people don&#8217;t even factor in when creating a conversational system. Let&#8217;s take the standard plan to spell it out.</p>
  611. <ul>
  612. <li>Unlimited API queries/month</li>
  613. <li><strong>Up to 20 workspaces</strong></li>
  614. <li>Up to 2000 intents</li>
  615. <li>Shared public cloud</li>
  616. </ul>
  617. <p>Yep, you have 20 workspaces to play with! Most people starting off just use it for development, testing and production. But there is so much more. Putting it in context.</p>
  618. <p><img data-attachment-id="1517" data-permalink="https://sodoherty.ai/2017/01/01/conversing-in-three-dimensions/complexityofconversation/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2016/11/complexityofconversation.png?w=756" data-orig-size="612,431" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="complexityofconversation" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2016/11/complexityofconversation.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2016/11/complexityofconversation.png?w=756?w=612" class="alignnone size-full wp-image-1517" src="https://sodohertydotcom.files.wordpress.com/2016/11/complexityofconversation.png?w=756" alt="complexityofconversation" srcset="https://sodohertydotcom.files.wordpress.com/2016/11/complexityofconversation.png 612w, https://sodohertydotcom.files.wordpress.com/2016/11/complexityofconversation.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2016/11/complexityofconversation.png?w=300 300w" sizes="(max-width: 612px) 100vw, 612px"   /></p>
  619. <h2>Functional Actions</h2>
  620. <p>For those new to Conversation, the first experience is normally the <a href="https://conversation-demo.mybluemix.net/?cm_mc_uid=03029485389514734364859&amp;cm_mc_sid_50200000=" target="_blank">car demo</a>. This is a good example of functional actions. In this case you know your application, and you want your end user to interact with it. So the user normally has prompts (conversational or visual) to allow them to refer to your user interface.</p>
  621. <p>These offer the least resistance of building. The user is taught the names of the interfaces by using it, and are unlikely to deviate from that language. In fact if they do use non-Domain language, it is more likely a fault of your user interface.</p>
  622. <h2>Question &amp; Answers</h2>
  623. <p>This is where you have collected questions from the end user, to determine what the answer/action that is needed to be taken.</p>
  624. <p>Often the end user does not understand the domain language of the documentation or business. So training on their language helps making the system better for them.</p>
  625. <h2>Process Flows</h2>
  626. <p>This is where you need to converse with the user, to collect more information to drive to meeting their needs.</p>
  627. <h2>Multiple Workspaces</h2>
  628. <p>Most see this as just creating a workspace for development, testing and production. But using these as part of your overall architecture can dramatically increase the functionality and accuracy of the system.</p>
  629. <p>Two main patterns that have been used are off-topic + drill down.</p>
  630. <h3>Off-Topic / Chit chat.</h3>
  631. <p>In this model we have a primary workspace which stores the main intents, as well as an intent for off-topic + chit-chat. Once one of these is detected, a second call is made out to the related workspace.</p>
  632. <p><img data-attachment-id="1564" data-permalink="https://sodoherty.ai/2017/01/01/conversing-in-three-dimensions/uml0117/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117.png?w=756" data-orig-size="491,206" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="uml0117" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117.png?w=756?w=491" class="alignnone size-full wp-image-1564 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117.png?w=756" alt="uml0117" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117.png 491w, https://sodohertydotcom.files.wordpress.com/2017/01/uml0117.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/01/uml0117.png?w=300 300w" sizes="(max-width: 491px) 100vw, 491px"   /></p>
  633. <p>From a price point of view this works well if you are calling out to a subject matter that is asked infrequently. If the user will often ask these questions though, then the drill down method is a better solution.</p>
  634. <h3>Drill Down.</h3>
  635. <p>This model is where the user asks a question which has a more expanded field going forward. For example when you enter a bank you may ask the information desk about mortgages, who will direct you to another desk to go into more detail on your questions.</p>
  636. <p><img data-attachment-id="1574" data-permalink="https://sodoherty.ai/2017/01/01/conversing-in-three-dimensions/uml0117a/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117a.png?w=756" data-orig-size="617,334" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="uml0117a" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117a.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117a.png?w=756?w=617" class=" size-full wp-image-1574 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117a.png?w=756" alt="uml0117a" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117a.png 617w, https://sodohertydotcom.files.wordpress.com/2017/01/uml0117a.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2017/01/uml0117a.png?w=300 300w" sizes="(max-width: 617px) 100vw, 617px"   /></p>
  637. <p>For this to work well, you need clear separation of what each workspace does. So that an off topic is triggered so as to pass back to the main workspace.</p>
  638. <p>When planning your model look for common processes vs entities. The example above might not be good to separate by pets, as they will share common questions with a different entity. But you could separate between purchasing, accessories, etc.</p>
  639. <p>As long as the conversation will not switch topics often then costs are kept down.</p>
  640. <h3>Multiple Workspace Calls.</h3>
  641. <p>This is not a recommended model as your costs go way up. It was originally used when there was a 500 intent limit per space (NLC).</p>
  642. <p><img data-attachment-id="1584" data-permalink="https://sodoherty.ai/2017/01/01/conversing-in-three-dimensions/uml0117b/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117b.png?w=756" data-orig-size="296,272" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="uml0117b" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117b.png?w=756?w=296" data-large-file="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117b.png?w=756?w=296" class="alignnone size-full wp-image-1584 aligncenter" src="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117b.png?w=756" alt="uml0117b" srcset="https://sodohertydotcom.files.wordpress.com/2017/01/uml0117b.png 296w, https://sodohertydotcom.files.wordpress.com/2017/01/uml0117b.png?w=150 150w" sizes="(max-width: 296px) 100vw, 296px"   /></p>
  643. <p>If money is no object, then this model works where you may have more then 2000 intents, or a number of intents that share similar patterns but you need to distinguish between them.</p>
  644. <p>You need to factor in if your conversation service is returning relative or absolute confidences. If relative, then responses are relative to their workspace and not to each other.</p>
  645. <p>If you do have numerous intents, it may be easier and better to use a different solution, like Discovery Service for example.</p>
  646. <p>&nbsp;</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1497/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1497/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1497&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  647. <wfw:commentRss>https://sodoherty.ai/2017/01/01/conversing-in-three-dimensions/feed/</wfw:commentRss>
  648. <slash:comments>5</slash:comments>
  649. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  650. <media:title type="html">simonodoherty</media:title>
  651. </media:content>
  652.  
  653. <media:content url="http://sodohertydotcom.files.wordpress.com/2016/11/complexityofconversation.png" medium="image">
  654. <media:title type="html">complexityofconversation</media:title>
  655. </media:content>
  656.  
  657. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/uml0117.png" medium="image">
  658. <media:title type="html">uml0117</media:title>
  659. </media:content>
  660.  
  661. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/uml0117a.png" medium="image">
  662. <media:title type="html">uml0117a</media:title>
  663. </media:content>
  664.  
  665. <media:content url="http://sodohertydotcom.files.wordpress.com/2017/01/uml0117b.png" medium="image">
  666. <media:title type="html">uml0117b</media:title>
  667. </media:content>
  668. </item>
  669. <item>
  670. <title>Data Science Experience</title>
  671. <link>https://sodoherty.ai/2016/12/09/data-science-experience/</link>
  672. <comments>https://sodoherty.ai/2016/12/09/data-science-experience/#respond</comments>
  673. <pubDate>Fri, 09 Dec 2016 18:08:32 +0000</pubDate>
  674. <dc:creator><![CDATA[sodoherty]]></dc:creator>
  675. <category><![CDATA[Data Science]]></category>
  676. <category><![CDATA[Python]]></category>
  677.  
  678. <guid isPermaLink="false">http://sodoherty.com/?p=1529</guid>
  679. <description><![CDATA[Apologies in my long time updating, life has been a bit crazy busy at the moment. I have a few entries cached to go, but couldn&#8217;t get around to finishing. As this year is nearly at an end for me, I should have some spare time to catch up. So this is a brief entry [&#8230;]<img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1529&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
  680. <content:encoded><![CDATA[<p>Apologies in my long time updating, life has been a bit crazy busy at the moment. I have a few entries cached to go, but couldn&#8217;t get around to finishing. As this year is nearly at an end for me, I should have some spare time to catch up.</p>
  681. <p>So this is a brief entry to talk about IBM Data Science experience. This is a new service which hooks into Bluemix. Using Spark it allows you build python/R/Scala notebooks.</p>
  682. <p>For those not familiar with notebooks, they are a really cool way to create prototyping code as documentation. It also has a whole host of extras that you can hook into to visualise and manipulate your data. As well as loads of datasets to play with.</p>
  683. <p><img data-attachment-id="1535" data-permalink="https://sodoherty.ai/2016/12/09/data-science-experience/dse/" data-orig-file="https://sodohertydotcom.files.wordpress.com/2016/12/dse.png?w=756" data-orig-size="757,545" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="dse" data-image-description="" data-medium-file="https://sodohertydotcom.files.wordpress.com/2016/12/dse.png?w=756?w=300" data-large-file="https://sodohertydotcom.files.wordpress.com/2016/12/dse.png?w=756?w=756" class="alignnone size-full wp-image-1535" src="https://sodohertydotcom.files.wordpress.com/2016/12/dse.png?w=756" alt="dse.png" srcset="https://sodohertydotcom.files.wordpress.com/2016/12/dse.png?w=756 756w, https://sodohertydotcom.files.wordpress.com/2016/12/dse.png?w=150 150w, https://sodohertydotcom.files.wordpress.com/2016/12/dse.png?w=300 300w, https://sodohertydotcom.files.wordpress.com/2016/12/dse.png 757w" sizes="(max-width: 756px) 100vw, 756px"   /></p>
  684. <p>You can <a href="https://apsportal.ibm.com/analytics" target="_blank">check it out for yourself</a>. Here is the<a href="https://apsportal.ibm.com/analytics/notebooks/cdc113db-1d13-4584-b894-4bbbc0559020/view?access_token=1436a23215501178e5d3ef208f86e755b1675e3f4117b76ffeed962a23b819d7"> example notebook</a> from above.</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/sodohertydotcom.wordpress.com/1529/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/sodohertydotcom.wordpress.com/1529/" /></a> <img alt="" border="0" src="https://pixel.wp.com/b.gif?host=sodoherty.ai&#038;blog=31007449&#038;post=1529&#038;subd=sodohertydotcom&#038;ref=&#038;feed=1" width="1" height="1" />]]></content:encoded>
  685. <wfw:commentRss>https://sodoherty.ai/2016/12/09/data-science-experience/feed/</wfw:commentRss>
  686. <slash:comments>0</slash:comments>
  687. <media:content url="http://0.gravatar.com/avatar/074e6aa01640f2d5425f42e8d7dca51a?s=96&#38;d=identicon&#38;r=G" medium="image">
  688. <media:title type="html">simonodoherty</media:title>
  689. </media:content>
  690.  
  691. <media:content url="http://sodohertydotcom.files.wordpress.com/2016/12/dse.png" medium="image">
  692. <media:title type="html">dse.png</media:title>
  693. </media:content>
  694. </item>
  695. </channel>
  696. </rss>
  697.  

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

  1. Download the "valid RSS" banner.

  2. Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)

  3. Add this HTML to your page (change the image src attribute if necessary):

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//sodoherty.com/feed/

Copyright © 2002-9 Sam Ruby, Mark Pilgrim, Joseph Walton, and Phil Ringnalda