Microsoft Bot Framework: using a LuisDialog for processing intents

In the previous post, I blogged about integrating a Holographic 2D app with Bot Framework and LUIS.

I also spent some time going through these great samples for some presentations and found a very nice implementation for handling the Bot messages when LUIS intents are recognised.

Instead of using the exposed LUIS endpoint and parsing the returned JSON, the framework already provides a specific LuisDialog<> type which can be used for handling the various intents, in order to make the code cleaner and more extensible.

I’ve then modified the HoloLensBotDemo sample and added a new RootLuisDialog:

And this is all the code now needed for the Bot.

The updated source code is available on GitHub.

Happy coding!

 

Experiments with HoloLens, Bot Framework, LUIS and Speech Recognition

Recently I had the opportunity to use a HoloLens device for some personal training and building some simple demos.

One of the scenarios that I find very intriguing is the possibility of integrating Mixed Reality and Artificial Intelligence (AI) in order to create immersive experiences for the user.

I decided to perform an experiment by integrating a Bot, Language Understanding Intelligent Services (LUIS), Speech Recognition and Mixed Reality via a Holographic 2D app.

The idea was to create a sort of “digital assistant” of myself that can be contacted using Mixed Reality: the first implementation contains only basic interactions (answering questions like “What are your favourite technologies” or “What’s your name”) but these could be easily be expanded in the future with features like time management (via the Graph APIs) or tracking projects status, etc.

Creating the LUIS application

To start, I created a new LUIS application in the portal with a list of intents that needed to be handled:

In the future, this could be further extended with extra capabilities.

After defining the intents and utterances, I trained and published my LUIS app to Azure and copied the key and URL for usage in my Bot:

Creating the Bot

I proceeded with the creation of the Bot using Microsoft Bot framework downloading the Visual Studio template and creating a new project:

The Bot template already defined a dialog named RootDialog so I extended the generated project with the classes required for parsing the JSON from the LUIS endpoint:

And then processed the various LUIS intents in RootDialog (another option is the usage of the LuisDialog and LuisModel classes as explained here):

Then, I tested the implementation using the Bot Framework Emulator:

And created a new Bot definition in the framework portal.

After that, I published it to Azure with an updated Web.config with the generated Microsoft App ID and password:

Since the final goal was the communication with an UWP HoloLens application, I enabled the Diret Line channel:

Creating the Holographic 2D app

Windows 10 UWP apps are executed on the HoloLens device as Holographic 2D apps that can be pinned in the environment.

I created a new project using the default Visual Studio Template:

And then added some simple text controls in XAML to receive the input and display the response from the Bot:

I decided to use the SpeechRecognizer APIs for receiving the input via voice (another option could be the usage of Cognitive Services):

The SendToBot() method makes use of the Direct Line APIs which permit communication with the Bot using the channel previously defined:

And then I got the app running on HoloLens and interfacing with a Bot using LUIS for language understanding and Speech recognition:

The source code of the project is available on GitHub here.

Happy coding!

Microsoft Bot Framework: showing a welcome message at the start of a new conversation

Recently I’ve worked on some projects related to Bot Framework and enjoyed the functionalities which permit to automate actions in response to user interactions.

It is important to provide the user with a great experience: one “nice touch” can be achieved by providing a welcome message at the beginning of a new conversation.

The first solution I tried was triggering the welcome message in the ConversationUpdate activity:

When I ran this, the message was presented twice in the Bot Framework Emulator:

After some investigation, I discovered that the ConversationUpdate activity is triggered both when the connection to the Bot is established and when a new user joins the conversation.

As explained on GitHub, the correct way to handle this case is by showing the welcome message only when a new user is added:

Using this approach the welcome message is displayed properly:

Happy coding!

Using Prism modularization in Xamarin.Forms

Recently, Prism for Xamarin.Forms 6.2.0 has been released with many notable improvements including a new bootstrapping process, AutoWireViewModel behaviour changes, deep-linking support,  modularity and Prism template pack enhancements (full release notes available here).

Today, I fired up Visual Studio to have a play with this new version and decided to try the Xamarin.Forms support for Prism Modules: this is a very useful feature which allows to separate clearly the various parts of the application and improve quality, extensibility and readability of the code.

After downloading the new template pack, I created a new solution selecting New Project => Templates => Visual C# => Prism => Forms => Prism Unity App:

PrismTemplatePack

The new wizard is very useful and permits to select the platforms to target in the project: I selected Android, iOS and UWP and the project was generated targeting the three platforms with a shared PCL. NuGet packages were already updated to the latest version so no need for further actions.

While exploring the new project structure and the new modularization stuff, I decided to create a new Xamarin.Forms portable class library containing a module with a single View/ViewModel (SamplePage SamplePageViewModel) visualised when a user interacts with a button on the home page.

The new module required the definition of the following class implementing the Prism IModule interface:

To keep the logic separated from the rest of the app, I decided to register the navigation type for SamplePage inside the Initialize() method of the module which will be triggered when the module loads.

I also applied Xamarin.Forms XAML compilation to the module to improve performance, which is always great to have 😉

It’s worth noting that in this new Prism release the default value for the attached property ViewModelLocator.AutowireViewModel is set to true, so we can omit it and the framework will automatically associate SampleViewModel as the BindingContext for the view:

I then explored the new breaking changes for the bootstrapping process: the application class now needs to inherit from the PrismApplication class and two new virtual methods OnInitialized() and RegisterTypes() permit respectively to specify the implementation for navigating to the home page and registering the known types / ViewModel’s for navigation:

The third overridden method, ConfigureModuleCatalog(), informs the app to initialize the catalog with the module we created previously and set the initialization mode to OnDemand which means the module is not activated when the application starts, but it must load explicitly. This feature is particularly useful in cases in which some functionalities of the app must initialise after some other requirements like, for instance, successful authentication or applications extended via modules.

The sample view was in place, so I proceeded with the implementation of the HomePage: I wrapped the existing one in a NavigationPage to allow the correct back stack and then created two commands for initializing the module and navigating to the SamplePage defined previously:

and the corresponding ViewModel:

The module is initialized by injecting the Prism ModuleManager and then calling the LoadModule() method:

The navigation to the page defined in the module is performed by:

The property IsSampleModuleRegistered permitted to control the CanExecute() for the button commands using this nice fluent syntax using ObservesProperty(()=>….) available in Prism:

Here we go: I found the Prism implementation in this new version very stable and working well with Xamarin.Forms. The modularization capabilities are useful to write clean, maintainable and extensible apps.

The source code is available for download as part of the official Prism samples on GitHub.

Looking forward to exploring all the other capabilities available in Prism and Xamarin.Forms. Happy coding!

Having fun with Xamarin.Forms and Multi-Touch Behaviors

Recently Xamarin has released preview support for the Universal Windows Platform in their Xamarin.Forms framework so I have been playing around with version 2.0 for testing its features and verify how easy is to target multiple platforms (iOS, Android, Windows 10 UWP, Windows Phone, Windows 8.1) with a single codebase.

One of the experiments I have done is related to custom multi-touch gestures: the idea to use a XAML Behavior is a common scenario to write well structured code so I started creating a new Cross-Platform Xamarin.Forms Portable project and upgraded the NuGet packages to the latest stable version of the framework (currently v2.0.1.6505).

I have then read the official documentation and analysed the samples available on GitHub: a very good example is the PinchGesture one so, starting from it, I created a new MultiTouchBehavior implementing this gesture and attached the same Behavior to an Image object added to a sample ContentPage, as described below in this lovely cross-platform XAML 🙂

The BindingContext=”{Binding}” is used to trigger the BindingContextChanged event and initialise correctly the GestureRecognizers  for the parent object since the AssociatedObject.Parent is set to null when the Behavior.OnAttachedTo() is called (I suppose that the XAML tree is not yet completely created when the behavior is attached in Xamarin.Forms):

Here is the project deployed to the Android, iOS and Windows 10 emulators:

MultiTouch_Android_Emulator   Simulator Screen Shot 14 Feb 2016, 18.25.00   MultiTouch_Windows10_Emulator

I’ve been particularly impressed by Xamarin.Forms: the possibility to target so many platforms using the same code is a killer feature and the development environment is also very comfortable to use.

The sample code used in this post is available on GitHub, I’m planning to add more functionalities in the future with particular regard to other common multi-touch gestures using Xamarin.Forms and XAML.

NDepend v6 available

Version 6 of NDepend is now available for download from the official site.

New features include:

  • enhanced Visual Studio integration;
  • support for Visual Studio 2015;
  • rule files shareable amongst projects;
  • default rules description and HowToFix;
  • default rules less false positives;
  • colored code metric view;
  • intuitive display of code coverage percentage;
  • compiler generated code removal;
  • async support;
  • analysis enhancements;
  • support for Visual Studio Blue, Dark, Light themes;
  • support for high DPI resolution;
  • integration with TFS, SonarQube and TeamCity.

A detailed description of the new capabilities is available here.

Windows 10 UWP: Using PlayReady content protection with live DASH

I’ve recently blogged about using PlayReady content protection with Smooth Streaming on Windows 10 UWP by initialising a MediaProtectionManager object in the player element.

Since PlayReady is a technology for handling protected media content, it is possible to use it with different stream formats like Dynamic Adaptive Streaming over HTTP (DASH).

In the previous days, I have been exploring the live DASH sample available in the Universal Windows Platform repository on GitHub: the example enables playback of streams containing the profile “urn:mpeg:dash:profile:isoff-live:2011“.

It is possible to just add PlayReady content protection to this live player by initialising the MediaExtensionManager and then handling the MediaProtectionManager setup / ServiceRequested event:

The updated Visual Studio solution is available here on GitHub.

NDepend v5.0 available

I’ve just received my copy of NDepend v5.0: this new version adds several great features to the product including

  • Support for Visual Studio 2013 (yeah!)
  • New Dashboard Panel
  • Focusing on Recent Rules violation
  • UI Enhancements
  • Trend Monitoring
  • Churning Rules and Queries
  • Report Enhancements

Click here to read a detailed description of the new capabilities and access the download links (14 days trial available).

Happy coding!

NDepend v4.1 adds CQLinq killer feature and support for Windows Store apps

I already blogged about NDepend basic features here: I was positively impressed by the capabilities of this tool by fellow MVP Patrick Smacchia.

NDepend permits you to examine .NET projects and highlight different metrics in the code in order to better analyse what areas of your solution needs refactoring for improving performance, code maintainability, application stability and applying best coding practices (just to name a few).

Some features, already available in the previous versions of NDepend, include tools for visually representing dependencies between classes, defining custom rules on code, explore code diffs and analyse code coverage. These ones are particularly helpful when the size of the project grows: a visual representation helps the developer to better identify areas that needs more work (and then save development time).

Recently Patrick contacted me for highlighting the new features available in NDepend v4.1. I was very pleased to find support for Windows Store apps but another capability got immediately my attention: CQLinq.

This is a killer feature: CQLinq (acronym for Code Query LINQ) enables the developer “to query the code” for retrieving information about it. Some simple examples of queries could be:

  • find me the methods too big / too complex;
  • find the methods added but poorly covered by tests;
  • get information about naming conventions;
  • any other query you like: the possibilities are not limited at all!

Technically these queries can be constructed using the traditional LINQ syntax and the types defined in the NDepend.CoreModel of the NDepend API.

The best place to start using CQLinq is here by exploring the dedicated section on the site which contains a ton of different examples to start: I recommend to try and run the available samples which are an excellent starting point and then modifying them for custom queries.

I’ve been really impressed by CQLinq and I can’t wait to use it on real projects to enhance the quality of the code and then reduce the time needed for new implementations.

Happy coding everyone!

Windows Phone 8 Multi-Touch Behaviors available on CodePlex

The last week has been very exciting for Microsoft developers: The //build/ conference in Redmond was full of amazing content including Windows 8Windows Phone 8 SDK, Microsoft Surface, Xbox SmartGlass, Kinect for Windows and the new language TypeScript.

Unfortunately I hadn’t the possibility to participate at the conference but had some free time during the week-end to watch some of the recorded sessions available here on Channel9: this is really a great resource and I suggest everyone interested in Microsoft technologies to download some videos and have fun with it.

I decided then to take a look at the new features available in Windows Phone 8 and the related SDK: Justin Angel has a great article about this topic in the Nokia Developer Community.

To get my hands dirty I moved then on my personal Open-Source project “Multi-Touch Behaviors” on CodePlex: this is a set of Epression Blend Behaviors for different platforms including Windows Phone, Silverlight and WPF for applying gestures and manipulations (like the one available in the “old” ScatterView control) to generic XAML elements.

The project was not yet updated to Windows Phone 8 so I took this opportunity and experimented with the new SDK: this was really a great experience since I was able to upgrade quickly all my samples to the new platform using the same codebase used for the Silverlight version.

Unfortunately I haven’t the possibility  to test the performance on a real Windows Phone 8 device at the moment but the first impressions are really positive.

I have already checked in the code for WP8 here, I’m planning more testing and development in the future.

Happy XAMLing everyone 🙂