Exploring Windows Mixed Reality, switching between 2D / 3D and embedding Web Views

In these days, I’m exploring the options for switching between Unity 3D and XAML 2D views integrating the access to UWP APIs for content hosted on Web Views.

This scenario could be particularly useful if an app needs to reuse existing code, perhaps available in a website, with the requirement to access the Windows Runtime when executed in Windows Mixed Reality devices and activated from a Unity 3D scene.

Creating the Unity project

To start, I created a new Unity scene and imported the HoloToolkit package downloaded from here.

I applied the Scene and Project settings from the HoloToolkit|Configure menu and added the following prefabs available from the imported package:

  • HoloLens Camera
  • InputManager
  • Cursor

Then, I added a simple cube which, when gazed and air-tapped, permits to trigger the view switching.

I defined a new script TapBehaviour to capture the event and call the 2D view:

The interesting part here is the one using the AppViewManager for switching views: after some research, I decided to use this class from GitHub as a starting point for handling the logic via CoreApplicationView.

I created a new script in Unity (AppViewManager.cs) and added the source code from the GitHub repository.

In this way, it is possible to retrieve the list of registered views or a specific one using for instance

And then simply transition to the new page by calling the Switch() or SwitchAsync() methods.

Creating and registering the 2D views

The 2D view named ContentPage needed to be defined and registered in the UWP project generated by Unity using the File|Build Settings|Build command after setting the UWP Build type to XAML to be able to define additional views in my project

Then, I opened the project in Visual Studio and added a new page called ContentPage.xaml containing a WebView and a button to switch back to Unity 3D

And the code makes use again of the AppViewManager previously imported in Unity:

We are now making use of two pages MainPage and ContentPage, respectively hosting the 3D and 2D content. Before using them, I needed to register both in the App.cs class right after initialising the Unity player:

Accessing UWP APIs from the hosted web page

The Web View is actually showing content from this web page hosted in my personal web-site which tries to access the UWP APIs if hosted with elevated permissions:

To enable Windows Runtime access, I modified the Package.appxmanifest and added the following under the Application section:

After this step, I was able to launch the app using an HoloLens device or emulator:

And then switch to the 2D view hosting the web page accessing UWP APIs on HoloLens by tapping the cube:

And return to the Unity 3D view using the “Click me” button.

As usual, the source code is available for download on GitHub.

Experiments with HoloLens, Bot Framework and LUIS: adding text to speech

Previously I blogged about creating a Mixed Reality 2D app integrating with a Bot using LUIS via the Direct Line channel available in the Bot Framework.

I decided to add more interactivity to the app by also enabling text to speech for the messages received by the Bot: this required the addition of a new MediaElement for the Speech synthesiser to the main XAML page:

Then I initialized a new SpeechSynthesizer at the creation of the page:

And added a new Speech() method using the media element:

When a new response is received from the Bot, the new Speech() method is called:

And then the recognition for a new phrase is started again via the MediaEnded event to simulate a conversation between the user and the Bot:

As usual, the source code is available for download on GitHub.

Validate input in Xamarin.Forms using INotifyDataErrorInfo, custom Behaviors, Effects and Prism

This post is also available in the Premier Developer blog.

I have recently been investigating the support available in Xamarin.Forms for validation and, in particular, researched the possibility of using INotifyDataErrorInfo to complement the traditional approach of using Behaviors.

In simple scenarios, it’s possible to perform validation by simply attaching a Behavior to the required view, as shown by the following sample code:

In this case, we are validating the input and modifying the UI if the number of characters entered is less than 5.

What about more articulated scenarios when multiple business rules are required to be checked for the input values?

In these cases, we could take advantage of other types available to make our code more structured and extensible:

  • INotifyDataErrorInfo: available in Xamarin.Forms. When implemented it permits specifying custom validation supporting multiple errors per property, cross-property errors and entity-level errors;
  • DataAnnotations decorate the data models using attributes which specify validation conditions to be applied to the specific field;
  • Forms Behaviors specify the specific UI to be applied to the specific validation scenarios, integrating with INotifyDataErrorInfo and DataAnnotations.

To start exploring this approach, I created a new Xamarin.Forms Prism solution using the Prism Template Pack which generated the following project structure:

Then, I added the following new model to be validated using DataAnnotations and INotifyDataErrorInfo:

The model uses attributes declared in the SystemComponentModel.DataAnnotations namespace which can be referenced in the solution modifying the Portable Class Library profile of the UsingValidation common project:

Quick tip: to be able to change the PCL profile I had to remove all the NuGet packages used by the common project, remove the Windows Phone 8 profile and then add back all the removed NuGet packages to the UsingValidation PCL.

To use the capability offered by INotifyDataErrorInfo, the model needs to implements 3 members defined in the interface:

  • GetErrors() returns an IEnumerable sequence of strings containing the error messages triggered by validation;
  • the HasErrors property returns a boolean value indicating if there are validation errors;
  • ErrorsChanged event can be triggered to Notify if the validation errors have been updated.

This interface is quite flexible and is designed to be customised depending on the different scenarios needed: I took as a starting point this implementation available on GitHub and modified it accordingly: I decided to separate the implementation of INotifyDataErrorInfo in a different base class called ValidationBase which contains the following code using a Dictionary<string, List<string>> needed for storing the generated validation errors:

The validation is performed by this method which evaluates the DataAnnotations decorating the model using the Validator available in the System.ComponentModel.DataAnnotations namespace and then stores the error messages in the dictionary:

At this stage, I needed a solution for linking the UI to the model, and modifying the visuals depending on the presence or not of validation errors.

The ViewModel for the sample page, contains only a property storing an instance of the item defined in the model:

Then, the corresponding XAML contains two Entry views used for input and a ListView used for showing the validation errors:

The sample page uses a Behavior called EntryValidationBehavior which take care of changing the colour of the Entry background views in the case validation errors are present:

The UI is also fine-tuned using a Xamarin.Forms effect applied only to the UWP platform, in order to change the colour of the Entry border when validation errors occur:

And this is the result when the application is executed on Android and UWP:

Conclusions

Xamarin.Forms provides a rich set of features for implementing validation: the usage of INotifyDataErrorInfo, Data Annotations, Behaviors and Effects permit the handling of complex scenarios including multiple conditions per field, cross-property validation, entity-level validation and custom UI depending on the platform.

The source code of the sample demo is available for download on GitHub here.

Happy coding!