Monthly Archives: May 2016

Back from Sweden! Slides and code for my #DevSum16 session

.NET, Conferences, HoloLens, Technical stuff, Universal Windows Platform UWP, Windows 10, XAML
No Comments

I am back from Stockholm and the DevSum conference! It was a great trip, what a beautiful city and lovely people! It was great to be there with so many friends from our beautiful community, and especially my colleague and friend Rene Schulte.

On Thursday, I started by helping my good friend Tim Huckaby with his keynote. My role was very limited, just monitoring the live stream coming from Tim’s HoloLens. Tim is a great speaker and his keynote was very interesting and funny. So nice to see him! Later, Rene did a talk about HoloLens 3D development. Always a great pleasure to see Rene show how to use Unity and build 3D apps!

image6

On Friday, it was my turn to speak. I have a Windows 10 Universal session and I decided to spice it up with a few demos of new platforms. It was a bit scary because there were quite a few moving pieces, and everything had to play together perfectly. And it almost did! The only thing I didn’t think about is that the Continuum dock is “protected” with HDCP. That means that you cannot connect the Dock to a projector. Such an annoying (and useless) feature! Thankfully, the conference center where DevSum took place had awesome technicians, and one of them saved my day by connecting an HDMI-to-VGA adapter which circumvented the issue.

The talk was quite beefy: We defined what a Universal app is, then we talked about Adaptive UI. We discussed the Centennial and Islandwood bridges before talking about Continuum and HoloLens. The final demo saw me switch my presentation to my Windows 10 mobile phone (950XL), show some slides in Powerpoint, then demo some Universal apps on this platform. Finally I started the HoloLens application on my phone, in Continuum mode. Since this is also a Universal app, it adapted to the big screen beautifully and I could stream what I was seeing through my HoloLens. I demonstrated how universal applications run on the holographic platform, including our own apps. It’s just as simple as deploying them to the HoloLens!

image4

We definitely live exciting times, and it’s really great to be working on these new platforms. You can find the slides and sample code for this talk on my website. The session was recorded, and I will tweet when the recording is available for your viewing please, so stay tuned to my Twitter feed. Thanks to everyone who came to see this session. I really hope it was informative and useful to you!

Happy coding
Laurent

GalaSoft Laurent Bugnion
Laurent Bugnion (GalaSoft)
Share on Facebook
 

My #HoloLens 101 notes

HoloLens, Technical stuff, Work
2 Comments

Yes finally I have a HoloLens device (thank you IdentityMine for footing the bill and facilitating the purchase). Even though IdentityMine has a long history of 3D development, and we have been working with HoloLens devices for the past few months in collaboration with Microsoft, so far my inputs have been limited to helping the team brainstorm concepts, test the apps, give feedback, and help with some presales activities. In the contrary of some of my colleagues, I am not a 3D developer yet, and I am eager to learn.

Here are a few notes I took during the learning stages. I am sure that many of you are in the same boat as I am, and these may come handy if you want to progress with HoloLens and Unity3D development.

A few preliminaries

As I mentioned, IdentityMine has been developing for HoloLens for a while now. You can see a session that my esteemed colleague Rene Schulte and I gave at the recent Build 2016 conference about an application that we built.

Even though you can code HoloLens applications with any 3D framework that supports Direct3D, the consensus these days (and the solution that Microsoft recommends) is to use Unity. This is a middleware (meaning that this is a set of APIs that will create code that then runs on a number of platforms, including HoloLens).

Rene Schulte has a very good blog post with recommendations. You should probably read it too!

Charging the device

You would think that charging the device is trivial! Well in fact… it charges with a micro-USB cable similar to what you use for your mobile phone (except if you have the latest generation which uses USB-C). Like for mobile phones, there are two kinds of cables: Charging only, and Charging+Data. Whichever you use, make sure that you test your cables and use the ones which charge the fastest. When testing cables to charge my mobile phones, I found huge discrepancies (some cables charge up too 3-4 times faster than others). A good choice is to use the cable and charger which come with the HoloLens device itself!

With a full charge you will be able to use the device for a few hours. I didn’t time it really, but when learning to code, I leave the device on and use it on and off for a whole evening without needing to charge. You can then charge it overnight, which is very convenient (it takes a while to get a full charge, so plan accordingly!).

Calibrating

Before you start using the device, you need to calibrate it. This is important so that your eyes can focus properly on the holograms. The Calibration app takes a couple of minutes to execute. The very first time you start the device, you will also get a tutorial about gestures. Later on, the Learn Gestures app can be started separately. Probably a good idea to run through this basic information! Note that you can modify and retrieve the Interpupillary distance (IPD) from the Device Portal (see below). This is an important value for a good experience!

Using apps

What better way to get started than to use apps! For a first try, the Holograms app (preinstalled) is cool: It allows you to place holograms anywhere in the room, and observe them from diverse angles. There are even some video holograms. It’s a good idea to use this app to learn to interact with the menus. Note that you will need an internet connection for this app to work!

In addition to the preinstalled apps, you have a choice of apps developed specifically for HoloLens in the store. I like the HoloTour, which gives you a guided tour of Rome and a high plateau in Peru. I hope we will see more content coming soon! Also available are games like Young Conker and Roboraid. Another app named Holo Anatomy will teach you about the human body. While HoloTour, Young Conker and Roboraid are what I would call full blown apps, Holo Anatomy is more of a proof of concept. Still cool to try!

Make sure to try the Fragments game too! It is probably the most compelling holographic experience at the moment, and integrates very nicely with the room you are in! One of the character was seen sitting on my office chair for example!!

There is also a great app developed on suggestions from the users’ community, called Galaxy Explorer. This app is available as source code on Github and you need to build it. See below in the “Learning to code” section.

You can also install 2D universal apps for Windows 10. Any UWP app should run fine though some (like Skype) have been customized for the HoloLens device. I installed OneDrive (which allows me to watch videos stored in the cloud), Groove Music (which lets me stream music from my OneDrive), Cast (a great universal podcast app which synchronizes content between all Windows 10 devices) and a few more.

Unfortunately I cannot find a way to download and save a file on the device, so for now the only media you can consume is online streamed media. This is OK but I hope we’ll be able to get some media on the drive for offline consumption at some point.

Here are a few screenshots. Note that some are a screen grab of the portal videostream so the quality is not great, but they are just illustrating the point. The actual quality is much better!

13173339_10154069664749651_6183228024374949655_o
The game “Fragments” is one of the most impressive HoloLens experiences at the moment.

20160512_133624_HoloLens
Groove Music and Skype pinned to my home office wall

2016-05-12_14-04-31
On a HoloSkype call with my colleague Rene. We both can draw on the scene (him in green and me in blue).

Finding your device’s IP address

In many occasions you will need your device’s IP address. To do this, wear the device, and then say “Hey Cortana”. After Cortana shows up and you hear the sound meaning that she is listening, ask “What is my IP address”. It will show up. Alternatively, you can open the Settings app, and then navigate to Network & Internet, Wi-Fi, Advanced Options and write down the IPv4 address. Note that you can also find it using your Router’s configuration menu, if you have access to it. It might be a good idea to configure your router to always use the same IP address for the device.

Using the device portal or the app

The device portal is a must-try. To open it is quite easy: First you need to find your device’s IP address (see above). Then you can enter this IP address in a web browser (ignore warnings about invalid security certificate), and it will connect to your device which acts as a web server. Note that on some networks, this will be prevented to work so make sure that the network you use is open to this.

Alternatively, you can try the HoloLens app available here. The cool thing is that it is a universal app, and so you can also let it run on a Windows 10 mobile phone. UWP FTW!!

The web portal lets you do the following operations:

  • Checking the device status and other information such as device name and Windows version, online status, temperature, battery level etc
  • Check your interpupillary distance (IPD) and modify this value
  • Change sleep settings. I recommend setting a high enough value so the device doesn’t go to sleep while you are coding, which can cause deployment to fail!
  • See in real time (almost) what the user is seeing. This is called Mixed Reality Capture (MRC). It even lets you make videos and take pictures (which include the holograms!). Note however that there is a lag of a couple of seconds between what the user sees and what is shown on the screen. Also, the screen refresh rate will drop to 30 FPS while the MRC is active (instead of ideally 60FPS). If you have fast animations, this can be a little disconcerting.
  • See the list of processes running on the device and getting performance information.
  • Seeing a list of installed apps and providing some management tools. You can even side load an app from this page.
  • Getting crash information.
  • Forcing the device to run in kiosk mode (i.e. disabling the start menu, disabling Cortana and hiding pinned applications). Great for demos!
  • Logging
  • Simulating rooms: You can take a capture of a given room and save it. Then you can pass this recording to another developer, who can use this page to load it to his device. Great when you need to test an app in some specific room conditions.
  • Networking settings and information such as device IP, MAC address etc
  • Virtual input, allowing the developer to simulate keyboard input. Note however that you can simply use a bluetooth keyboard if you want to easily enter text.

The universal app seems to show the live stream with less lag. Definitely worth a try!

Taking screenshots and videos

You can take screenshots and videos of what you see in two different ways. Note however that the resolution of the screenshots and the videos is not going to be as good as the real thing, because they downplay the device resolution when screen capture is on.

Using the device portal

If you are an operator for someone else trying the headset, launch the device portal (see above) and navigate to the Mixed Reality Capture section. You will see buttons allowing you to record, take photo or even see the live stream (with a few seconds delay).

Using Cortana

More spontaneously, if you are in a great experience on HoloLens and want to share with the world, you can take a screenshot by saying “Hey Cortana” and then “Take a picture”. Similarly, for videos, say “Hey Cortana” and then “Take a video”. When you are done filming, say “Hey Cortana” and then “Stop video”. This takes a bit of exercise to get right, so make sure to train before you do it for real. The pictures and videos will then show up in the Photos application on the device, as well as in the Device Portal, where they can be downloaded from.

Letting people try the HoloLens

For a better experience, you should always get a new user’s Interpupillary distance IPD! Use the Calibration app to do that. Alternatively, there are some devices one can purchase, but they require some know how so make sure to learn how to use them. Once a user’s IPD has been determined, you can always modify it or retrieve it from the Device Portal (see above).

One thing I noticed after letting a few “newbies” try it out: The “tap” gesture is not as easy as it sounds. A good friend never got it right. So in addition to the calibration, it is interesting to get some time to do the Learn Gestures applications too. At first it is a bit hard to guide them to start the Calibration application. What I did to help is this:

  • Connect the UWP app to the device and observe what they are seeing. Sure it drops the frame rate but it is really helpful to know what they are currently seeing, in the beginning.
  • You can do some gestures for them. Simply put your own hand in front of the visor and bloom or air tap. For example to teach my friend how to air tap, at first I told her to simply look at a tile and then I air tapped myself. This helped her to understand the gaze gesture better.
  • There is a clicker that comes with the HoloLens device, that you can use instead of an “air tap” gesture.

It is pretty overwhelming at first, and for us it is easy to forget that it is pretty overwhelming at first I was happy to have the occasion to observe a few people trying it out for the first time and learn from this experience.

About Unity

It’s easy to confuse Unity (the 3D middleware) with Unity (the Inversion-of-control framework). The fact that both of these are used by Microsoft makes it even more confusing. In case of doubts, make sure to use Unity3D instead of just Unity. In this post, I will simply talk about Unity and this is NOT the IOC one ;)

When you install a new version of Unity3D, it might install side by side with older version. DO NOT GET CONFUSED! If you open a HoloLens Unity project with a version of Unity that is not suitable, you will get VERY confusing error messages. Make absolutely certain that you have the correct version of Unity open. Currently, the version is

Learning to code

The HoloLens academy is a Microsoft offering and has a growing number of tutorials. You can find all the information on their webpage.

Note that you don’t strictly need a device to get started. There is a free emulator which works with Visual Studio and lets you try your code out (see below). It’s a good way to wait for a device, as getting one can take a while due to overwhelming demand.

Learning to deploy

Before you even get to coding, it would be a good idea to learn to build and deploy a project. For instance you can follow these steps to download and build an existing project, the open source Galaxy Explorer:

  • Go to the Galaxy Explorer GitHub repo
  • Fork or download the code as Zip file, and extract it.
  • Start Unity3D
  • In the start dialog, press Open
  • Navigate to the folder GalaxyExplorer and press Select
  • Wait until Unity loads the project
  • Open the menu File / Build Settings
  • Make sure that the Windows Store platform is selected
  • Set SDK to Universal 10
  • Set UWP Build Type to D3D
  • Make sure that Build and Run is set to Local Machine.
  • Press the Build button
  • In the Build Windows Store folder selection menu, create a New Folder and name it App.
  • Make sure that the App folder is selected.
  • Press Select Folder.

This will start the build process. Note that this only creates the Visual Studio project with all necessary files. You will still need to open the created Solution file. Follow the steps:

  • Navigate to the App folder that you created earlier.
  • Open the GalaxyExplorer.sln file in Visual Studio 2015.
  • Make sure to select the following configuration: Release / x86 / Remote Machine.
  • If the Remote Connections dialog shows up, enter your device’s IP address in the Address field and make sure that “Universal (Unencrypted Protocol)” is the Authentication Mode selected. Then press Select to establish a connection to your device.
  • If this is the first time you deploy, you will need to enter a pin. The pin should be shown on your device, but if it is not, go to Settings, Update, For Developers, Pair. Then enter the pin into the Visual Studio dialog.
  • Finally, select Debug / Start without Debugging.

If everything is configured correctly, the application should start, and you can feel confident that deploying works for your future studies.

Don’t let errors like “System.Object doesn’t exist” startle you

If you run the examples of the HoloLens academy, you might encounter some weird errors when you generate the Visual Studio code from Unity, and then open the resulting SLN file in Visual Studio. You will see a LOT of errors because the Nuget packages have not been restored yet. Do not fret, you can just build, which will force the Nuget packages to get restored, and all the ugly errors should go away.

Match the file name and the class name

In C#, it is highly recommended that the file name and the class name match, and that you have only one class by file. But these are recommendations only. When coding in Unity however, I had some compilation errors because the file name didn’t match the class name. If you follow the early tutorials (for example the 101), they will ask you to create a new C# script file with a certain name (for example GazeGestureManager). Make sure to enter that name as you create the script. If you don’t do this, but instead you create a new script file with the default name, and then rename the script file to GazeGestureManager, Unity will refuse to do anything with this file. This is because the class name inside the file and the file name don’t match. So be careful to follow the steps exactly.

Sometimes Unity starts MonoDevelop instead of Visual Studio

In the course of the tutorials, you will sometimes have to edit some C# script files. Unity comes preinstalled with the MonoDevelop code editor, but using Visual Studio is more comfortable. To ensure that Visual Studio is launched, check the following settings: Edit / Preferences / External Tools / External Script Editor and make sure it is set to Visual Studio 2015.

Even with this setting properly configured, it can happen that Unity gets confused and starts MonoDevelop anyway. In my experience the file opens in Visual Studio anyway and you can just close MonoDevelop and move on.S

Starting the emulator

The emulator is a good way to get started when you don’t have a device with you. Strictly speaking, it’s not really faster to deploy to the emulator than to a real device, and of course the experience is not comparable, so you will probably prefer an actual device if you have a chance. You can also simulate gazing at objects, tapping them, walking around etc (see below).

The emulator can be started from Visual Studio directly (it is called HoloLens Emulator in the list of all the emulators and devices). If you don’t see it, you might have to install it first :)

I had some cases where the emulator refused to start, because my machine didn’t have enough memory (for example on a Surface Pro 3 with 8GB RAM, the emulator requires 2GB RAM and Windows decided there wasn’t enough left… annoying!). In that case, you should do the following:

  • Create a BAT file.
  • Enter the following command:
    “C:\Program Files (x86)\Microsoft XDE\10.0.11082.0\XDE.exe” /sku HDE /video 1268×720 /vhd “C:\Program Files (x86)\Windows Kits\10\Emulation\HoloLens\10.0.11082.1039\flash.vhd”
  • Save the BAT file and restart your machine
  • Before you do anything else, run the BAT file to start the emulator.
  • Then only start Visual Studio and deploy to the emulator, which is already running.

Issues when deploying to the device

In theory you can deploy to the device using a USB cable attached to your laptop, or using an internet connection and “Remote Machine” (not Remote Device like stated in some locations in the official documentation).

To deploy using your internet connection, you must make sure that the device is connected to the same network as your PC. I had a few issues when trying to deploy at the hotel, and I suspect that there was a firewall or something which was preventing me from successfully deploying. I am still investigating this. At home it seems to work fine.

Make sure the device is awake…

This may sound silly but it happened to me a few times… I remove the device, place it on the table upside down (which seems the safest way), and then after I make changes to the code I hit Ctrl-F5… and it fails. What happens is that after a few minutes, the device goes to sleep. Even if you wake it up quickly after you start deploying, it doesn’t seem to work (I think there is a delay for the device to reconnect to the network). So really, make sure that it is up and running before you hit Ctrl-F5!

FAT vs NTFS

I seem to have issues deploying to the device when the source code is on a FAT filesystem (SD card). If I copy the same source code to my main SDD (which is NTFS), then it works. I will investigate more. It’s not completely surprising because I had the same issue before with Windows 10 UWP applications, and the HoloLens is a Windows 10 device after all. It would be nice to be able to do this from an SD card though, because a typical HoloLens project is pretty large. I’ll update this if I find a solution.

Deploying over USB

Here too I am having issues when I am trying to deploy over USB. I am using the original HoloLens USB cable, and I am getting a cryptic error (Unexpected Error: -2145648626). Here too I will try to find more information and update this. So far I didn’t manage to solve it, but here are a few indications from other people in the community that might help you:

  • If your HoloLens device goes to sleep, and then you wake it up and try to deploy to it, it might fail. In this case, restart Visual Studio and try again.
  • Sometimes the USB connection keeps connecting and disconnecting continuously. In this case, shut the device off, then plug the USB cable in. This will wake the device up. Then you can try deploying again.

Overall, it seems that deploying over Wifi is more reliable than over USB. That’s a shame because USB works faster. My guess is that it’s a firmware issue and I hope it will get fixed some time in the future.

Conclusion

Well that’s a pretty long post… so I will stop here and publish. I am not stopping my investigations and learnings however, so you can probably expect more such notes in the future. I hope you enjoyed reading and that it is useful!

Happy holocoding!
Laurent

GalaSoft Laurent Bugnion
Laurent Bugnion (GalaSoft)
Share on Facebook
 

Slides and sample code for #XamarinEvolve and #Techorama

.NET, MVVM, Techorama, WPF, Xamarin, XAML
4 Comments

These past weeks have been busy with travel and speaking. After the wonderful time in San Francisco for Build 2016, I had a few precious weeks to prepare for Xamarin Evolve (Orlando, FL) and Techorama (Mechelen, Belgium). I just came back and here is the time to post the slides, sample code, and for Xamarin Evolve we even have a video of the talk!

Xamarin Evolve

2016-01-03_19-25-58

Evolve took place in tropical Orlando, and it was pretty nice to see sun, warm temperature and even some pool time on the day after the conference ended. I had a great time there. I talked about the DataBinding system in MVVM Light, which applies to Xamarin.iOS and Xamarin.Android. This critical part of all MVVM applications is there to ensure the connection between the ViewModel layer (typically in a portable class library and shared across all platforms) and the View layer. In Xamarin.Forms and on Windows, we don’t need an external databinding framework because we already have these (this is what you use when you write Text=”{Binding MyProperty}” in XAML). But in Android and iOS, there is no such concept, and this is where the MVVM Light platform-specific extensions come handy.

Here is the abstract (which was modified by Xamarin themselves… I normally don’t really use this marketing-y tone ;):

An In-Depth Study of the MVVM Light Databinding System

Living in the dark ages and still wiring up properties manually between your user interface and data source? Databinding is a technique to automatically synchronize a user interface with it’s data source and can vastly simplify how your app displays and interacts with data. While databinding is available out of the box for Xamarin.Forms and Windows applications, additional components are needed in Xamarin.Android and Xamarin.iOS. In this session, learn how to leverage databinding in your cross-platform applications as you master MVVM Light databinding and the MVVM pattern.

I created a page for this presentation on my website. There you will find the slides, video recording as well as the sample code.

Note: At the moment, some of the Xamarin Evolve videos are not working properly. Xamarin is informed. Thanks for your patience.

Techorama

techorama

Techorama is one of my favorite conferences, created by the community for the community after the cancellation of TechDays Belgium. Gill, Pieter and Kevin created a hell of a show, which grew to host more than 1000 visitors these days. The venue is awesome too, it is a movie theater and we get to project our slides and code on a huge screen. This year there were quite a few renowned speakers from the US and the whole world in fact. Even though I got to spend only one night at home after coming back from Orlando before flying again, which was quite tiring and a bit stressful, I was really looking forward to go to Mechelen. I hope you enjoyed my session there about WPF. It was a fun session where I talked about the differences between WPF and the Windows 10 Universal platform, about new development in WPF (especially tools such as the Visual Tree, the Live Property Explorer, and XAML Edit and Continue), about the Windows Bridge “Project Centennial” which takes a classic Windows app and “packs” it to transform it into a Universal application. We finished with an exciting demo of a new feature shown at Xamarin Evolve the week before: the Xamarin Workbooks, which allow you to create a rich document (using Markdown) with titles, subtitles, images etc, and allows including snippets of C# that will be executed by the Xamarin Inspector tool. Because the tool supports Android, iOS and WPF, it was a great find and it fitted well in my session which aimed to show that WPF is still very current and state of the art. So I happily changed my presentation to include it in the demos.

Windows Presentation Foundation (WPF) 4.6

Windows Presentation Foundation is what people are using to build real applications for the enterprise, the industry, the workplace, and for every situation where Windows 10 Universal is not quite ready yet. Far from being dead, WPF is 10 years old this year, alive and kicking, and gives Universal Applications a run for their money! In this session, Laurent Bugnion, a Microsoft Windows Developer MVP and Microsoft Regional Director, WPF expert since 2006, will show you what is new in Windows Presentation Foundation, where it is going in the future, and what you can achieve with WPF that Universal Application developers can only dream of.

The presentation’s page is on my website, and will give you access to the slides and the demo source code. Make sure to check the last couple of slides for more resources!

One more thing

I recently discovered (not sure how I missed that) that my session about Windows 10 UWP at the Future Decoded conference in London last year had been recorded. I added the video to the presentation’s page. So in case you want to know how to adapt your UWP app on multiple platforms, this is where you can go!

Happy coding!
Laurent

GalaSoft Laurent Bugnion
Laurent Bugnion (GalaSoft)