Building an App for Windows 10 IOT using Angular 2

Window 10 Raspberry Pi

I’m late to the Raspberry Pi fun.  It was on my “to learn” list for the last year, and as usual work gets busy, the kids get busy, and life in general just doesn’t provide enough free time to play with technology. 

When the kids were in middle and elementary school we had time to play experiment with Lego robotics. I even had the opportunity to co-teach Lego NXT programming in my daughter’s 8th grade class.   So a couple weeks back when my youngest was browsing through Pintrest and discovered an article on how to build a “magic mirror” with a Raspberry Pi, her excitement became too infectious to ignore. It took me all of two-minutes to place the order on Amazon and select the next day shipping option.

The Pi arrived on a quiet weekend morning and we dug in fast. We ripped into the box and put it together without issue. Getting the Windows 10 IOT image installed on the micro SD card took a bit longer, but was a simple process with the Windows 10 IOT Core Dashboard application.  We followed the instructions on the Windows IOT Dev Center and didn’t really run into many issues or problems with the documentation.  Within about 10 minutes we had our Pi booted up and running over an HDMI cable to our living room TV. There was a minor scare as the Window 10 boot seemed to hang for about three minutes during setup, after which we both sighed in relief as we saw the dashboard app start.

Getting the Right Display Resolution

The first thing we noticed was the awful resolution. It was very low at 640×480 and the image was over-scanning quite a bit making the UI hard to read. To fix this, we quickly did some searching and read up on the Raspberry Pi’s config.txt file at the root of our SD card.

The config.txt file controls various basic settings on the Raspberry Pi but the most important for us was the HDMI mode.  Read more about the config.txt file here. We ended up making adjustments to the hdmi_group and overscan settings to get the picture to show up correctly on our HDTV. To edit the config.txt file, just insert the SD card back into your laptop and edit the file directly in a text editor. The following settings are what worked for us:


gpu_mem=64  
framebuffer_ignore_alpha=1 
framebuffer_swap=1    
disable_overscan=0  # Disable overscan 1
overscan_left=0
overscan_right=0
overscan_top=0
overscan_bottom=0
hdmi_group=1  # CEA=1 or 2 for DMT
hdmi_drive=2  # 2 enables sound over HDMI

I found that setting the hdmi_group to 1, which sets it to CEA mode, did the trick. Other values that I tested out included the gpu_mem, which I increased to 64Mb and the various overscan settings to adjust the overscan value on the display. Once this was all dialed in, I popped the SD card back into the Pi and rebooted.  Now I had full 1920×1080 resolution with no overscan issues. If I were to move plug the Pi into a computer monitor later I would likely have to re-adjust this config file back to using hdmi_group=2 for VESA Display Mode.

Creating a Windows JavaScript Project in Visual Studio

The most excruciating part of our first day with Windows 10 IOT was getting all of the SDK’s installed.  I already had Visual Studio 2015 installed on my laptop, but recently I have been using VS Code instead so it’s bigger brother was not up-to-date.  I unfortunately spent about 3 hours downloading and installing the long list of required SDKs (with my poor daughter completely bored out of her mind next to me waiting) to see a simple text “Hello World” show on the TV screen.  This was the most disappointing part of the experience.  I had high hopes of getting a basic IOT app up and running and then switching over to teaching some basic Javascript and setting her free to play. It would be nice to see this as simple as using just VS Code or Sublime, a couple “npm install” modules and then simple xcopy deploy to the device. Hopefully the IOT team will spend a few sprints optimizing this workflow to its simplest form.

If you are starting from scratch like I did,  head over to the Getting Started page and follow all 6 steps of the “set up your PC” section very closely. Don’t miss a step!  Grab a beverage of choice and immediately lower your expectations on how much you will actually get done today. It’s also a good time to read ahead on the Angular2 Getting Started tutorial.

After installing the requirements (and have a lot less storage then you began with on your laptop), fire up Visual Studio 2015 and hit the New Project menu (Ctrl-Shift-N.)
Under the Templates, select the Javascript Blank App for Universal Windows.

image

 

At this point you can hit F5 and see the application launch locally. It will load the default.html file and show you a screen with the default text- “Content goes here”. To make ourselves feel good – we edited the default file to say “Hello World” instead. After hours of struggling, we were both very ready to see this actually show up on the TV screen in front of us.

image

Configuring Remote Deployment

Now that we had a good base project, we were ready to deploy it.  Luckily this part was pretty simple for a change. In Visual Studio 2015, we just went to the Project->Properties menu (or we could have also right clicked on the Solution in the Solution Explorer window and selected Properties.) In this dialog, you can select the Debugging page and then select Remote Debugging in the dropdown.

image

 

On the Debugging Properties page, you can then set the the machine name to your Raspberry Pi’s machine name (if you changed yours at installation make sure to use the correct name or IP Address.)

Change the Require Authentication mode to Universal or you will not be able to connect to the device.

image

 

Now, back in the editor, you only need to select the dropdown on the menu bar to switch between Local Machine or Remote Machine debugging. Selecting Remote Machine and pressing F5 will build, package up the app and deploy it to the Raspberry Pi device.  In addition, it will connect the debugger and set you up for remote script debugging.

image

 

I turned to my daughter and startled her away from her phone with a loud shout, “it works!”  She returned my enthusiasm with slow clapping. I had already lost my daughter to Twitter or Tumblr at this point. Oh well, I’ll pull her back into this another day once I know what I’m doing.

Setting up Angular 2

Determined to kill even more time on this weekend morning, I decided for some reason to see if I could get the latest Angular 2 beta running on the Pi. Honestly, I don’t know what compelled me to do this and I should have left well enough alone. I think my reasoning was the complete lack of documentation on the subject. I could have comfortably stayed with WinJS and built an interface for the “Magic Mirror” project that had gotten my daughter so excited in the first place. But, since I had lost her for the rest of the day, I decided it was a good opportunity to take myself down the rabbit hole.

The blank project in my Visual Studio editor was uncomfortable to me. I preferred working in VS Code with Node.js and npm modules. It was where I had been working lately with Node.js and some other JavaScript projects. VS 2015 just seems too “heavy” to me lately. So I switched over to the lightweight Visual Studio Code editor.

To do this, I fired up my PowerShell console and navigated to my project directory. To launch VS Code, I just entered “code .” from the command line.  My goal was to use NPM to install the Angular 2 module and it’s dependencies into my project and follow along with the latest getting started tutorial on Angular.io

To get started I initialized a new package.json file by typing:

npm init

After accepting the default settings I was now ready to install the modules I needed. To make sure that TypeScript was installed globally on  my machine I entered:
npm install –g typescript

In the Angular 2 Quickstart there is a section that outlines what is needed in the package.json file to run Angular 2. I made my life easier by just copying the package.json file settings and dependencies.  The important parts of this include the dependency versions for es-6shim, reflect-metadata, rxjs, zone.js, and the developer dependencies.  Arguably I won’t really need the lite-server or concurrently dependencies, as I plan to use features of the Universal Windows runtime eventually which will not be available to me in just the browser.  I kept it for now, as it was handy to test out and see if the project would load in the browser by typing only  npm run start.

{
  "name": "angular2-quickstart",
  "version": "1.0.0",
  "scripts": {
    "tsc": "tsc",
    "tsc:w": "tsc -w",
    "lite": "lite-server",
    "start": "concurrent \"npm run tsc:w\" \"npm run lite\" "
  },
  "license": "ISC",
  "dependencies": {
    "angular2": "2.0.0-beta.3",
    "systemjs": "0.19.6",
    "es6-promise": "^3.0.2",
    "es6-shim": "^0.33.3",
    "reflect-metadata": "0.1.2",
    "rxjs": "5.0.0-beta.0",
    "zone.js": "0.5.11"
  },
  "devDependencies": {
    "concurrently": "^1.0.0",
    "lite-server": "^2.0.1",
    "typescript": "^1.7.5"
  }
}

After copying and pasting just the “scripts”, “dependencies”, and “devDependencies” from the sample package.json above, I entered:

npm install

This installed all of the required dependencies for Angular 2 beta (latest at the time of writing this blog post).   I recommend at this point to complete the rest of the Getting Started tutorial for Angular 2.  I won’t go into any details on what is happening in the tutorial here, but you may want to pay close attention to how you structure your files. In the tutorial the default folder structure creates an “app” folder where the TS files are stored, and an index.html file.

I went into my project and changed the “default.html” file to be “index.html” to match the tutorial.  In doing so, I also needed to go into .jsproj file and update the name of the content item to  <Content Include=”index.html” /> .  I also made certain that the package.json file’s main file was pointing to “index.html”.

Adding Angular Scripts to the Project file

Since your Windows Universal application is a self contained project when deployed to the device, you need to package all of the required scripts into the application.  To do this, you just need to make certain that all of your script files required for Angular 2 are listed in the .jsproj file for your project.   In VS Code, simply open the .jsproj file and add the following script paths to your project to include the relevant Angular 2 sample’s required files.  As you add more Angular 2 features to your project, you will have to maintain this list of packaged resources.   There may be a way to link out to the minimized versions of these on a trusted CDN root in the application, but I didn’t get around to figuring that out yet (if you figure it out, comment below please and I’ll update this post.)


    <Content Include="node_modules\angular2\bundles\angular2-polyfills.js" />
    <Content Include="node_modules\angular2\bundles\angular2-polyfills.min.js" />
    <Content Include="node_modules\angular2\bundles\angular2.dev.js" />
    <Content Include="node_modules\angular2\bundles\angular2.js" />
    <Content Include="node_modules\angular2\bundles\angular2.min.js" />
    <Content Include="node_modules\angular2\bundles\http.dev.js" />
    <Content Include="node_modules\angular2\bundles\http.js" />
    <Content Include="node_modules\angular2\bundles\http.min.js" />
    <Content Include="node_modules\angular2\bundles\router.dev.js" />
    <Content Include="node_modules\angular2\bundles\router.min.js" />
    <Content Include="node_modules\angular2\bundles\testing.dev.js" />
    <Content Include="node_modules\angular2\bundles\upgrade.dev.js" />
    <Content Include="node_modules\angular2\core.js" />
    <Content Include="node_modules\angular2\platform\browser.js" />
    <Content Include="node_modules\angular2\platform\browser_static.js" />
    <Content Include="node_modules\angular2\platform\common_dom.js" />
    <Content Include="node_modules\angular2\platform\server.js" />
    <Content Include="node_modules\angular2\platform\testing\browser.js" />
    <Content Include="node_modules\angular2\platform\testing\browser_static.js" />
    <Content Include="node_modules\angular2\platform\testing\server.js" />
    <Content Include="node_modules\angular2\platform\worker_app.js" />
    <Content Include="node_modules\angular2\platform\worker_render.js" />
    <Content Include="node_modules\es6-shim\es6-shim.js" />
    <Content Include="node_modules\es6-shim\es6-shim.min.js" />
    <Content Include="node_modules\rxjs\bundles\Rx.js" />
    <Content Include="node_modules\rxjs\bundles\Rx.min.js" />
    <Content Include="node_modules\systemjs\dist\system-polyfills.js" />
    <Content Include="node_modules\systemjs\dist\system-polyfills.src.js" />
    <Content Include="node_modules\systemjs\dist\system.js" />
    <Content Include="node_modules\systemjs\dist\system.src.js" />

Modify the Package to Launch in WebView or Fail

In order to get Angular 2 scripts to run properly in the application, I discovered that I needed to launch the app in WebView.  There is a good talk from //Build about the WebView and how it is used here if you want the details.

To make the index.html file the Start page for your application and launch in WebView, you have to update the package.appxmanifest as shown below.  Setting the StartPage property to “ms-appx-web///index.html”, and also adding a uap:AppllicationContentUriRules section to allowForWebOnly.

image

This is the setting I used (for easy copy/paste). As shown above, it is stored under the <Application> element in the schema.


  <Applications>
    <Application Id="App" StartPage="ms-appx-web:///index.html">
      <uap:VisualElements DisplayName="MagicMirror" Description="MagicMirror" BackgroundColor="transparent" Square150x150Logo="images\Square150x150Logo.png" Square44x44Logo="images\Square44x44Logo.png">
        <uap:DefaultTile Wide310x150Logo="images\Wide310x150Logo.png">
        </uap:DefaultTile>
        <uap:SplashScreen Image="images\splashscreen.png" />
      </uap:VisualElements>
      <uap:ApplicationContentUriRules>
        <uap:Rule Match="ms-appx-web://" Type="include" WindowsRuntimeAccess="allowForWebOnly" />
      </uap:ApplicationContentUriRules>
    </Application>
  </Applications>

Deploy your Angular 2 App to Windows 10 IOT

That’s it, you are finally ready  (yeah exhausting, I know) to deploy your very basic Angular 2 beta project to your Raspberry Pi device running Windows IOT. Go back to Visual Studio 2015 and reload the project.

It’s best to Rebuild the solution now and test it locally to make sure that everything worked out ok from the steps above, and that you missed nothing.  Set the Debug mode back to Local Machine from the menu dropdown and press F5 to build, deploy, and run locally. After the Splash screen loads, you should see the basic Angular 2 Getting Started project load successfully.   If for any reason you do not see it working at this point, carefully rewind and check all of your steps.  Make absolutely certain that you have all of the required Scripts in the .jsproj file as indicated above.  You can also check the debugger console window for any errors or scripts that failed to load.   Another step to double check would be setting the WebView to load by using ms-appx-web in the package.appxmanifest file.

Once you are successfully able to run the Angular application locally, change the debug menu dropdown to Remote Machine and hit F5 again.  At this point, Visual Studio will connect to the Raspberry Pi, deploy the application and launch it.

If everything worked locally, you should have no issue seeing the project running on Windows 10 IOT at this time.   Congratulations, and good luck with your Angular 2 projects on Windows 10 IOT.

Next up on the learning list for my daughter and I is to begin creating a UX layout and writing some Angular 2 Components for our “Magic Mirror” project.  We’re going to explore using some various UX frameworks including Bootstrap and Material Design.

We have lots more to learn together and I’ll post progress  and the finished app code in future blogs. On our list is to learn more about the Universal APIs in Windows IOT to support the camera, speech recognition, and possibly add in some Face Recognition from my friends over in the Oxford team.   Of course, I’ll also find a way to bring in my own Azure Media Services API as well for live and on-demand video playback – but that’s later.

The Project Oxford team has a bunch of interesting (and free) APIs for your IOT applications like Speech to Text conversion, Speech Intent Recognition, Computer Vision, Face Detection and Emotion APIs. My team is  also working closely with Project Oxford to deliver new Video API’s powered by the Azure Media Services platform.  I’ll be posting more details about those Video API’s in our platform soon.

Leave a Reply

Your email address will not be published. Required fields are marked *