Eric Schmidt shares with us a great project, using some of our favorite tools. How can you go wrong when LEGOs are in the mix!
Some of our related past posts;
LEGO, IR and Netduino?
LEGO Mindstorms EV3 API, Windows 8, Windows Phone 8, Desktop and you!
LEGONXT, Windows 8 and Bluetooth: a How-to
Below I've pulled enough from the original post to hopefully wet your appetite. Make sure you click through for the full details, code, tips and more...
Talking to robots (or other devices) using Bluetooth from a Windows Runtime app
Microsoft recently sponsored a team to complete in the “Build for Good” LEGO® MINDSTORMS® Robo Competition, a team that represented the Xbox brand. I had the distinct honor of working on this team of developers, testers, and one business manager to create a pretty awesome exhibit.
...
Each of the robots executed code written using the LEGO® MINDSTORMS® EV3 Software. The code listened for messages from the app and then reacted in a specific way.
From the app’s perspective, it connected three separate and distinct Bluetooth-enabled devices to which it sent various messages. The difficulty lay in establishing the Bluetooth connection with each robot and then sending and receiving messages. To accomplish this, the app used several of the Windows Runtime APIs designed for interacting with devices – including the Windows.Devices.Bluetooth.Rfcomm namespace introduced in Windows 8.1 and Windows Phone 8.1.
In this blog post, I’ll provide step-by-step instructions for how you can talk to a MINDSTORMS robot from a Windows Runtime app. I’ll show you how to build a very simple app that sends and receives messages from an EV3 robot. I’ll focus on the specific Windows Runtime APIs we used to connect to the robots, so I’ll be glossing over a lot of the basics about building a Windows Runtime app using HTML, CSS, and JavaScript. If you need to learn more about the fundamentals, see Create your first Windows Store app using JavaScript on MSDN.
On the other hand, keep in mind that this blog post provides more explicit instructions around managing project files in a Visual Studio solution than we normally provide in our posts. I know that many of the loyal readers of our blog are professional developers who are deeply familiar with the basics of Visual Studio. For you, I encourage you to skip ahead to some of the details about connecting to Bluetooth devices from a Windows Runtime app.
Get the hardware and software
Before I get any further, I assume that you’ve already got the required devices and software for this process. I’m using Visual Studio Express 2013 for Windows, running on Windows 8.1, to write my code. I’m connecting to a MINDSTORMS EV3 Intelligent Brick (the ‘brain’ of the robot) that has been updated with the 1.03H firmware version of the EV3 operating system. (It’s easiest to update the firmware from the MINDSTORMS EV3 Software.)
IMPORTANT: Your EV3 Intelligent Brick needs to be paired and connected at least once to your Windows PC before they will be able to receive and send messages to each other. This can be a tricky process:
On the settings page for the EV3 Intelligent Brick, make sure that Bluetooth is turned on. When you select the Bluetooth option from the screen, ensure that Visibility and Bluetooth are selected. Also, make sure that iPhone/iPad/iPod are not selected.
Pair the EV3 Intelligent Brick and your Windows PC.
From the Start screen, type Devices and Printers, and then select Devices and Printers. In the Control Panel, on the Devices and Printers page, click Add a device. In the list, find the name of your EV3 Intelligent Brick (if you haven’t changed it in the EV3 Software, the name will be ‘EV3’) and then follow the on-screen instructions.
t this point, you’ll be prompted on the EV3 Intelligent Brick to acknowledge the connection with your PC and to accept a passcode. On the PC, you’ll be asked to supply this passcode.
Make sure you remember your brick’s name!This becomes important later on when you try to connect to the brick.
Now you can connect to the brick from your PC. You can do this with a single click from the EV3 Software.
Open Devices and Printers again. If your brick and PC are paired, the brick will show up in the “Unspecified” category under its own name. (You may see it twice if you connected to the brick previously by USB. Select the entry that has the details ‘Bluetooth Peripheral Device’.) Right-click the device and select Properties. In the device Properties window, go to the Services tab and ensure that Serial Port (SPP) ‘Serial Port’ is selected. Apply any changes and close the settings dialog box.
Make sure that the EV3 Software is not connected to your brick by Bluetooth when you attempt to connect to the brick from your app. This can block the Window Runtime app from connecting to the EV3 brick.
For more info about how to troubleshoot Bluetooth connectivity problems, download the EV3 Bluetooth guide.
Write the EV3 listening and receiving code
To demonstrate sending and receiving messages, the brick runs a simple listening program written using the MINDSTORMS EV3 Software. A picture of the program running on the robot is shown below.
...
Create the user interface for the Windows Runtime app
Before I start writing more code, I’ll take a quick inventory of the features I need to build for this simple Windows Runtime app. The app has to be able to do a few things programmatically:
It needs to establish a connection to the brick. (For Team Xbox, the code needed to be extensible so that the app could connect to three bricks.)
It needs to convert a user-generated message into the message protocol that the EV3 brick can understand.
It needs to listen for and decode incoming messages from the brick.
...
Write the code to interact with the robot
Now I can begin to write the JavaScript code that talks to the EV3 robot. I’ll start by adding a new JavaScript file called “Robot.js” to the solution. (In Solution Explorer, right click the ‘js’ folder, choose Add, New JavaScript File. In the Add New Item dialog box, name the file “Robot.js” and click Add.) Before I continue, I’ll add a reference to the new JavaScript file to my default.html file.
...
Test the app
Before running and testing the Windows Runtime app, you’ll want to start the program on the EV3 brick. I highly recommend that you download the code to the brick, close all instances of the EV3 Software on your PC, and then run the program directly on the brick.
With the EV3 program running, run the Windows Store app code from Visual Studio Express (press F5). The user interface for the app should look similar to this:
...
Other resources
You might be wondering if there’s an easier way to accomplish all this. Well, the good news is that there are a couple of CodePlex projects designed to help you communicate with the LEGO MINDSTORMS EV3 brick over Bluetooth:
LEGO MINDSTORMS EV3 API. This project includes a Windows Runtime component (.winmd file) that can be added to a Windows Runtime project (Windows, Windows Phone, and universal Windows apps), a library for Windows Phone apps built with Silverlight, and a library that can be used in a Win32 application. This library is very powerful and provides a significant range of functionality. You can use it to read sensor values, run motors, even play sounds or read files on the EV3 brick. However, it won’t allow you to connect to multiple EV3 bricks from a Windows Runtime app.
EV3Messenger. This Win32 project (a simple Windows Forms application) allows you to connect directly to a specific EV3 brick by specifying the serial port assigned to the brick. If you have more than one brick connected to your PC, you need only specify different serial ports to connect to each / all of them. One of the best parts of this project is that the message protocol for the EV3 brick is broken down into a very readable table. Also, the project contains some C# code that can be reused to encode and decode messages from a Windows Runtime app (with some slight modifications). The only drawback is that the connection code for this app uses the System.IO.Ports namespace, which you can’t use in a Windows Runtime app or in a Windows Runtime component.
Unfortunately, neither of these libraries supply both of two key features that Team Xbox needed. We wanted to have the form factor of a Windows Runtime app, but we needed to send messages to and receive messages from multiple (in our case, three) EV3 bricks. We produced some code that was heavily influenced by the two CodePlex projects mentioned earlier, but tailored for our needs. The “secret sauce” lies in the Robot.connectToRobotAsync function. As I mentioned previously, it can be easily modified to accept a unique name for an EV3 brick and the same function can be called repeatedly at runtime (allowing us to connect to more than one block during one app session).
All in all, Team Xbox had a blast building Lego robots and writing the code to talk to them. Our goal was to inspire more people – particularly young adults – to engage and explore the world of science, technology, engineering, and mathematics. I think we achieved our goal; furthermore, I hope that this blog post will help you build a Windows Runtime app that talks to an EV3 robot … or whatever other Bluetooth-enabled device you’re working with.
Thank you for reading and keep on developing apps!
[Click through for the source and all...]
Follow @CH9
Follow @Coding4Fun
Follow @gduncan411