Finished Design, Pics, and Video!

Okay, enough content from our paper - if you want to actually see the design in action, this post is for you.

Here's a few pics of us actually using this thing:

Paul demoing in Nazi Zombies on Call of Duty: Black Ops

Henry owning in Spec Ops mode on Call of Duty: Modern Warfare 2

And finally, the video Roop made for us:

This shows that the design not only works, but even more amazingly, is actually quite fun to play!  I was able to make it through round 5 on Kino der Toten, and finish a few levels of Spec Ops in Modern Warfare 2.  The ultimate goal?  Actually get some kills online in one of these games.

That's all for now - I hope to have some time to play around with this project (or possibly even recreate it) over the summer, and if I make any improvements, I will be sure to post updated content.

Paul

Receiver (Controller) Design

Hardware Design

A schematic describing the design of the circuit of the receiver is shown in the picture below.

Schematic showing the wiring of the receiver.  The DAC modules on the right side are implicitly connected to the mbed’s 3.3V and ground pins.  Note that the interface for one trigger and one button is shown: the pins used are listed, but the external interface (relay and soldering to button contacts) needs to be replicated for each pin.

In total, the receiver needs to implement four analog outputs (one for each of the original joystick axes).  The mbed has one built-in digital output on pin 18, so that will be used to implement one.  Since this DAC has a 3.3V range, software will ensure that the most significant bit is 0, keeping the range of analog values in the lower half, or below about 1.65V, which is the proper range for the analog joystick.  The mbed also features two I2C buses, and on each of these we put a MCP4725 I2C DAC, which provided two additional analog outputs.  Unfortunately, because the least significant address pin of the MCP4725 was hard-coded (wired) to 0 by the breakout board, we could not put multiple DAC board slaves on a single I2C bus; therefore, to implement the final analog output, we constructed a 4-bit resistor ladder DAC, as shown above.  Again, in order to prevent the voltage from going over 1.65V, the most significant bit was hard-coded to 0 (connected to ground).  This DAC only gives three bits of precision (for a total of 8 states of that joystick), however, because it is used for the strafe axis (arguably the one with the least need for precision), the user usually will not notice this limitation.

To actually connect these software-controllable analog values to the Xbox 360 controller, both analog joysticks were removed (desoldered) from the controller PCB and in place of the middle pin contacts (recall that an analog joystick is simply a potentiometer) we soldered a wire leading to one of the four analog outputs above.

To implement software-controllable buttons without the need to manipulate and implement the matrix layout of the Xbox 360 controller in software on the mbed, we elected to implement each button as a relay.  A relay is an electrically-controlled switch in which driving a current through an electromagnet closes a mechanical switch.  These relays are controlled using GPIO pins on the mbed (the current draw is immeasurably small).  On the Xbox 360 controller, we scraped away each button contact with a blade to reach the copper underneath, and soldered a wire to each contact (two per button).  The wires for each button were then connected across the terminals of a relay that is controlled by the mbed.  Therefore, if we want to make a connection that presses the ‘A’ button from the mbed, all we need to do is turn the corresponding GPIO pin to high, which will close the switch inside the relay, and convince the controller that the ‘A’ button is being pressed.  For brevity and clarity of important details, only one button’s implementation is shown above.  However, it is important to note that one relay, one GPIO pin, and two wire leads to the Xbox 360 controller are necessary for each button that you wish to control.

Note that while we implement the two triggers as binary (on or off) switches, because these are analog triggers on the Xbox 360 controller, they each have their own dedicated line, and we can merely drive this line to the desired voltage.  Because the mbed uses 3.3V levels, and the trigger is designed to vary between 1.65V (unpressed) and 0.3V (pressed, will work properly if given 0V as well), we use the voltage divider shown above on the trigger output pins (p15 and p16 for RT and LT, respectively), which gives a value of 1.65V to the controller contact when the output pin is high (assuming a low current draw from the output) and 0V when the input is low.

Finally, the receiving mbed is connected to the data output pin from an XBee wireless module through pin 14 (serial RX) as shown above.

Software Design

The C++ source for this project can be found in its entirety at this link.  For a detailed description of how the software works, please consult the paper (also available at the link above), or the comments in the code (most of it is fairly straightforward to follow, except possibly the "packetization" code).

Finished Product

  

A picture of the completed receiver can be seen in the first picture below.  This picture highlights the relays, DACs, mbed, and XBee wireless module.  The second gives a close-up of the Xbox 360 controller PCB.

Completed Receiver module (shown running and connected to Xbox 360).  The black DIP packages are relays, the red PCBs are the I2C DAC chips, the device with LEDs is the mbed, and the resistors are used to implement the custom DAC.

Close-up on the de-soldered Xbox 360 controller.  Button contacts are on the reverse - this shows leads for the strafe axis as well as the left and right triggers.

Transmitter (Gun) Design

The transmitter design is essentially a plastic toy gun that has been physically modified to feature the Xbox 360 controller interface.  A gyroscope is used to sense the angular velocity at which the gun is moving.  When the gun is moved or rotated, the gyroscope will sense rotation in two separate axes, and cause the voltage on one of two pins to deviate from the reference voltage accordingly.  This deviation from the reference voltage is used to generate a digital value that corresponds approximately to the deflection of an analog stick that would cause a similar effect in-game.

The remainder of the Xbox 360 controller interface (consisting of two triggers, 10 buttons, and a second analog joystick) is implemented in the same manner as the controller; however, these features are spread around the gun for ease of use.  Unlike the Xbox 360 triggers, which are analog (see section II.B), the triggers for this design are binary.  As a result, while the 360 controller features 10 buttons, we will have 12 (counting the two triggers).  For the intended games (first-person shooters), this is acceptable, as there are only two in-game states for the trigger: shoot and don’t shoot.

Hardware Design

A hardware schematic describing the design of the circuitry of the transmitter (gun) is shown in the following picture.

Schematic showing the wiring of the internals of the gun transmitter.  Note that all 12 buttons are connected to a common power, and due to the nature of software buses of GPIO pins on the mbed, can be connected to arbitrary pins.  Power and ground connections to devices on the right side are omitted for clarity.

The mbed shown above makes use of five analog inputs.  Two (p18 and p17) are used for the two axes of the movement joystick.  The other three are used to interface with the gyroscope.  The gyroscope used (the LPR530AL dual-axis gyro) has 4x amplified outputs for each of the X and Y-axes, so we connected those to ADC pins 19 and 20.  As previously discussed, because we wish to calculate the difference between these readings and the specified reference voltage, we also use ADC pin 15 to capture the reference voltage.

The buttons on the gun are implemented simply using GPIO.  Because the mbed will automatically pull down the line for a high-asserting button, there is no need to add a pull-down network to these buttons: they only need to make a connection to 3.3V from the mbed.  Inside the gun, these buttons are almost all in the middle compartment on a single soldered circuit board.  These share a common connection to 3.3V (to one side of each button) and on the other side of each is a wire leading back to the mbed.

Finally, the XBee wireless module is connected to the serial output (TX) pin on pin 28.  Additionally, to prevent the user from having to maneuver into uncomfortable positions, we added a button (labeled “Gyro disable” above) to essentially zero out the gyro in software: the result is that when the button is pressed, moving the gun will not affect the in-game character.

Software Design

The C++ source for this project can be found in its entirety at this link.  For a detailed description of how the software works, please consult the paper (also available at the link above), or the comments in the code (most of it is fairly straightforward to follow, except possibly the "packetization" code).

Finished Product

The entirety of the circuitry shown above was packed in to the three internal hollow compartments of a plastic toy gun provided from last year’s ESE 350 projects.  The completed gun (without display, showing PVC mounting rails) can be seen in the following picture:

Completed, closed gun (transmitter) without display.

The regular Xbox 360 controller buttons (A, B, X, Y, Start, Back) can be seen on an exposed circuit board on the front of the center compartment of the gun (above the trigger).  The trigger can be seen immediately below that.  The gyroscope is located immediately above the trigger, held in place (to maintain the proper organization) by tape.  The left analog joystick (a genuine Xbox 360 joystick that was removed from the wired controller that we ripped apart) can be seen on the bottom left.  The mbed microcontroller itself is in the compartment right above the joystick, which holds a small breadboard with wires leading from it to the rest of the components.  A picture of the contents of the front compartment (including the mbed) can be seen in the following picture.

Insides of the front compartment, containing mbed and wiring to buttons, gyroscope, and XBee.

Finally, the XBee wireless antenna is located in the very back of the gun (the stock).  The finished product, with screen attached, can be seen here:

Completed gun with screen attached and mounted.  Game shown is Call of Duty: Modern Warfare 2.

The button layout is consistent with the Xbox 360 controller.  In fact, the button layout was measured to exactly match that of the controller (we actually have a cut up faceplate that we can put on, however, we ran out of time).

Project Introduction and Overview

Background Info

In today’s video game market, the current trend for hardware and software engineers alike is motion capture.  Following the launch of the Nintendo Wii in 2006, which brought with it an unprecedented level of motion sensing in a retail console through the Wii Remote controllers, Sony and Microsoft have been quick to respond with the PlayStation Move and Xbox Kinect, respectively.  Surprisingly, despite these recent trends, vendors have not made substantial progress towards expanding the motion sensing market towards the best-selling titles over the last five years: specifically, first-person shooters from the Halo and Call of Duty franchises.  FPS 360 fills this void by demonstrating how to create an enjoyable and immersive platform for motion capture in the first-person shooter (FPS) genre.

The focus of this project was the use of a plastic toy gun as the primary form of user input to the Xbox 360.  The user can aim the gun and a dual-axis gyroscope senses the changes in the guns orientation.  An mbed microcontroller placed inside the gun realizes the analog gyroscope data, along with an analog joystick and buttons for the rest of the Xbox 360 controller interface (see picture below).  Using an XBee point-to-point wireless chip, the mbed in the gun builds packets of controller data and sends those over a serial interface to the XBee.  Another receiving mbed (connected to a second XBee) receives and decodes each packet.  It then uses four DACs (one internal, two I2C, and one custom) to generate realistic analog values for the two joysticks on the Xbox 360 controller.  Each button contact on the Xbox 360 controller is also attached through a relay that is controlled by GPIO on the mbed: whenever a button is pressed, that relay will be closed, otherwise, it will be open.  Finally, in order to complete the design, a display was mounted on the top of the gun and the software was calibrated to allow the user to experience the feeling of moving and looking around in the virtual game environment.

Design Overview

An overview of the hardware design of the system can be seen in the following picture.

Block diagram giving an overview of the design

This project can be divided into two largely distinct functional units: the transmitter, which is the right mbed in the picture above (and everything connected to it), and the receiver, which is the left mbed.  The hardware and software design for each of these will be discussed separately in sections III and IV, respectively.  Functionally, the two blocks can be treated as distinct.  The transmitter essentially converts the user input from the gun to a packet format that is fundamentally the same as the standard Human Interface Device (HID) protocol, while the receiver will use this HID-like data to configure relays and DACs connected to an Xbox 360 controller to control the character in game.