Project
Repurpose a piece of commodity handheld gaming hardware for audio and video performance use by developing an app for Nintendo’s DS devices.
Challenges
What is possible to create within the limits of the graphics and sprite capabilities of the hardware?
Is it possible to install a test app onto a real Nintendo DS in order to fully test and prototype the design concept?
Role
I was the sole designer and technologist working on this project. I researched the development platform and how to get it working, how to compile apps to the DS, the features and specifications of the hardware, and designed an interface for switching screens and elements to interact with on each screen.
Process
The idea for this project came about through a combination of events. I became aware of the release of an expensive multitouch hardware controller, generally used for musical performance control and then learned about the creation of a library for Nintendo DS homebrew development that enabled wifi communication. The thought of being able to use the cheap handheld gaming device that I already owned as a visuals performance controller was compelling.
As a concept, I liked the initial hardware controller device and so designed this as a little brother that could go alongside. I changed the design and graphics to fit within the hardware limitations of the DS, as well as the limited screen resolution.
My process for laying out screens was iterative. I would create initial elements and array them across a Photoshop area of the size of the DS screen to figure out how many elements would fit comfortably. I could then compile the app so that I could drag elements around on the actual DS. This allowed me to see how usable a size or shape would be in practice. I was anticipating using this in dark music venues, so I aimed for low light readability, and made the touch targets large enough to afford easy interaction.
One strange aspect of the Nintendo DS hardware is that while the bottom screen has a touch screen overlay, the top screen does not. I decided to use the shoulder buttons (furthest from the user when the device is used normally) to switch what was on each screen. Left shoulder button to switch between the current screen and the menu, or the right shoulder button to cycle forward through the screens.
Another design direction would be to have the menu screen hidden until pressing a button. This would allow the top screen to be used to show other information. I could see how it would be extremely useful to have the top screen showing MIDI values as you move a slider or tap a button.
Learnings & Future
Setup Process: The hardest part of creating an app that I could test on hardware was getting the development environment set up on my computer. If further development were in the cards, I would set aside a computer specifically for this task, as well as document every technical specification and step in the setup process in order to be able to repeat it. It would be useful to make a disk image of the drive when it was in this state in order to be able to restore it to a working state more easily.
Get It Installed: It was extremely useful to be able to get my concept app onto actual Nintendo DS hardware. This let me play with the stylus and figure out if the touch targets were large enough or not, if the length of the sliders would be meaningful enough to be useful as controllers. I also realized that the multislider and multiball concepts were going to be harder to develop, based on my understand of the way that the sprite graphic system works on the device. It’s possible that these graphic elements would have to be hand-coded in c in order to build them up programmatically vs designing the pieces using images like I was doing.
Outcome
DSLemur is a Nintendo DS homebrew app design. I was excited about the possibilities of using this cheap portable gaming console as a feature-rich controller for visuals and other software over a wifi connection when performing at events. At the point I was working on this, having a wireless controller that I could walk around the room with was a dream. I researched the graphical system of the Nintendo DS to make sure the design would be viable, though it's at the limits for the visual frameworks on the device. Using Photoshop I created a multilayer design to fit within the sprite and object limitations of the system. I would require a developer partner to make further progress a reality.
