What have I done?
Artificial Intelligence
Reinforcement Learning and Adaptive Kanerva Coding in XPilot
Beginning in my spare time during the summer of 2009, and continuing on as a side project over the course of the 2009-2010 academic school year, I have been working on a project in reinforcement learning that takes advantage of the XPilot-AI game interface. It is an open source project designed to take the Astroids-like XPilot game and make it controllable by programs / scripts written in C, Java, Python, or Scheme.
The eventual goal of the project was to write a reinforcement learning program that would control multiple cooperative agents in a non-omniscient environment, as in no agent wouldn't be able to simply pull up any or all information on the rest of the game board without "seeing" it themself. Our first step was simple: control one agent. Given the complex nature of the environment and the baby-steps that computationally intensive artificial intelligence methods are capable of taking, we wanted to start small.
Initially we tried with a trial-and-error like approach to determining our proper state space and available actions. Not so surprisingly, this was ineffective at best. As a result, we have started to work with an adaptive method based on Kanerva coding. The idea behind this is that it takes the agent's representation of the world and shrinks it while attempting to maximize its effectiveness, all while allowing the computer to essentially learn for itself how to improve this state space when it is not quite as effective as we would like it to be.
A paper was written based on the results of our findings and was accepted to the IEEE CEC 2011 conference for oral presentation.
Computational Intelligence
Bio-Inspired Genetic Algorithm
For my final project in an upper level computational intelligence course at Connecticut College, another student (Alex Ellison) and I researched extensively into biologically inspired genetic algorithms. We were interested in studying the apparent disconnect between a classical genetic algorithm and the way in which biological evolution and DNA reproduction occurs.
In a genetic algorithm, when mating two parents together, a simple crossover method is often employed to model the joining of the two parents DNA. The crossover may happen around one point or two, but that is generally it. Similarly, mutation often occurs in simple fashion as well. The standard mutation for a classical GA is to simply set a very small probability for each and every bit of the resultant child to be flipped. While these methods do occur naturally, they are far from the only things that do happen.
Biologically speaking, a number of other types of mutation and mating can occur. For example, a section of one parent can simply be inserted into the second parent, increasing the length of the chromosome, with this new altered chromosome being considered the child of the two. Or during mutation, it is possible that entire sections of the chromosome might be deleted, moved, or even duplicated in another location on the chromosome.
Further still is the concept of coding versus non-coding sequences found in your DNA. In the genome, there are many sections of your DNA that are never directly accessed at any given moment. They may be turned on or turned off by random mutation. The existence of these sections often produces interesting results in terms of how a generation will evolve, yet in a genetic algorithm this entire concept is ignored and does not exist.
Our project was to examine the viability of a genetic algorithm that implemented all of these biological concepts that have been more or less removed from the realm of a genetic algorithm.
Punctuated Anytime Learning for Evolving Hexapod Gaits
This project was the simultaneous love and bane of my existence from summer 2008 until spring 2010. Though I am glad to have had the opportunity to work on this project, I am equally glad that it is over. As a result of this project, I have garnered a great respect for those who work with robots on a regular basis while simultaneously realizing that, though I have a deep passion for artificial intelligence and evolutionary learning, my passion does not extend itself to robots as well.
After my Freshman year, I went to my advisor, Professor Gary Parker, looking to get involved in some research in the Computer Science department. We decided I should work on a Punctuated Anytime Learning system. It's an idea that he developed a number of years ago, but one that never quite came to complete fruition. It had been implemented and proven to work, but thus far only with the continuous assistance of a human. It had been discussed in great length in theory as well, but was lacking a truly autonomous implementation. On a very basic level, the idea is to create an automated learning system for a robot that would constantly improve, and never require human interaction or interference.
Punctuated Anytime Learning in XPilot for Evolving Combat Agents
As an off shoot of my original PAL project, Prof. Parker and I have decided to implement Punctuated Anytime Learning in XPilot for my undergraduate thesis. We intend to exploit an artifact in XPilot to model the dissimilarity that almost always occurs between a simulation and the real environment of a robot. When learning for a robot, a simulation of the robot's environment is often used to speed up the process of learning. The issue with this is that it is almost impossible to perfectly model the real world- servos will get weak, legs will break, and chances are the programmer will not have perfectly created the simulation down to the smallest detail. This problem is one that PAL strives to create a robust work around for.
In using the XPilot environment, one thing we have discovered is that if we run the game at a speed any faster than 64 frames per second (FPS), artifacts occur. The game acts strangely and oddities arise to the point where you cannot trust a learning algorithm that is run at any faster speeds. We tend to ignore this and run the game at a significantly higher FPS anyway and use this as our "simulation" model. Simultaneously, we will run another agent on another computer at 16 FPS, the standard FPS easily playable by humans.
In creating this dual system, we will run the learning algorithm on the faster computer while periodically updating the static model on the slower computer with learned results. As the static model is periodically updated, information will be sent back to the learning computer to bias the learning algorithm towards the slower computer's environment with the hopes that any oddities that arise from running the game too fast will be taken care of.
A paper was written based on the results of our findings and was accepted to the IEEE CEC 2011 conference for oral presentation.
Mechanical Engineering
Psuedo Segway
Forgive the name because, to be honest, I'm not sure what else to call it. In the spring of 2009, I took a course in robotics. Most of the class ends up building a hexapod robot of some kind. There may or may not be variations, but they all tend to be pretty similar. The main differences are in the various sensors and applications of the different robots. This particular class's robots ranged from a robot that detects and moves towards sound to biped robots to hybrid wheeled and legged robots.
In this class, though, there was also a student from the Coast Guard Academy across the street. As he was currently building a highly complicated autonomous vehicle for his senior capstone project and wanted to do something a little more applicable to real life and I had just built my own hexapod for my PAL research, we decided to work together on something a little different. In the robotics lab on campus, we have had the frame for a two-wheeled self-balancing vehicle (for the sake of ease, I'll call it a pseudo Segway) that a student started building three years ago. He never got it working, though, so we decided to work on it. Parker's only stipulation was that our goal be not just to make it safe enough for a human to ride, but to try and make it balance and move entirely on its own. After all, it's for a class in robotics, right?
If you're curious about more of the details, you can check out the project page here.
Software Development
Google Person Finder
During the summer of 2010 and the fall semester of my senior year (2010), I was fortunate enough to get an internship at Google under their .org wing. During the latter half of this internship, I helped develop features for the Google Person Finder application. Its purpose is to help locate and track information pertaining to missing people. Though we primarily serve it up as a tool for use in disaster situations, it need not be restricted to only that environment.
Please see Google's blog post, landing page, or simply visit Person Finder for more information.
Google Resource Finder
In addition to working on Person Finder while interning at Google, I also worked on a similarly-minded application, Resource Finder. It is a tool designed to help gather, track, and spread information in times of disaster. Alongside Person Finder, it helps make up Google's disaster management "suite". With luck, they will be used together whenever a crisis hits to help manage the disaster by way of tracking information.
Again, please reference Google's blog post, landing page, or simply visit Resource Finder for more information.
LOLPython
For my programming languages class [Fall 2010], we were asked to complete a project -- any project. Basically the assignment was to just do something interesting (educational video, write a paper, program something, whatever). I decided to implement a LOLCODE → Python translator. After about a day's worth of hacking, this is what I came up with. Documentation is a bit spotty, so feel free to just play around with it. Right now, it's still in "first draft" mode, as the project guidelines were pretty loose, so don't expect perfection from this piece of code. This was just a quick, fun, and easy project to work on :)
For some example LOLPython files, try these: anagrams.lol, beer.lol, fib.lol, guess_number.lol, hello_world.lol, lzw.lol.
Need Comms Now
Need Comms Now is an Android application resulting from the Random Hacks of Kindess event in Portland in December, 2011 [we won Best in Show!]. It is a system designed to track cell phone and wifi signal strengths in areas with damaged infrastructure during disaster scenarios. In disaster environments, infrasture is often weak or nonexistant. Having a method of finding cell phone service on the fly is essential. If someone is in danger and needs to call in help ASAP, they often won't have time to spend a half an hour wandering in the wrong direction because they can't find cell service to get help. Having a quick and effective method of finding out where they and others have received cell service can make a drastic difference.
Though this project is still very much a work in progress, I and the other developers have committed to spending what spare time we can on it to continue fleshing out the system and making it more robust and reliable for field usage. You can find a video describing the project from RHoK here. Need Comms Now is discussed from the start until about 10 minutes into the video.
POSIT - Portable Open Search and Identification Tool
In the summer of 2009, I was lucky enough to get an internship with the HFOSS Summer Institute program. The Humanitarian FOSS program is a collaborative, community-building project created by faculty from Connecticut College, Trinity College, and Wesleyan University. It was created with the goal of building a community of academic computing departments, IT corporations, and local and global humanitarian organizations dedicated to creating free and open source software.
While working at the Summer Institute at Connecticut College, I led a small team of students in a project built on the Google Android mobile phone operating system. We worked with students from Trinity College on a program called POSIT, which stands for Portable Open Search and Identification Tool.
It's a smartphone-based tool for search and rescue missions, scientific field work, and other applications. For example, in a disaster situation, emergency workers can use POSIT to create notes containing a location, add additional information such as a name and description, and take pictures, videos, and audio clips of a situation with the built-in phone camera. These notes can be synchronized from device to device using a central server. Because the notes are tagged with the device's GPS location, POSIT can generate a map of markers for a given incident, overlaid with satellite imagery or street data. The project was started by Prasanna Gautam, an HFOSS intern, in 2008.
This program be useful for any number of things. In the disaster management scenario, it could be used to keep track of missing people by taking pictures of various people and sending them to the centralized server to be inserted into a database used to help re-connect families. Or, for example, it might be used to track supplies in various areas so that people are able to obtain the materials they need to survive. On a different note, it could be used for field research. A biologist tracking the change in quantity of birds in a specific area might use the program to keep track of the birds he comes across in the field. These are only a few of the many possible applications of POSIT.
Whacky Moles
During my winter break, senior year of college, a friend and I were pretty bored. We had nothing to do and plenty of time that we wanted to not continue doing nothing in, so we decided to develop an Android game. We messed around a bit and eventually ended up with a whack-a-mole game called Whacky Moles. Feel free to check it out and give us a review in the app store! We're pretty proud of it, had almost 1,000 installs in the first few days, and so far are getting some awesome reviews.
At its best, the app was ranked #17 of all games and #36 of all free apps on the Amazon app store following the Kindle Fire launch. As of January 1, 2012, it had been downloaded by well over 200,000 people.