My Summer in Sunny San Diego

Let me preface this post with a warning: it's gonna get technical. It's just not possible to talk about an entire summers worth of software development without getting too technical for the average reader, so I apologize in advance. (Google is your friend!)

ViaSat's very military logo

So I had an amazing experience doing software development for ViaSat Inc this past summer, and I want to share some of the awesome technologies I got to work with! I was working with a team of two other interns---one of which, also goes to Harvey Mudd. Our initial task was to develop the equivalent of Netflix's "Chaos Monkey" but for ViaSat's core switch infrastructure. We had to switch gears completely when we got hit with the ridiculous policy that no interns could be signed into the core switch's NDA because of the 'likelihood we may work for a competitor in the future.' ... Wait? Isn't that the entire purpose of an NDA? Hmm... So we came up with a solution: Instead of writing a program like Chaos Monkey that would run various tests on the development infrastructure, we would write a highly extensible framework that allows the developers to write tests that would be run on the development infrastructure. Write code to run other people's code!

Even though our users were other developers, we wanted to make things as easy as possible for them to increase the likelihood of our project actually being useful. So we decided to allow the users to run ANYTHING! ... (that runs on Linux). Figuring out how to make this work involved learning a slew of shiny new system administration and cloud tools including Docker, Ansible, OpenStack, and did I mention Docker?

In the end, our final product was split amongst three repositories.

1. The Deployment Repository

This repository mainly consisted of Ansible playbooks and a Dockerfile. Ansible playbooks are similar to bash scripts in what they can accomplish, but they are wildly different in how they are written and how much simpler their syntax is. The playbooks would boot up a configurable number of slave and master VMs. They would configure these VMs according to their tasks, and finally pull and deploy our web interface repository, and our scheduler repository. In the end of a deploy--which could take up to 30 minutes--our entire infrastructure would be up and fully functioning. Ansible needs a few dependencies to be able to interface with Openstack, so I wrote a Dockerfile with all the dependencies that would automatically run the deployment process. This way, users would only need to have Docker installed to deploy our VM cluster.

2. The Scheduler Repository

The view from my office (just imagine a beautiful sunny day)

This repository took was deployed in the form of another job run on the infrastructure. It's task was to communicate with the web app and the database server, schedule the jobs to run randomly based on each job's configuration, and then go to sleep until it was needed next. In order to wake the scheduler up early, I had to write a signal handler, and a signal sender, that would ssh into the container that the scheduler job was running, and send the unix alarm signal (SIGALRM) to the running process. It was kind of hacky, but also very cool to figure out!

3. The Web Interface Repository

This repository contained mostly Python code, HTML, and Javascript. Our web framework was called Flask, and for pretty javascript forms and buttons we used Foundation. I'm most proud of my docker backend that I wrote to allow the web app to use the docker api and to start images for the ssh configuration feature. I used docker-py to do the api integration, and tutumcloud's images for the ssh serving docker images. This task was the most challenging part of the project. Some functions required communicating between various VMs, generating and serving secure ssh keys, and carefully testing for and catching errors. I needed to discern between server errors and user's dockerfile errors, which the system would need to handle. In the end, I am incredibly happy with how it turned out.

In hindsight, I would say that our team did a good job of avoiding hitting too many major roadblocks. We managed to be quite productive all summer. I had an incredibly fun experience at ViaSat and I'm looking forward to doing more software development in the future.

Thanks for reading!