FetLife

BDSM & fetish community
for kinksters by kinksters.

Open Source Projects

Open source software is behind virtually every aspect of FetLife's operations. Without all of the great community projects that we use, there would be no FetLife.

Some notable ones include:

  • Our operating system: CentOS.
  • Our database server: Percona Server.
  • Our in-memory database server/outsourced heap: Redis.
  • Our primary programming language: Ruby.
  • Our secondary programming language: Scala.
  • Our tertiary programming language: erlang.
  • Our web server: nginx.
  • Our web development framework: Ruby on Rails.
  • Our RPC mechanism: Apache Thrift.
  • Our caching layer: memcached.
  • Our configuration management system: chef.

Naturally, though, sometimes we actually write our own code (at least, that's what we tell John). We're proud to give back to the community by open sourcing the vast majority of our projects (not to mention contributing patches and support to other projects). Here are some of the cool things we've opened up.

Our Projects

Code that powers FetLife that we have proudly given back to the community.

objectify

FetLife is a complex application with a lot of functionality and our team is tiny. So, our ability to continue to improve the site rests on the maintainability of our code.

The best practices commonly followed in the industry failed us in that regard, so we went looking for answers in the literature on software design. Ever since, we've been using what we found to refine a methodology for writing more maintinable code.

Objectify is a framework we built that implements and codifies our methodology.

Links: Github/Code | Release Announcement

rollout

While features are still baking in the oven, we like to be able to test them on the live site. But until they're ready to rock, not everybody gets to see them.

Rollout is the tool we use to determine who gets access to features. It allows us to give access to groups, like the caretakers and greeters, specific people, like John or James, or a percentage of the total FetLife population. We currently use rollout to control access to the activity feed and chat. We'll use it for anything new that we build.

Links: Github/Code

degrade

Sometimes shit is broken — you might have noticed. One of the ways that we're mitigating broken shit is by turning off features when their error rates become too high. If the activity feed is broken, you might see a FetLife fail whale. Degrade is how we're keeping track of what is broken. It monitors the ratio of failed requests to successful ones. If it exceeds a threshold, we show an error page until somebody resets the circuit breaker manually.

Links: Github/Code

Friendly

When you have nearly half a million kinksters posting pics, posting in groups, commenting, and just generally having a good time, you end up with an activity feed that generates a hell of a lot of data (nearly one billion rows per month). Adding fields and indexes to MySQL tables that contain that much data is a losing game, which means that adding functionality to the activity feed was nearly impossible.

Out of this challenge (and inspiration from this article), Friendly was born. Friendly stores all of its data in a single, binary field in the database. Inverted indexes are stored in separate, optimized tables for querying. Adding and removing fields is simple, and building indexes can be done offline.

Since Friendly's queries are much less powerful, caching data stored in Friendly is easy. We see cache hit rates of 99.9%+ on the feed pages. Friendly makes FetLife fast.

Note that we're not using this anymore — not because it's no longer useful, but because it no longer applies to our problem set.

Links: Github/Code | Release Announcement

munin_plugin

We use the awesome munin to trend important system metrics. That helps us get a good look at what's happening in the depths of FetLife's server farm. After writing about a billion munin plugins, we decided to put together a little domain-specific language to make the job easier.

Link: Github/Code

scala-resque-worker

For background work, we use the awesome resque, written by our friends over at Github. Nearly every aspect of FetLife depends on resque, and our installation processes over 10 million jobs per month. Crazy, eh?

One of our important background jobs is handled by scala code because scala is Really Fucking Fast (tm). Resque is a ruby system, but it was easy for us to implement a worker in scala and we released it.

Link: Github/Code

Want to see some of our technical presentations?

Check out our presentations page were we share a couple of JamesLennon's talks where he talks shop.


Want to Contribute?

If you want to get involved with any of these projects, join Github, fork the project and send us some pull requests!

^ going up? ^