6 minute read + video
Trigger warnings: computer jargon, monster movie references
If you're a regular reader of the Huffington Post you might not have given much thought to the technology behind the news articles that you read, share, and comment upon on our site. Since 2005 the tech team at HuffPost has been working hand-in-glove with our editors to create the ultimate digital content delivery system. We call this system "MT" because it was originally based on Movable Type. Over the last eight years we have enhanced MT and its features to the point that they have morphed into a tool that puts every feature a modern tech-savvy journalist needs at her finger tips. MT won't write a great story for you, but it levels the playing field in an increasingly competitive, crowded, and careening Internet. MT is like those giant robots in the movie Pacific Rim: Tremendous monster fighting firepower with a human team at its heart.
Eight years is a long time for an Internet application to live. Since HuffPost MT was forked from the original Movable Type the technological landscape has changed. Entire technology paradigms have fallen in and out of favor with the digerati. It's time for a new version of HuffPost's MT, with a modernized codebase, to build upon the success of the past and prepare us for a future where mobile devices are more powerful than desktop computers, social media is the only media, and everyone with an Internet connection can stream content 24/7. We have to get MT ready for the next eight years and the next wave of users who create thousands of hours of video, billions of Likes, and millions of tweets each day.
Earlier this year HuffPost assembled a team of battle hardened software developers and product managers to work with our editors on the next generation of MT -- codenamed Athena. I'd like to introduce you to one member of that team: Kailuo Wang. Kai is not only an accomplished coder and dev lead of HuffPost's Athena project but a photographer and blogger as well. My favorite Kai quote is this:
"Forget about all the coding principles that people (through the Internet) try to shove into your head -- SOLID, DRY, Code smells, Design Patterns, etc. You can write good code without them. One principle is enough: have sympathy towards other developers in the team..." -- from Why Write Good Code (in a Simple Way)
Developers love to get into flamewars about coding styles and design principles but the idea that you should write your code with "sympathy" is as brilliant as it is practical. And it's one of the core goals of HuffPost's Athena project: After eight years it's not only time to update the MT codebase and take advantage of modern programming languages and frameworks but to write code that is fun and easy to work with. Let's call it Kai's Law of Humanistic Software Development. A codebase written by considerate coders is one that will be easily maintained and extended and one that that will survive whatever the Internet has to throw at it.
As a reader of HuffingtonPost.com you don't get a chance to see much of the technology that is used behind the scenes to create all the news articles, blog posts, original videos and slide shows that live on our site. In the past we've used PHP, MySQL, HTML, and JavaScript but those technologies, the workhorses of Internet applications, are no longer yearlings. To build the next version of MT, Kai and the Athena team, wanted to take advantage of a whole new "stack" of modern tech to write our humanistic code.
MySQL is a database, one of the best. (I'm old enough to remember having to fight grumpy old enterprise operation managers to switch from commercial databases to MySQL.) For Athena the team wanted to make it super easy to model the complex documents that news stories have become in the 21st century. I'm sure HuffPost readers would be disappointed to find a news story that didn't have relevant images, videos, tweets, and Facebook posts embedded inside it. For that task we're using MongoDB. While MySQL requires lots of up front planning and modeling to translate a news story into a collection of data fields and datatypes, MongoDB is a new kind of database that natively stores documents with flexibility. As our idea of what a news story evolves over the next eight years, MongoDB will help us keep pace.
PHP is a programming language, and like MySQL, a foundational component of today's Internet. Facebook is build upon PHP as is HuffPost and millions of websites across the world. But PHP has its warts and its limits, especially when it comes to handling a high volume of users. Facebook has worked miracles around the scalability issues of PHP. But to write humanistic code, code that isn't micromanaged by merciless engineering managers, MT needed a new programming language at her heart. The Athena team chose Scala (and its Play framework).
Scala is a relatively young programming language with several modern ideas built into its design. Scala was built on top of the environment of the older and well tested language Java. So Scala gets the benefits of Java's performance and stability while enabling developers like Kai to write simpler, more robust, and more reusable code. While keeping things simple, safe, and reusable can be done in PHP it takes a tremendous amount of effort. Scala enforces humanistic coding by default. (You can write bad code with Scala but you have to work hard to do it.)
JavaScript is the most unlikely success story in the history of computer programming languages. Once derided and abhorred by "real programmers" JavaScript has become an elegant and powerful tool built into every web browser today. But JavaScript is only good if you avoid its bad parts. There are so many bad parts to JavaScript that the book entitled "JavaScript the Good Parts" has an appendix devoted to its "awful parts." Kai will tell you that the secret to using JavaScript well is to use it through a great framework or tool. The next generation of MT has to be real time and fully dynamic. The Athena team chose AngularJS to tame JavaScript and HTML. AngularJS extends the vocabulary of HTML and the interaction of JavaScript to keep data on the server and data on a webpage automatically synchronized. It's a more humanistic approach because with AngularJS there is far less code to write and maintain.
By now you're probably wondering what is Athena going to do with MongoDB, Scala, and AngularJS? The video created by Kai below is just one example. It doesn't have a sound track so I'll walk you through it.
Say that two HuffPost editors want to collaborate on a story in real time. They don't want to email a text document around -- who has time for that? They don't want to use Google Doc to collaborate because it takes time and tweaking to move a body of text from one system to another.
So these two editors, writing under the names John Stewart and Stephen Cobert (no relation to anyone famous) both log into the Athena beta system. John opens up an article about Hank Azaria and starts editing it. Stephen opens the same article. John is informed by a message in the lower right of his screen that Stephen has started to work on the same article. As Stephen edits the article in his web browser, John is given the opportunity to "fight" with Stephen for control. John wisely chooses to allow Stephen to continue writing and sees the changes Stephen has made to the article in realtime.
The sweet thing about all this realtime collaboration is that the developer just has to set it up--the AngularJS framework (and Scala) are doing all the hard work. AngularJS monitors the text field in Stephen's web browser, notifies the Scala code back on the server of a change, and the Scala code notifies the text field in John's web browser. (It's a little more complex than this but you can chat with Kai for all the details.) The amount of MySQL, PHP, HTML, and JavaScript code required to make this realtime collaboration happen would have been staggering and hard to reuse. Athena's modern tech stack with MongoDB, Scala, and AngularJS, makes it simpler for the developer and easier for the editors: No need to "check out" documents, no arbitrary limits on the number of collaborators, minimal locking of the shared document, no submit button required. A much more human way doing things for everyone involved.
I hope you enjoyed this "sneak peak" behind the curtain at HuffPost. If you're a reader, you'll be happy to know that we're already testing part of Athena and hope to get a faster and more dynamic HuffingtonPost.com to you soon. If you're a developer just into coding tweet me or Kai for more information on how to use this tech stack for fun and profit. If you're a competitor, watch the video at 1080p and full screen so you can get some valuable intel.
Resources and more info:
MongoDB
Scala and Play
AngularJS
JavaScript the Good Parts
Kai's Blog