On April 4, 2017, Amita Ekbote and Susan Huynh, both of Mesosphere, Inc., presented on DC/OS, Mesosphere’s datacenter operating system. Mesosphere hosted the event at their San Francisco headquarters.

Over pizza and beer, attendees got the “10,000 foot” introduction to DC/OS and its components as well as a detailed demo.

Amita Ekbote began with a quick history of Mesosphere, from Apache Mesos’ origins in 2009 as founder Ben Hindman’s grad school project, through the debut of Mesos at Twitter, and on to the creation of Mesosphere and the DC/OS stack built on top of Mesos.

Then she explained Mesos’ resource offer process and introduced the audience to the four levels of the DC/OS stack. Mesos sits on top of existing infrastructure, like physical servers or a public cloud. DC/OS runs on top of Mesos, offering the user a genuine operating system that takes care of things like container orchestration and monitoring, and provides a UI and CLI. On top of DC/OS, a user can install a vast number of services, like HDFS or Spark.

After Amita’s DC/OS overview, Susan Huynh took the stage with the ambitious purpose of setting up a data pipeline on DC/OS — from soup to nuts — in 20 minutes, while also pausing to explain what she was doing.

First she explained the logic of data pipelines in general, then introduced us to the one she’d be demo-ing. A simple Twitter clone called “tweeter” posts data from the user as well as Shakespeare quotes from a “post-tweets” script. The data is processed by the Kafka distributed message bus and stored in the Cassandra distributed database. Then it’s sent to Spark for analytics and, finally, Zeppelin creates a visualization of the analyzed data.

In 20 minutes Susan installed DC/OS on a fresh cluster, installed all of the services she needed, and ran the Zeppelin visualization!

– Learn more about DC/OS: https://dcos.io/docs/1.9/overview/

– Try out Susan’s demo yourself: https://dcos.io/docs/1.9/usage/tutorials/iot_pipeline/

– Amita’s slides: https://www.slideshare.net/AmitaEkbote/introduction-to-dcos-74562699

– Susan’s slides: https://www.slideshare.net/huynh14/big-data-on-dcos