Containers and Languages

@xav-b|April 12, 2017 (8y ago)3 views

In march 2013, Solomon Hykes presents Docker, and democratizes access to linux containers. The underlying technology, control groups, is already incubating for a few years at Google. But Docker abstracts away the complexity of containers' lifecycle and adoption skyrockets among developers. In June 2016, Datadog published some compelling statistics about Docker adoption: the industry as a whole increasingly adopts containers for production.

Since everybody is talking about how to containarize everything, I would like to take a step back and study how it is influencing the development of our most fundamental medium : programming languages. Golang rise, Java8 release, Python 3.6 improvements, … How both languages development and containarisation market maturation play together in 2017 ?

Container technologies scope

Let's define the scope of what we call container technologies. Way back in 2006, two Google engineers started to work on a new technology for partition hierarchical group of tasks. They called it cgroups and submitted the code to the Linux Kernel. This lightweight approach of virtualization (sorry Mike) was an opportunity for infrastructure-heavy companies and Heroku, Google, among others, took advantage of it to orchestrate so-called containers. Put simply, they were now able to think of application deployment as the dynamique manipulation of theses determinist runtimes. Whatever the code or the business logic, it was encapsulated into a uniforme execution format.

Cgroups are very low-level though, and tooling around the original primitives quickly emerged, like LXC backed by Canonical. Then Solomon Hykes came in and made the technology widely accessible with Docker. Possibilities were endless and indeed developers and startups alike rushed in all directions. Lately however the hype seems to cool down. Docker market share is being questioned while the company precises its business strategy. At the end of the day, developers start forgetting about vendor/technology and just want simple tooling for more efficient coding. Docker-compose, Red Hat Container Development Kit, GC Container Builder or local Kubernetes are very sophisticated pieces of technologies that hide the details of the underlying container mechanics. What they give to engineers is powerful primitives for advanced development practices : development/production environment parity, transparent services replication, predictable runtime configuration, ...

But this is not just about development correctness or convenience, considering how containers are eating the IaaS landscape. It is also about deployment optimizations and resilience. Tech giants who operate crazy large infrastructures developed incredible frameworks, often in the open, to push how fast they could deploy auto-scalable, self-healing, zero-downtime fleets. Apache Mesos backed by Microsoft, or Kubernetes by Google, make at least two promises:

• Safe and agile deployments at the (micro-)service level
• Reliable orchestration with elegant service discovery, load-balancing and failure management (because one have to accept that production always go wrong at one point)

Kubernetes Overview - lost

Containers enabled us to manage complexity with infrastructure design patterns like micro-services or serverless. Behind the hype of these buzzwords, engineers try to improve team collaboration, safe and agile deployments, large project maintainance and monitoring. But we quickly came to realize it was sold with a DevOps tax. Fortunately the software industry has a hard-won experience of such balance and we start to see it converges toward the most robust approaches.

This container landscape overview hopefully gave us the requirements to now study how it impacted the development of programming languages. We will take a look first at their eco-systems, and then we will dive in languages designs themselves.

Languages Ecosystems and Usages

Most developers are now aware of how invasive container technologies can be. It makes its way in your development toolbox or how your company manages its servers. Some will argue that the fundamentals of coding did not evolve much but the trend is hard to ignore anyway.

Docker trend over time - lost

While we are free of course to stay away from Silicon Valley latest fashions, I think containers tackle a problem most languages struggle with : dependencies and packaging. Go for example got packaging right but is still trying to figure out how to handle dependencies versioning and vendoring. Javascript, on the other hand, has npm to manage with fine grained third party codes, but build tools are scattered all over Github. Containers won't spare you the pain of setting things up (they target runtimes, not build steps), but it can lower the bar of language adoption. Official images can run most standard language projects and one can both give a try and deploy a basic hello world in no time. When you realize that Go1.5+ needs Go1.4 to be compiled, it can be a relief to just docker run your 5-lines-long main.go.

Growing a language community is a sure way to develop its tooling, libraries, … But containers also proved to be able to influence how we design those components. They are kind of the cloud counterparts of the the current functional trend. We tend to embrace a world where both functions and servers are immutable and single-purpose. We want predictable, pure primitives (in the mathematical sense). All of that to match increasingly distributed and intensive workloads.

I hope those approaches come from products need but obviously having the right technology at hand drives the innovation. As software engineers in 2017 we also design libraries and tools with containers in mind : high performance networking, distributed process management, Data pipelines, and so on.

Languages design

Languages Trends over time - lost

What about languages design themselves ? To get things straight, I don't think containers influcences how Guido Van Rossum designs Python. And that is the point of containers. They abstract the runtime to let you focus on your code™  (it is literally on every Docker-based PaaS landing page). You should be able to design whatever logic implementation you need, and containers will come handy to help you run it when needed.

I do believe however that both languages last evolutions and the rise of containers serve the same maturation of ideas in the tech community.

• Correctness at compile time : Both Python 3.6, ELM, and Javascript ES7 are bringing back typing to their language (see type hints or Typescripts) / An application running locally will launch just the same in production. You can even run tests against multiple runtimes without complex scripts or heavy setup.
• Simplicity : Go won a lot of its market share thanks to its initial simplicity, taking a lot of decisions for you / Containers try their best to offer one unified way to run code, whatever the stack.
• Functional: Scala, javascript, Elixir, … all enforce immutable states, function compositions with support for lambda expressions, function purity / It echoes the serverless trend that promote function as a service. Most of the providers leverage some kind of container technology to bring the required agility to their platforms.

There is something elegant having both language features, programmation design patterns, and infrastructure operations going hand in hand. While I don't think one of them influences the others, I certainely believe that their development smooth others' innovations.

Conclusion

Container technologies and the fame around are finally starting to converge toward fewer and more robust usages. At the same time, infrastructure designs, new languages and evolutions of existing ones seem to promote the same underlying patterns : simple, functional, decoupled components. I think this coincidence comes from industry maturity and openness, more than, as I said, one technology influencing the others.

Containers, however, are shaking how we collaborate and design tools for the languages we love. It changes the way we on-board developers learning a new language. It change how we setup local development environments, with micro-replicates of production topology. It changes the way we package and deploy code. And most importantly, it enabled architectures like micro-services or lambdas that influence how we design our programs.

I my opinion programming languages design should continue to evolve decoupled from containers. They serve different purposes, and given the pace of the tech industry, major languages should never depends on new shining tools. That being said, the evolution of languages now comes a lot with the activity of its community. What they build, how they use it, how they spread it in companies, … Coping with containers is an opportunity to bring new developers, improve production robustness, and accelerate both technical and human growth.