Is it Possible? – The New Stack

In the past decade, weve seen the rise, standardization and meme-ification of as code: Infrastructure-as-Code, Monitoring-as-Code, Policy-as-Code and soon perhaps Data-as-Code.

Jan Van Bruggen

Jan is responsible for developer relations at itopia Spaces. He co-developed itopia’s distributed engineering group and architected itopia’s website. He enjoys optimizing his setup, and sharing what he has learned. He has developed DevOps tools to assist a variety organizations including Google, NASA, startups, and open-source projects. But his most proud accomplishment is the code he deleted after finding a simpler solution.

Stuff-as-Code, in essence, is the practice to statelessly automate the management of stuff using version-controlled, declarative configura files. It is worth asking whether the same DevOps practices can be applied to a set unstable resources that developers are more close to than their backends or local coding environments.

While the common problem of developer environment drift can be a minor inconvenience when revisiting older projects or following outdated tutorials it can cause professional developers to lose days of progress each month in order to restore/update their environment(s), particularly if they work with a diverse team.

Terraform cannot save us unfortunately local dependency hell, and Kubernetes is unable to help Windows and macOS laptops cooperate on a Codebase Right? Is it possible for these local environment problems to be solved with declarative configurations, version control, and a practice stateless automation

Let’s take a look at everything local that a person relies on when writing code.

• Integrated development environments (IDEs)
• Personal settings
• Library for specific projects
• Packages that cover the entire system
• Operating system (OS).
• Computer hardware
• Internet connection
• Electricity
• Keep a cup of water near their keyboard

Local environments resist configuration due to the dependence of each layer on the layers below. Any attempt to statelessly automate a layer above will be undermined by stateful chaos beneath.

For example, the dependencies of a Python project can be defined in declarative. requirements.txtEach execution will result in a different file depending on what environment variables have been set and which OS packages have been installed. VS Code’s support for containerized environments is a major advance, but it still requires manual maintenance of a Docker setup (as a well as a compatible OS subscription and subscription to Docker).

Is it possible to statelessly automate the entire local environment? No.

Create an Environment

Recent progress has been made in the stateless configuration of these chaotic middle layers. Developer workflows are evolving to reflect this. Let’s look at the current configurability for each of the software-defined layers in local environments.

Personal Settings

Personalization is the highest-level layer under an IDE. This includes configuration files that can override default settings in tools and apps, such as for.

• IDEs: settings.json, options/*.xml, .vimrc, .Editorconfig, etc.
• Terminals: .bashrc, .zshrc, .tmux.conf, etc.
• Other: .gitconfig, psqlrc, etc.

Fun fact: Many of these file names begin with a period, and are therefore called dotfiles. Hidden files were accidentally invented in 1997.

Personalization is key to IDEs. This includes keybindings, shortcuts, accessibility, layout, and aesthetics. These files should always be unique from one developer to the next, since everyone has their own needs and habits. How can we automate personalization across devices and environments while not treating developers in the same way?

The most popular answer to this question is to use a Dotfile manager such as chezmoi, Dotbot, yadm, or others. Dotfile managers Each has its own unique featuresThey all automate the process for updating personalization files in order to reach a desired, version-controlled status. Some even use declarative configuration files!

It seems that personalization-as-code is a solved problem. Dotfile managers are not yet widely used, but professional developers who work on multiple devices and accounts should consider using one. It will save you a lot of time setting up, context switching, and debugging these environments.

Project Libraries

Every software project imports third party libraries from Java to Rust. This layer is already managed as code. Usually, a declarative configuration file is included in the project source code as part of version control.

This list continues, with a standard protocol for almost every programming language.

Fun fact: C++ was the exception to this trend for years. This is due to their unique integration of system packages (see next subsection) and (presumably), the inertia caused by legacy workflows. Today, however, both ConanAnd vcpkgPopularity is growing and seem to be a popular choice.

These libraries are often used by IDEs to format, analyze, and execute source code. It is a boon for developers that these configuration files can be found within source code. Library dependencies-ascode have been solved so thoroughly that:

• Nearly every project begins statelessly managing its library dependencies from day one.
• Almost all programming languages are expected to have a library management protocol.
• It’s a common phrase that is often overlooked, but it makes sense.
• Most developers take this for granted until someone notices its presence when discussing other stuff-ascode.

System Packages

Unlike project-specific libraries that are limited to a single OS, system-wide (and per user) packages can be used across multiple/all developers projects and are tailored for each OS. These include:

• Interpreters and compilers like gccAnd Python.
• Clients and servers, alike curlAnd apache.
• Terminal shells, and GUIs, such as xtermAnd Gnome.

IDEs are not only packages, but also depend on other packages to render views and connect to services. IDEs can also be used to analyze/execute code. Each project requires a specific set of packages. Therefore, each developer installs a different collection. However, only one package can be installed at once, and sometimes, different packages are incompatible with or compete with each others. This causes frustrating dependency mismatches within and conflicts within environments. Is there a way for a project to automatically manage its package dependencies across devices?

Over the past decades, a stateful solution is very popular: using a package manger like APT, Pacman, HomebrewOr ChocolateyTo install specific packages one by one. These tools allow for powerful setup scripts that are more reliable and manageable when combined with the sterility of containerization (more about that in the next section).

Unfortunately, without containerization, a stateful configuration is inherently less scalable than a stateless configuration, which can know at any point in the future if it has become invalid/conflicted and then efficiently repair itself; just ask anyone who chooses Terraforms approach to Infrastructure-as-Code for this reason!

This is why it’s so exciting to see. NixSlowly Being More PopularBecause Nix is a stateless solution for the same problem, Nix allows you to future-proof your environment against all forms of bitrot by installing one or more immutable packages and translating package dependencies (direct and indirect) into a DAG. This makes it much easier to collaborate with developers who use different OSes.

Any developer who is suffering OS-level dependency hell should install Nix to see if it helps. nix-shellBrightens their entire week.

Operating Systems

The operating system (OS) is the core of every developer’s software stack. It includes many subsystems.

• Application platform
• File management
• Drivers
• Resource (processors, memory, etc.) Management

It is impossible to statelessly configure an OS installed on most developer computers. Therefore, the best way to manage this layer is to use containerized guest OSes that host developer environments. Guest OSes are environments that are isolated from one another, while containers allow each environment to be version-controlled.

Fun fact: There are a few stateless OSes, but they are not very popular. NixOSIt is a stateless OS, managed entirely by Nix the package manager. Guix SystemIt is a libre copycat for NixOS. NixOS might be a favorite of adventurous developers, but it may not support all their development tools. This is due to its radical design.

DockerIt has been the best platform for specifying and building containers since the inception of most of the concepts and jargon related to containers. It implements a stateful system to specify build steps but a built container image can be referenced in declarativeconfig files.

Recommended Approach

Although it is ideal to have stateless automation at every layer, it is possible to incrementally improve your environment by adding new automation tools. Nix is an excellent addition to projects and chezmoi is a friendly assistant to developers.

However, if your goal is to create and maintain a fully automated local environment, you can do so by following these steps:

• When creating an environment
  • Docker allows you to statefully configure a guest operating system.
  • Configure some system packages using your OSs package manager.
• When managing an environment
  • Statelessly configure a container using a built container image.
  • Statelessly configure most system package with Nix
  • Statelessly configure project libraries using your language tool of choice.
  • You can statelessly create personal settings with chezmoi.

Depending on your use case you may want to move some of the stateless configuration steps into build to cache their results in container images layers.

Run an IDE

We have many options to create a local environment that is reliable and reproducible using the tools we have. This environment can be a performant and efficient daily driver. It can also be isolated from personal files, messaging applications, and Docker itself. However, it is not obvious how to interact within an IDE that is running inside a Container. This is unless NixOS is used, which doesn’t require containers.

An IDE installed locally can be used locally since it acts as its client. However, most containerized IDEs require a separate client for communication between the OSes. These IDE clients are notable, and they include both native (hybrid IDE), and web-based (online IDE).

VS Code

VS Code is a well-known IDE and was the first to offer both online and hybrid solutions.

Remote DevelopmentThis set of first-party VS Code Extensions allows an existing VS Code installation, to be used as a hybrid IDE. It can also be connected to a VS Code Server running somewhere. Although this is an easy solution, it requires the user to either create a VS Code server within their container or to use it. the container’s extensionYou can host the entire development environment. This tool is intended for solo developers who would like to use their local VS Code IDE Client.

Alternatively, code-serverThis third-party VS Code webserver renders the VS Code client as an online IDE in a web application. It is connected to a VS Code Server. This flexible online solution requires advanced web hosting skills. It cannot support extensions from Microsofts Marketplace. This tool is for solo developers who wish to use VS Code not-locally.

JetBrains IDEs

ProjectorThis is a first-party containerization solution that works with the JetBrains suite IDEs: PyCharm. IntelliJ IDEA. PhpStorm. This allows you to use any JetBrains IDE online, but the UX is inconsistent in their native apps. Projector implements an HTML5 connector that allows for Swing-based GUIs.

Other IDEs

Selkies-gstreamerIt is a DIY containerization solution that allows you to run any Linux compatible IDE/GUI in a container using a browser-based user interface. We will now discuss the broader implications. SelkiesPlatform will be covered in a later section as it is a unique solution. However, this standalone component can be a powerful tool for solo programmers who need to run a specific IDE/GUI without a web frontend or containerization tool.

Scaling Problems

These solutions are perfect for solo developers, although they do require some Docker tinkering. It is not easy to deploy and scale them to support a large development team. This is because of the need for specialized skills in networking and container orchestration. It is not an easy technical challenge to set up and maintain Kubernetes servers for per-user streaming sessions. Remote teams may also require clusters on multiple continents. This adds complexity.

Scale to Serve Your Team

We at ItopiaIt seemed like this kind of automation challenge would be more fun to our remote workstations teams than for the average developer team. Most teams have higher priorities than maintaining workstation clusters. Itopia has been managing remote enterprise servers for nearly a decade so we have some insight into how large teams adopt, scale up, and maintain productive environments.

Designed with Large Team Members in Mind

These preferences are common for large teams when it comes to remote workstations.

• Portability: Browser based IDE clients offer more convenience than OS-native IDE customers.
• Performance: Web apps should run all heavy computations and network in the cloud. This will prevent resource bottlenecks.
• Flexibility: All IDEs must be supported with the same UX to local installations in order to minimize transition costs and maximize developer productivity.
• Security: Protect intellectual property with high-security endpoints
• Maintenance: A fully managed service is often more practical for enterprises than a self hosted one. This is due to the nuanced upkeep that self-hosting requires.
• Transparency. Open source tech is more flexible than closed source tech and can be audited. This is important to both CTOs (computer administrators) and sysadmins (sysadmins).

We were shocked to discover that there was no solution to all the above preferences. This meant that teams had a to make compromises. Both browser-rendered and native IDE client solutions cover most of the solutions. However, streaming source code is required for all development devices. There is little protection against IP exfiltration. Opinionated services such as GitHub CodespacesThey are limited in their support of one IDE or version control system. Self-hosting is an option, but it adds labor and security.

Itopia Spaces: Local Environment-as-Code at Scale

itopia SpacesThis checklist checks all the boxes for both teams and companies; it’s free! test driveIt is easy to see why. It can stream any Linux-compatible GUI/IDE without any custom clients. This is one of its most promising features. See Our open source collection of itopia-managed imagesFor ideas and starting points for creating custom-designed images, see the following.

The core technology is open source, so be sure to check it out. SelkiesThe open sourceWe’ve created a cloud native streaming platform In partnership with Google. If you like what is seen today, Join our communityLet us know what you would like to see tomorrow! We love to see Selkies grow and we want to establish a new standard in remote app streaming.

It’s gratifying to see how much has been done to improve developer environments. It’s also exciting to imagine what the future of coding will look like in a few more years. Developers will be able, at the very minimum, to configure their environments reliably to meet their needs without having to spend a week manually typing commands or editing files every time a project or angle of their cup of water changes.

The New Stack is a wholly-owned subsidiary of Insight Partners. It is an investor in the following companies: Docker.

Feature image via Nappy