More

Introducing The Tech Stack File

By  Yonas Beshawred

Today we’re excited to launch a new open source file format - "The Tech Stack File." With input from the Cloud Native Computing Foundation (CNCF) leadership and project maintainers, our goal with this new file format is to create the universal standard for tech stack data to make it easier for teams everywhere to access, share, and gain insights from the full range of their technology data. To help the StackShare community leverage this new file format, we’re also announcing two new products: StackShare AI and StackShare Connect.

Tech stack data on which software engineering technologies are being used typically lives in spreadsheets and wikis. Populating these sources and keeping them up to date is manual and time consuming, not to mention that this data is outdated almost immediately after it’s gathered. This makes migrations, codebase onboarding, and due diligence tasks all slower than they need to be.

Many traditional large companies are undergoing digital transformations and moving more and more workloads and applications to the cloud as interest in AI grows by the day. This means that tech stack data is becoming outdated at an increasing rate as tech sprawl continues to spread.

The best software engineering organizations in the world have robust and accurate data on the technologies they’re using across their software engineering teams. With the introduction of the Tech Stack File, we’re bringing this capability to all companies through a universal standard file that’s easily accessible. Having tech stack files will help these organizations dynamically organize and visualize their technology stack, and even bring this context to applications and tools they use every day.

The Tech Stack File, called techstack.yml, is a YML file that’s available now at stackshare.io/tech-stack-file. It lists out the entire tech stack and software supply chain of a given Git repo, with metadata and details about each technology. It details which open source packages and dependencies are being used in a repo, but it also includes closed source (commercial) software, data stores, Infrastructure as Code (IaC), Software as a Service (SaaS), and other types of technologies that are all part of the tech stack.

Organizations can not only leverage their tech stack files to organize and visualize their tech stacks - they can bring the context of their stack with them to applications and services they use every day to make them more useful. As such, we’re announcing two more new products that will be available in the coming months that leverage this new file format: StackShare AI and StackShare Connect.

StackShare AI allows you to query the entire knowledgebase of StackShare’s 1M+ developer community and personalize the results by connecting your techstack.yml files. It’s an AI agent that brings StackShare’s data to your team combined with personalization based on your org’s tech stack data. This means organizations will be able to use their tech stack files and query StackShare AI to understand redundancies, generate reports, and ask questions like “who are the top React experts across the company?” or “how many applications are using Stripe in production?” All of this will be accessible through a conversational Slack bot. You can sign up to join the waitlist for StackShare AI here.

StackShare Connect allows teams to take their tech stack data with them by connecting their techstack.yml files to the websites and apps they already use. Sharing your team’s tech stack files with external websites and applications makes them stack-aware. Imagine seeing your company’s tech stack data in the engineering, security, and product management tools your teams already use. To this end, we’re excited to announce FOSSA, the developer-native Open Source Management Platform, as our StackShare Connect launch partner. FOSSA and StackShare customers will be able to connect their tech stack files to their FOSSA accounts to easily generate comprehensive dependency inventory and audit-grade reports and institute sophisticated policy governance.

“We’re excited to partner with StackShare on this because we’ve seen demand from our customers asking for more robust tech stack data that goes beyond open source dependencies. Customers want to know what infrastructure, SaaS, and other technologies are being used and then apply governance policies to those. With StackShare Connect we’re going to enable this and drive more value for our enterprise customers.” - Kevin Wang, Founder & CEO, FOSSA.

How to get The Tech Stack File

The Tech Stack File can be automatically generated and visualized through a new GitHub Application: Stack File. The GitHub app uses StackShare APIs to automatically generate techstack.yml files for every repo you connect.

Here's a sample of what a techstack.yml file looks like:

The GitHub app also generates a second file for every repo that visualizes the YML file through a Markdown file called techstack.md. Here's a sample of what it looks like:

You even get a small Markdown snippet that lets you list the tech stack of a repo in your README file:

The app generates these two files and opens a Pull Request on every repo you connect it to. The files get updated via a new PR once per week:

Stack File’s main features:

  1. Fully Automated: Stack File automates the entire file generation process, eliminating manual input inefficiencies and ensuring accuracy.
  2. GitHub Integration: Seamlessly integrated with GitHub, the Stack File GitHub App creates the files and updates them in your GitHub repos, ensuring your tech stack data is fresh and always up-to-date without manual effort.
  3. Easily Accessible In Your Codebase: The YML and Markdown files live in your GitHub repos, which provides easy access without requiring any workflow changes.

Insights from the Cloud Native Computing Foundation (CNCF)

There’s no shortage of standardized ways to define infrastructure, services, and libraries.

But until now, there was no universal standard for how to describe the entire tech stack and what technologies are being used in a code repository. Since most developers already think of StackShare as a place to discover, compare, and research tech stacks, it made sense for us to attempt to create this universal standard to describe tech stacks that can be adopted by others and improved upon.

As we thought about organizations to work with on this, it became clear that CNCF was the ideal engagement to help us refine and test the format. CNCF is one of the world’s largest open source foundations and is home to over 150 of the most popular open source projects in the world like Kubernetes, Prometheus, Istio, Helm, Backstage, and many more.

CNCF understands that knowing which companies and organizations are using which open source projects is an important consideration for potential adopters of open source projects because it builds trust. Knowing other organizations that you recognize rely on the solution you’re considering oftentimes inherently answers many questions you may have about the project (reliability, scalability, etc). This helps you better decide whether or not the solution could be a good fit for your organization.

It’s because of this dynamic that they’ve seen community-driven efforts like ADOPTERS.md files. These files organically appeared in projects without any official standards. Projects that have these files typically ask end users to self-identify and add themselves to a list of adopters of that project. Since there is no standard, many projects do it in many different ways and it usually ends up being more work for the maintainers of these projects.

The Tech Stack File enables organizations to easily keep track of and share open source projects and other technologies they’re using with others, including CNCF projects. CNCF projects and other open source maintainers can benefit from Tech Stack File data in multiple ways including being able to automatically ingest techstack.yml files into ADOPTERS.md files for their projects.

More details on The Tech Stack File

Rather than being prescriptive about the format, we think of techstack.yml as the opposite: no matter what Infrastructure as Code (IaC), package manager, containers, or other technologies are being used, the format we’re introducing simply takes those as inputs to create an output.

We set out with a few basic requirements for the tech stack file. The YML file needed to be:

  • Human Readable: Tech Stack Files are designed to be human-readable and easily understood at a glance.
  • Exhaustive & Accurate: These files contain comprehensive information about an organization’s technology stack and are kept up to date via automation.
  • Unique Per Repo: Each repository has its own unique Tech Stack File, providing specific insight into the technologies used in that particular repo.

We also knew that it would be valuable to be able to easily visualize the data file so we thought a Markdown file would be the best way to do this. We decided the MD file needed to be:

  • Generated based on the YML file
  • Visually appealing and clear

Use Cases

Having easily accessible and up to date tech stack data ultimately saves repo maintainers time by automating all the manual work of gathering and updating this data.

Tech stack files are not just about visualizing your tech stack. Here are a few ways you can leverage them:

  1. Spinning up new projects - Ensure that new projects are kicked off on a solid foundation of best practices from the outset by documenting what’s being used.
  2. Onboarding new developers - Giving a developer an overview of the tech stack being used helps them better understand what they’re dealing with and who they should talk to about which technologies.
  3. Implementing migrations across multiple repos - Implementing migrations across multiple repos involves identifying the right repositories based on the versions of libraries being used and identifying who, if anyone, is responsible for those libraries and repos. Instead of having to dig through repos to get this data you can simply go to the tech.yml file in each repo.
  4. Due diligence for financing/M&A - During corporate transactions such as financing or M&A companies are often required to provide a comprehensive list of tech stacks and open source software (OSS) licenses in use. Instead of tasking a developer with this, they can simply grab all the tech.yml files that exist.

The Future of The Tech Stack File

We’re starting off by working with CNCF project contributors to gather feedback and refine the format. In the coming months, we’re going to enable CNCF project maintainers to automate the ingestion of techstack.yml files into ADOPTERS.md files for their projects.

We’re also going to build out an ecosystem of Tech Stack File partners through StackShare Connect. This will allow other companies and platforms to leverage Tech Stack Files in their products and services, which in turn will make their offerings more valuable to their customers.

Finally, we’re planning on releasing StackShare AI (Beta) to StackShare Enterprise customers in early 2024. This will enable these organizations to leverage their tech stack files in conjunction with the entire knowledgebase of StackShare’s community to get personalized AI-driven insights.

Get your own tech stack files now by visiting stackshare.io/tech-stack-file.

More Posts
Modernizing StackShare's Front End: The React + Rails Stack
The Evolution of The New York Times Tech Stack
View all posts