So, SBOMs matter, now what?
<2022-10-28 Fri> Software Bill of Materials (SBOM) Speakers are from VMware which make paketo buildpacks (https://paketo.io/):
- an alternative to dockerfiles.
- part of “could foundry foundation”
Only 25% of containerized apps generate sboms for containerized apps (2021 anchore survey)
Not all SBOMs are actually useful.
Format versions seem to matter.
- xml and json versions aren’t necessarily equivalent.
SPDX has good license support. CycloneDX has good support for things like purls, SWID tags, etc.. which are good for vuln detection.
Their group used syft to generate their sboms, I believe.
The different languages don’t necessarily have equivalent support for things like license support.
They suggest storing sboms alongside the images in the OCI registry.
One of the things that buildpacks seems to do is.. given a language.. generate a full image. I think that this is good that it doesn’t need to differentiate the artifact and the later-generated image.
source -> container build -> oci image.
- Given peanut allergy, source = ingredients. oci-image = final result. container build = “manufactured on equipment that processes tree nuts”
They suggest generating the sbom during the build-process itself.
Multi-stage build files are a concern, b/c things that are installed may result in security issues. Minimal images, while good for security, erase information about how the compiled source was generated.
The “node” base image has the same problem where you don’t really know the compiled things that went into it.. right?
What data tells us about sofware supply chain security and what to do about it?
Q: “an sbom” but in an earlier talk (sboms matter, now what).. they talked about the chain of sboms (similar to provenance).
Do you think our tooling and industry is looking at a plurality of sboms? Or is still in a single-sbom mindset?
Trip report
I went to KubeCon 2022 this year. It was a fantastic networking opportunity, in that I got to meet several folks that are active in projects that we rely on or are interested in (kubernetes, OPA, open feature, etc). The vendor list was also great, if only to see what folks are doing in the industry.
My kubernetes knowledge going into the conference was that I was comfortable deploying things into k8s, but haven’t really built infrastructural components (operators, CRDs, etc) before. I got a much stronger conceptual model of k8s, which is that it’s effectively a yaml-defined API service with an internal eventing system that pushes towards convergence on the yaml-defined spec.
In general, there was a lot of buzz about multi-cluster deploys, large scale deployment, observability, and software supply chain.
On the SBOM (software bill of materials) side, one tidbit that was helpful to me was that it’s not adequate to have a single SBOM for an artifact. Especially in containers where we compile out a big pile of node_modules into some minified javascript.. we’ll miss those dependencies because they’re removed from the artifact before deploying. This means that we actually need to capture several SBOMs for a given artifact (think: one per layer/stage of the docker image).
There were a few things that I’m hoping to follow up on, especially related to our engagement in the open source world.
First, there was an awesome demo of this thing called backstage. It’s basically an open sourced version of cloud console that was released 2 years ago. I think their developer ergonomics are probably a little nicer than ours. We’ve had cloud console for like 6 years now. This would have been an awesome thing to open source. There was a standing room only crowd to listen to the talk about it.
Secondly, there are a lot of conversations happening in the k8s community around things that would be interesting to eBay. Most notably, multicluster deployments. I think folks are trying to solve problems similar to those we’ve solved with tess (kudos to that group for being ahead of the times!). If we’re not involved in those community discussions, there’s a real risk that the k8s community could head in a different direction than we’ve gone, which would make tess harder to support/train people on. I also think there’s an opportunity for eBay to be a thought leader in this space, given we’ve been operating multi-cluster deploys at scale for a while now.
Lastly, the ergonomics of k8s are really not that hard and there are products that make it simpler. I think we should probably be exposing a little bit of k8s into the dev stack. Not to say everyone has to go sling some yaml to deploy.. but it’s not magic under the hood. We need to increase our training here and help developers understand a little bit of what’s going on there. I think we broadly want to operate in a gitops model of software delivery, which should push us more in that direction.
There were a few cool vendors that I wanted to highlight:
First https://tracetest.io/ allows you to author and run tests based on OpenTelemetry traces. I don’t love this, because it’s more e2e tests which are brittle and slow, but it’s a pretty neat use of traces. I think this would be most useful in the mobile space.
Second, https://firehydrant.com/ is a runbooks-as-a-service company. They provide really fancy looking runbooks that trigger when your pager goes off (do pagers go off here? I don’t hear about it much). It also supports automation around incident management things. https://unskript.com/ was also another vendor in this space, which operate on top of Jupyter notebooks.
While an open source project and not a vendor, I was pretty impressed with https://crossplane.io/ which provides k8s-like yaml APIs for infrastructure (think: setting up new nodes, etc). I don’t think we have much infrastructure like this, but perhaps some of our data stores? Interesting all the same.