CloudFoundry BOSH

We plan to remove the powerdns job from the bosh director release due to its purpose being replaced by bosh-dns several years ago. If anybody still thinks this is important, please let us know.

Due to an incorrectly configured cleanup process, we've accidentally deleted the AMIs for light Jammy stemcells 1.55 through 1.75 in the us-gov-west-1 region. All other regions are unaffected. If you are using these versions in this region ... [More] , we would suggest either upgrading the stemcell to a newer unaffected version, or uploading the heavy version of the stemcell with the "--fix" flag to ensure your bosh director has a working copy of the stemcell. [Less]

FYI, if anyone wants to comment, there's a Google Doc which has the contents of the top email in this thread ("BOSH Director Performance on Jammy Stemcells FAQ") updated to reflect Benjamin's points: https://docs.google.com/document/d/1umE4SwaE0wxXNsQs3D4VdFufoBn06KjnHrzNltKIba8/edit?usp=sharing

Hi Benjamin, On Fri, Oct 28, 2022 at 1:32 PM Benjamin Gandon <benjamin@...> wrote: When you intend the Bosh Agent to remove the Clang compiler on non-compilation VMs, I’m wondering about how consistent this would be with ... [More] the env.bosh.remove_dev_tools option [1] in instance groups (defaulting to false), that already controls the removal of GCC on runtime VMs. I believe the mechanism we would use would be the env.bosh.remove_dev_tools property that you describe; if you removed GCC, you also would remove Clang.   With the design you expose, we would end up with GCC being kept on runtime VMs by default, with a possibility to remove it, whereas Clang would be systematically be removed. Good point! All the more reason to incorporate Clang-removal under the env.bosh.remove_dev_tools umbrella.   In order to have a consistent behavior about compilers removal, what about those two alternate solutions: 1. Add a new env.bosh.remove_clang option that would default to true. 2. Have Clang removal be controlled by the env.bosh.remove_dev_tools option, and make it true by default. After all, There are no Bosh Releases that need the compilers to be kept on runtime VMs. I love that idea! I'd like to remove the dev_tools by default. I'll bring it up with the team to see if there's any reason why it shouldn't default to true.   Second remark, there hasn’t been any public CentOS 7 Stemcell generated for ≈3 years, as the latest v3763.61 tag [2] was set in Dec 2019 [3]. I can’t reasonably think that any organization would actually run software on such old OS. That's good news for us.   It seems that the pipeline has been stopped back then, and might not have been maintained since then. Therefore in the Stemcells page [4], would it be possible to add a Deprecation Warning for CentOS 7 Stemcells? We always are willing to review pull requests! 😉 -- Brian Cunnie, 650.968.6262 [Less]

Thank you for the info, Brian.When you intend the Bosh Agent to remove the Clang compiler on non-compilation VMs, I’m wondering about how consistent this would be with the env.bosh.remove_dev_tools option [1] in instance groups (defaulting to false) ... [More] , that already controls the removal of GCC on runtime VMs.With the design ou expose, we would end up with GCC being kept on runtime VMs by default, with a possibility to remove it, whereas Clang would be systematically be removed.In order to have a consistent behavior about compilers removal, what about those two alternate solutions:1. Add a new env.bosh.remove_clang option that would default to true.2. Have Clang removal be controlled by the env.bosh.remove_dev_tools option, and make it true by default. After all, There are no Bosh Releases that need the compilers to be kept on runtime VMs.Second remark, there hasn’t been any public CentOS 7 Stemcell generated for ≈3 years, as the latest v3763.61 tag [2] was set in Dec 2019 [3]. I can’t reasonably think that any organization would actually run software on such old OS.It seems that the pipeline has been stopped back then, and might not have been maintained since then.Therefore in the Stemcells page [4], would it be possible to add a Deprecation Warning for CentOS 7 Stemcells?Best,Benjamin [1]: https://bosh.io/docs/manifest-v2/#instance-groups [2]: https://bosh.io/stemcells/#centos-7 [3]: https://github.com/cloudfoundry/bosh-linux-stemcell-builder/tree/centos-7/v3763.61 [4]: https://bosh.io/stemcells/ [Less]

BOSH Director Performance on Jammy Stemcells FAQ What’s the problem? Ruby programs using Ruby compiled with GCC (GNU Compiler Collection) on Jammy stemcells have a much larger RSS (Resident Set Size) memory footprint, which can cause memory ... [More] pressure. This affects Ruby-based programs such as the BOSH Director and the BOSH Azure, AWS, and vSphere CPIs. This can cause BOSH operations such as “bosh deploy” to take much longer and even time out. What’s the fix? We plan to compile the Ruby interpreter on Jammy with Clang (an Apple-sponsored GCC-compatible compiler). Ruby interpreters compiled with Clang don’t appear to have the same memory bloat when running the BOSH Director or the Ruby-based CPIs. How will we accomplish that? We plan to include the Clang compiler on the Jammy stemcells. We also plan to modify the Ruby BOSH package to use Clang if it’s available, otherwise fall back to GCC. Doesn’t the Clang compiler take up a lot of disk space? Yes, the Clang compiler takes up 700-800 MB of disk space; however, we plan to instruct the BOSH agent to remove the Clang compiler on boot unless the VM is a compilation VM. In other words, the Clang compiler won’t take up precious space on the root disk for the typically deployed VM. What about the Xenial, Bionic, and CentOS stemcells? We don’t plan to install Clang on Xenial and Bionic stemcells; they don’t exhibit the performance problem, so Clang has little to offer.We’re not sure whether the CentOS stemcell is affected. Other than memory, is there a performance impact of using Clang-based Ruby? In our testing, it appears that a Jammy-based Clang-based Ruby Director with a Clang-based vSphere CPI offers 5-25% performance boost over a Xenial-based GCC-based Ruby Director. What is the root cause of the problem? We’re not sure; it appears that the problem is related to the version of GCC used to compile Ruby. We notice that the memory footprint of Ruby’s threads grew from Ubuntu Disco to Ubuntu Eoan: 204 kiB → 10400 kiB. -- Brian Cunnie, 650.968.6262 [Less]

That makes sense, thanks!

It's derived from Borg, Google's internal cluster management system.BOSH was also created by 2 former Google engineers, which would explainthe influence :-)BOSH: borg++ (r+1=s, g+1=h)Taken from:https://tanzu.vmware.com/content/blog/comparing-bosh-ansible-chef-part-1

hi everyone,what does bosh stand for?thanks!

We've started tracking the work to finish up the removal of the registry from the bosh director: https://github.com/cloudfoundry/bosh/discussions/2355